CN115150415B - Control method, control device, electronic equipment and storage medium - Google Patents

Control method, control device, electronic equipment and storage medium Download PDF

Info

Publication number
CN115150415B
CN115150415B CN202210761970.8A CN202210761970A CN115150415B CN 115150415 B CN115150415 B CN 115150415B CN 202210761970 A CN202210761970 A CN 202210761970A CN 115150415 B CN115150415 B CN 115150415B
Authority
CN
China
Prior art keywords
value
bandwidth
data recovery
service bandwidth
parameter
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210761970.8A
Other languages
Chinese (zh)
Other versions
CN115150415A (en
Inventor
张人元
吴长福
夏俊明
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Lenovo Beijing Ltd
Original Assignee
Lenovo Beijing Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Lenovo Beijing Ltd filed Critical Lenovo Beijing Ltd
Priority to CN202210761970.8A priority Critical patent/CN115150415B/en
Publication of CN115150415A publication Critical patent/CN115150415A/en
Application granted granted Critical
Publication of CN115150415B publication Critical patent/CN115150415B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/50—Queue scheduling
    • H04L47/52—Queue scheduling by attributing bandwidth to queues

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

The application provides a control method, a control device, electronic equipment and a storage medium, wherein the control method comprises the following steps: acquiring a data recovery parameter and a service bandwidth value of the distributed file system, wherein the service bandwidth value is determined under the condition that the value of the data recovery bandwidth of the distributed file system meets a set condition; determining the influence degree of the data recovery bandwidth on the service bandwidth based on the service bandwidth value; and adjusting the data recovery parameters based on the influence degree to obtain adjusted data recovery parameters, so that the distributed file system sets the data recovery bandwidth based on the adjusted data recovery parameters.

Description

Control method, control device, electronic equipment and storage medium
Technical Field
The present application relates to the field of computer technologies, and in particular, to a control method, an apparatus, an electronic device, and a storage medium.
Background
With the increasing development of internet technology, a large amount of data is generated in the process of using the internet, and in order to conveniently store and process the data, a distributed storage system is generated. When the data is lost or damaged, the lost or damaged data can be recovered through the corresponding backup data, so that the safety and reliability of the data are greatly improved.
However, in the scenario that the process of recovering lost or damaged data and the process of writing and reading data to and from the distributed storage system by the service client exist at the same time, the bandwidth occupied by data recovery affects the bandwidth occupied by data writing and reading by the service client, so as to affect service execution, and how to control the bandwidth occupied by data recovery to reduce the effect on the bandwidth occupied by data writing and reading by the service client becomes a problem.
Disclosure of Invention
The application provides the following technical scheme:
in one aspect, the present application provides a control method, including:
acquiring a data recovery parameter of a distributed file system and a service bandwidth value, wherein the service bandwidth value is determined under the condition that the value of the data recovery bandwidth of the distributed file system meets a set condition;
Determining the influence degree of the data recovery bandwidth on the service bandwidth based on the service bandwidth value;
And adjusting the data recovery parameters based on the influence degree to obtain adjusted data recovery parameters, so that the distributed file system sets the data recovery bandwidth based on the adjusted data recovery parameters.
The determining, based on the service bandwidth value, the influence degree of the data recovery bandwidth on the service bandwidth includes:
Under the condition that the number of the acquired service bandwidth values reaches a set number, carrying out filtering processing on the service bandwidth values with the set number to obtain service bandwidth values after the filtering processing;
and determining the influence degree of the data recovery bandwidth on the service bandwidth based on the service bandwidth value after the filtering processing.
The service bandwidth value includes: a first traffic bandwidth value and a second traffic bandwidth value;
if the service bandwidth value is the first service bandwidth value, the value of the data recovery bandwidth of the distributed file system meets a set condition, including: the value of the data recovery bandwidth of the distributed file system is lower than a first set threshold;
If the service bandwidth value is the second service bandwidth value, the value of the data recovery bandwidth of the distributed file system meets the set condition, including: the value of the data recovery bandwidth of the distributed file system is higher than a second set threshold.
The determining, based on the service bandwidth value, the influence degree of the data recovery bandwidth on the service bandwidth includes:
if the second service bandwidth value is smaller than the first service bandwidth value and the difference value between the first service bandwidth value and the second service bandwidth value is larger than a first set service bandwidth threshold value, determining that the influence degree of the data recovery bandwidth on the service bandwidth is higher than a first set influence degree threshold value;
and if the second service bandwidth value is not smaller than the first service bandwidth value, determining that the influence degree of the data recovery bandwidth on the service bandwidth is lower than a second set influence degree threshold.
The method further comprises the steps of:
Determining a target service bandwidth interval to which the first service bandwidth value belongs from the set service bandwidth interval, and acquiring a recovery parameter template corresponding to the target service bandwidth interval;
and adjusting the data recovery parameter based on the influence degree to obtain an adjusted data recovery parameter, including:
And adjusting the data recovery parameters based on the influence degree and the recovery parameter template to obtain adjusted data recovery parameters.
Based on the influence degree and the recovery parameter template, adjusting the data recovery parameter comprises the following steps:
Determining at least one target parameter from the data recovery parameters based on the influence degree, wherein the parameter value of the target parameter and the parameter value of the target parameter in the recovery parameter template meet a set size condition;
Determining a processing strategy corresponding to the influence degree, wherein the processing strategy is a strategy for carrying out a widening process or a reducing process on the target parameter based on the association relation between the target parameter and the data recovery bandwidth;
And adjusting the parameter value of the target parameter based on the processing strategy.
If the distributed file system is Ceph, the data recovery parameters include data recovery parameters of a plurality of OSDs, and the determining at least one target parameter from the data recovery parameters based on the influence degree includes:
If the parameter value of at least one parameter in the data recovery parameters of the OSD and the parameter value of the corresponding data recovery parameter in the recovery parameter template meet the set size condition based on the influence degree, determining that the OSD is a target OSD;
Selecting a set number of target OSD from a plurality of target OSD under the condition that the target OSD is a plurality of target OSD;
and determining the parameters of the target OSD with the set number as target parameters.
Another aspect of the present application provides a control apparatus including:
The system comprises an acquisition module, a service bandwidth acquisition module and a control module, wherein the acquisition module is used for acquiring data recovery parameters of a distributed file system and service bandwidth values, and the service bandwidth values are determined under the condition that the values of the data recovery bandwidths of the distributed file system meet set conditions;
the determining module is used for determining the influence degree of the data recovery bandwidth on the service bandwidth based on the service bandwidth value;
And the adjusting module is used for adjusting the data recovery parameters based on the influence degree to obtain adjusted data recovery parameters so that the distributed file system sets the data recovery bandwidth based on the adjusted data recovery parameters.
A third aspect of the present application provides an electronic apparatus, comprising:
A memory and a processor;
The memory is used for storing at least one group of instruction sets;
The processor is configured to call and execute the instruction set in the memory, and execute the control method according to any one of the above by executing the instruction set.
A fourth aspect of the present application provides a storage medium storing a computer program for implementing the control method according to any one of the above, the computer program being executed by a processor to implement the control method according to any one of the above.
In the application, the influence degree of the data recovery bandwidth on the service bandwidth is determined based on the service bandwidth value by acquiring the data recovery parameter and the service bandwidth value of the distributed file system, and the data recovery parameter is adjusted based on the influence degree, so that the data recovery bandwidth is set by the distributed file system based on the adjusted data recovery parameter, the data recovery bandwidth is adjusted according to the service bandwidth value, the influence of the data recovery bandwidth on the service bandwidth is reduced, and the reliability of service execution is ensured.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort to a person skilled in the art.
Fig. 1 is a schematic flow chart of a control method provided in embodiment 1 of the present application;
FIG. 2 is a schematic diagram of an implementation scenario of a control method provided by the present application;
Fig. 3 is a flow chart of a control method provided in embodiment 2 of the present application;
FIG. 4 is a schematic diagram illustrating an adjustment of data recovery parameters according to the present application;
FIG. 5 is a schematic diagram of another adjustment of data recovery parameters provided by the present application;
FIG. 6 is a schematic diagram of a control device according to the present application;
fig. 7 is a schematic structural diagram of an electronic device provided by the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
In order that the above-recited objects, features and advantages of the present application will become more readily apparent, a more particular description of the application will be rendered by reference to the appended drawings and appended detailed description.
Referring to fig. 1, a flow chart of a control method provided in embodiment 1 of the present application is shown in fig. 1, and the method may include, but is not limited to, the following steps:
step S101, acquiring a data recovery parameter and a service bandwidth value of the distributed file system, wherein the service bandwidth value is determined when the value of the data recovery bandwidth of the distributed file system meets a set condition.
The distributed file system may include, but is not limited to: a plurality of nodes, each node may include at least one disk for storing data. The distributed file system can carry out redundant backup on data on the disk, and when the data is lost or lost due to the reasons of disk damage and the like, the lost or damaged data can be subjected to data recovery through the backup data.
The data recovery parameters of the distributed file system can be used for setting the data recovery bandwidth of the distributed file system. The data recovery bandwidth can be understood as: the distributed file system performs the bandwidth required for data recovery.
Specifically, the service bandwidth value is a value of a service bandwidth of the distributed file system determined in a case where a value of a data recovery bandwidth of the distributed file system satisfies a set condition. The traffic bandwidth can be understood as: the service client performs the bandwidth required by data reading and writing on the distributed file system.
In this embodiment, the distributed file system may include, but is not limited to: ceph. Accordingly, the data recovery parameters of the distributed file system may include: the data recovery parameters for each OSD in Ceph.
The data recovery parameters of the OSD may include, but are not limited to: any one or more of osd_recovery_sleep, osd_recovery_max_active, osd_max_ backfills, and osd_recovery_single_start. Wherein, osd_recovery_sleep represents Sleep time after osd dequeues, namely, the time interval of two Recovery operations; osd_recovery_max_active represents how many pg is doing recovery at the same time as one osd at most; osd_max_ backfills indicates how many pgs can be done simultaneously on one osd backfill (full backfill); osd_recovery_single_start represents the maximum number that one pg can initiate a data recovery operation.
Step S102, based on the service bandwidth value, determining the influence degree of the data recovery bandwidth on the service bandwidth.
The specific implementation of determining the influence degree of the data recovery bandwidth on the service bandwidth based on the service bandwidth value is not limited in the present application.
Specifically, this step may include, but is not limited to:
s1021, when the number of the acquired service bandwidth values reaches the set number, filtering the service bandwidth values with the set number to obtain the service bandwidth values after the filtering, wherein the set number is larger than 1.
In this embodiment, in the service client executing the service, the service client may cause fluctuation of the traffic of the distributed file system, which may cause abnormality in the collected service bandwidth values, so that a plurality of service bandwidth values may be collected, and the filtering process is performed on the set number of service bandwidth values when the number of the obtained service bandwidth values reaches the set number.
The filtering process for the set number of service bandwidth values may include, but is not limited to: and deleting the abnormal business bandwidth value in the business bandwidth values of the set number.
S1022, determining the influence degree of the data recovery bandwidth on the service bandwidth based on the service bandwidth value after the filtering process.
And step S103, adjusting the data recovery parameters based on the influence degree to obtain adjusted data recovery parameters so that the distributed file system sets the data recovery bandwidth based on the adjusted data recovery parameters.
It can be appreciated that the corresponding distributed file system includes Ceph, and the data recovery parameters of Ceph include: embodiments of data recovery parameters for multiple OSDs, adjusting the data recovery parameters based on the degree of impact may include: and respectively adjusting the data recovery parameters of each OSD based on the influence degree.
In this embodiment, an initialization module, a data acquisition driving module, a data recovery parameter configuration module, a filtering module, a cache module, and an adaptive recovery bandwidth regulation module may be set in the distributed file system. Next, a process of adjusting the data recovery parameter and further setting the data recovery bandwidth will be described in conjunction with the above modules. As shown in fig. 2, the initialization module initializes the data acquisition driving module, the data recovery parameter configuration module, the filtering module, the cache module and the adaptive recovery bandwidth regulation module; the data acquisition driving module acquires data recovery parameters and service bandwidth values of the distributed file system, the acquired data recovery parameters and service bandwidth values are sent to the filtering module, the filtering module can carry out filtering processing on the service bandwidth values with the set number under the condition that the number of the acquired service bandwidth values reaches the set number, and the data recovery parameters and the service bandwidth values after the filtering processing of the distributed file system are sent to the cache module;
The caching module caches the data recovery parameters of the distributed file system and the service bandwidth value after the filtering processing;
The self-adaptive recovery bandwidth regulation and control module acquires data recovery parameters of the distributed file system and service bandwidth values after filtering processing from the cache module, determines the influence degree of the data recovery bandwidth on the service bandwidth based on the service bandwidth values after filtering processing, adjusts the data recovery parameters based on the influence degree to obtain adjusted data recovery parameters, and sends the adjusted data recovery parameters to the data recovery parameter configuration module;
The data recovery parameter configuration module sets a data recovery bandwidth based on the adjusted data recovery parameter. The data recovery bandwidth affects the traffic bandwidth, and thus the traffic bandwidth value may also change after the data recovery bandwidth is reset.
It should be noted that, the data acquisition driving module may periodically acquire the data recovery parameter and the service bandwidth value of the distributed file system, so after the data recovery parameter configuration module sets the data recovery bandwidth based on the adjusted data recovery parameter, the data acquisition driving module may acquire the latest service bandwidth value, and under the condition that the data recovery parameter and the service bandwidth value are acquired each time, trigger the filtering module, the buffering module, the adaptive recovery bandwidth regulation module and the data recovery parameter configuration module to execute in sequence, so as to realize dynamic adjustment of the data recovery bandwidth.
Of course, in this embodiment, an initialization module, a data acquisition driving module, a data recovery parameter configuration module, a cache module, and an adaptive recovery bandwidth regulation module may also be set in the distributed file system. Wherein,
The data acquisition driving module acquires data recovery parameters and service bandwidth values of the distributed file system and sends the acquired data recovery parameters and service bandwidth values to the cache module;
The caching module caches the data recovery parameters and the service bandwidth values of the distributed file system;
The self-adaptive recovery bandwidth regulating module acquires data recovery parameters and service bandwidth values of the distributed file system from the cache module, can carry out filtering processing on the service bandwidth values of the set number under the condition that the number of the acquired service bandwidth values reaches the set number, determines the influence degree of the data recovery bandwidth on the service bandwidth based on the service bandwidth values after the filtering processing, adjusts the data recovery parameters based on the influence degree, acquires the adjusted data recovery parameters, and sends the adjusted data recovery parameters to the data recovery parameter configuration module;
The data recovery parameter configuration module sets a data recovery bandwidth based on the adjusted data recovery parameter.
In this embodiment, by acquiring the data recovery parameter and the service bandwidth value of the distributed file system, determining the influence degree of the data recovery bandwidth on the service bandwidth based on the service bandwidth value, and adjusting the data recovery parameter based on the influence degree to obtain the adjusted data recovery parameter, so that the distributed file system sets the data recovery bandwidth based on the adjusted data recovery parameter, the data recovery bandwidth is adjusted according to the service bandwidth value, the influence of the data recovery bandwidth on the service bandwidth is reduced, and the reliability of service execution is ensured.
As another alternative embodiment of the present application, referring to fig. 3, a schematic flow chart of an embodiment 2 of a control method provided by the present application is mainly a refinement of the control method described in the foregoing embodiment 1, and the method may include, but is not limited to, the following steps:
Step S201, acquiring a data recovery parameter of the distributed file system, a first service bandwidth value and a second service bandwidth value, where the first service bandwidth value is determined when the value of the data recovery bandwidth of the distributed file system is lower than a first set threshold, and the second service bandwidth value is determined when the value of the data recovery bandwidth of the distributed file system is higher than a second set threshold.
In this embodiment, the distributed file system may be controlled to be in a state where data recovery is not performed, so that the value of the data recovery bandwidth of the distributed file system is lower than a first set threshold, and the first service bandwidth value is determined to be obtained when the value of the data recovery bandwidth of the distributed file system is lower than the first set threshold.
On the basis of determining the first service bandwidth value, the distributed file system can be controlled to be in a state capable of data recovery, the value of the data recovery bandwidth of the distributed file system is higher than a second set threshold value under the condition that the distributed file system is in the state capable of data recovery, and the second service bandwidth value is determined to be obtained under the condition that the value of the data recovery bandwidth of the distributed file system is higher than the second set threshold value.
It is understood that the data recovery parameter of the distributed file system is determined in case the value of the data recovery bandwidth of the distributed file system is higher than the second set threshold.
The first set threshold value and the second set threshold value may be set as needed, and are not limited in the present application.
Step S201 is a specific implementation of step S101 in example 1.
Step S202, if the second service bandwidth value is smaller than the first service bandwidth value and the difference between the first service bandwidth value and the second service bandwidth value is larger than the first set service bandwidth threshold, determining that the influence degree of the data recovery bandwidth on the service bandwidth is higher than the first set influence degree threshold.
This step may include, but is not limited to:
s2021, when the number of the obtained first service bandwidth values reaches the set number, performing filtering processing on the set number of the first service bandwidth values to obtain the first service bandwidth values after the filtering processing;
S2022, if the second service bandwidth value is smaller than the first service bandwidth value after each filtering process, and the difference value between the first service bandwidth value and the second service bandwidth value after each filtering process is larger than the first set service bandwidth threshold, determining that the influence degree of the data recovery bandwidth on the service bandwidth is higher than the first set influence degree threshold.
Step S203, if the second service bandwidth value is not smaller than the first service bandwidth value, determining that the influence degree of the data recovery bandwidth on the service bandwidth is lower than a second set influence degree threshold.
Step S202 and step S203 are a specific implementation of step S102 in example 1.
Step S204, determining a target service bandwidth interval to which the first service bandwidth value belongs from the set service bandwidth interval, and obtaining a recovery parameter template corresponding to the target service bandwidth interval.
In this embodiment, the service bandwidth may be divided into a plurality of service bandwidth intervals according to the service requirement of the service client in advance, so as to obtain a set service bandwidth interval. For example, the traffic bandwidth is divided into 4 sections, namely [800MB/s,900MB/s ], [700MB/s,800MB/s ], [600MB/s,700MB/s ] and [500MB/s,600MB/s ], respectively, that is, the set traffic bandwidth sections include: [800MB/s,900MB/s ], [700MB/s,800MB/s ], [600MB/s,700MB/s ] and [500MB/s,600MB/s ].
Each service bandwidth interval in the set service bandwidth interval corresponds to a recovery parameter template respectively. The recovery parameter template includes a reference value of the data recovery parameter. For example, if the distributed file system is Ceph, as shown in table 1, the recovery parameter templates corresponding to [800,900] are service priority templates, the service priority templates include ：osd_max_backfills＝1,osd_recovery_max_active＝1,osd_recovery_single_start＝1,osd_recovery_sleep＝1;[700,800] that the recovery parameter templates corresponding to service sub-priority templates, the service sub-priority templates include ：osd_max_backfills＝50,osd_recovery_max_active＝5,osd_recovery_single_start＝5,osd_recovery_sleep＝0.25;[600,700] that the recovery parameter templates corresponding to service recovery balance templates, the service recovery balance templates include ：osd_max_backfills＝50,osd_recovery_max_active＝5,osd_recovery_single_start＝5,osd_recovery_sleep＝0.25;[500,600] that the recovery parameter templates corresponding to recovery priority templates include: osd_max_ backfills =50, osd_recovery_max_active=5, osd_recovery_single_start=5, osd_recovery_sleep=0.
TABLE 1
And step S205, adjusting the data recovery parameters based on the influence degree and the recovery parameter template to obtain adjusted data recovery parameters so that the distributed file system sets the data recovery bandwidth based on the adjusted data recovery parameters.
Step S205 is a specific implementation of step S103 in example 1.
Based on the influence degree and the recovery parameter template, the data recovery parameters are adjusted, and the adjusted data recovery parameters can include, but are not limited to:
S2051, determining at least one target parameter from the data recovery parameters based on the influence degree, wherein the parameter value of the target parameter and the parameter value of the target parameter in the recovery parameter template meet the set size condition.
If the influence degree is higher than the first set influence degree threshold, if the association relation between one parameter in the data recovery parameters and the data recovery bandwidth is that the larger the parameter value is, the smaller the data recovery bandwidth is, and if the parameter value of the parameter is smaller than the parameter value of the parameter in the recovery parameter template, the parameter is determined to be a target parameter; if the association relation between one of the data recovery parameters and the data recovery bandwidth is that the larger the parameter value is, the larger the data recovery bandwidth is, and if the parameter value of the parameter is larger than the parameter value of the parameter in the recovery parameter template, the parameter is determined to be the target parameter.
If the influence degree is lower than the second set influence degree threshold, if the association relation between one parameter in the data recovery parameters and the data recovery bandwidth is that the larger the parameter value is, the smaller the data recovery bandwidth is, and if the parameter value of the parameter is larger than the parameter value of the parameter in the recovery parameter template, the parameter is determined to be a target parameter; if the association relation between one of the data recovery parameters and the data recovery bandwidth is that the larger the parameter value is, the larger the data recovery bandwidth is, and if the parameter value of the parameter is smaller than the parameter value of the parameter in the recovery parameter template, the parameter is determined to be the target parameter.
Corresponding to the embodiment of the distributed file system being Ceph, in the case that the data recovery parameter includes data recovery parameters of a plurality of OSDs, determining at least one target parameter from the data recovery parameters based on the influence degree may include:
S20511, if the parameter value of at least one parameter in the data recovery parameters of the OSD and the parameter value of the corresponding data recovery parameter in the recovery parameter template meet the set size condition based on the influence degree, determining the OSD as the target OSD.
S20512, when there are a plurality of target OSDs, selecting a set number of target OSDs from the plurality of target OSDs.
S20513, determining the parameters of the target OSD with the set number as target parameters.
For example, if the influence degree is higher than the first set influence degree threshold, if the recovery parameter template corresponding to the target service bandwidth interval to which the first service bandwidth value belongs is a service priority template, if the osd_recovery_sleep of a certain OSD is smaller than 1, the osd_recovery_sleep of the OSD is the target parameter of the OSD; if the osd_max_ backfills of the OSD is greater than 1, the osd_max_ backfills of the OSD is the target parameter of the OSD; if the OSD osd_recovery_max_active is greater than 1, the osd_recovery_max_active of the OSD is the target parameter of the OSD; if the osd_recovery_single_start of the OSD is greater than 1, the osd_recovery_single_start of the OSD is the target parameter of the OSD; and, the OSD is a target OSD.
If the influence degree is lower than the second set influence degree threshold, if the recovery parameter template corresponding to the target service bandwidth interval to which the first service bandwidth value belongs is a recovery priority template, if the OSD osd_recovery_sleep of a certain OSD is greater than 0, the sd_recovery_sleep of the OSD is the target parameter of the OSD; if the OSD osd_max_ backfills is less than 50, the OSD osd_max_ backfills is the target parameter of the OSD; if the OSD osd_recovery_max_active is less than 5, the osd_recovery_max_active of the OSD is the target parameter of the OSD; if the osd_recovery_single_start of the OSD is less than 5, the osd_recovery_single_start of the OSD is the target parameter of the OSD; and, the OSD is a target OSD.
In this embodiment, the parameters of the target OSD with the set number are determined as the target parameters, so that the data recovery bandwidth can be prevented from being reduced or raised too fast, and the data recovery bandwidth is ensured to be adjusted more stably.
S2052, determining a processing strategy corresponding to the influence degree, wherein the processing strategy is a strategy for carrying out the enlargement processing or the reduction processing on the target parameter based on the association relation between the target parameter and the data recovery bandwidth.
In this embodiment, the processing strategies corresponding to the influence degree include: and a strategy for processing each target parameter. The policy for processing the target parameter may be a policy for performing a scaling-up process or a scaling-down process on the target parameter based on an association relationship between the target parameter and the data recovery bandwidth.
If the influence degree is higher than a first set influence degree threshold, if the association relation between the target parameter and the data recovery bandwidth is that the larger the parameter value of the target parameter is, the smaller the data recovery bandwidth is, the target parameter is determined to be subjected to the enlarging processing; and if the association relation between the target parameter and the data recovery bandwidth is that the larger the parameter value of the target parameter is, the larger the data recovery bandwidth is, determining to carry out reduction processing on the target parameter.
If the influence degree is lower than a second set influence degree threshold, if the association relation between the target parameter and the data recovery bandwidth is that the larger the parameter value of the target parameter is, the smaller the data recovery bandwidth is, the smaller the target parameter is determined to be subjected to the reduction processing; and if the association relation between the target parameter and the data recovery bandwidth is that the larger the parameter value of the target parameter is, the larger the data recovery bandwidth is, the target parameter is subjected to the enlarging processing.
S2053, adjusting the parameter value of the target parameter based on the processing strategy.
If the influence degree is higher than the first set influence degree threshold, adjusting the parameter value of the target parameter based on the processing strategy, so that the value of the data recovery bandwidth can be reduced, and the influence degree on the service bandwidth is reduced;
if the influence degree is lower than the second set influence degree threshold, the parameter value of the target parameter is adjusted based on the processing strategy, so that the value of the data recovery bandwidth can be increased, and the data recovery speed is improved.
For example, the adjustment priority of the osd_recovery_sleep is higher than the adjustment priority of the osd_max_ backfills, the adjustment priority of the osd_max_ backfills is higher than the adjustment priority of the osd_recovery_max_active, and the adjustment priority of the osd_recovery_max_active is higher than the adjustment priority of the osd_recovery_single_start.
If the influence degree is higher than the first set influence degree threshold, the processing strategy corresponding to the influence degree is to carry out the adjustment processing on the parameter value of the osd_recovery_sleep, and the adjustment processing is carried out on the parameter values of the osd_max_ backfills, osd _recovery_max_active and the osd_recovery_single_start.
If, as shown in fig. 4, if the recovery parameter template corresponding to the target service bandwidth interval to which the first service bandwidth value belongs is a service priority template, it is determined that the osd_recovery_sleep is a target parameter if the osd_recovery_sleep is less than 1, and then the osd_recovery_sleep is enlarged;
If the osd_recovery_sleep is not less than 1, the osd_recovery_sleep is a non-target parameter, whether the osd_max_ backfills is greater than 1 is continuously determined, if the osd_max_ backfills is greater than 1, the osd_max_ backfills is determined to be a target parameter, and the osd_max_ backfills is subjected to reduction processing;
If the osd_max_ backfills is not greater than 1, continuing to determine whether the osd_recovery_max_active is greater than 1, if the osd_recovery_max_active is greater than 1, determining that the osd_recovery_max_active is a target parameter, and performing reduction processing on the osd_recovery_max_active;
If the osd_recovery_max_active is not greater than 1, continuing to determine whether the osd_recovery_single_start is greater than 1, if so, determining that the osd_recovery_single_start is a target parameter, and performing reduction processing on the osd_recovery_single_start.
For example, if the influence level is lower than the second set influence level threshold, the processing policy corresponding to the influence level is to perform a scaling-down process on the parameter value of osd_recovery_sleep, and to perform a scaling-up process on the parameter values of osd_max_ backfills, osd _recovery_max_active and osd_recovery_single_start. If, as shown in fig. 5, if the recovery parameter template corresponding to the target service bandwidth interval to which the first service bandwidth value belongs is a recovery priority template, it is determined that the osd_recovery_sleep is the target parameter if the osd_recovery_sleep is greater than 0, and then the osd_recovery_sleep is reduced;
if the osd_recovery_sleep is not greater than 0, the osd_recovery_sleep is a non-target parameter, whether the osd_max_ backfills is smaller than 50 is continuously determined, if the osd_max_ backfills is smaller than 50, the osd_max_ backfills is determined to be a target parameter, and the osd_max_ backfills is subjected to the enlargement processing;
If the osd_max_ backfills is not less than 50, continuing to determine whether the osd_recovery_max_active is less than 5, if the osd_recovery_max_active is less than 5, determining that the osd_recovery_max_active is a target parameter, and performing the enlarging process on the osd_recovery_max_active;
If the osd_recovery_max_active is not less than 5, continuously determining whether the osd_recovery_single_start is less than 5, if the osd_recovery_max_active is less than 5, determining that the osd_recovery_single_start is a target parameter, and performing the scaling-up processing on the osd_recovery_single_start.
Corresponding to the embodiment of the control method provided by the application, the application provides an embodiment of a control device.
Referring to fig. 6, the control device may include: the device comprises a first acquisition module 100, a first determination module 200 and an adjustment module 300.
The first obtaining module 100 is configured to obtain a data recovery parameter of the distributed file system and a service bandwidth value, where the service bandwidth value is determined when a value of the data recovery bandwidth of the distributed file system meets a set condition.
The first determining module 200 is configured to determine, based on the service bandwidth value, a degree of influence of the data recovery bandwidth on the service bandwidth.
The adjustment module 300 is configured to adjust the data recovery parameter based on the influence degree, so as to obtain an adjusted data recovery parameter, so that the distributed file system sets the data recovery bandwidth based on the adjusted data recovery parameter.
In this embodiment, the first determining module 200 may specifically be configured to:
Under the condition that the number of the acquired service bandwidth values reaches the set number, filtering the service bandwidth values with the set number to obtain the service bandwidth values after the filtering;
And determining the influence degree of the data recovery bandwidth on the service bandwidth based on the service bandwidth value after the filtering process.
In this embodiment, the service bandwidth value may include: a first traffic bandwidth value and a second traffic bandwidth value;
If the service bandwidth value is the first service bandwidth value, the data recovery bandwidth value of the distributed file system meets the set condition, including: the value of the data recovery bandwidth of the distributed file system is lower than a first set threshold;
If the service bandwidth value is the second service bandwidth value, the data recovery bandwidth value of the distributed file system meets the set condition, including: the value of the data recovery bandwidth of the distributed file system is above a second set threshold.
In this embodiment, the first determining module 200 may specifically be configured to:
If the second service bandwidth value is smaller than the first service bandwidth value and the difference value between the first service bandwidth value and the second service bandwidth value is larger than the first set service bandwidth threshold value, determining that the influence degree of the data recovery bandwidth on the service bandwidth is higher than the first set influence degree threshold value;
and if the second service bandwidth value is not smaller than the first service bandwidth value, determining that the influence degree of the data recovery bandwidth on the service bandwidth is lower than a second set influence degree threshold.
In this embodiment, the control device may further include:
The second determining module is used for determining a target service bandwidth interval to which the first service bandwidth value belongs from the set service bandwidth interval;
And the second acquisition module is used for acquiring a recovery parameter template corresponding to the target service bandwidth interval.
The adjustment module 300 may be specifically configured to:
And adjusting the data recovery parameters based on the influence degree and the recovery parameter template to obtain adjusted data recovery parameters.
In this embodiment, the adjusting module 300 adjusts the data recovery parameter based on the influence degree and the recovery parameter template, which may specifically include:
Determining at least one target parameter from the data recovery parameters based on the influence degree, wherein the parameter value of the target parameter and the parameter value of the target parameter in the recovery parameter template meet the set size condition;
determining a processing strategy corresponding to the influence degree, wherein the processing strategy is a strategy for carrying out the enlarging processing or the reducing processing on the target parameter based on the association relation between the target parameter and the data recovery bandwidth;
And adjusting the parameter value of the target parameter based on the processing strategy.
In this embodiment, if the distributed file system is Ceph, the data recovery parameters include data recovery parameters of a plurality of OSDs, and the process of determining at least one target parameter from the data recovery parameters by the adjustment module 300 based on the influence degree may specifically include:
if the parameter value of at least one parameter in the data recovery parameters of the OSD and the parameter value of the corresponding data recovery parameter in the recovery parameter template meet the set size condition based on the influence degree, determining the OSD as a target OSD;
Selecting a set number of target OSD from the target OSD under the condition that the target OSD has a plurality of target OSD;
and determining the parameters of the target OSD with the set number as target parameters.
Corresponding to the embodiment of the control method provided by the application, the application also provides an embodiment of the electronic equipment applying the control method.
As shown in fig. 7, a schematic structural diagram of an embodiment 1 of an electronic device according to the present application may include the following structures:
a memory 10 and a processor 20.
A memory 10 for storing at least one set of instructions;
a processor 20 for calling and executing the instruction set in said memory 10, by executing the instruction set, performing the control method as described in embodiment 1 or 2.
The present application also provides an embodiment of a storage medium corresponding to the above-mentioned embodiment of a control method provided by the present application.
In this embodiment, a storage medium stores a computer program implementing the control method according to any one of the foregoing embodiments, the computer program being executed by a processor to implement the control method according to any one of the foregoing embodiments.
It should be noted that, in each embodiment, the differences from the other embodiments are emphasized, and the same similar parts between the embodiments are referred to each other. For the apparatus class embodiments, the description is relatively simple as it is substantially similar to the method embodiments, and reference is made to the description of the method embodiments for relevant points.
Finally, it is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
For convenience of description, the above devices are described as being functionally divided into various units, respectively. Of course, the functions of each element may be implemented in the same piece or pieces of software and/or hardware when implementing the present application.
From the above description of embodiments, it will be apparent to those skilled in the art that the present application may be implemented in software plus a necessary general hardware platform. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the embodiments or some parts of the embodiments of the present application.
The above describes in detail a control method, apparatus, electronic device and storage medium provided by the present application, and specific examples are applied herein to illustrate the principles and embodiments of the present application, where the above examples are only used to help understand the method and core idea of the present application; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present application, the present description should not be construed as limiting the present application in view of the above.

Claims (8)

1. A control method, comprising:
acquiring a data recovery parameter of a distributed file system and a service bandwidth value, wherein the service bandwidth value is determined under the condition that the value of the data recovery bandwidth of the distributed file system meets a set condition;
Determining the influence degree of the data recovery bandwidth on the service bandwidth based on the service bandwidth value;
Based on the influence degree, adjusting the data recovery parameters to obtain adjusted data recovery parameters, so that the distributed file system sets the data recovery bandwidth based on the adjusted data recovery parameters; if the influence degree is higher than a first set influence degree threshold, adjusting the parameter value of the target parameter to reduce the value of the data recovery bandwidth so as to reduce the influence degree on the service bandwidth; if the influence degree is lower than a second set influence degree threshold, adjusting the parameter value of the target parameter to increase the value of the data recovery bandwidth so as to increase the data recovery speed;
Wherein the service bandwidth value includes: a first traffic bandwidth value and a second traffic bandwidth value;
if the service bandwidth value is the first service bandwidth value, the value of the data recovery bandwidth of the distributed file system meets a set condition, including: the value of the data recovery bandwidth of the distributed file system is lower than a first set threshold;
if the service bandwidth value is the second service bandwidth value, the value of the data recovery bandwidth of the distributed file system meets the set condition, including: the value of the data recovery bandwidth of the distributed file system is higher than a second set threshold;
Wherein the determining, based on the service bandwidth value, the extent of the influence of the data recovery bandwidth on the service bandwidth includes:
if the second service bandwidth value is smaller than the first service bandwidth value and the difference value between the first service bandwidth value and the second service bandwidth value is larger than a first set service bandwidth threshold value, determining that the influence degree of the data recovery bandwidth on the service bandwidth is higher than a first set influence degree threshold value;
and if the second service bandwidth value is not smaller than the first service bandwidth value, determining that the influence degree of the data recovery bandwidth on the service bandwidth is lower than a second set influence degree threshold.
2. The method of claim 1, the determining, based on the traffic bandwidth value, a degree of impact of the data recovery bandwidth on the traffic bandwidth, comprising:
Under the condition that the number of the acquired service bandwidth values reaches a set number, carrying out filtering processing on the service bandwidth values with the set number to obtain service bandwidth values after the filtering processing;
and determining the influence degree of the data recovery bandwidth on the service bandwidth based on the service bandwidth value after the filtering processing.
3. The method of claim 1, the method further comprising:
Determining a target service bandwidth interval to which the first service bandwidth value belongs from the set service bandwidth interval, and acquiring a recovery parameter template corresponding to the target service bandwidth interval;
and adjusting the data recovery parameter based on the influence degree to obtain an adjusted data recovery parameter, including:
And adjusting the data recovery parameters based on the influence degree and the recovery parameter template to obtain adjusted data recovery parameters.
4. A method according to claim 3, wherein adjusting the data recovery parameters based on the degree of influence and the recovery parameter template comprises:
Determining at least one target parameter from the data recovery parameters based on the influence degree, wherein the parameter value of the target parameter and the parameter value of the target parameter in the recovery parameter template meet a set size condition;
Determining a processing strategy corresponding to the influence degree, wherein the processing strategy is a strategy for carrying out a widening process or a reducing process on the target parameter based on the association relation between the target parameter and the data recovery bandwidth;
And adjusting the parameter value of the target parameter based on the processing strategy.
5. The method of claim 4, wherein if the distributed file system is Ceph, the data recovery parameters include data recovery parameters of a plurality of OSDs, and determining at least one target parameter from the data recovery parameters based on the degree of influence comprises:
If the parameter value of at least one parameter in the data recovery parameters of the OSD and the parameter value of the corresponding data recovery parameter in the recovery parameter template meet the set size condition based on the influence degree, determining that the OSD is a target OSD;
Selecting a set number of target OSD from a plurality of target OSD under the condition that the target OSD is a plurality of target OSD;
and determining the parameters of the target OSD with the set number as target parameters.
6. A control apparatus comprising:
The system comprises an acquisition module, a service bandwidth acquisition module and a control module, wherein the acquisition module is used for acquiring data recovery parameters of a distributed file system and service bandwidth values, and the service bandwidth values are determined under the condition that the values of the data recovery bandwidths of the distributed file system meet set conditions;
the determining module is used for determining the influence degree of the data recovery bandwidth on the service bandwidth based on the service bandwidth value;
The adjusting module is used for adjusting the data recovery parameters based on the influence degree to obtain adjusted data recovery parameters so that the distributed file system sets the data recovery bandwidth based on the adjusted data recovery parameters; if the influence degree is higher than a first set influence degree threshold, adjusting the parameter value of the target parameter to reduce the value of the data recovery bandwidth so as to reduce the influence degree on the service bandwidth; if the influence degree is lower than a second set influence degree threshold, adjusting the parameter value of the target parameter to increase the value of the data recovery bandwidth so as to increase the data recovery speed;
Wherein the service bandwidth value includes: a first traffic bandwidth value and a second traffic bandwidth value;
if the service bandwidth value is the first service bandwidth value, the value of the data recovery bandwidth of the distributed file system meets a set condition, including: the value of the data recovery bandwidth of the distributed file system is lower than a first set threshold;
if the service bandwidth value is the second service bandwidth value, the value of the data recovery bandwidth of the distributed file system meets the set condition, including: the value of the data recovery bandwidth of the distributed file system is higher than a second set threshold;
Wherein the determining, based on the service bandwidth value, the extent of the influence of the data recovery bandwidth on the service bandwidth includes:
if the second service bandwidth value is smaller than the first service bandwidth value and the difference value between the first service bandwidth value and the second service bandwidth value is larger than a first set service bandwidth threshold value, determining that the influence degree of the data recovery bandwidth on the service bandwidth is higher than a first set influence degree threshold value;
and if the second service bandwidth value is not smaller than the first service bandwidth value, determining that the influence degree of the data recovery bandwidth on the service bandwidth is lower than a second set influence degree threshold.
7. An electronic device, comprising:
A memory and a processor;
The memory is used for storing at least one group of instruction sets;
The processor is configured to invoke and execute the instruction set in the memory, and execute the control method according to any of claims 1-5 by executing the instruction set.
8. A storage medium storing a computer program implementing the control method according to any one of claims 1-5, the computer program being executed by a processor implementing the control method according to any one of claims 1-5.
CN202210761970.8A 2022-06-30 2022-06-30 Control method, control device, electronic equipment and storage medium Active CN115150415B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210761970.8A CN115150415B (en) 2022-06-30 2022-06-30 Control method, control device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210761970.8A CN115150415B (en) 2022-06-30 2022-06-30 Control method, control device, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN115150415A CN115150415A (en) 2022-10-04
CN115150415B true CN115150415B (en) 2024-04-19

Family

ID=83411067

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210761970.8A Active CN115150415B (en) 2022-06-30 2022-06-30 Control method, control device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN115150415B (en)

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107026708A (en) * 2016-02-02 2017-08-08 星辰天合（北京）数据科技有限公司 Data processing method and device
CN107391317A (en) * 2017-09-14 2017-11-24 郑州云海信息技术有限公司 A kind of method, apparatus of data recovery, equipment and computer-readable recording medium
CN107943617A (en) * 2017-11-17 2018-04-20 北京联想超融合科技有限公司 Restorative procedure, device and the server cluster of data
CN109213637A (en) * 2018-11-09 2019-01-15 浪潮电子信息产业股份有限公司 Data recovery method, device and medium for cluster nodes of distributed file system
CN110417677A (en) * 2019-07-29 2019-11-05 北京易捷思达科技发展有限公司 A kind of QoS control method based on Ceph distributed storage Osd end data Recovery
CN111917823A (en) * 2020-06-17 2020-11-10 烽火通信科技股份有限公司 Data reconstruction method and device based on distributed storage Ceph
CN112015326A (en) * 2019-05-28 2020-12-01 浙江宇视科技有限公司 Cluster data processing method, device, equipment and storage medium
WO2020258746A1 (en) * 2019-06-28 2020-12-30 京信通信系统（中国）有限公司 Data transmission processing method, device, and communication apparatus
CN113946274A (en) * 2020-07-15 2022-01-18 浙江宇视科技有限公司 Data processing method, device, equipment and medium

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10498815B2 (en) * 2015-05-21 2019-12-03 Netapp Inc. Data recovery objective modeling

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107026708A (en) * 2016-02-02 2017-08-08 星辰天合（北京）数据科技有限公司 Data processing method and device
CN107391317A (en) * 2017-09-14 2017-11-24 郑州云海信息技术有限公司 A kind of method, apparatus of data recovery, equipment and computer-readable recording medium
CN107943617A (en) * 2017-11-17 2018-04-20 北京联想超融合科技有限公司 Restorative procedure, device and the server cluster of data
CN109213637A (en) * 2018-11-09 2019-01-15 浪潮电子信息产业股份有限公司 Data recovery method, device and medium for cluster nodes of distributed file system
CN112015326A (en) * 2019-05-28 2020-12-01 浙江宇视科技有限公司 Cluster data processing method, device, equipment and storage medium
WO2020258746A1 (en) * 2019-06-28 2020-12-30 京信通信系统（中国）有限公司 Data transmission processing method, device, and communication apparatus
CN110417677A (en) * 2019-07-29 2019-11-05 北京易捷思达科技发展有限公司 A kind of QoS control method based on Ceph distributed storage Osd end data Recovery
CN111917823A (en) * 2020-06-17 2020-11-10 烽火通信科技股份有限公司 Data reconstruction method and device based on distributed storage Ceph
CN113946274A (en) * 2020-07-15 2022-01-18 浙江宇视科技有限公司 Data processing method, device, equipment and medium

Also Published As

Publication number Publication date
CN115150415A (en) 2022-10-04

Similar Documents

Publication Publication Date Title
CN110996352B (en) Flow control method, device, computer equipment and storage medium
JP6457447B2 (en) Data center network traffic scheduling method and apparatus
US8719297B2 (en) System for managing data collection processes
CN110399102B (en) Data writing method, device and equipment of distributed storage system and storage medium
US12001289B2 (en) Data recovery method, apparatus and device, and readable storage medium
CN112866136B (en) Service data processing method and device
CN108564951A (en) The method that intelligence reduces voice control device false wake-up probability
CN112165436B (en) Flow control method, device and system
CN111212114B (en) Method and device for downloading resource file
CN107402863B (en) Method and equipment for processing logs of service system through log system
CN111291252B (en) Method and device for adjusting query rate per second, electronic equipment and storage medium
CN115150415B (en) Control method, control device, electronic equipment and storage medium
US11140089B2 (en) Data writing method, system, device and computer-readable storage medium
CN107040566B (en) Service processing method and device
US11599184B2 (en) Throughput-optimized, quality-of-service aware power capping system
CN116863980B (en) Dynamic adjusting circuit and method for gating signals
WO2018228323A1 (en) Service level control method and system for on-line service system, and readable storage medium
CN102143206A (en) Storage pool regulation method, device and system for cluster storage system
CN111309480A (en) Method and equipment for dynamic power consumption capping regulation and control
CN111708799A (en) Spark task processing method and device, electronic equipment and storage medium
CN115774605A (en) Kubernetes prediction type elastic expansion method and system
US20230214725A1 (en) Method and apparatus for multiple reinforcement learning agents in a shared environment
CN111930304B (en) Data acquisition speed control method, equipment and storage medium
CN110278233B (en) Load adjusting method and device
CN107196962B (en) User access control method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant