CN111274230B - Data migration management method, device, equipment and storage medium - Google Patents
Data migration management method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN111274230B CN111274230B CN202010224116.9A CN202010224116A CN111274230B CN 111274230 B CN111274230 B CN 111274230B CN 202010224116 A CN202010224116 A CN 202010224116A CN 111274230 B CN111274230 B CN 111274230B
- Authority
- CN
- China
- Prior art keywords
- data migration
- occupation amount
- bandwidth occupation
- bandwidth
- time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000005012 migration Effects 0.000 title claims abstract description 283
- 238000013508 migration Methods 0.000 title claims abstract description 283
- 238000007726 management method Methods 0.000 title abstract description 32
- 230000004044 response Effects 0.000 claims abstract description 84
- 238000012549 training Methods 0.000 claims description 60
- 238000000034 method Methods 0.000 claims description 36
- 238000004891 communication Methods 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 11
- 230000002159 abnormal effect Effects 0.000 abstract description 7
- 230000008569 process Effects 0.000 description 8
- 230000001186 cumulative effect Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000012423 maintenance Methods 0.000 description 3
- 239000004065 semiconductor Substances 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000003795 chemical substances by application Substances 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006116 polymerization reaction Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/214—Database migration support
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0876—Network utilisation, e.g. volume of load or congestion level
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
- H04L67/62—Establishing a time schedule for servicing the requests
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Environmental & Geological Engineering (AREA)
- Computing Systems (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The embodiment of the invention provides a data migration management method, a device, equipment and a storage medium. The data migration management method is applied to a management platform of a data migration task, and after the data migration task is obtained, the first bandwidth occupation amount of the data migration task for the special line broadband to be utilized at the appointed moment is determined; wherein the specified time belongs to a time period between the earliest starting time and the latest starting time of the data migration task; determining a service response task utilizing a dedicated line broadband; predicting the bandwidth occupation amount of the service response task for the dedicated line broadband at the appointed moment to obtain a second bandwidth occupation amount; and stopping executing the data migration task at a designated moment when the sum of the first bandwidth occupation amount and the second bandwidth occupation amount meets a preset flow control condition corresponding to the special line broadband. The scheme can reduce the abnormal execution of service response tasks using the same special line broadband with data migration.
Description
Technical Field
The present invention relates to the field of data migration technologies, and in particular, to a method, an apparatus, a device, and a storage medium for managing data migration.
Background
To cope with the characteristics of mass data of internet services, the data may be stored by a storage cluster and service response tasks with respect to implementing the internet services may be performed by an execution cluster. On this basis, if the execution cluster needs to use the data stored in the storage cluster when executing the service response task, the data migration task needs to be executed before the task is executed. Among these, the so-called data migration tasks are: the data stored by the storage cluster is migrated to the task of executing the cluster.
However, the inventors found in the course of implementing the present invention that:
because of the massive data features of internet services, it is likely that a large amount of data will need to be migrated when performing data migration tasks, thus occupying a large amount of bandwidth. Then, in the scenario that the data migration task and the service response task share the same special line broadband, the problem that the service response task is abnormal in execution is easily caused in the special line broadband occupied by the data migration task, and the available bandwidth is not enough for the service response task to occupy.
Disclosure of Invention
The embodiment of the invention aims to provide a data migration management method, a device, equipment and a storage medium, so as to realize the effect of reducing the execution exception of a service response task using the same special line broadband with a data migration task. The specific technical scheme is as follows:
In a first aspect, an embodiment of the present invention provides a method for managing data migration, which is applied to a platform for managing a task of data migration, where the method includes:
after a data migration task is obtained, determining a first bandwidth occupation amount of the data migration task at a designated moment for a special line broadband to be utilized; wherein the specified time belongs to a time period between the earliest starting time and the latest starting time of the data migration task;
determining a service response task utilizing the dedicated line broadband;
predicting the bandwidth occupation amount of the service response task for the dedicated line broadband at the appointed moment to obtain a second bandwidth occupation amount;
and stopping executing the data migration task at the appointed moment when the sum value of the first bandwidth occupation amount and the second bandwidth occupation amount meets the preset flow control condition corresponding to the special line broadband.
Optionally, the sum of the first bandwidth occupation amount and the second bandwidth occupation amount meets a flow control condition corresponding to the dedicated line broadband, including:
and the sum value of the first bandwidth occupation amount and the second bandwidth occupation amount is larger than the bandwidth safety threshold corresponding to the special line broadband.
Optionally, the predicting the bandwidth occupation amount of the service response task for the dedicated line broadband at the designated time to obtain a second bandwidth occupation amount includes:
based on the first historical bandwidth occupation amount of the special line broadband at a plurality of continuous first historical moments of the service response task, predicting the bandwidth occupation amount of the special line broadband at the appointed moment by using a first time sequence prediction model obtained by pre-training, so as to obtain a second bandwidth occupation amount;
wherein any one first history time is before the appointed time, and the adjacent first history time is spaced by an appointed time step;
the first time sequence prediction model is a model which is obtained by training based on a first training sample set and is used for predicting bandwidth occupation amount, the first training sample set comprises a first sample sequence, the first sample sequence comprises first sample bandwidth occupation amount of the service response task for the special bandwidth at a plurality of continuous first sample moments, and the adjacent first sample moments are separated by the appointed time step.
Optionally, the plurality of first sample bandwidth occupation amounts are obtained by adopting the following steps:
And searching for the historical bandwidth occupation amount occupying the dedicated line broadband and taking the occupied time as the first sample time from a plurality of prestored historical bandwidth occupation amounts aiming at the service response task aiming at each first sample time to obtain the plurality of first sample bandwidth occupation amounts.
Optionally, the determining the first bandwidth occupation amount of the data migration task at the designated time for the dedicated line broadband to be utilized includes:
if the data migration task is started, acquiring the bandwidth occupation amount of the data migration task on the special line broadband by using a designated interface, and taking the bandwidth occupation amount of the data migration task on the special line broadband to be utilized at a designated moment as a first bandwidth occupation amount of the data migration task on the special line broadband to be utilized; the specified interface is an interface for recording the bandwidth occupation amount in a module for executing the data migration task;
the stopping the execution of the data migration task at the designated time includes:
and stopping data migration aiming at the data migration task at the appointed moment.
Optionally, the determining the first bandwidth occupation amount of the data migration task at the designated time for the dedicated line broadband to be utilized includes:
if the data migration task is not started, based on the second historical bandwidth occupation amount of the dedicated line broadband to be utilized at a plurality of continuous second historical moments of the data migration task, predicting and obtaining the first bandwidth occupation amount of the data migration task at a designated moment by using a second time sequence prediction model obtained by training in advance; wherein any second historical moment is before the earliest starting moment, and the adjacent second historical moment is separated by a designated time step;
The second time sequence prediction model is a model which is obtained based on training of a second training sample set and used for predicting bandwidth occupation amount, the second training sample set comprises a second sample sequence, the second sample sequence comprises second sample bandwidth occupation amounts of the data migration task for the to-be-utilized special bandwidth at a plurality of continuous second sample moments, and adjacent second sample moments are separated by appointed time steps;
the stopping the execution of the data migration task at the designated time includes:
if the data migration task is not started at the appointed time, stopping starting the data migration task at the appointed time;
and stopping data migration aiming at the data migration task at the appointed time if the data migration task is started at the appointed time.
In a second aspect, an embodiment of the present invention provides a management apparatus for data migration, which is applied to a management platform for a data migration task, where the apparatus includes:
the first bandwidth occupation amount determining module is used for determining the first bandwidth occupation amount of the data migration task at the appointed moment for the special line broadband to be utilized after the data migration task is obtained; wherein the specified time belongs to a time period between the earliest starting time and the latest starting time of the data migration task;
The second bandwidth occupation prediction module is used for determining a service response task using the dedicated line broadband; predicting the bandwidth occupation amount of the service response task for the dedicated line broadband at the appointed moment to obtain a second bandwidth occupation amount;
and the flow control module is used for stopping executing the data migration task at the appointed moment when the sum value of the first bandwidth occupation amount and the second bandwidth occupation amount accords with a preset flow control condition corresponding to the special line broadband.
Optionally, the sum of the first bandwidth occupation amount and the second bandwidth occupation amount meets a flow control condition corresponding to the dedicated line broadband, including:
and the sum value of the first bandwidth occupation amount and the second bandwidth occupation amount is larger than the bandwidth safety threshold corresponding to the special line broadband.
Optionally, the second bandwidth occupation amount prediction module is specifically configured to:
based on the first historical bandwidth occupation amount of the special line broadband at a plurality of continuous first historical moments of the service response task, predicting the bandwidth occupation amount of the special line broadband at the appointed moment by using a first time sequence prediction model obtained by pre-training, so as to obtain a second bandwidth occupation amount;
Wherein any one first history time is before the appointed time, and the adjacent first history time is spaced by an appointed time step;
the first time sequence prediction model is a model which is obtained by training based on a first training sample set and is used for predicting bandwidth occupation amount, the first training sample set comprises a first sample sequence, the first sample sequence comprises first sample bandwidth occupation amount of the service response task for the special bandwidth at a plurality of continuous first sample moments, and the adjacent first sample moments are separated by the appointed time step.
Optionally, the plurality of first sample bandwidth occupation amounts are obtained by adopting the following steps:
and searching for the historical bandwidth occupation amount occupying the dedicated line broadband and taking the occupied time as the first sample time from a plurality of prestored historical bandwidth occupation amounts aiming at the service response task aiming at each first sample time to obtain the plurality of first sample bandwidth occupation amounts.
Optionally, the first bandwidth occupation amount determining module is specifically configured to:
if the data migration task is started, acquiring the bandwidth occupation amount of the data migration task on the special line broadband by using a designated interface, and taking the bandwidth occupation amount of the data migration task on the special line broadband to be utilized at a designated moment as a first bandwidth occupation amount of the data migration task on the special line broadband to be utilized; the specified interface is an interface for recording the bandwidth occupation amount in a module for executing the data migration task;
The flow control module is specifically configured to:
and stopping data migration aiming at the data migration task at the appointed moment.
Optionally, the first bandwidth occupation amount determining module is specifically configured to:
if the data migration task is not started, based on the second historical bandwidth occupation amount of the dedicated line broadband to be utilized at a plurality of continuous second historical moments of the data migration task, predicting and obtaining the first bandwidth occupation amount of the data migration task at a designated moment by using a second time sequence prediction model obtained by training in advance; wherein any second historical moment is before the earliest starting moment, and the adjacent second historical moment is separated by a designated time step;
the second time sequence prediction model is a model which is obtained based on training of a second training sample set and used for predicting bandwidth occupation amount, the second training sample set comprises a second sample sequence, the second sample sequence comprises second sample bandwidth occupation amounts of the data migration task for the to-be-utilized special bandwidth at a plurality of continuous second sample moments, and adjacent second sample moments are separated by appointed time steps;
the flow control module is specifically configured to:
If the data migration task is not started at the appointed time, stopping starting the data migration task at the appointed time;
and stopping data migration aiming at the data migration task at the appointed time if the data migration task is started at the appointed time.
In a third aspect, an embodiment of the present invention provides an electronic device, including:
the device comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are in communication with each other through the bus; a memory for storing a computer program; and a processor, configured to execute a program stored in a memory, and implement the steps of the data migration management method provided in the first aspect.
In a fourth aspect, an embodiment of the present invention provides a computer readable storage medium having stored therein a computer program which, when executed by a processor, implements the steps of the method for managing data migration provided in the first aspect.
In the scheme provided by the invention, after the management platform of the data migration task obtains the data migration task, the first bandwidth occupation amount of the data migration task for the special line broadband to be utilized at the appointed moment is determined, and the service response task utilizing the special line broadband is determined; further predicting the bandwidth occupation amount of the service response task at the appointed time for the dedicated line broadband to obtain a second bandwidth occupation amount, and stopping executing the data migration task at the appointed time when the sum value of the first bandwidth occupation amount and the second bandwidth occupation amount accords with the flow control condition corresponding to the preset dedicated line bandwidth; and, the specified time belongs to a time period between the earliest start time and the latest start time of the data migration task. Therefore, the execution of the data migration task is stopped at the appointed moment, the expected execution of the data migration task can be considered, the bandwidth available for the service response task exists in the special line broadband used by the data migration task, and the service response task can be normally executed, so that the abnormal execution of the service response task which uses the same special line broadband as the data migration task is reduced.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below.
FIG. 1 is a flow chart illustrating a method for managing data migration according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a method for managing data migration according to another embodiment of the present invention;
FIG. 3 is an exemplary diagram of an application example of a method for managing data migration according to another embodiment of the present invention;
FIG. 4 is a schematic diagram of a management apparatus for data migration according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In order to enable those skilled in the art to better understand the technical solutions of the present invention, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The following first describes a method for managing data migration according to an embodiment of the present invention.
The data migration management method provided by the embodiment of the invention can be applied to a data migration task management platform, and the platform can specifically comprise a desktop computer, a portable computer, an internet television, an intelligent mobile terminal, a wearable intelligent terminal, a server and the like, is not limited herein, and any electronic device capable of realizing the embodiment of the invention can be used as the data migration task management platform, and belongs to the protection scope of the embodiment of the invention.
As shown in fig. 1, a flow of a method for managing data migration according to an embodiment of the present invention may include:
s101, after obtaining a data migration task, determining the first bandwidth occupation amount of the data migration task at a designated moment for the dedicated line broadband to be utilized. Wherein the specified time belongs to a time period between the earliest start time and the latest start time of the data migration task.
In a specific application, the data migration task may be acquired in multiple ways. The management platform of the data migration task may receive the data migration task submitted by the operation and maintenance personnel, or when the management platform of the data migration task is in communication connection with the execution cluster and the storage cluster and is used for creating the data migration task, the management platform of the data migration task may directly read the data migration task created by the management platform itself. Any manner of obtaining a data migration task may be used in the present invention, and this embodiment is not limited thereto.
The data migration task generally indicates the identity of the dedicated line broadband for the agent that is used by the data migration task, and the earliest and latest start times at which the data migration task is started. Therefore, in order to enable the data migration task to be executed as expected in the following process, and ensure that the dedicated line broadband used by the data migration task has a bandwidth available for the service response task, the service response task can be normally executed, and the control time for executing the data migration task can be a designated time belonging to a time period between the earliest starting time and the latest starting time of the data migration task. Thus, the first bandwidth occupation amount of the data migration task at the designated time for the dedicated line bandwidth to be utilized can be determined, so that it is subsequently determined in step S104 whether to stop executing the data migration task at the designated time.
The first bandwidth occupation amount may be determined in various manners. For example, since the bandwidth occupation amount of any data migration task at any moment is the same in a complete execution process, if the data migration task is already started, the bandwidth occupation amount of the data migration task can be collected in real time as the bandwidth occupation amount at the designated moment. Or, for example, since the bandwidth occupation amount of the same data migration is likely to have a certain rule in different execution processes, the first bandwidth occupation amount can be predicted by using a second time sequence prediction model based on the second historical bandwidth occupation amount of the data migration task before the earliest starting time. The manner in which the first bandwidth occupancy is determined is described in the form of alternative embodiments in the following for ease of understanding and rational layout.
Any manner of determining the first bandwidth occupancy may be used in the present invention, and the present embodiment is not limited thereto.
S102, determining service response tasks using the dedicated line broadband.
In particular, the task of determining service response utilizing dedicated line broadband may be varied. The service response task submitted by the operation and maintenance personnel can be received, or the service response task corresponding to the dedicated line broadband to be utilized can be searched from the corresponding relation between the pre-stored dedicated line broadband and the service response task.
S103, predicting the bandwidth occupation amount of the service response task at the appointed moment for the special line broadband to obtain a second bandwidth occupation amount.
In a specific application, predicting the bandwidth occupation amount of the service response task for the dedicated line broadband at the appointed time to obtain a second bandwidth occupation amount, which may be specifically based on the first historical bandwidth occupation amounts of the service response task for the dedicated line broadband at a plurality of continuous first historical time, and predicting the bandwidth occupation amount of the service response task for the dedicated line broadband at the appointed time by using a first time schedule prediction model obtained by pre-training to obtain the second bandwidth occupation amount; wherein, any one first history time is before the appointed time, and the adjacent first history time is spaced by an appointed time step; the first time sequence prediction model is a model which is obtained by training based on a first training sample set and is used for predicting bandwidth occupation amount, the first training sample set comprises a first sample sequence, the first sample sequence comprises first sample bandwidth occupation amount of a service response task for a special bandwidth at a plurality of continuous first sample moments, and time steps are designated at intervals of adjacent first samples.
This will be described in detail in the fig. 2 embodiment of the present invention for ease of understanding and rational layout.
And S104, stopping executing the data migration task at the appointed moment when the sum of the first bandwidth occupation amount and the second bandwidth occupation amount meets the flow control condition corresponding to the preset special line broadband.
In an optional implementation manner, the sum of the first bandwidth occupation amount and the second bandwidth occupation amount accords with a flow control condition corresponding to the dedicated line broadband, which specifically may include:
and the sum value of the first bandwidth occupation amount and the second bandwidth occupation amount is larger than the bandwidth safety threshold corresponding to the special line broadband.
For example, 10:00 of the specified time of 3 months and 2 days is predicted at 3 months and 1 day, and the service response task, such as a video playing task, occupies 800mb/s for a second bandwidth of the dedicated line broadband 'dedicated line broadband from Shanghai xx machine room to Beijing xx machine room'. The total usable bandwidth of the special line broadband from the Shanghai xx machine room to the Beijing xx machine room is 2000mb/s, the bandwidth safety line is 90%, namely the bandwidth safety threshold is 1800mb/s. When the sum of the first bandwidth occupation amount of the data migration task and the second bandwidth occupation amount of the service response task is greater than 1800mb/s, for example, 1200mb/s+800 mb/s=2000 mb/s > 1800mb/s, the sum of the first bandwidth occupation amount and the second bandwidth occupation amount is indicated to meet the flow control condition corresponding to the preset special line broadband. Thus, 10:00 of day 2, 3 months at the specified time, may stop performing data migration tasks.
Moreover, the manner in which the execution of the data migration task is stopped at a specified time may be varied in correspondence with different start states of the data migration task. The following detailed description is of alternative embodiments for proper layout and for easy understanding.
In the scheme provided by the invention, after the management platform of the data migration task obtains the data migration task, the first bandwidth occupation amount of the data migration task for the special line broadband to be utilized at the appointed moment is determined, and the service response task utilizing the special line broadband is determined; further predicting the bandwidth occupation amount of the service response task at the appointed time for the dedicated line broadband to obtain a second bandwidth occupation amount, and stopping executing the data migration task at the appointed time when the sum value of the first bandwidth occupation amount and the second bandwidth occupation amount accords with the flow control condition corresponding to the preset dedicated line bandwidth; and, the specified time belongs to a time period between the earliest start time and the latest start time of the data migration task. Therefore, the execution of the data migration task is stopped at the appointed moment, the expected execution of the data migration task can be considered, the bandwidth available for the service response task exists in the special line broadband used by the data migration task, and the service response task can be normally executed, so that the abnormal execution of the service response task which uses the same special line broadband as the data migration task is reduced.
In an optional implementation manner, the determining the first bandwidth occupation amount of the data migration task at the designated time for the dedicated line broadband to be utilized specifically may include the following steps:
if the data migration task is started, acquiring the bandwidth occupation amount of the data migration task on the special line broadband by using a designated interface, and taking the bandwidth occupation amount of the data migration task on the special line broadband to be utilized at a designated moment as a first bandwidth occupation amount of the data migration task on the special line broadband to be utilized; the interface is designated as an interface for recording the bandwidth occupation amount in a module for executing the data migration task;
correspondingly, the step of stopping executing the data migration task at the designated time may specifically include the following steps:
and stopping data migration for the data migration task at a specified time.
When the management platform of the data migration task is specifically MapReduce, the above specified interface may be an API (Application Programming Interface, application program interface) provided by the MapReduce authority for obtaining traffic, to obtain the bandwidth occupation amount of the data migration task at the current moment, as the first bandwidth occupation amount of the data migration task for the dedicated line broadband to be utilized at the specified moment. Wherein MapReduce is a cluster-based high-performance parallel computing platform for parallel operation of large-scale data sets (greater than 1 TB). Further, since the data migration task is already started at this time, the execution of the data migration task is stopped at a specified timing, specifically, the data migration for the data migration task is stopped at a specified timing.
The optional embodiment collects real bandwidth occupation amount of the special line broadband to be utilized by the data migration task in real time by utilizing the designated interface, and is beneficial to improving the accuracy of judging whether to stop executing the task when the data migration task is managed later.
In another optional implementation manner, the determining the first bandwidth occupation amount of the data migration task for the dedicated line broadband to be utilized at the designated time may specifically include the following steps:
if the data migration task is not started, based on the second historical bandwidth occupation amount of the data migration task at a plurality of continuous second historical moments of the dedicated line broadband to be utilized, predicting the first bandwidth occupation amount of the data migration task at the appointed moment by using a second time sequence prediction model obtained through pre-training; wherein, any second history time is before the earliest starting time, and the adjacent second history time is separated by a designated time step;
the second time sequence prediction model is a model which is obtained based on second training sample set training and is used for predicting bandwidth occupation amount, the second training sample set comprises a second sample sequence, the second sample sequence comprises a second sample bandwidth occupation amount of a data migration task for a to-be-utilized special bandwidth at a plurality of continuous second sample moments, and adjacent second sample moments are separated by appointed time steps;
Correspondingly, the step of stopping executing the data migration task at the designated time may specifically include the following steps:
if the data migration task is not started at the appointed time, stopping starting the data migration task at the appointed time;
and stopping data migration for the data migration task at the appointed time if the data migration task is started at the appointed time.
In a specific application, since the designated time belongs to a time period between the earliest starting time and the latest starting time of the data migration task, there are two cases that the data migration task is started or not started at the designated time. In contrast, if the data migration task is not started at the designated time, the data migration task can be stopped to be started at the designated time; if the data migration task is started at the designated time, the data migration for the data migration task may be stopped at the designated time. For the case of non-startup, the method and the device for data migration in the embodiment can reduce resource waste caused by stopping data migration after the data migration task is started by predicting the first bandwidth occupation amount obtained before the data migration task is started and comparing with the first bandwidth occupation amount obtained by real-time acquisition.
And similar to the second bandwidth occupation amount obtained by prediction, the first bandwidth occupation amount of the data migration task at the appointed moment can be obtained by prediction by using a second time sequence prediction model obtained by pre-training based on the second historical bandwidth occupation amount of the data migration task at a plurality of continuous second historical moments of the special line broadband to be utilized. The second time sequence prediction model is similar to the first time sequence prediction model, and the difference between the second time sequence prediction model and the first time sequence prediction model is that the sample training set utilized in the training process is different, and the same parts are not described herein again, and detailed description of the embodiment of fig. 2 and the optional embodiment of the present invention will be described later. The second sample bandwidth occupation amount at the successive second sample times is a bandwidth occupation amount obtained by using the designated interface.
As shown in fig. 2, a flow of a method for managing data migration according to another embodiment of the present invention may include:
s201, after obtaining the data migration task, determining the first bandwidth occupation amount of the data migration task at the appointed time for the dedicated line broadband to be utilized. Wherein the specified time belongs to a time period between the earliest start time and the latest start time of the data migration task.
S202, determining service response tasks using the dedicated line broadband.
The steps S201 to S202 are the same as the steps S101 to S102 in the embodiment of fig. 1 of the present invention, and are not repeated here, and detailed descriptions of the embodiment of fig. 1 of the present invention are described above.
S203, based on the first historical bandwidth occupation amount of the service response task on the dedicated line broadband at a plurality of continuous first historical moments, predicting the bandwidth occupation amount of the service response task on the dedicated line broadband at a designated moment by utilizing a first time sequence prediction model obtained by pre-training to obtain a second bandwidth occupation amount;
wherein, any one first history time is before the appointed time, and the adjacent first history time is spaced by an appointed time step; the first time sequence prediction model is a model which is obtained by training based on a first training sample set and is used for predicting bandwidth occupation amount, the first training sample set comprises a first sample sequence, the first sample sequence comprises first sample bandwidth occupation amount of a service response task for a special bandwidth at a plurality of continuous first sample moments, and time steps are designated at intervals of adjacent first samples.
In an alternative embodiment, the plurality of first sample bandwidth occupation amounts is obtained by:
and searching for the historical bandwidth occupation amount occupying the dedicated line broadband and taking the occupied time as the first sample time from a plurality of prestored historical bandwidth occupation amounts aiming at the service response task aiming at each first sample time to obtain a plurality of first sample bandwidth occupation amounts.
In a specific application, the pre-storing manner of the plurality of historical bandwidth occupation amounts may include the following steps:
the bandwidth occupation amount of the service response task on each special line broadband at each occupation moment is collected;
and correspondingly storing each occupied time, the identification record of each dedicated line broadband and the acquired bandwidth occupation amount to obtain a pre-stored historical bandwidth occupation amount.
For example, the bandwidth occupation amount of each dedicated line broadband at each occupation time can be collected at each occupation time by the service response task, and the pre-stored historical bandwidth occupation amount in the storage form of "< current time, source machine room, target machine room, and outlet flow >" is correspondingly stored in each occupation time, the identification record of each dedicated line broadband and the collected bandwidth occupation amount. The source machine room and the target machine room form a special line broadband identifier. In addition, the historical bandwidth occupation amount can be any database, wherein if the database is an analysis type database, for example, a guide database, the first sample bandwidth occupation amounts can be more efficiently found out from the prestored historical bandwidth occupation amounts. For example, pre-stored historical bandwidth occupancy includes:
< time (polymerization in minutes), source machine room, destination machine room, sum of outlet flows >
< 2019/10/08:00, "Shanghai xx machine room", "Beijing xx machine room", 800 mb/s)
< 2019/10/08:00, "Shanghai xx machine room", "Chongqing xx machine room", 700 mb/s-
< 2019/10/08:10, "Shanghai xx machine room", "Beijing xx machine room", 600 mb/s)
< 2019/10/08:10, "Shanghai xx machine room", "Chongqing xx machine room", 500 mb/s-
Based on the method, the bandwidth occupation amount of any two machine rooms, namely any special line broadband, in any period of time can be inquired. And, when the historical bandwidth occupation is recorded in an aggregate of minutes, and the designated time step of the first sample time is one hour, for example, 2019/10/08 10:00-2019/10/09 11:00, the queried historical bandwidth occupation can be averaged: sum of the outlet flows +.. When the first sample time is the same as the occupation time of the historical bandwidth occupation amount, the queried historical bandwidth occupation amount can be directly used as the first sample bandwidth occupation amount. Similarly, the second sample bandwidth occupancy may be obtained in the same manner, except that the bandwidth-occupied task is a data migration task.
The first timing prediction model may specifically be the following formula one:
wherein Y is t The second bandwidth occupancy at time t, mu is a constant term,for autoregressive cumulative values, a first historical bandwidth occupancy y for describing time t-i separated from time t by i time steps t-i Relation with second bandwidth occupancy at time t, p is hysteresis number of first historical bandwidth occupancy, gamma i As autocorrelation coefficient epsilon t Error term for autoregressive cumulative value corresponding to time t,/->For moving average cumulative value, for accumulating error term of autoregressive cumulative value, q is hysteresis number of error term, θ i Is a moving average coefficient; wherein, gamma i 、ε t θ i The parameters obtained for training.
And S204, stopping executing the data migration task at a designated moment when the sum of the first bandwidth occupation amount and the second bandwidth occupation amount meets the flow control condition corresponding to the preset special line broadband.
The step S204 is the same as the step S104 in the embodiment of fig. 1 of the present invention, and is not repeated here, and the detailed description of the embodiment of fig. 1 of the present invention is described above.
For ease of understanding, the process from acquisition of a training sample set to determining whether to stop performing a data migration task is described below in an exemplary form. As shown in fig. 3, since the service response task is executed by the execution cluster, when the server in the execution cluster executes the service response task, the bandwidth occupation amount of the server in the execution cluster to the dedicated line broadband may be collected in advance, so as to obtain and store a plurality of historical bandwidth occupation amounts, and further obtain a training sample set from the prestored plurality of historical bandwidth occupation amounts. On the basis, a first time sequence prediction model can be obtained by training a training sample set, and the first time sequence prediction model is provided for a data migration task management platform for use. When the data migration task management platform receives a data migration task submitted by an operation and maintenance person, the first bandwidth occupation amount of the data migration task can be determined, and the second bandwidth occupation amount is obtained by utilizing the first time sequence prediction model; and judging whether to stop executing the data migration task or not by using the first bandwidth occupation amount and the second bandwidth occupation amount.
Corresponding to the embodiment of the method, the embodiment of the invention also provides a management device for data migration.
As shown in fig. 4, a management device for data migration according to an embodiment of the present invention is applied to a management platform for data migration tasks, where the device may include:
the first bandwidth occupation amount determining module 401 is configured to determine, after obtaining a data migration task, a first bandwidth occupation amount of the data migration task at a designated time for a dedicated line broadband to be utilized; wherein the specified time belongs to a time period between the earliest starting time and the latest starting time of the data migration task;
a second bandwidth occupation prediction module 402, configured to determine a service response task using the dedicated line broadband; predicting the bandwidth occupation amount of the service response task for the dedicated line broadband at the appointed moment to obtain a second bandwidth occupation amount;
and the flow control module 403 is configured to stop executing the data migration task at the specified time when the sum of the first bandwidth occupation amount and the second bandwidth occupation amount meets a predetermined flow control condition corresponding to the dedicated line broadband.
In the scheme provided by the invention, after the management platform of the data migration task obtains the data migration task, the first bandwidth occupation amount of the data migration task for the special line broadband to be utilized at the appointed moment is determined, and the service response task utilizing the special line broadband is determined; further predicting the bandwidth occupation amount of the service response task at the appointed time for the dedicated line broadband to obtain a second bandwidth occupation amount, and stopping executing the data migration task at the appointed time when the sum value of the first bandwidth occupation amount and the second bandwidth occupation amount accords with the flow control condition corresponding to the preset dedicated line bandwidth; and, the specified time belongs to a time period between the earliest start time and the latest start time of the data migration task. Therefore, the execution of the data migration task is stopped at the appointed moment, the expected execution of the data migration task can be considered, the bandwidth available for the service response task exists in the special line broadband used by the data migration task, and the service response task can be normally executed, so that the abnormal execution of the service response task which uses the same special line broadband as the data migration task is reduced.
Optionally, the sum of the first bandwidth occupation amount and the second bandwidth occupation amount meets a flow control condition corresponding to the dedicated line broadband, including:
and the sum value of the first bandwidth occupation amount and the second bandwidth occupation amount is larger than the bandwidth safety threshold corresponding to the special line broadband.
Optionally, the second bandwidth occupation amount prediction module 402 is specifically configured to:
based on the first historical bandwidth occupation amount of the special line broadband at a plurality of continuous first historical moments of the service response task, predicting the bandwidth occupation amount of the special line broadband at the appointed moment by using a first time sequence prediction model obtained by pre-training, so as to obtain a second bandwidth occupation amount;
wherein any one first history time is before the appointed time, and the adjacent first history time is spaced by an appointed time step;
the first time sequence prediction model is a model which is obtained by training based on a first training sample set and is used for predicting bandwidth occupation amount, the first training sample set comprises a first sample sequence, the first sample sequence comprises first sample bandwidth occupation amount of the service response task for the special bandwidth at a plurality of continuous first sample moments, and the adjacent first sample moments are separated by the appointed time step.
Optionally, the plurality of first sample bandwidth occupation amounts are obtained by adopting the following steps:
and searching for the historical bandwidth occupation amount occupying the dedicated line broadband and taking the occupied time as the first sample time from a plurality of prestored historical bandwidth occupation amounts aiming at the service response task aiming at each first sample time to obtain the plurality of first sample bandwidth occupation amounts.
Optionally, the first bandwidth occupation amount determining module 401 is specifically configured to:
if the data migration task is started, acquiring the bandwidth occupation amount of the data migration task on the special line broadband by using a designated interface, and taking the bandwidth occupation amount of the data migration task on the special line broadband to be utilized at a designated moment as a first bandwidth occupation amount of the data migration task on the special line broadband to be utilized; the specified interface is an interface for recording the bandwidth occupation amount in a module for executing the data migration task;
the flow control module 403 is specifically configured to:
and stopping data migration aiming at the data migration task at the appointed moment.
Optionally, the first bandwidth occupation amount determining module 401 is specifically configured to:
if the data migration task is not started, based on the second historical bandwidth occupation amount of the dedicated line broadband to be utilized at a plurality of continuous second historical moments of the data migration task, predicting and obtaining the first bandwidth occupation amount of the data migration task at a designated moment by using a second time sequence prediction model obtained by training in advance; wherein any second historical moment is before the earliest starting moment, and the adjacent second historical moment is separated by a designated time step;
The second time sequence prediction model is a model which is obtained based on training of a second training sample set and used for predicting bandwidth occupation amount, the second training sample set comprises a second sample sequence, the second sample sequence comprises second sample bandwidth occupation amounts of the data migration task for the to-be-utilized special bandwidth at a plurality of continuous second sample moments, and adjacent second sample moments are separated by appointed time steps;
the flow control module 403 is specifically configured to:
if the data migration task is not started at the appointed time, stopping starting the data migration task at the appointed time;
and stopping data migration aiming at the data migration task at the appointed time if the data migration task is started at the appointed time.
Corresponding to the above embodiment, the embodiment of the present invention further provides an electronic device, as shown in fig. 5, where the device may include:
the device comprises a processor 501, a communication interface 502, a memory 503 and a communication bus 504, wherein the processor 501, the communication interface 502 and the memory complete communication with each other through the communication bus 504;
a memory 503 for storing a computer program;
The processor 501 is configured to implement the steps of the data migration management method according to any one of the above embodiments when executing the computer program stored in the memory 503.
In a specific application, the electronic device is a management platform for managing data migration tasks.
In the scheme provided by the invention, after the management platform of the data migration task obtains the data migration task, the first bandwidth occupation amount of the data migration task for the special line broadband to be utilized at the appointed moment is determined, and the service response task utilizing the special line broadband is determined; further predicting the bandwidth occupation amount of the service response task at the appointed time for the dedicated line broadband to obtain a second bandwidth occupation amount, and stopping executing the data migration task at the appointed time when the sum value of the first bandwidth occupation amount and the second bandwidth occupation amount accords with the flow control condition corresponding to the preset dedicated line bandwidth; and, the specified time belongs to a time period between the earliest start time and the latest start time of the data migration task. Therefore, the execution of the data migration task is stopped at the appointed moment, the expected execution of the data migration task can be considered, the bandwidth available for the service response task exists in the special line broadband used by the data migration task, and the service response task can be normally executed, so that the abnormal execution of the service response task which uses the same special line broadband as the data migration task is reduced.
The Memory may include RAM (Random Access Memory ) or NVM (Non-Volatile Memory), such as at least one magnetic disk Memory. Optionally, the memory may be at least one memory device located remotely from the processor.
The processor may be a general-purpose processor, including a CPU (Central Processing Unit ), NP (Network Processor, network processor), etc.; but also DSP (Digital Signal Processor ), ASIC (Application Specific Integrated Circuit, application specific integrated circuit), FPGA (Field-Programmable Gate Array, field programmable gate array) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components.
An embodiment of the present invention provides a computer readable storage medium, including an electronic device, where the computer readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the management method for data migration in any of the foregoing embodiments are implemented.
In the scheme provided by the invention, after the management platform of the data migration task obtains the data migration task, the first bandwidth occupation amount of the data migration task for the special line broadband to be utilized at the appointed moment is determined, and the service response task utilizing the special line broadband is determined; further predicting the bandwidth occupation amount of the service response task at the appointed time for the dedicated line broadband to obtain a second bandwidth occupation amount, and stopping executing the data migration task at the appointed time when the sum value of the first bandwidth occupation amount and the second bandwidth occupation amount accords with the flow control condition corresponding to the preset dedicated line bandwidth; and, the specified time belongs to a time period between the earliest start time and the latest start time of the data migration task. Therefore, the execution of the data migration task is stopped at the appointed moment, the expected execution of the data migration task can be considered, the bandwidth available for the service response task exists in the special line broadband used by the data migration task, and the service response task can be normally executed, so that the abnormal execution of the service response task which uses the same special line broadband as the data migration task is reduced.
In yet another embodiment of the present invention, a computer program product containing instructions that, when executed on a computer, cause the computer to perform the method of managing data migration described in any of the above embodiments is also provided.
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present invention, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wired (e.g., coaxial cable, fiber optic, DSL (Digital Subscriber Line), or wireless (e.g., infrared, radio, microwave, etc.) means, or semiconductor media (e.g., SSD State Disk, solid State Disk), etc., the computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage cluster comprising one or more available media integrated servers, data centers, etc., the available media may be magnetic media (e.g., floppy Disk, hard Disk, magnetic tape), optical media (e.g., DVD (Digital Versatile Disc, digital versatile Disk)), or semiconductor media (e.g., SSD State Disk, solid State Disk), etc.
In this document, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
In this specification, each embodiment is described in a related manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for the management apparatus and the electronic device embodiments of data migration, since they are substantially similar to the method embodiments, the description is relatively simple, and the relevant points are referred to in the description of the method embodiments.
The foregoing description is only of the preferred embodiments of the present invention and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.
Claims (14)
1. A method for managing data migration, which is applied to a management platform of a data migration task, the method comprising:
after a data migration task is obtained, determining a first bandwidth occupation amount of the data migration task at a designated moment for a special line broadband to be utilized; wherein the specified time belongs to a time period between the earliest starting time and the latest starting time of the data migration task;
determining a service response task utilizing the dedicated line broadband;
predicting the bandwidth occupation amount of the service response task for the dedicated line broadband at the appointed moment to obtain a second bandwidth occupation amount;
and stopping executing the data migration task at the appointed moment when the sum value of the first bandwidth occupation amount and the second bandwidth occupation amount meets the preset flow control condition corresponding to the special line broadband.
2. The method of claim 1, wherein the sum of the first bandwidth occupancy and the second bandwidth occupancy meets a flow control condition corresponding to the dedicated line broadband, comprising:
And the sum value of the first bandwidth occupation amount and the second bandwidth occupation amount is larger than the bandwidth safety threshold corresponding to the special line broadband.
3. The method according to claim 1 or 2, wherein predicting the bandwidth occupation amount of the service response task for the dedicated line broadband at the designated time to obtain a second bandwidth occupation amount includes:
based on the first historical bandwidth occupation amount of the special line broadband at a plurality of continuous first historical moments of the service response task, predicting the bandwidth occupation amount of the special line broadband at the appointed moment by using a first time sequence prediction model obtained by pre-training, so as to obtain a second bandwidth occupation amount;
wherein any one first history time is before the appointed time, and the adjacent first history time is spaced by an appointed time step;
the first time sequence prediction model is a model which is obtained based on training of a first training sample set and used for predicting bandwidth occupation amount, the first training sample set comprises a first sample sequence, the first sample sequence comprises first sample bandwidth occupation amount of the service response task for the dedicated line bandwidth at a plurality of continuous first sample moments, and the adjacent first sample moments are separated by the appointed time step.
4. A method according to claim 3, wherein the plurality of first sample bandwidth footprints are obtained by:
and searching for the historical bandwidth occupation amount occupying the dedicated line broadband and taking the occupied time as the first sample time from a plurality of prestored historical bandwidth occupation amounts aiming at the service response task aiming at each first sample time to obtain the plurality of first sample bandwidth occupation amounts.
5. The method according to claim 1 or 2, wherein the determining a first bandwidth occupancy of the data migration task for a dedicated line bandwidth to be utilized at a specified time comprises:
if the data migration task is started, acquiring the bandwidth occupation amount of the data migration task on the special line broadband by using a designated interface, and taking the bandwidth occupation amount of the data migration task on the special line broadband to be utilized at a designated moment as a first bandwidth occupation amount of the data migration task on the special line broadband to be utilized; the specified interface is an interface for recording the bandwidth occupation amount in a module for executing the data migration task;
the stopping the execution of the data migration task at the designated time includes:
and stopping data migration aiming at the data migration task at the appointed moment.
6. The method according to claim 1 or 2, wherein the determining a first bandwidth occupancy of the data migration task for a dedicated line bandwidth to be utilized at a specified time comprises:
if the data migration task is not started, based on the second historical bandwidth occupation amount of the dedicated line broadband to be utilized at a plurality of continuous second historical moments of the data migration task, predicting and obtaining the first bandwidth occupation amount of the data migration task at a designated moment by using a second time sequence prediction model obtained by training in advance; wherein any second historical moment is before the earliest starting moment, and the adjacent second historical moment is separated by a designated time step;
the second time sequence prediction model is a model which is obtained based on training of a second training sample set and used for predicting bandwidth occupation amount, the second training sample set comprises a second sample sequence, the second sample sequence comprises second sample bandwidth occupation amounts of the data migration task for the dedicated line bandwidth to be utilized at a plurality of continuous second sample moments, and adjacent second sample moments are separated by appointed time steps;
the stopping the execution of the data migration task at the designated time includes:
If the data migration task is not started at the appointed time, stopping starting the data migration task at the appointed time;
and stopping data migration aiming at the data migration task at the appointed time if the data migration task is started at the appointed time.
7. A management device for data migration, wherein the device is applied to a management platform for data migration tasks, and comprises:
the first bandwidth occupation amount determining module is used for determining the first bandwidth occupation amount of the data migration task at the appointed moment for the special line broadband to be utilized after the data migration task is obtained; wherein the specified time belongs to a time period between the earliest starting time and the latest starting time of the data migration task;
the second bandwidth occupation prediction module is used for determining a service response task using the dedicated line broadband; predicting the bandwidth occupation amount of the service response task for the dedicated line broadband at the appointed moment to obtain a second bandwidth occupation amount;
and the flow control module is used for stopping executing the data migration task at the appointed moment when the sum value of the first bandwidth occupation amount and the second bandwidth occupation amount accords with a preset flow control condition corresponding to the special line broadband.
8. The apparatus of claim 7, wherein the sum of the first bandwidth occupancy and the second bandwidth occupancy meets a flow control condition corresponding to the dedicated line bandwidth, comprising:
and the sum value of the first bandwidth occupation amount and the second bandwidth occupation amount is larger than the bandwidth safety threshold corresponding to the special line broadband.
9. The apparatus according to claim 7 or 8, wherein the second bandwidth occupation amount prediction module is specifically configured to:
based on the first historical bandwidth occupation amount of the special line broadband at a plurality of continuous first historical moments of the service response task, predicting the bandwidth occupation amount of the special line broadband at the appointed moment by using a first time sequence prediction model obtained by pre-training, so as to obtain a second bandwidth occupation amount;
wherein any one first history time is before the appointed time, and the adjacent first history time is spaced by an appointed time step;
the first time sequence prediction model is a model which is obtained based on training of a first training sample set and used for predicting bandwidth occupation amount, the first training sample set comprises a first sample sequence, the first sample sequence comprises first sample bandwidth occupation amount of the service response task for the dedicated line bandwidth at a plurality of continuous first sample moments, and the adjacent first sample moments are separated by the appointed time step.
10. The apparatus of claim 9, wherein the plurality of first sample bandwidth footprints are obtained by:
and searching for the historical bandwidth occupation amount occupying the dedicated line broadband and taking the occupied time as the first sample time from a plurality of prestored historical bandwidth occupation amounts aiming at the service response task aiming at each first sample time to obtain the plurality of first sample bandwidth occupation amounts.
11. The apparatus according to claim 7 or 8, wherein the first bandwidth occupancy determination module is specifically configured to:
if the data migration task is started, acquiring the bandwidth occupation amount of the data migration task on the special line broadband by using a designated interface, and taking the bandwidth occupation amount of the data migration task on the special line broadband to be utilized at a designated moment as a first bandwidth occupation amount of the data migration task on the special line broadband to be utilized; the specified interface is an interface for recording the bandwidth occupation amount in a module for executing the data migration task;
the flow control module is specifically configured to:
and stopping data migration aiming at the data migration task at the appointed moment.
12. The apparatus according to claim 7 or 8, wherein the first bandwidth occupancy determination module is specifically configured to:
If the data migration task is not started, based on the second historical bandwidth occupation amount of the dedicated line broadband to be utilized at a plurality of continuous second historical moments of the data migration task, predicting and obtaining the first bandwidth occupation amount of the data migration task at a designated moment by using a second time sequence prediction model obtained by training in advance; wherein any second historical moment is before the earliest starting moment, and the adjacent second historical moment is separated by a designated time step;
the second time sequence prediction model is a model which is obtained based on training of a second training sample set and used for predicting bandwidth occupation amount, the second training sample set comprises a second sample sequence, the second sample sequence comprises second sample bandwidth occupation amounts of the data migration task for the dedicated line bandwidth to be utilized at a plurality of continuous second sample moments, and adjacent second sample moments are separated by appointed time steps;
the flow control module is specifically configured to:
if the data migration task is not started at the appointed time, stopping starting the data migration task at the appointed time;
and stopping data migration aiming at the data migration task at the appointed time if the data migration task is started at the appointed time.
13. The electronic equipment is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the bus; a memory for storing a computer program; a processor for executing a program stored on a memory, implementing the method steps of any one of claims 1-6.
14. A computer-readable storage medium, characterized in that the storage medium has stored therein a computer program which, when executed by a processor, implements the method steps of any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010224116.9A CN111274230B (en) | 2020-03-26 | 2020-03-26 | Data migration management method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010224116.9A CN111274230B (en) | 2020-03-26 | 2020-03-26 | Data migration management method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111274230A CN111274230A (en) | 2020-06-12 |
| CN111274230B true CN111274230B (en) | 2024-03-08 |
Family
ID=71002725
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010224116.9A Active CN111274230B (en) | 2020-03-26 | 2020-03-26 | Data migration management method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111274230B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106528270A (en) * | 2016-11-16 | 2017-03-22 | 航天信息股份有限公司 | Automatic migration method and system of virtual machine based on OpenStack cloud platform |
| WO2018177107A1 (en) * | 2017-03-29 | 2018-10-04 | 腾讯科技（深圳）有限公司 | Data migration method, migration server, and storage medium |
| CN108900358A (en) * | 2018-08-01 | 2018-11-27 | 重庆邮电大学 | Virtual network function dynamic migration method based on deepness belief network resource requirement prediction |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104516767B (en) * | 2013-09-27 | 2018-01-02 | 国际商业机器公司 | The method and system of the re-transmission time of applications client during setting virtual machine (vm) migration |
| US20150200833A1 (en) * | 2014-01-10 | 2015-07-16 | Seagate Technology Llc | Adaptive Data Migration Using Available System Bandwidth |
| US9558005B2 (en) * | 2014-05-19 | 2017-01-31 | Intel Corporation | Reliable and deterministic live migration of virtual machines |
| CN107220263B (en) * | 2016-03-22 | 2021-09-03 | 阿里巴巴集团控股有限公司 | Optimization method, evaluation method, processing method and device for data migration |
-
2020
- 2020-03-26 CN CN202010224116.9A patent/CN111274230B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106528270A (en) * | 2016-11-16 | 2017-03-22 | 航天信息股份有限公司 | Automatic migration method and system of virtual machine based on OpenStack cloud platform |
| WO2018177107A1 (en) * | 2017-03-29 | 2018-10-04 | 腾讯科技（深圳）有限公司 | Data migration method, migration server, and storage medium |
| CN108664496A (en) * | 2017-03-29 | 2018-10-16 | 腾讯科技（深圳）有限公司 | Data migration method and device |
| CN108900358A (en) * | 2018-08-01 | 2018-11-27 | 重庆邮电大学 | Virtual network function dynamic migration method based on deepness belief network resource requirement prediction |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111274230A (en) | 2020-06-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108173905B (en) | Resource allocation method and device and electronic equipment | |
| CN107748790B (en) | Online service system, data loading method, device and equipment | |
| CN108243032B (en) | Method, device and equipment for acquiring service level information | |
| CN110083475B (en) | Abnormal data detection method and device | |
| CN109543891B (en) | Method and apparatus for establishing capacity prediction model, and computer-readable storage medium | |
| CN108762885B (en) | Virtual machine creating method and device, management equipment and terminal equipment | |
| CN113591068B (en) | Online login device management method and device and electronic device | |
| CN105337783B (en) | Monitor the method and device of communication equipment non-normal consumption flow | |
| CN111427628A (en) | Software function module configuration method, device, software product and storage medium | |
| CN111835536B (en) | Flow prediction method and device | |
| CN115269108A (en) | Data processing method, device and equipment | |
| CN111143070A (en) | Resource scheduling method and device | |
| CN109284193B (en) | Distributed data processing method based on multithreading and server | |
| CN110389817B (en) | Scheduling method, device and computer readable medium of multi-cloud system | |
| CN111061588A (en) | Method and device for locating database abnormal source | |
| CN111800807A (en) | Method and device for alarming number of base station users | |
| CN110928636A (en) | Virtual machine live migration method, device and equipment | |
| CN110543509B (en) | Monitoring system, method and device for user access data and electronic equipment | |
| CN111274230B (en) | Data migration management method, device, equipment and storage medium | |
| CN112650940A (en) | Recommendation method and device of application program, computer equipment and storage medium | |
| CN110955587A (en) | Method and device for determining equipment to be replaced | |
| CN113220705A (en) | Slow query identification method and device | |
| CN110708361A (en) | System, method and device for determining grade of digital content publishing user and server | |
| US9268809B2 (en) | Method and system for document update | |
| CN112491939B (en) | Multimedia resource scheduling method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |