WO2016197706A1 - Procédé et dispositif de migration de données - Google Patents

Procédé et dispositif de migration de données Download PDF

Info

Publication number
WO2016197706A1
WO2016197706A1 PCT/CN2016/079915 CN2016079915W WO2016197706A1 WO 2016197706 A1 WO2016197706 A1 WO 2016197706A1 CN 2016079915 W CN2016079915 W CN 2016079915W WO 2016197706 A1 WO2016197706 A1 WO 2016197706A1
Authority
WO
WIPO (PCT)
Prior art keywords
nodes
node
node set
data
initial cluster
Prior art date
Application number
PCT/CN2016/079915
Other languages
English (en)
Chinese (zh)
Inventor
毛刘刚
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Publication of WO2016197706A1 publication Critical patent/WO2016197706A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/56—Provisioning of proxy services
    • H04L67/563—Data redirection of data network streams
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28—Databases characterised by their database models, e.g. relational or object models
    • G06F16/284—Relational databases
    • G06F16/285—Clustering or classification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor

Definitions

  • the present application relates to, but is not limited to, the field of communications, and in particular, to a data migration method and apparatus.
  • each node communication device
  • the service processing capability of the node ensures better network quality. It is necessary to migrate the corresponding configuration data to nodes with fast startup rate and small space utilization, so that the overall communication node service processing is more balanced.
  • a data migration task to be scheduled in each source node is determined according to a quality of service QOS of a source node to a destination node in the network. And sending a scheduling command to the target source node corresponding to the to-be-scheduled data migration task, where the scheduling command is used to schedule the data migration task, which improves data migration efficiency and avoids network congestion, but the method only
  • the "time" factor of quality of service between nodes is used as the criterion for judging the priority of data migration. It does not take into account the "space" factor of storage between nodes, which may lead to excessive storage pressure of some nodes with large space utilization. Reduce the life of your hard drive.
  • a method for calculating an average value of the space utilization ratio of the same type of hard disk is adopted, and space utilization of each hard disk in any one of the storage pools is obtained, and the space of the same type is used.
  • the data in the hard disk having a utilization greater than the average value is migrated to the hard disk in the same type where the space utilization is less than the average value. It reduces the pressure on the hard disk with large data storage capacity and prolongs the service life of the hard disk with large data storage capacity.
  • this method only takes the storage space utilization of the node as the priority judgment criterion for data migration, and does not take care of the storage. The "time" factor of the media object is judged.
  • the main purpose of the present application is to provide a method and an apparatus for migrating data, so as to at least solve the problem of one of the time factors such as space utilization and service quality, such as space utilization and other operational factors during data migration in the related art. .
  • a data migration method including: acquiring a startup rate of a plurality of nodes in a communication network and a storage space utilization ratio of the plurality of nodes; according to a preset rule, the startup rate, and The storage space utilization classifies the priorities of the multiple nodes to obtain a set of nodes classified according to priorities; and migrates data between the set of nodes according to priorities.
  • the step of acquiring a startup rate of the multiple nodes in the communication network and the storage space utilization of the multiple nodes includes: acquiring a time difference between a power-on time and a normal running time of the multiple nodes, where And using the time difference as a startup rate of the multiple nodes; acquiring a proportion of a space size of all files in the storage space of the multiple nodes to a total space size, and using the specific gravity as the storage space rate.
  • the step of classifying the priorities of the multiple nodes according to the preset rule, the startup rate, and the storage space utilization, and obtaining the node set classified according to the priority includes: Generating the plurality of node feature vectors by using the startup rate and the storage space utilization; classifying the priorities of the plurality of feature vectors by cluster analysis, and dividing the plurality of nodes according to the result of the cluster analysis A first node set and a second node set, wherein the nodes in the first node set have a higher priority than the nodes in the second node set.
  • the step of classifying the priorities of the multiple feature vectors by cluster analysis, and dividing the multiple nodes into the first node set and the second node set according to the result of the cluster analysis includes Presetting a feature vector of the first initial cluster center and the second initial cluster center, wherein the first initial cluster center belongs to the first node set, and the second initial cluster center belongs to the first a two-node set; obtaining a distance value between a feature vector of the plurality of nodes and a feature vector of the first initial cluster center and the second initial cluster center; respectively, according to the distance value and the closest distance a matching principle, the plurality of nodes are allocated to the first node set or the second node set that are closer to each other; and all nodes in the first node set are acquired relative to the first initial cluster a distance average of the center, and obtaining a distance average of all nodes in the second node set relative to the second initial cluster center, and adjusting the first initial cluster center or the initial according to the distance mean respectively The second initial cluster center.
  • the step of migrating data between the set of nodes according to the priority includes: migrating data to be migrated of the migrating node in the second set of nodes to the migrating node of the first set of nodes.
  • the method further includes: when the system of the migrating node needs to access the migrated data, The migrated data in the migrated node is restored to the migrated node.
  • the application further provides a computer readable storage medium storing computer executable instructions that are implemented when the computer executable instructions are executed.
  • a data migration apparatus including: an acquisition module configured to acquire a startup rate of a plurality of nodes in a communication network and a storage space utilization ratio of the plurality of nodes; a classification module, setting Sorting the priorities of the multiple nodes according to a preset rule, the startup rate, and the storage space utilization, and obtaining a node set classified according to the priority; the migration module is set to be based on the priority, The data is migrated between the set of nodes.
  • the acquiring module includes: a first acquiring unit, configured to acquire a time difference between a power-on time and a normal running time of the multiple nodes, and use the time difference as a starting rate of the multiple nodes
  • the second obtaining unit is configured to obtain a proportion of a space size of all files in the storage space of the plurality of nodes to a total space size, and use the specific gravity as the storage space utilization rate.
  • the classification module includes: a generating unit, configured to generate the multiple node feature vectors according to the startup rate and the storage space utilization rate; and a classification unit configured to perform the multiple The priority of the feature vector is classified, and the plurality of nodes are divided into a first node set and a second node set according to a result of the cluster analysis, wherein the nodes in the first node set have higher priority than the first node set The priority of the nodes in the second set of nodes.
  • the classification unit includes: a preset subunit, configured to preset in the first initial cluster a feature vector of the heart and the second initial cluster center, wherein the first initial cluster center belongs to the first node set, the second initial cluster center belongs to the second node set; and the subunit is acquired And configured to obtain a distance value between the feature vector of the plurality of nodes and the feature vector of the first initial cluster center and the second initial cluster center respectively; the allocation subunit is configured to be allocated according to the distance value and the closest distance
  • the plurality of nodes are allocated to the first node set or the second node set that are closer to themselves;
  • the adjusting subunit is configured to acquire all nodes in the first node set relative to the a distance average of the first initial cluster center, and obtaining a distance average of all nodes in the second node set relative to the second initial cluster center, and respectively adjusting the first initial cluster according to the distance mean Center or the initial second initial cluster center.
  • the priority of multiple nodes is classified according to a preset rule, a startup rate, and a storage space utilization, and a node set classified according to priority is obtained, and is migrated between the node sets according to the priority.
  • the data that is, the startup rate and the storage space utilization ratio of the nodes in the communication network are combined in the present application, and the data between the nodes is migrated in a priority manner, and the related art only considers the data migration when performing data migration.
  • the problem of one of the time factors such as space utilization or service efficiency of service quality, and thus the effect of improving system efficiency.
  • FIG. 1 is a flowchart of a method of migrating data according to an embodiment of the present invention
  • FIG. 2 is a structural block diagram of a data migration apparatus according to an embodiment of the present invention.
  • FIG. 3 is a block diagram showing an optional structure of an obtaining module 22 of a data migration apparatus according to an embodiment of the present invention
  • FIG. 4 is a block diagram 1 of an optional structure of a classification module 24 of a data migration apparatus according to an embodiment of the present invention
  • FIG. 5 is an optional structural block of the classification module 24 of the data migration apparatus according to an embodiment of the present invention.
  • FIG. 6 is a block diagram 1 of an optional structure of a data migration apparatus according to an embodiment of the present invention.
  • FIG. 7 is a block diagram showing the structure of a multipoint priority data migration apparatus according to an alternative embodiment of the present invention.
  • FIG. 8 is a flowchart of a multi-point priority data migration method in accordance with an alternative embodiment of the present invention.
  • FIG. 9 is a flow chart of clustering analysis of a communication node in accordance with an alternative embodiment of the present invention.
  • FIG. 10 is a schematic diagram of migration data backup and recovery in accordance with an alternate embodiment of the present invention.
  • FIG. 1 is a flowchart of a method for migrating data according to an embodiment of the present invention. As shown in FIG. 1, the steps of the method include:
  • Step S102 Acquire a startup rate of multiple nodes in the communication network and a storage space utilization ratio of the multiple nodes.
  • Step S104 classify the priorities of the multiple nodes according to a preset rule, a startup rate, and a storage space utilization, and obtain a node set classified according to the priority;
  • Step S106 migrating data between the node sets according to the priority.
  • the priority of the multiple nodes is classified according to the startup rate and the storage space utilization by using a preset rule, and the node set classified according to the priority is obtained, and the priority is determined according to the priority.
  • the data is migrated between the node sets, that is, in this embodiment, the startup rate and the storage space utilization rate of the nodes in the communication network are combined, and the data is migrated between the nodes according to the priority, and the migration data in the related art is solved.
  • the time factor such as space utilization or service quality operation efficiency is considered, and the effect of improving system efficiency is achieved.
  • the method of obtaining the startup rate of the multiple nodes in the communication network and the storage space utilization ratio of the multiple nodes in the step S102 of the embodiment may be implemented by the following steps in the optional implementation manner of the embodiment:
  • Step S11 Obtain a time difference between a power-on time and a normal running time of the multiple nodes, and use the time difference as a starting rate of the multiple nodes.
  • Step S12 Acquire the proportion of the space size of all files in the storage space of the plurality of nodes to the total space size, and use the specific gravity as the storage space utilization rate.
  • the startup rate and the storage space utilization involved in the foregoing step S11 and the step S12 may be:
  • the startup rate of the node is 5 minutes.
  • the node space utilization is 50%.
  • the priority of the multiple nodes is classified according to the startup rate and the storage space utilization by using a preset rule, and the node set classified according to the priority is obtained.
  • a preset rule a preset rule
  • Step S22 Generate multiple node feature vectors according to the startup rate and the storage space utilization rate
  • Step S23 classifying the priorities of the plurality of feature vectors by cluster analysis, and dividing the plurality of nodes into a first node set and a second node set according to a result of the cluster analysis, wherein the first node The nodes in the set have a higher priority than the nodes in the second set of nodes.
  • the priority of the plurality of feature vectors is classified by cluster analysis, and the plurality of nodes are divided into a first node set and a second node set according to the result of the cluster analysis.
  • Step S31 Presetting the feature vectors of the first initial cluster center and the second initial cluster center, wherein the first initial cluster center belongs to the first node set, and the second initial cluster center belongs to the second node set;
  • Step S32 acquiring feature vectors of the plurality of nodes and the first initial cluster center and the first The distance value between the feature vectors of the two initial cluster centers;
  • Step S33 assigning a plurality of nodes to the first node set or the second node set that are closer to each other according to the distance value and the nearest distance allocation principle;
  • Step S34 Acquire a distance average value of all nodes in the first node set relative to the first initial cluster center, and obtain a distance average value of all nodes in the second node set relative to the second initial cluster center, and adjust according to the distance average value respectively.
  • an application scenario in this embodiment may be:
  • the obtained pattern feature vector is used as the input of the unit for cluster analysis. First, the distance between each feature vector is counted. The distance between the node x i and the node x j is expressed as follows:
  • K initial cluster centers are selected.
  • the sample x i in the sample set is assigned to the nearest neighbor cluster z j according to the principle of minimum distance, which can be expressed as follows:
  • the step of migrating data between the node sets includes: migrating the data to be migrated of the migrating node in the second node set to the migrating node of the first node set.
  • the method in this embodiment further includes: the system in the migrating node needs to access When the data is migrated, the data moved in the moved node is restored to the evicted node.
  • Embodiments of the present invention further provide a computer readable storage medium storing a computer executable The instructions, when the computer executable instructions are executed, implement the above method.
  • a data migration device is further provided, and the device is used to implement the foregoing embodiments and preferred embodiments, and details are not described herein.
  • the term “module” may implement a combination of software and/or hardware of a predetermined function.
  • the apparatus described in the following embodiments is preferably implemented in software, hardware, or a combination of software and hardware, is also possible and contemplated.
  • the apparatus includes: an acquisition module 22 configured to acquire a startup rate of a plurality of nodes in a communication network and the plurality of nodes.
  • the classification module 24 is coupled to the acquisition module 22, and is configured to classify the priorities of the multiple nodes according to the preset rules, the startup rate, and the storage space utilization, and obtain the node sets classified according to the priority.
  • the migration module 26 is coupled to the classification module 24 and configured to migrate data between sets of nodes in accordance with priority.
  • FIG. 3 is a block diagram showing an optional structure of the acquiring module 22 of the data migration apparatus according to the embodiment of the present invention.
  • the obtaining module 22 includes: a first obtaining unit 32 configured to obtain power-on of multiple nodes. The time difference between the time and the normal running time, and the time difference is used as the starting rate of the plurality of nodes; the second obtaining unit 34 is configured to obtain the space size of all the files in the storage space of the plurality of nodes. The proportion of the size of the space, and the specific gravity as the storage space utilization.
  • the classification module 24 includes: a generation unit 42 configured to be based on a startup rate and a storage space utilization rate. Generating a plurality of node feature vectors; the classifying unit 44 is coupled to the generating unit 42 and configured to classify the plurality of feature vectors by cluster analysis, and divide the plurality of nodes according to the result of the cluster analysis. The first node set and the second node set, wherein the nodes in the first node set have higher priority than the node priority in the second node set.
  • FIG. 5 is a block diagram 2 of an optional structure of a classification module 24 of a data migration apparatus according to an embodiment of the present invention.
  • the classification unit 44 includes: a preset subunit 52 configured to preset a first initial aggregation.
  • the obtaining sub-unit 54, and the preset sub-unit 52 a coupling connection, configured to acquire feature vectors of the plurality of nodes and first a distance value between the feature vector of the initial cluster center and the second initial cluster center;
  • the assigning subunit 56 is coupled with the obtaining subunit 54 and configured to allocate the plurality of nodes according to the distance value and the nearest distance allocation principle.
  • the adjusting subunit is coupled to the assigning subunit 56, configured to obtain a distance average of all nodes in the first node set relative to the first initial cluster center, and obtain The distance between all nodes in the second node set relative to the second initial cluster center is averaged, and the first initial cluster center or the initial second initial cluster center is respectively adjusted according to the distance mean.
  • the migration module 26 is further configured to migrate the data to be migrated of the migrated node in the second node set to the migrated node of the first node set.
  • FIG. 6 is a block diagram of an optional structure of a data migration apparatus according to an embodiment of the present invention. As shown in FIG. 6, the apparatus further includes: a recovery module 62 coupled to the migration module 26 and configured to be required by the system of the migration node. When accessing the migrated data, the migrated data in the migrated node is restored to the migrated node.
  • a recovery module 62 coupled to the migration module 26 and configured to be required by the system of the migration node.
  • the present invention provides a multi-point priority data migration method and apparatus.
  • the configuration data with low priority weight can be migrated based on the comprehensive consideration of the time factor and the spatial factor of the communication node.
  • the node with low priority weight effectively integrates system resources and improves the overall service processing capability of the communication node.
  • FIG. 7 is a structural block diagram of a multi-point priority data migration apparatus according to an alternative embodiment of the present invention.
  • the apparatus includes: a data pre-processing module (the acquisition module 22 and the classification module 24 in the foregoing embodiment). a data backup module (corresponding to the migration module 26 in the above embodiment) and a data recovery module (corresponding to the recovery module 62 in the above embodiment), wherein the data preprocessing module includes: a statistical startup rate unit ( Corresponding to the first acquisition unit 32), the statistical space utilization unit (corresponding to the second acquisition unit 34 in the above embodiment), the mode feature processing unit (corresponding to the generation unit 42 of the above embodiment), and the right in the above embodiment.
  • a value priority processing unit (corresponding to the classification unit 44 in the present embodiment described above).
  • FIG. 8 is a flowchart of a multi-point priority data migration method according to an alternative embodiment of the present invention. Based on the module of the apparatus in FIG. 7, the steps of the method shown in FIG. 8 include:
  • Step S802 number each node in the communication network layout as a subsequent pattern clustering Collection of pattern samples
  • Step S804 Counting the startup rate of each node in the communication network layout
  • Step S806 Counting the space utilization rate of the memory card of each node in the communication network layout
  • Step S808 Taking the startup rate and the space utilization rate of each node as the classification feature vector of the node;
  • Step S810 Perform cluster analysis on feature vectors of each node, and classify the sample set of the communication node into two categories: high and low data migration priority weights;
  • Step S812 The configuration data of the corresponding node with a lower priority weight is migrated to the classification node with a higher priority weight.
  • step S804 includes: calculating, by using a statistical start rate unit, a difference between a power-on time and a normal running time of each node in the communication layout as a startup rate of the node.
  • step S806 includes: obtaining a space utilization ratio of the node by using a ratio of a space size of all files of the memory card of each node calculated by the space utilization unit to a total space size.
  • step S808 includes: using a value obtained by each node statistical start rate unit and a statistical space utilization unit as a priority processing unit by the mode feature processing unit to obtain a corresponding mode feature vector.
  • step S810 includes: performing cluster analysis on the pattern feature vectors of the respective nodes by using the weight priority processing unit to obtain two sets of high priority weights and low priority weights. .
  • step S804 to step S810 optionally,
  • Step S804 includes: counting the power-on time of each node in the communication network layout and the time after the node is working normally by using the statistical start rate unit, and using the power-on time and the normal working time as the input of the statistical start rate unit, and outputting the node as the corresponding Start rate.
  • Step S806 includes: counting the size of all the files of the memory cards of each node in the communication network layout and the total capacity of the memory card by using the statistical space utilization unit, and using all the file sizes and the total capacity of the memory card as the statistical space utilization unit.
  • Input, output is the corresponding empty space of the node Utilization rate.
  • Step S808 includes: passing, by the mode feature processing unit, an output of the statistical start rate unit and an output of the statistical space utilization unit as an input of the mode feature processing unit, and the output of the mode feature processing unit is a mode feature vector of the corresponding node.
  • Step S810 includes: inputting, by the weight priority processing unit, an output of the pattern feature processing unit as a weight priority processing unit, and processing, by the unit, the output of the weight priority processing unit is a priority weight and priority Two sets of low-level nodes.
  • the processing method of the weight priority processing unit includes:
  • the utilization rate is large and the startup rate of the node is low.
  • An initial cluster center is determined for the two classification sets respectively, the sample nodes are allocated to the nearest cluster set according to the nearest distance allocation principle, and the sample mean of each cluster set is used as the new cluster center. Repeat the above steps so that the cluster centers of the two cluster sets no longer change. If the last sample points are the same distance from the two cluster centers, then the sample points are removed, indicating that the node does not need data migration. . Finally, two cluster sets with corresponding priority weights and low priority weights are obtained.
  • FIG. 9 is a flowchart of performing cluster analysis on a communication node according to an alternative embodiment of the present invention. As shown in FIG. 9, the steps of the process include:
  • Step S902 obtaining a feature vector of the mode sample
  • Step S904 Heuristically selecting initial values of two cluster centers
  • Step S906 each data point is classified into a classification to which the cluster center closest to it belongs;
  • Step S908 Calculate a new cluster center of each category
  • Step S910 determining whether the cluster center changes, and determining that there is no change, executing step S906, determining that there is a change, executing step S912;
  • Step S912 The node set is divided into two categories according to the weight of the priority.
  • step S812 The method steps involved in the foregoing step S812 in the present embodiment involving performing backup by the data backup module include:
  • Step S41 determining, according to the output of the data migration pre-processing unit, that is, the classification set of the two nodes, the data migration node and the data migration node;
  • Step S42 determining basic configuration data to be migrated by the data migration node and a storage path of the data to be migrated;
  • Step S43 determining the IP address of the node where the data is moved into the node, the slot number of the rack, and the disk storage path of the migrated data;
  • Step S44 The corresponding basic configuration data in the source node (the node where the data is migrated) is migrated to the corresponding storage path of the destination node (the node into which the data is moved).
  • the data recovery method of the data migration may be implemented by using the data recovery module in the optional embodiment, and the method includes:
  • Step S51 determining the source node IP address of the data migration, the slot number of the rack, and the storage path of the data;
  • Step S52 determining an IP address of the data migration destination node, a slot number of the rack, and a storage path of the data;
  • Step S53 When the system of the data migration node needs to access the migrated configuration data, the corresponding configuration data in the node where the data is migrated is restored to the node where the data is moved out.
  • FIG. 10 is a schematic diagram of migration data backup and recovery according to an alternative embodiment of the present invention.
  • the priority weight of the node is determined according to the node startup rate and the storage space utilization, and the node basic configuration data with the lower priority weight is set. It is migrated to a node with a higher priority weight. This method not only improves the service life of the memory of a node with a large storage capacity, but also improves the system efficiency.
  • the data migration method provided in this alternative embodiment is applied to the communication network layout.
  • Each node in the communication network layout may be a communication board, and the migrated storage unit is a memory card in the board.
  • the data migration method includes the following steps:
  • Step S61 numbering each node in the communication network layout
  • n nodes in the communication network layout can be expressed as follows:
  • Step S62 the rate of starting the node is counted by the statistics start rate unit
  • the startup rate of the node is 5 minutes.
  • the statistical start rate unit may be a program running by the node. If the node power-on time is t i1 , when the node is put into normal operation and the corresponding working time t i2 is counted again, the startup rate of the node x i may be expressed as follows. :
  • Step S63 collecting the space utilization rate of the node memory card by using the statistical space utilization unit
  • the space utilization rate of the node is 50%.
  • the statistical space utilization unit can be used as a program running in the node. If the file size of the memory card in the node is s 1 and the total storage capacity of the node is s 2 , the storage space utilization of the node can be expressed as follows:
  • a feature vector pattern node x i may be expressed as follows:
  • Step S64 the priority weight processing unit performs cluster analysis on the mode feature vector obtained by the mode feature processing unit as an input of the unit;
  • the distance between each feature vector is first counted, and the distance between the node x i and the node x j is expressed as follows:
  • Step S65 selecting K initial cluster centers according to clustering criteria of priority weights
  • the sample x i in the sample set is assigned to the nearest neighbor cluster z j according to the principle of minimum distance, which can be expressed as follows:
  • step S66 using the distance average of the samples in each cluster relative to the initial cluster center as the new cluster center, step S64 and step S65 are repeated until the cluster center point of the sample center does not change.
  • the classification with high priority is characterized by high node space utilization and slow node startup rate.
  • the classification feature with low priority weight is low node space utilization and node startup. The rate is fast.
  • Step S67 The data backup unit migrates the data migrated out of the set and the data migration set by the data migration preprocessing module to perform corresponding backup;
  • the IP address of the node where the data is migrated is determined.
  • the board in the communication node is the same as the standby board and the standby board.
  • the IP address is the same. You need to continue to determine the slot number of the rack. Based on the IP address and the slot number, a node where data is migrated can be determined, and then the data migration unit of the data migration node is determined, and the storage path of the migration data is counted. For example, if the IP address is 192.100.90.1 and the slot number is 2, the basic data in the storage path /home/sd is migrated out.
  • Step S68 determining an IP address, a slot number, and a data storage path of the data moving into the node
  • the data migration node IP address is 192.100.90.3
  • the destination node slot number is 1
  • the data migration path is /home/sd/bak.
  • Step S69 the corresponding configuration data of the data migration node is migrated to the storage location corresponding to the corresponding storage path of the data migration node through the network;
  • the IP address is 192.100.90.1 and the slot number is 2
  • the data in the /home/sd directory is migrated to the IP address of 192.100.90.3 and the slot number is 1.
  • Step S70 When the system of the data migration node needs to access the migrated configuration data, the data recovery process is triggered;
  • the IP address of the source node and the destination node, the slot number of the rack, and the storage path of the data unit are first restored.
  • step S71 the corresponding configuration data in the node where the data is moved is restored to the node where the data is moved out.
  • the configuration data of the storage path is /home/sd/bak is restored to the IP address of 192.100.90.1 and the storage path of the node with slot number 2 is / The storage location corresponding to home/sd.
  • a storage medium is further provided, wherein the software includes the above-mentioned software, including but not limited to: an optical disk, a floppy disk, a hard disk, an erasable memory, and the like.
  • modules or steps of the present application can be implemented by a general computing device, which can be concentrated on a single computing device or distributed over a network composed of multiple computing devices. Alternatively, they may be implemented by program code executable by the computing device such that they may be stored in the storage device for execution by the computing device and, in some cases, may be performed in a different order than herein.
  • the steps shown or described are either made separately into individual integrated circuit modules, or a plurality of modules or steps are fabricated as a single integrated circuit module.
  • the application is not limited to any particular combination of hardware and software.
  • each module/unit in the above embodiment may be implemented in the form of hardware, for example, by implementing an integrated circuit to implement its corresponding function, or may be implemented in the form of a software function module, for example, executing a program stored in the memory by a processor. / instruction to achieve its corresponding function.
  • Embodiments of the invention are not limited to any particular form of hardware and The combination of software.
  • the startup rate, and the storage space utilization the priorities of the multiple nodes are classified, the node sets classified according to the priority are obtained, and the data is migrated between the node sets according to the priority, that is, in the present
  • the application combines the startup rate and storage space utilization of nodes in the communication network, and migrates data between nodes in a priority manner, which solves the space in the related art when only data utilization is considered.
  • the problem of one of the factors such as factors or service quality operation efficiency, and thus the effect of improving system efficiency.

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Mobile Radio Communication Systems (AREA)

Abstract

L'invention concerne un procédé et un dispositif de migration de données. Le procédé consiste : à acquérir un débit de début d'une pluralité de nœuds dans un réseau de communication et un taux d'utilisation d'espace de stockage de la pluralité de nœuds; à classifier la priorité de la pluralité de nœuds selon une règle prédéfinie, le débit de début et le taux d'utilisation d'espace de stockage, de façon à obtenir des ensembles de nœuds classifiés selon la priorité; et à faire migrer des données entre les ensembles de nœuds selon la priorité. Au moyen de la solution, le problème dans l'état pertinent de la technique selon lequel uniquement un aspect d'un facteur d'espace, tel qu'un taux d'utilisation d'espace, ou d'un facteur de temps, tel qu'un taux d'utilisation de qualité de service, est pris en considération, est résolu, permettant ainsi d'obtenir l'effet d'amélioration de l'efficacité de système.
PCT/CN2016/079915 2015-06-09 2016-04-21 Procédé et dispositif de migration de données WO2016197706A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201510311624.XA CN106254403A (zh) 2015-06-09 2015-06-09 数据的迁移方法及装置
CN201510311624.X 2015-06-09

Publications (1)

Publication Number Publication Date
WO2016197706A1 true WO2016197706A1 (fr) 2016-12-15

Family

ID=57503167

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/079915 WO2016197706A1 (fr) 2015-06-09 2016-04-21 Procédé et dispositif de migration de données

Country Status (2)

Country Link
CN (1) CN106254403A (fr)
WO (1) WO2016197706A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108009204A (zh) * 2017-11-02 2018-05-08 深圳市网心科技有限公司 基于扩展名分类并去冗余的方法及系统

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109325016B (zh) * 2018-09-12 2021-04-20 杭州朗和科技有限公司 数据迁移方法、装置、介质及电子设备
CN111897962B (zh) * 2020-07-27 2024-03-15 绿盟科技集团股份有限公司 一种物联网资产标记方法及装置
CN114510742B (zh) * 2022-04-15 2022-07-15 纬创软件(武汉)有限公司 一种基于隐私安全的混合云数据迁移方法及系统

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103034615A (zh) * 2012-12-07 2013-04-10 无锡美森微电子科技有限公司 一种适用于流应用多核处理器的存储管理方法
EP2692197A1 (fr) * 2011-03-31 2014-02-05 Telefonaktiebolaget L M Ericsson (PUBL) Procédé et dispositif dans un système de communications pour la gestion d'un support gbr
CN103747069A (zh) * 2013-12-27 2014-04-23 王之杰 一种利用gdp成分编制股票指数的云计算系统
CN104539730A (zh) * 2015-01-16 2015-04-22 浙江大学 一种hdfs中面向视频的负载均衡方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2692197A1 (fr) * 2011-03-31 2014-02-05 Telefonaktiebolaget L M Ericsson (PUBL) Procédé et dispositif dans un système de communications pour la gestion d'un support gbr
CN103034615A (zh) * 2012-12-07 2013-04-10 无锡美森微电子科技有限公司 一种适用于流应用多核处理器的存储管理方法
CN103747069A (zh) * 2013-12-27 2014-04-23 王之杰 一种利用gdp成分编制股票指数的云计算系统
CN104539730A (zh) * 2015-01-16 2015-04-22 浙江大学 一种hdfs中面向视频的负载均衡方法

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108009204A (zh) * 2017-11-02 2018-05-08 深圳市网心科技有限公司 基于扩展名分类并去冗余的方法及系统

Also Published As

Publication number Publication date
CN106254403A (zh) 2016-12-21

Similar Documents

Publication Publication Date Title
CN110301128B (zh) 基于学习的资源管理数据中心云架构的实现方法
KR101827369B1 (ko) 데이터 스트림 분산 병렬 처리 서비스 관리 장치 및 방법
WO2021179462A1 (fr) Procédé de planification de tâches pour plate-forme spark basé sur un algorithme amélioré de colonie de fourmis quantiques
US10534542B2 (en) Dynamic core allocation for consistent performance in a non-preemptive scheduling environment
WO2017167025A1 (fr) Procédé et dispositif servant à réaliser une planification de tâche, et support de stockage informatique
TW475153B (en) Processing channel subsystem pending I/O work queues based on priorities
Cohen et al. Almost optimal virtual machine placement for traffic intense data centers
US20190245924A1 (en) Three-stage cost-efficient disaggregation for high-performance computation, high-capacity storage with online expansion flexibility
JP6241300B2 (ja) ジョブスケジューリング装置、ジョブスケジューリング方法、およびジョブスケジューリングプログラム
CN109710405B (zh) 区块链智能合约管理方法、装置、电子设备及存储介质
US20130290957A1 (en) Efficient execution of jobs in a shared pool of resources
US20120221730A1 (en) Resource control system and resource control method
US10394606B2 (en) Dynamic weight accumulation for fair allocation of resources in a scheduler hierarchy
WO2016197706A1 (fr) Procédé et dispositif de migration de données
US8898422B2 (en) Workload-aware distributed data processing apparatus and method for processing large data based on hardware acceleration
WO2021136137A1 (fr) Procédé et appareil de planification de ressources, et dispositif associé
CN105607952B (zh) 一种虚拟化资源的调度方法及装置
US10360267B2 (en) Query plan and operation-aware communication buffer management
US20160062929A1 (en) Master device, slave device and computing methods thereof for a cluster computing system
US11157323B2 (en) Multiple metric based load prediction and resource allocation in an active stream processing job
JP6993495B2 (ja) クラウド・ネットワーキングにおけるスケーラブルな統計及び分析メカニズム
CN116391175A (zh) 自动缩放用于企业级大数据工作负载的查询引擎
CN111625367A (zh) 一种动态调整文件系统读写资源的方法
Sontakke et al. Optimization of hadoop mapreduce model in cloud computing environment
WO2020124488A1 (fr) Procédé de mise en correspondance de processus d'application, dispositif électronique et support d'informations lisible par ordinateur

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16806604

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16806604

Country of ref document: EP

Kind code of ref document: A1