CN112799785B - Virtual machine cluster migration method, device, equipment and medium - Google Patents
Virtual machine cluster migration method, device, equipment and medium Download PDFInfo
- Publication number
- CN112799785B CN112799785B CN202110138501.6A CN202110138501A CN112799785B CN 112799785 B CN112799785 B CN 112799785B CN 202110138501 A CN202110138501 A CN 202110138501A CN 112799785 B CN112799785 B CN 112799785B
- Authority
- CN
- China
- Prior art keywords
- metadata
- virtual machine
- migration
- target
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000013508 migration Methods 0.000 title claims abstract description 103
- 230000005012 migration Effects 0.000 title claims abstract description 103
- 238000000034 method Methods 0.000 title claims abstract description 32
- 238000013515 script Methods 0.000 claims abstract description 101
- 238000012544 monitoring process Methods 0.000 claims abstract description 78
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 27
- 238000001514 detection method Methods 0.000 claims abstract description 20
- 238000012216 screening Methods 0.000 claims abstract description 18
- 230000002159 abnormal effect Effects 0.000 claims description 29
- 230000005856 abnormality Effects 0.000 claims description 24
- 238000006243 chemical reaction Methods 0.000 claims description 17
- 239000013598 vector Substances 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 10
- 239000000463 material Substances 0.000 claims description 7
- 238000010276 construction Methods 0.000 claims description 6
- 238000010586 diagram Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 238000002347 injection Methods 0.000 description 2
- 239000007924 injection Substances 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000013475 authorization Methods 0.000 description 1
- 238000007621 cluster analysis Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 239000000243 solution Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012706 support-vector machine Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3409—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/362—Software debugging
- G06F11/3636—Software debugging by tracing the execution of the program
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
- G06F9/5088—Techniques for rebalancing the load in a distributed system involving task migration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a virtual machine cluster migration method, which comprises the following steps: acquiring an open virtual machine format file, and configuring a dynamic monitoring script for monitoring a server and a virtual machine cluster according to the open virtual machine format file; executing a dynamic monitoring script to monitor and collect data flow packet metadata of a local data center and system logs of a virtual machine cluster; screening out flow packet metadata and basic metadata in a system log according to a preset anomaly detection algorithm, and constructing a dependency graph according to the basic metadata; and creating a migration script according to the dependency graph and the basic metadata, and running the migration script to migrate the virtual machine cluster to the target cloud platform. The invention can reduce the workload and time when migrating to public cloud. In addition, a virtual machine cluster migration device, equipment and a storage medium are also provided.
Description
Technical Field
The present invention relates to the field of cloud computing technologies, and in particular, to a virtual machine cluster migration method, device, equipment, and medium.
Background
The cloud platform can provide a "cloud" based service for use by a developer in creating an application. While virtualization technology is the core technology of the cloud platform. Wherein a virtual machine refers to a complete system with complete hardware system functions that runs in a completely isolated environment through software emulation.
In promoting basic cloud computing services to enterprise customers, it is often desirable to migrate existing systems or resources into a cloud computing environment provided by a cloud facilitator. In the prior art, a manual configuration mode is often adopted when the virtual machine cluster is migrated. And the file formats of the target cloud platform operating system and the source cloud platform disk are not completely consistent. Firstly, environmental parameters are required to be collected, and the disk file format is converted according to the mode of the target cloud platform. And then checking whether the migrated virtual machine is matched with the disk drive of the target cloud platform, and solving the problem of interface drive compatibility in the migration process. And finally, importing the disk file of the virtual machine into a target cloud platform, and generating the virtual machine. All configuration items required for migration need to be manually operated, and the operation process is very complicated. Therefore, how to reduce the tedious workload caused by manual operation is a problem to be solved.
Disclosure of Invention
In view of the above, it is desirable to provide a virtual machine cluster migration method, apparatus, device, and medium that can reduce the workload and time in migrating to public clouds.
A method of virtual machine cluster migration, the method comprising:
acquiring an open virtual machine format file, and configuring a dynamic monitoring script for a monitoring server and a virtual machine cluster according to the open virtual machine format file;
Executing the dynamic monitoring script to monitor and collect the data flow packet metadata of the local data center and the system log of the virtual machine cluster;
Screening out the flow packet metadata and the basic metadata in the system log according to a preset anomaly detection algorithm, and constructing a dependency graph according to the basic metadata;
And creating a migration script according to the dependency graph and the basic metadata, and running the migration script to migrate the virtual machine cluster to a target cloud platform.
In one embodiment, the executing the dynamic monitoring script to monitor and collect data traffic packet metadata of the local data center and system logs of the virtual machine cluster includes:
Periodically executing the dynamic monitoring script, and acquiring the metadata of the data flow packet and the system log according to the data source configured by the dynamic monitoring script;
And according to the data type configured by the dynamic monitoring script, carrying out corresponding type conversion on the acquired data flow packet metadata and the system log, acquiring conversion data, and storing the conversion data.
In one embodiment, the data traffic packet metadata includes at least one of descriptive metadata, structural metadata, and administrative metadata;
the system log includes at least one of a memory log, a storage log, and a processor log.
In one embodiment, the screening the flow packet metadata and the basic metadata in the system log according to a preset anomaly detection algorithm includes:
Acquiring total distance values of the flow packet metadata and the target metadata in the system log and K pieces of neighbor metadata, wherein the total distance between the K pieces of neighbor metadata and the target metadata is smaller than the total number of preset distances;
Determining an abnormality score according to the total distance value and the total number, and taking target metadata corresponding to the abnormality score as abnormality metadata when the abnormality score is larger than an abnormality score threshold;
And removing the flow packet metadata and the abnormal metadata in the system log to obtain the basic metadata.
In one embodiment, the building a dependency graph from the base metadata includes:
acquiring data relations among the basic metadata, taking the basic metadata as a vertex set, and taking the data relations as a relation set connected with the vertex set;
And constructing a graph data set with vectors and weights according to the vertex set and the relation set recursion, wherein the graph data set is used for showing the dependency relation between the virtual machine and the application program in the dependency graph.
In one embodiment, the creating a migration script from the dependency graph and the base metadata includes:
Determining a correlation sequence among virtual machines according to the vector sequence and the weight sequence of the graph dataset, and determining a target migration sequence of the virtual machine cluster according to the correlation sequence;
Acquiring a power consumption peak value and average power consumption of the virtual machine cluster, and determining target configuration of the virtual machine cluster on a target cloud platform according to the power consumption peak value and the average power consumption;
And acquiring a target software version of each virtual machine on the target cloud platform, and creating the migration script according to the target migration sequence, the target configuration and the target software version.
In one embodiment, after the generating the migration script according to the dependency graph and the base metadata, the method further includes:
And generating feasibility reports for different cloud platforms, wherein the feasibility reports comprise cloud platform month basic expenses, a system topological graph, local virtual machine cluster configuration and flow data materials.
A virtual machine cluster migration apparatus, the apparatus comprising:
The monitoring script configuration module is used for acquiring an open virtual machine format file and configuring dynamic monitoring scripts for the monitoring server and the virtual machine cluster according to the open virtual machine format file;
The monitoring and collecting module is used for executing the dynamic monitoring script to monitor and collect the data flow packet metadata of the local data center and the system log of the virtual machine cluster;
The dependence graph construction module is used for screening out the flow packet metadata and the basic metadata in the system log according to a preset abnormality detection algorithm, and constructing a dependence graph according to the basic metadata;
and the migration module is used for creating a migration script according to the dependency graph and the basic metadata, and running the migration script to migrate the virtual machine cluster to the target cloud platform.
A computer readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the steps of:
acquiring an open virtual machine format file, and configuring a dynamic monitoring script for a monitoring server and a virtual machine cluster according to the open virtual machine format file;
Executing the dynamic monitoring script to monitor and collect the data flow packet metadata of the local data center and the system log of the virtual machine cluster;
Screening out the flow packet metadata and the basic metadata in the system log according to a preset anomaly detection algorithm, and constructing a dependency graph according to the basic metadata;
And creating a migration script according to the dependency graph and the basic metadata, and running the migration script to migrate the virtual machine cluster to a target cloud platform.
A virtual machine cluster migration apparatus comprising a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to perform the steps of:
acquiring an open virtual machine format file, and configuring a dynamic monitoring script for a monitoring server and a virtual machine cluster according to the open virtual machine format file;
Executing the dynamic monitoring script to monitor and collect the data flow packet metadata of the local data center and the system log of the virtual machine cluster;
Screening out the flow packet metadata and the basic metadata in the system log according to a preset anomaly detection algorithm, and constructing a dependency graph according to the basic metadata;
And creating a migration script according to the dependency graph and the basic metadata, and running the migration script to migrate the virtual machine cluster to a target cloud platform.
The invention provides a virtual machine cluster migration method, a device, equipment and a medium, wherein basic metadata is obtained through dynamic monitoring and data tracking of a local server and a virtual machine cluster and screening through an anomaly detection algorithm, so that unnecessary data migration work is reduced, a dependence chart is built again, a migration script is created based on the dependence chart, the data cluster can be automatically migrated to a public cloud by executing the migration script, and the workload of manual operation is greatly reduced.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Wherein:
FIG. 1 is a flow chart of a virtual machine cluster migration method in one embodiment;
FIG. 2 is a schematic diagram illustrating a virtual machine cluster migration apparatus according to an embodiment;
FIG. 3 is a block diagram of a virtual machine cluster migration apparatus, in one embodiment.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
As shown in fig. 1, fig. 1 is a flow chart of a virtual machine cluster migration method in an embodiment, where the steps provided by the virtual machine cluster migration method in the embodiment include:
Step 102, obtaining an open virtual machine format file, and configuring dynamic monitoring scripts for monitoring the server and the virtual machine clusters according to the open virtual machine format file.
The open virtual machine format file (OVF, open Virtualization Format) is a file package formed by a plurality of files of different types, and the file package can be used as a standard and reliable format file between different subsequent virtual machines, so that the universality between different virtual machines is met. The open virtual machine format file is used as a dependency injection framework infra to realize dependency injection, the open virtual machine format file comprises an mf file and a Cert file, wherein the mf file plays a role in monitoring whether the file is tampered by an illegal user, the Cert file plays a role in verifying the legitimacy of the file, the monitoring matters of the dynamic monitoring script are configured based on the files, and the dynamic monitoring script is uniformly configured on different virtual machines by an infrastructure, namely a code (Infrastructure as Code).
Step 104, executing a dynamic monitoring script to monitor and collect data flow packet metadata of the local data center and system logs of the virtual machine cluster.
The data traffic packet Metadata (Metadata) is data (data about other data) describing other data, or is structural data (structural data) for providing information about a certain resource, and specifically includes at least one of descriptive Metadata, structural Metadata, and administrative Metadata. Wherein descriptive metadata is metadata describing the discovery and discernment of meaningful resources, which specifically includes elements such as titles, summaries, authors, and keywords. Structural metadata is metadata describing the material container that indicates how to sort objects composited therein, such as how pages are organized into chapters. Administrative metadata is metadata that manages resource information, such as the time and manner in which the material was created, the type of file, and other technical information, and who has access rights. The system log is divided into a memory log, a storage log and a processor log according to different parts of log data sources, and at least one of the memory log, the storage log and the processor log is monitored and collected.
In one embodiment, the dynamic monitoring script configures a data source, such as an ip address, and obtains data traffic packet metadata and system logs pertaining to the data source. And according to the data type configured by the dynamic monitoring script, carrying out corresponding type conversion on the acquired data flow packet metadata and the system log so as to meet the description requirement of the metadata, and storing the converted conversion data. By collecting the metadata of the I/O data of each virtual machine, the data security privacy is protected, and only effective data is obtained. Specifically, the dynamic monitoring script comprises a system batch log scanner script monitor. Sh and an I/O metadata monitoring script iom. Sh, and the data is collected by periodically executing the dynamic monitoring script every 10 minutes, and the whole process is three weeks to five weeks, so that the most complete and accurate data is ensured to be obtained. The monitor is written by a linux shell script table, for example, the memory log and the starting log used for recording the starting condition in the processor log are read through dynamically read authorization and command lines such as/var/log/boot. Log, the memory log and the log used for recording the invaded condition in the processor log are read through/var/log/auth. Log, and each server configured with the monitor.sh script has an independent monitor.log monitoring log to dynamically store all information returned to the monitoring storage nodes in batches on the monitoring log, namely, the starting log, the log of the invaded condition and other monitoring information. The metadata monitoring script iom.sh monitors and processes metadata through monitoring a custom IO port, for example, monitors 1234 a port through sudo tcpflow-i any-C-J port 1234, and the obtained data is analyzed and processed through python script of agg.py, and then stored after noise reduction and unified format processing. Each server configured with iom.sh will have an independent io.log file to store the metadata after processing analysis.
And 106, screening out the flow packet metadata and the basic metadata in the system log according to a preset abnormality detection algorithm, and constructing a dependency graph according to the basic metadata.
In this embodiment, the screening is performed by using an anomaly detection algorithm without supervised learning, including a support vector machine algorithm, a cluster analysis algorithm, or a nearest neighbor algorithm, to screen out related data flows, such as system update, security check, and the like, which are not strongly associated with each other. And then removing the part of the data collected by monitoring and unnecessary data flow, thereby retaining the basic metadata with strong correlation and dependency used for transmitting data among the data clusters.
In a specific embodiment, based on a nearest neighbor anomaly detection algorithm, firstly, acquiring total distance values L of target metadata and K pieces of neighbor metadata (K pieces of neighbor metadata are K pieces of nearest distances to the target metadata selected from all metadata) in flow packet metadata and system logs, and total number N of the K pieces of neighbor metadata, wherein the relative distances between the K pieces of neighbor metadata and the target metadata are smaller than a preset distance d. And determining the abnormality score of the target metadata according to the total distance value L and the total number N, wherein the greater the total distance value L is, the higher the abnormality score is, and the lesser the total number N is, the higher the abnormality is. When the abnormality score is greater than the abnormality score threshold, the target metadata is regarded as abnormality metadata. And judging whether the metadata in the flow packet metadata and the metadata in the system log are abnormal metadata or not through abnormal scores, and removing the abnormal metadata to obtain basic metadata.
For example, for the clustering analysis algorithm, if the data sample size of an abnormal cluster is found to be much smaller than that of other clusters after clustering, and the value such as the characteristic mean distribution of the data in the abnormal cluster is also very different from that of the other clusters, the metadata in the abnormal cluster are mostly abnormal metadata, and the abnormal metadata are removed.
A system dependency graph is then constructed recursively through the graph theory data (DEPENDENCY GRAPH). The dependency graph intuitively exposes the interactions and dependencies of each virtual machine and application through the graph dataset (GRAPH DATASET).
In one embodiment, data relationships between base metadata are obtained, the base metadata is used as a vertex set, and the data relationships are used as a relationship set connecting the vertex sets. These data sets are written to dataframe format and applied to open source distributed cluster algorithm frameworks APACH SPARK, spark MLib in conjunction with the graph computation library SPARK GRAPHFRAMES to recursively construct a graph dataset with vertices (Vertices), edges (edges), and attributes (properties).
And step 108, creating a migration script according to the dependency graph and the basic metadata, and running the migration script to migrate the virtual machine cluster to the target cloud platform.
According to the embodiment, through the constructed dependency graph and the optimal configuration analysis of the computing resources of the target cloud platform, the open source tool teraform is used for safely and efficiently pre-configuring and managing the infrastructure of the target cloud platform. Data including the ip address of the virtual machine cluster, the links of the server clusters, the target migration sequence, the target configuration, the target software version of each virtual machine on the target cloud platform and the like can be obtained from the dependency graph and the basic metadata. The vector sequence and the weight sequence of the graph dataset can determine the correlation sequence among the virtual machines, the virtual machines with the correlation sequence share one server, data communication among the virtual machines can be communicated in one server without communicating through a network, and under the migration condition, the virtual machines with the correlation sequence are simultaneously migrated to another server, so that the target migration sequences of the virtual machines with the correlation sequence are the same. And obtaining the migration priority among the virtual machines without the correlation sequence through a topological sorting algorithm and a strong relation algorithm chain, thereby obtaining the target migration sequence. The target configuration at the target cloud platform is determined by the peak consumption and the average power consumption of the virtual machine cluster. The target software version includes a software use license version for each virtual machine and a potentially alternate version of the software in a different public cloud. the teragen. Py will automatically read these data and generate a migration script that can configure the cloud environment with one key, and by executing the migration script, the target cloud platform that can be migrated includes public cloud platforms such as ali cloud, tencent cloud, hua as cloud, etc.
The following are examples of partial migration scripts, such as those created in the alicloud:
Cloud computing virtual machine clusters are configured in the Alicloud:
the user defines the cloud environment, for example, creates a storage space in the messenger cloud:
furthermore, after the migration script is created, a feasibility report facing different cloud platforms can be generated, the migration report can display data such as cloud platform month foundation expenses of different public clouds, a system topology diagram, local virtual machine cluster configuration, data flow data and the like in a visual form, and the feasibility report can help a user to reduce risks and cost in migration and increase system safety.
According to the virtual machine cluster migration method, the dynamic monitoring and data tracking of the local server and the virtual machine clusters are performed, the basic metadata are obtained through screening by the anomaly detection algorithm, migration work on unnecessary data is reduced, the dependency graph is built again, the migration script is created based on the dependency graph, the data clusters can be automatically migrated to public cloud by executing the migration script, and the workload of manual operation is greatly reduced.
In one embodiment, as shown in fig. 2, a virtual machine cluster migration apparatus is provided, which includes:
the monitoring script configuration module 202 is configured to obtain an open virtual machine format file, and configure dynamic monitoring scripts for the monitoring server and the virtual machine cluster according to the open virtual machine format file;
a monitoring collection module 204, configured to execute a dynamic monitoring script to monitor and collect metadata of data traffic packets of a local data center and system logs of a virtual machine cluster;
the dependency graph construction module 206 is configured to screen out the flow packet metadata and the basic metadata in the system log according to a preset anomaly detection algorithm, and construct a dependency graph according to the basic metadata;
And the migration module 208 is used for creating a migration script according to the dependency graph and the basic metadata, and running the migration script to migrate the virtual machine cluster to the target cloud platform.
According to the virtual machine cluster migration device, the dynamic monitoring and data tracking of the local server and the virtual machine clusters are performed, the basic metadata are obtained through screening by the anomaly detection algorithm, the migration work of unnecessary data is reduced, the dependency graph is built again, the migration script is created based on the dependency graph, the data clusters can be automatically migrated to public cloud by executing the migration script, and the workload of manual operation is greatly reduced.
In one embodiment, the monitoring collection module 204 is specifically configured to: periodically executing a dynamic monitoring script, and acquiring metadata of a data flow packet and a system log according to a data source configured by the dynamic monitoring script; and according to the data type configured by the dynamic monitoring script, carrying out corresponding type conversion on the acquired data flow packet metadata and the system log, acquiring conversion data, and storing the conversion data.
In one embodiment, the dependency graph construction module 206 is specifically configured to: acquiring total distance values of the flow packet metadata and the target metadata and K pieces of neighbor metadata in the system log, wherein the total distance between the target metadata and the K pieces of neighbor metadata is smaller than the preset distance; determining an abnormal score according to the total distance value and the total number, and taking target metadata corresponding to the abnormal score as abnormal metadata when the abnormal score is larger than an abnormal score threshold value; and removing the flow packet metadata and the abnormal metadata in the system log to obtain the basic metadata.
In one embodiment, the dependency graph construction module 206 is further specifically configured to: acquiring data relations among the basic metadata, taking the basic metadata as a vertex set, and taking the data relations as a relation set for connecting the vertex sets; and constructing a graph data set with vectors and weights according to the vertex set and the relation set recursion, wherein the graph data set is used for showing the dependency relationship between the virtual machine and the application program in the dependency graph.
In one embodiment, the migration module 208 is further specifically configured to: determining a correlation sequence among the virtual machines according to the vector sequence and the weight sequence of the graph dataset, and determining a target migration sequence of the virtual machine cluster according to the correlation sequence; acquiring a power consumption peak value and average power consumption of the virtual machine cluster, and determining target configuration of the virtual machine cluster on a target cloud platform according to the power consumption peak value and the average power consumption; and acquiring a target software version of each virtual machine on the target cloud platform, and creating a migration script according to the target migration sequence, the target configuration and the target software version.
In one embodiment, the virtual machine cluster migration apparatus further includes a report generation module configured to: and generating feasibility reports for different cloud platforms, wherein the feasibility reports comprise cloud platform month basic expenses, a system topology diagram, local virtual machine cluster configuration and flow data materials.
FIG. 3 illustrates an internal block diagram of a virtual machine cluster migration apparatus in one embodiment. As shown in fig. 3, the virtual machine cluster migration apparatus includes a processor, a memory, and a network interface connected through a system bus. The memory includes a nonvolatile storage medium and an internal memory. The non-volatile storage medium of the virtual machine cluster migration device stores an operating system, and may also store a computer program that, when executed by a processor, causes the processor to implement a virtual machine cluster migration method. The internal memory may also store a computer program that, when executed by the processor, causes the processor to perform a virtual machine cluster migration method. It will be appreciated by those skilled in the art that the architecture shown in fig. 3 is merely a block diagram of a portion of the architecture associated with the present inventive arrangements and is not limiting of the virtual machine cluster migration apparatus to which the present inventive arrangements are applied, and that a particular virtual machine cluster migration apparatus may include more or fewer components than shown, or may combine certain components, or may have a different arrangement of components.
A virtual machine cluster migration apparatus comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the steps of when executing the computer program: acquiring an open virtual machine format file, and configuring a dynamic monitoring script for monitoring a server and a virtual machine cluster according to the open virtual machine format file; executing a dynamic monitoring script to monitor and collect data flow packet metadata of a local data center and system logs of a virtual machine cluster; screening out flow packet metadata and basic metadata in a system log according to a preset anomaly detection algorithm, and constructing a dependency graph according to the basic metadata; and creating a migration script according to the dependency graph and the basic metadata, and running the migration script to migrate the virtual machine cluster to the target cloud platform.
In one embodiment, executing a dynamic monitoring script to monitor system logs collecting data traffic packet metadata of a local data center and a virtual machine cluster, comprises: periodically executing a dynamic monitoring script, and acquiring metadata of a data flow packet and a system log according to a data source configured by the dynamic monitoring script; and according to the data type configured by the dynamic monitoring script, carrying out corresponding type conversion on the acquired data flow packet metadata and the system log, acquiring conversion data, and storing the conversion data.
In one embodiment, screening the metadata of the traffic packet and the basic metadata in the system log according to a preset anomaly detection algorithm includes: acquiring total distance values of the flow packet metadata and the target metadata and K pieces of neighbor metadata in the system log, wherein the total distance between the target metadata and the K pieces of neighbor metadata is smaller than the preset distance; determining an abnormal score according to the total distance value and the total number, and taking target metadata corresponding to the abnormal score as abnormal metadata when the abnormal score is larger than an abnormal score threshold value; and removing the flow packet metadata and the abnormal metadata in the system log to obtain the basic metadata.
In one embodiment, building a dependency graph from base metadata includes: acquiring data relations among the basic metadata, taking the basic metadata as a vertex set, and taking the data relations as a relation set for connecting the vertex sets; and constructing a graph data set with vectors and weights according to the vertex set and the relation set recursion, wherein the graph data set is used for showing the dependency relationship between the virtual machine and the application program in the dependency graph.
In one embodiment, creating a migration script from a dependency graph and underlying metadata includes: determining a correlation sequence among the virtual machines according to the vector sequence and the weight sequence of the graph dataset, and determining a target migration sequence of the virtual machine cluster according to the correlation sequence; acquiring a power consumption peak value and average power consumption of the virtual machine cluster, and determining target configuration of the virtual machine cluster on a target cloud platform according to the power consumption peak value and the average power consumption; and acquiring a target software version of each virtual machine on the target cloud platform, and creating a migration script according to the target migration sequence, the target configuration and the target software version.
In one embodiment, after generating the migration script from the dependency graph and the base metadata, further comprising: and generating feasibility reports for different cloud platforms, wherein the feasibility reports comprise cloud platform month basic expenses, a system topology diagram, local virtual machine cluster configuration and flow data materials.
A computer readable storage medium storing a computer program which when executed by a processor performs the steps of: acquiring an open virtual machine format file, and configuring a dynamic monitoring script for monitoring a server and a virtual machine cluster according to the open virtual machine format file; executing a dynamic monitoring script to monitor and collect data flow packet metadata of a local data center and system logs of a virtual machine cluster; screening out flow packet metadata and basic metadata in a system log according to a preset anomaly detection algorithm, and constructing a dependency graph according to the basic metadata; and creating a migration script according to the dependency graph and the basic metadata, and running the migration script to migrate the virtual machine cluster to the target cloud platform.
In one embodiment, executing a dynamic monitoring script to monitor system logs collecting data traffic packet metadata of a local data center and a virtual machine cluster, comprises: periodically executing a dynamic monitoring script, and acquiring metadata of a data flow packet and a system log according to a data source configured by the dynamic monitoring script; and according to the data type configured by the dynamic monitoring script, carrying out corresponding type conversion on the acquired data flow packet metadata and the system log, acquiring conversion data, and storing the conversion data.
In one embodiment, screening the metadata of the traffic packet and the basic metadata in the system log according to a preset anomaly detection algorithm includes: acquiring total distance values of the flow packet metadata and the target metadata and K pieces of neighbor metadata in the system log, wherein the total distance between the target metadata and the K pieces of neighbor metadata is smaller than the preset distance; determining an abnormal score according to the total distance value and the total number, and taking target metadata corresponding to the abnormal score as abnormal metadata when the abnormal score is larger than an abnormal score threshold value; and removing the flow packet metadata and the abnormal metadata in the system log to obtain the basic metadata.
In one embodiment, building a dependency graph from base metadata includes: acquiring data relations among the basic metadata, taking the basic metadata as a vertex set, and taking the data relations as a relation set for connecting the vertex sets; and constructing a graph data set with vectors and weights according to the vertex set and the relation set recursion, wherein the graph data set is used for showing the dependency relationship between the virtual machine and the application program in the dependency graph.
In one embodiment, creating a migration script from a dependency graph and underlying metadata includes: determining a correlation sequence among the virtual machines according to the vector sequence and the weight sequence of the graph dataset, and determining a target migration sequence of the virtual machine cluster according to the correlation sequence; acquiring a power consumption peak value and average power consumption of the virtual machine cluster, and determining target configuration of the virtual machine cluster on a target cloud platform according to the power consumption peak value and the average power consumption; and acquiring a target software version of each virtual machine on the target cloud platform, and creating a migration script according to the target migration sequence, the target configuration and the target software version.
In one embodiment, after generating the migration script from the dependency graph and the base metadata, further comprising: and generating feasibility reports for different cloud platforms, wherein the feasibility reports comprise cloud platform month basic expenses, a system topology diagram, local virtual machine cluster configuration and flow data materials.
It should be noted that the foregoing virtual machine cluster migration method, apparatus, device and computer readable storage medium belong to a general inventive concept, and the content in the embodiments of the virtual machine cluster migration method, apparatus, device and computer readable storage medium may be mutually applicable.
Those skilled in the art will appreciate that implementing all or part of the above-described methods in accordance with the embodiments may be accomplished by way of a computer program stored in a non-transitory computer-readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (SYNCHLINK) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The foregoing examples illustrate only a few embodiments of the application, which are described in detail and are not to be construed as limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of protection of the present application is to be determined by the appended claims.
Claims (9)
1. A method for virtual machine cluster migration, the method comprising:
acquiring an open virtual machine format file, and configuring a dynamic monitoring script for a monitoring server and a virtual machine cluster according to the open virtual machine format file;
executing the dynamic monitoring script to monitor and collect data flow packet metadata of a local data center and system logs of the virtual machine cluster;
Screening out the flow packet metadata and the basic metadata in the system log according to a preset anomaly detection algorithm, and constructing a dependency graph according to the basic metadata;
creating a migration script according to the dependency graph and the basic metadata, and running the migration script to migrate the virtual machine cluster to a target cloud platform;
The step of screening out the flow packet metadata and the basic metadata in the system log according to a preset anomaly detection algorithm includes: acquiring total distance values of the flow packet metadata and the target metadata in the system log and K pieces of neighbor metadata, wherein the total distance between the K pieces of neighbor metadata and the target metadata is smaller than the total number of preset distances; determining an abnormality score of the target metadata according to the total distance value and the total number, and taking the target metadata corresponding to the abnormality score as the abnormality metadata when the abnormality score is greater than an abnormality score threshold; and removing the flow packet metadata and the abnormal metadata in the system log to obtain the basic metadata.
2. The method of claim 1, wherein the executing the dynamic monitoring script to monitor the collection of data traffic packet metadata of the local data center and the system log of the virtual machine cluster comprises:
Periodically executing the dynamic monitoring script, and acquiring the metadata of the data flow packet and the system log according to the data source configured by the dynamic monitoring script;
And according to the data type configured by the dynamic monitoring script, carrying out corresponding type conversion on the acquired data flow packet metadata and the system log, acquiring conversion data, and storing the conversion data.
3. The method of claim 1, wherein the data traffic packet metadata comprises at least one of descriptive metadata, structural metadata, and administrative metadata;
the system log includes at least one of a memory log, a storage log, and a processor log.
4. The method of claim 1, wherein said building a dependency graph from said base metadata comprises:
acquiring data relations among the basic metadata, taking the basic metadata as a vertex set, and taking the data relations as a relation set connected with the vertex set;
And constructing a graph data set with vectors and weights according to the vertex set and the relation set recursion, wherein the graph data set is used for showing the dependency relation between the virtual machine and the application program in the dependency graph.
5. The method of claim 4, wherein the creating a migration script from the dependency graph and the base metadata comprises:
Determining a correlation sequence among virtual machines according to the vector sequence and the weight sequence of the graph dataset, and determining a target migration sequence of the virtual machine cluster according to the correlation sequence;
Acquiring a power consumption peak value and average power consumption of the virtual machine cluster, and determining target configuration of the virtual machine cluster on a target cloud platform according to the power consumption peak value and the average power consumption;
And acquiring a target software version of each virtual machine on the target cloud platform, and creating the migration script according to the target migration sequence, the target configuration and the target software version.
6. The method of claim 1, further comprising, after said generating a migration script from said dependency graph and said base metadata:
And generating feasibility reports for different cloud platforms, wherein the feasibility reports comprise cloud platform month basic expenses, a system topological graph, local virtual machine cluster configuration and flow data materials.
7. A virtual machine cluster migration apparatus, the apparatus comprising:
The monitoring script configuration module is used for acquiring an open virtual machine format file and configuring dynamic monitoring scripts for the monitoring server and the virtual machine cluster according to the open virtual machine format file;
The monitoring and collecting module is used for executing the dynamic monitoring script to monitor and collect data flow packet metadata of the local data center and system logs of the virtual machine cluster;
The dependence graph construction module is used for screening out the flow packet metadata and the basic metadata in the system log according to a preset abnormality detection algorithm, and constructing a dependence graph according to the basic metadata;
The migration module is used for creating a migration script according to the dependency graph and the basic metadata, and running the migration script to migrate the virtual machine cluster to a target cloud platform;
The dependency graph construction module is further specifically configured to: acquiring total distance values of the flow packet metadata and the target metadata in the system log and K pieces of neighbor metadata, wherein the total distance between the K pieces of neighbor metadata and the target metadata is smaller than the total number of preset distances; determining an abnormality score of the target metadata according to the total distance value and the total number, and taking the target metadata corresponding to the abnormality score as the abnormality metadata when the abnormality score is greater than an abnormality score threshold; and removing the flow packet metadata and the abnormal metadata in the system log to obtain the basic metadata.
8. A computer readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the steps of the method of any one of claims 1 to 6.
9. A virtual machine cluster migration apparatus comprising a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to perform the steps of the method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110138501.6A CN112799785B (en) | 2021-02-01 | 2021-02-01 | Virtual machine cluster migration method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110138501.6A CN112799785B (en) | 2021-02-01 | 2021-02-01 | Virtual machine cluster migration method, device, equipment and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112799785A CN112799785A (en) | 2021-05-14 |
| CN112799785B true CN112799785B (en) | 2024-07-19 |
Family
ID=75813488
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110138501.6A Active CN112799785B (en) | 2021-02-01 | 2021-02-01 | Virtual machine cluster migration method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112799785B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113296835B (en) * | 2021-05-25 | 2024-04-16 | 平安养老保险股份有限公司 | Program script migration method, device, computer equipment and storage medium |
| CN113485755B (en) * | 2021-06-30 | 2023-08-18 | 深圳市科力锐科技有限公司 | Device driver verification method, device and storage medium |
| CN113721976B (en) * | 2021-07-31 | 2024-02-06 | 远光软件股份有限公司 | Data migration method and device based on BI analysis software, storage medium and electronic equipment |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9665849B2 (en) * | 2015-02-26 | 2017-05-30 | Red Hat, Inc. | Employing dependency graph in software build projects |
| CN107589982B (en) * | 2017-09-12 | 2021-01-15 | 国云科技股份有限公司 | Method for migrating virtual machine from VMware to KVM |
| KR101959055B1 (en) * | 2017-10-02 | 2019-07-04 | 광주과학기술원 | Cloud-to-Cloud migration of multiple virtual machines |
| US10924346B1 (en) * | 2019-07-24 | 2021-02-16 | Vmware, Inc. | System and method for migrating network policies of software-defined network components |
-
2021
- 2021-02-01 CN CN202110138501.6A patent/CN112799785B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN112799785A (en) | 2021-05-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112799785B (en) | Virtual machine cluster migration method, device, equipment and medium | |
| US11327749B2 (en) | System and method for generating documentation for microservice based applications | |
| US20160269423A1 (en) | Methods and systems for malware analysis | |
| US11671506B2 (en) | Microservice management system for recommending modifications to optimize operation of microservice-based systems | |
| US10437996B1 (en) | Classifying software modules utilizing similarity-based queries | |
| CN109885624B (en) | Data processing method, data processing device, computer equipment and storage medium | |
| CN112989330B (en) | Container intrusion detection method, device, electronic equipment and storage medium | |
| CN105122212A (en) | Periodicity optimization in an automated tracing system | |
| CN105122234A (en) | Deploying trace objectives using cost analyses | |
| CN105283866A (en) | Optimization analysis using similar frequencies | |
| US11379221B2 (en) | Version control mechanisms augmented with semantic analysis for determining cause of software defects | |
| US20200250587A1 (en) | Framework for multi-tenant data science experiments at-scale | |
| CN110830483B (en) | Webpage log attack information detection method, system, equipment and readable storage medium | |
| Ezzati-Jivan et al. | A stateful approach to generate synthetic events from kernel traces | |
| CN111859047A (en) | Fault solving method and device | |
| US20240004993A1 (en) | Malware detection in containerized environments | |
| CN113836237A (en) | Method and device for auditing data operation of database | |
| US11860727B2 (en) | Data quality-based computations for KPIs derived from time-series data | |
| CN116226865A (en) | Security detection method, device, server, medium and product of cloud native application | |
| US11106525B2 (en) | Systems and methods for classifying and predicting the cause of information technology incidents using machine learning | |
| Ali et al. | [Retracted] Classification and Prediction of Software Incidents Using Machine Learning Techniques | |
| WO2022042126A1 (en) | Fault localization for cloud-native applications | |
| US11782938B2 (en) | Data profiling and monitoring | |
| US12093245B2 (en) | Temporal directed cycle detection and pruning in transaction graphs | |
| Michalas et al. | MemTri: A memory forensics triage tool using bayesian network and volatility |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20220607 Address after: Unit J, 3 / F, far east building, 5-6 Middle Road, Tsim Sha Tsui, Hong Kong, China Applicant after: HAOGU Group Co.,Ltd. Address before: Unit J, 3 / F, far east building, 5-6 Middle Road, Tsim Sha Tsui, Hong Kong, China Applicant before: Sugar paper umbrella Technology Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20240426 Address after: 201209 8 Chuansha Road, Pudong New Area, Shanghai, 1098 Applicant after: SHANGHAI MAGIC VALLEY IC TECHNOLOGY Co.,Ltd. Country or region after: China Address before: Unit J, 3rd Floor, Far East Building, 5-6 Middle Road, Tsim Sha Tsui, Kowloon, Hong Kong, China Applicant before: HAOGU Group Co.,Ltd. Country or region before: Hong-Kong |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |