CN115562805A

CN115562805A - Resource migration method and device and electronic equipment

Info

Publication number: CN115562805A
Application number: CN202211216818.8A
Authority: CN
Inventors: 高云登; 房立坤; 井小飞; 郭朝辉; 何帅
Original assignee: Nsfocus Technologies Inc; Nsfocus Technologies Group Co Ltd
Current assignee: Nsfocus Technologies Inc; Nsfocus Technologies Group Co Ltd
Priority date: 2022-09-30
Filing date: 2022-09-30
Publication date: 2023-01-03

Abstract

The application relates to the field of cloud and native, in particular to a resource migration method, a resource migration device and electronic equipment, which are suitable for a container arrangement system and are used for realizing automatic migration control of container application. The method comprises the following steps that a source node acquires migration information of a migration resource object, wherein the migration information at least comprises: the method comprises the steps that a Pod to be migrated, a Pod memory migration type, a source node and a destination node are obtained, then after migration information is obtained, the Pod mirror image information in the Pod to be migrated is collected and sent to the destination node, then the destination node receives the Pod mirror image information and creates the destination Pod in response to the destination node, a migration rule corresponding to the Pod memory migration type is determined to be an online migration rule, and the Pod memory information in the Pod to be migrated is migrated to the destination Pod by adopting the online migration rule. Based on the method, automatic migration control of containerized applications on a plurality of hosts in the cluster can be realized.

Description

Resource migration method and device and electronic equipment

Technical Field

The present application relates to the field of cloud-native technologies, and in particular, to a method and an apparatus for resource migration, and an electronic device.

Background

The container Live Migration (Live Migration) is to ensure that a container application is migrated from one physical node to another physical node on the basis that the container application is not perceived by a user, and ensure that the migrated container application is restored to a state before Migration.

For a container orchestration system, such as K8s (kubernets, i.e., for managing container applications on multiple hosts in a cloud platform), no function/service of container live migration is currently provided for using objects, so that a traditional manual migration manner is currently required for container live migration.

In view of this, there is still no method for automatically implementing migration control of container applications in a container arrangement system.

Disclosure of Invention

The application provides a resource migration method, a resource migration device and electronic equipment, which are used for realizing automatic migration control of containerization applications on a plurality of hosts in a cluster.

In a first aspect, the present application provides a method for resource migration, where the method includes:

a source node acquires migration information of a migration resource object; wherein the migration information at least comprises: the Pod to be migrated, the container memory migration type, the source node and the destination node;

after the migration information is obtained, collecting container mirror image information in the Pod to be migrated, and sending the container mirror image information to the destination node;

responding to the target node to receive the container mirror image information and create a target Pod, and determining that a migration rule corresponding to the container memory migration type is an online migration rule;

and migrating the container memory information in the Pod to be migrated to the target Pod by adopting the online migration rule.

In one possible design, the migration information further includes: the current state of the migration resource object and the container information of each container in the Pod to be migrated, where the container information at least includes a container attribute and the current state, then:

after the migration information is obtained, collecting container mirror image information in the Pod to be migrated, and sending the container mirror image information to the destination node, including:

after the migration information is obtained, in response to that the current state of the migration resource object is a standby state, collecting container mirror image information in the Pod to be migrated, and sending the container mirror image information to the destination node;

and if the container mirror image information is sent to the destination node, modifying the current state of the migration resource object into a starting state so that the destination node creates a destination Pod and deploys the container mirror image information on the destination Pod.

the migrating the container memory information in the Pod to be migrated to the destination Pod by using the online migration rule includes:

in response to that the current state of the migration resource object is a migration state, migrating the container memory information in the Pod to be migrated to the target Pod by using the online migration rule;

and if it is determined that all the container memory information in the Pod to be migrated is migrated to the destination Pod, modifying the current state of the migrated resource object to be a migration completion state.

In a possible design, the migrating, by using the online migration rule, container memory information in the Pod to be migrated to the destination Pod includes:

compressing and transmitting all memory pages in the Pod to be migrated to the target Pod;

after all the memory pages are compressed and transmitted, performing multi-round iterative transmission on dirty pages generated in the transmission process; in the M +1 round of iterative transmission process, the following operations are executed:

calculating a predicted dirty page rate corresponding to the M +1 th iteration transmission based on the predicted dirty page rate and the actual dirty page rate corresponding to each previous M iteration transmission, and taking the product of the predicted dirty page rate and the transmission time spent by the M +1 th iteration transmission as a predicted dirty page amount corresponding to the M +1 th iteration transmission;

and if the predicted dirty page amount corresponding to the M +1 th iteration transmission is larger than the dirty page amount generated by the M +1 th iteration transmission, or the total time length of the previous M +1 th iteration transmission is larger than a preset threshold value, copying the dirty page generated by the Pod to be migrated in the M +1 th iteration transmission to the target Pod.

In one possible design, after the compression transmission of all the memory pages, performing multiple rounds of iterative transmission on the dirty pages generated in the transmission process includes:

after all the memory pages are compressed and transmitted, N rounds of transmission are carried out on dirty pages generated in the summary of the transmission process; wherein N is an integer greater than 0;

judging whether the actual dirty page amount corresponding to the N-wheel transmission is smaller than a preset dirty page amount or not;

if so, copying the dirty page generated by the Pod to be migrated in the Nth iteration transmission to the target Pod;

if not, determining to perform multi-round iterative transmission again on the dirty pages generated in the transmission process.

In one possible design, the calculating the predicted dirty page rate corresponding to the M +1 th iteration transmission based on the predicted dirty page rate and the actual dirty page rate corresponding to each of the previous M iteration transmissions includes:

aiming at the previous M rounds of iterative transmission, calculating an error value between a predicted dirty page rate and an actual dirty page rate corresponding to each round of iterative transmission to obtain M error values corresponding to the previous M rounds of iterative transmission;

calculating a weighted average value of the M error values, and taking the weighted average value as a target correction value corresponding to the M +1 th round of iterative transmission;

and determining an initial dirty page rate predicted by the M +1 th iteration transmission, and correcting the initial dirty page rate by using the target correction value to obtain a predicted dirty page rate corresponding to the M +1 th iteration transmission.

In a possible design, after the container memory information in the Pod to be migrated is migrated to the destination Pod, the method further includes:

in response to the success of the service recovery of the target Pod, deleting the Pod to be migrated;

and recovering the service of the Pod to be migrated in response to the failure of the service recovery of the target Pod.

after the service recovery in response to the destination Pod is successful and the Pod to be migrated is deleted, the method further includes:

modifying the current state of the migration resource object into a migration success state;

after the service of the Pod to be migrated is recovered in response to the failure of the service recovery of the destination Pod, the method further includes:

and modifying the current state of the migration resource object into a migration failure state.

In summary, the present application provides a method for resource migration, which is used to implement automatic migration control on node resources, and based on the definition of migration resources designed in the present application, in combination with a unified asynchronous message processing mechanism (List-Watch) of a container orchestration system, implement synchronization of each module and component in the container orchestration system on the migration resources, thereby implementing automatic control on a migration process.

In a second aspect, the present application provides an apparatus for resource migration, the apparatus comprising:

the acquisition module acquires migration information of the migration resource object by the source node; wherein the migration information at least comprises: the Pod to be migrated, the container memory migration type, the source node and the destination node;

the sending module is used for collecting container mirror image information in the Pod to be migrated after the migration information is acquired, and sending the container mirror image information to the destination node;

the determining module is used for responding to the receiving of the container mirror image information and the creation of a destination Pod by the destination node and determining that the migration rule corresponding to the container memory migration type is an online migration rule;

and the migration module is used for migrating the container memory information in the Pod to be migrated to the target Pod by adopting the online migration rule.

after the migration information is obtained, collecting container mirror image information in the Pod to be migrated, and sending the container mirror image information to the destination node, where the sending module is specifically configured to:

the migration module is specifically configured to migrate the container memory information in the Pod to be migrated to the destination Pod according to the online migration rule, and the migration module is configured to:

In a possible design, the migrating module is specifically configured to migrate, by using the online migration rule, container memory information in the Pod to be migrated to the destination Pod, and:

and if the predicted dirty page amount corresponding to the M + 1-th iteration transmission is larger than the dirty page amount generated by the M + 1-th iteration transmission, or the total duration of the previous M + 1-th iteration transmission is larger than a preset threshold, copying the dirty page generated by the Pod to be migrated in the M + 1-th iteration transmission to the target Pod.

In a possible design, after the all memory pages are compressed and transmitted, performing multiple rounds of iterative transmission on the dirty pages generated in the transmission process, where the migration module is specifically configured to:

after all the memory pages are compressed and transmitted, carrying out N-round transmission on dirty pages generated in the summary of the transmission process; wherein N is an integer greater than 0;

and if not, determining to perform multi-round iterative transmission on the dirty pages generated in the transmission process again.

In one possible design, the predicted dirty page rate corresponding to the M +1 th iteration transmission is calculated based on the predicted dirty page rate and the actual dirty page rate corresponding to each of the previous M iteration transmissions, and the migration module is specifically configured to:

calculating a weighted average value of the M error values, and taking the weighted average value as a target correction value corresponding to the M +1 th iteration transmission;

In a possible design, after the container memory information in the Pod to be migrated is migrated to the destination Pod, the migration module is further configured to:

and responding to the failure of the service recovery of the target Pod, and recovering the service of the Pod to be migrated.

after the Pod to be migrated is deleted in response to the successful service restoration of the destination Pod, the migration module is further configured to:

after the service of the Pod to be migrated is recovered in response to the failure of the service recovery of the destination Pod, the migration module is further configured to:

In a third aspect, the present application provides an electronic device, comprising:

a memory for storing a computer program;

the processor is configured to implement the above-mentioned method steps of resource migration when executing the computer program stored in the memory.

In a fourth aspect, the present application provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs one of the method steps of resource migration described above.

For each of the second to fourth aspects and possible technical effects of each aspect, please refer to the above description of the first aspect or the possible technical effects of each of the possible solutions in the first aspect, and no repeated description is given here.

Drawings

In order to more clearly illustrate embodiments of the present application or technical solutions in the prior art, drawings required for the embodiments of the present application will be briefly described below.

FIG. 1 is a schematic view of a migration lifecycle of a migrated resource object provided herein;

FIG. 2 is a schematic diagram of an architecture of a resource migration system provided in the present application;

FIG. 3 is a schematic diagram of a resource migration method provided herein;

FIG. 4 is a flow chart of a method for resource migration provided herein;

FIG. 5 is a schematic diagram of an online migration rule based on time series prediction according to the present application;

FIG. 6 is a flow chart illustrating an online migration rule based on time series prediction according to the present application;

FIG. 7 is a schematic diagram of an apparatus for resource migration provided herein;

fig. 8 is a schematic diagram of a structure of an electronic device provided in the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more clear, the present application will be further described in detail with reference to the accompanying drawings. The particular methods of operation in the method embodiments may also be applied in device embodiments or system embodiments.

In the description of the present application "plurality" is understood to mean "at least two". "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. A is connected with B and can represent: a and B are directly connected and A and B are connected through C. In addition, in the description of the present application, the terms "first," "second," and the like are used for descriptive purposes only and are not intended to indicate or imply relative importance nor order to be construed.

To facilitate those skilled in the art to better understand the technical solutions provided in the embodiments of the present application, the following brief descriptions of the terms used in the embodiments of the present application are provided:

resource objects to be migrated: in a resource migration scenario, an abstract result of specified content in the system, such as a Pod, a service, a node, and other resources, is arranged for a container.

In this embodiment of the application, if a resource migration scenario of K8s is taken as an example, the K8s cluster system represents the running state of the whole cluster through an instantiated object, and the resource object to be migrated is an abstract result designated as a migration state in the K8s cluster system.

Migration information of the resource object to be migrated: in a resource migration scenario, migration information of a resource object to be migrated is a migration resource (which is not described repeatedly below).

In the embodiment of the application, in order to implement migration control of an automated container application based on a container arrangement system, a resource type of the container arrangement system is expanded, and then a definition of migration resources is designed, so that synchronization of migration resources among modules and components in the container arrangement system is achieved, and then automatic control of a migration flow is completed.

The following definitions of migration resources are specifically set forth, and refer to the following table 1:

TABLE 1

The Spec field is used for containing basic information of the resource to be migrated, and the Status field is used for transmitting the current state of the resource object to be migrated. Specifically, the Spec field includes: name (the Name of the Pod to be migrated, namely the Pod to be migrated), type (container memory migration Type, such as iterative migration, direct migration and the like), sourceNode (source node) and DestinationNode (destination node); the Status field includes: phase (current state of migration resource object) and container (container attribute and current state of each container in Pod to be migrated), where the container attribute at least includes an Identity (Identity) ID.

Further, the current state of the migration resource object is used for indicating the current execution link of the migration resource object in the whole migration life cycle. In other words, in this embodiment of the present application, the whole migration lifecycle may be represented by different states of the migration resource object, and the states of the migration resource object may include: prepare (Ready state), ready (start state), migrating (migration state), migrated (migration complete state), succaded (migration successful state), failed (migration Failed state).

Referring to fig. 1, which is a schematic diagram of a migration life cycle of a migration resource object, if a current state of the migration resource object is a "preparation state" and a Pod to be migrated of a source node completes preprocessing, and if container mirror image information is collected for the Pod to be migrated, the current state is switched to a "start state"; if the current state of the resource object is a 'starting state', and the target Pod of the target node completes the resource application work, and if the target Pod is rebuilt, the current state is switched to be a 'migration state'; if the current state of the resource object is a migration state and the container memory of the Pod to be migrated is migrated to the target Pod, switching the current state to be a migration completion state; if the current state of the resource object is a migration completion state and the service recovery of the target Pod is determined to be successful, switching the current state to be a migration success state, deleting the Pod to be migrated and operating the target Pod; and if the current state of the migration resource object is the migration completion state and the service recovery of the target Pod is determined to be failed, switching the current state to be the migration failure state, recovering the service of the Pod to be migrated, and deleting the target Pod.

After the above brief description of the specific terms designed in the embodiments of the present application, some brief descriptions are provided below for application scenarios to which the technical solutions of the embodiments of the present application can be applied.

The application scenario provided by the embodiment of the application relates to a cloud native technology, in particular to a container arrangement system.

Referring to fig. 2, an architecture schematic diagram of a resource migration system provided in the embodiment of the present application is shown, where the system architecture may implement automated migration control on node resources, and the automated migration control may include: a client 21 (Kubectl), a Master node 22 (Master), a source node 23 (i.e., a source working node), and a destination node 24 (i.e., a destination working node).

The client 21 is configured to obtain a migration instruction sent by a user, and send the migration instruction to the host node 22.

Specifically, in response to a migration instruction given by a user operating at the client 21, the client 21 obtains the migration instruction, and sends the migration instruction to the host node 22 after obtaining the migration instruction.

The main node 22 is configured to receive a migration instruction sent by the client 21, create, store, and manage a migration resource object based on the migration instruction, and perform synchronization processing on the migration resource object to complete migration flow control on the migration resource object.

Specifically, the master node 22 includes a cluster core module (API Server), a migration control module (i.e., a migration controller), and a data storage module (Etcd).

In the master node 22, the cluster core module is configured to receive a migration instruction sent by the client 21, extract migration information of the migration resource object from the migration instruction, where the migration information at least includes a Pod to be migrated, a container memory migration type, a source node, and a destination node, and store the extracted migration information in the data storage module. Further, the cluster core module may also receive a modification instruction for the migration resource object sent by the client 21, and perform addition, deletion, modification, and check on the corresponding migration resource object based on the modification instruction. In addition, the cluster core module is further configured to perform information transmission interaction with each working node (e.g., the source node 23 and the destination node 24), for example, in a resource migration scenario, to perform synchronous processing on migration resources on the source node 23 and the destination node 24, for example, receive respective monitoring data of the source node 23 and the destination node 24, which are reported by a monitoring component in the cluster, and store the monitoring data in the data storage module, where the monitoring data may include a current state of a migration resource object, container attributes of each container in the Pod to be migrated, and a current state of each container.

In the primary node 22, the migration control module is configured to obtain migration information of the migration resource object from the cluster core module, and determine migration information of the migration resource object based on the migration information, where the migration control module includes: the method comprises the steps of Pod to be migrated, container memory migration type, source node, destination node, current state of a migration resource object, container attribute of each container in the Pod to be migrated and current state of each container.

In the master node 22, the data storage module is configured to store migration information of the migrated resource object, so that the cluster core module may obtain the corresponding migration information, and in addition, the migration information in the data storage module may be updated based on an instruction issued by the cluster core module.

The source node 23 is configured to periodically obtain migration information of the migration resource object from the host node 22, and then perform pre-processing work before migration on the Pod to be migrated based on the Pod to be migrated in the migration information, that is, synchronize the migration resource to the host node 22, for example, collect container mirror image information in the Pod to be migrated, and then synchronize the collected information to the host node 22 and send the collected information to the destination node 24.

Specifically, the source node 23 includes a first control module (Kubelet), at least one Pod (including at least a Pod to be migrated), and a first container runtime module (container runtime).

In the source node 23, the first control module may be understood as a component that executes an operation of the source node 23, further, the source node 23 periodically acquires migration information of a migration resource object from a cluster core module of the master node 22, and if the source node in the acquired migration information is matched with itself, performs pre-processing before migration for a Pod to be migrated based on the Pod to be migrated and a container memory migration type in the migration information, specifically, on one hand, synchronizes the migration resource to the cluster core module in the master node 22, so that the cluster core module updates data in the data storage module based on the received migration resource, and in addition, the cluster core module also forwards the migration resource to the migration control module, so that the migration control module can control a migration flow in real time; on the other hand, the container mirror image information, pod static information, container file system information, etc. of the Pod to be migrated are collected, and then the collected information is sent to the destination node 24, so that the destination node 24 performs the reconstruction work of the destination Pod for the Pod to be migrated.

In the source node 23, a plurality of pods may be included, where the plurality of pods at least include a Pod to be migrated, and if a single Pod may include one or more application containers and shared resources of the containers, in a load balancing scenario or a fault tolerance processing scenario of resource migration, what needs to be paid attention to the source node 23 is the Pod to be migrated.

In the source node 23, the first container operation module is configured to receive a migration instruction issued by the first control module for migration information of an object to be migrated (a migration resource object is in a ready state), collect, based on the received migration instruction, container mirror image information, pod static information, container file system information, and the like in a Pod to be migrated, send the collected information to the destination node 24, so that the destination node 24 generates a destination Pod based on the received information, and synchronize the collected information to the master node 22, so that the master node 22 completes automation control on a migration process. Further, the migration resource object is modified to a start state.

The destination node 24 is configured to periodically obtain migration information of the migration resource object from the host node 22, then receive container mirror image information and the like in the Pod to be migrated, which are sent by the source node 23, and generate the destination Pod based on the received information.

Specifically, the destination node 24 includes a second control module, at least one Pod, and a second container operation module.

In the destination node 24, the second control module may be understood as a component for executing an operation of the destination node 24, the destination node 24 acquires migration information of a migration resource object from a cluster core module of the master node 22 periodically through the second control module, and in addition, the destination node 24 may also receive information sent by the source node 23 through the second control module, for example, receive container image information, pod static information, container file system information, and the like of the Pod source node 23 to be migrated, which are sent by the source node 23, and generate the destination Pod based on the received information. Further, the destination node 24 may synchronize the migration resource to the cluster core module in the host node 22, so that the cluster core module updates the data in the data storage module based on the received migration resource, and in addition, the cluster core module also forwards the migration resource to the migration control module, so that the migration control module can control the migration flow in real time, for example, in response to generation of the destination Pod, the current state of the migration resource object needs to be changed, and at this time, the updated state information may be synchronized to the cluster core module of the host node 22.

In the destination node 24, a plurality of Pod may be included, and the plurality of Pod should include a destination Pod generated based on container mirror information, pod static information, container file system information, and the like of the Pod to be migrated.

In the destination node 24, the second container operation module is configured to receive a Pod reestablishment instruction issued by the second control module for migration information of the to-be-migrated object (the migration resource object is in a starting state), and generate the destination Pod for migration based on the received reestablishment instruction, and container mirror image information, pod static information, container file system information, and the like in the Pod to be migrated. Further, the migration resource object is modified into a migration state.

It should be noted that the first Container Runtime module and the second Container Runtime module may be called through CRI (Container Runtime Interface).

In addition, for container memory migration during operation of a bottom container in a Pod to be migrated, in order to implement automatic control of a migration flow, in the embodiment of the present application, a runC (container generation and operation tool) is used to replace a Docker (open source application container engine) during design of a resource migration system, and a runC container can implement functions such as dirty page saving, memory tracking, iterative migration, and the like during operation, and can snapshot the container, thereby avoiding a problem that a function expansion is required when an existing Docker is applied to a container live migration scenario.

Based on the resource migration system, the embodiment of the present application provides a method for resource migration, which is used to implement automatic migration control on node resources, and based on the definition of migration resources designed in the embodiment of the present application, in combination with a unified asynchronous message processing mechanism of a container arrangement system, the synchronization of each module and component in the container arrangement system on the migration resources is implemented, thereby implementing automatic control on a migration flow.

As shown in fig. 3, the master node creates a migration resource object and stores migration information of the migration resource object, and it is worth explaining that the migration information includes a current state of the migration resource object; the source node and the destination node regularly acquire migration information of the migration resource object from the main node; if the state of the migration resource object acquired by the source node based on the migration information is a 'preparation state', preprocessing is carried out on the Pod to be migrated, including the steps of collecting mirror image information in the Pod to be migrated and sending the mirror image information to the destination node, and the source node also synchronizes the current flow to the main node so that the state of the main node updating the migration resource object is a 'starting state'; if the destination node acquires that the state of the migrated resource object is a 'starting state' based on the migration information, generating a destination Pod based on the container mirror image information in the Pod to be migrated, and synchronizing the current flow to the master node by the destination node so that the master node updates the state of the migrated resource object to be the 'migration state'; if the state of the migration resource object obtained by the source node based on the migration information is a "migration state", online migrating the container memory in the Pod to be migrated to the destination node until all migration is completed, and synchronizing the current flow to the master node so that the master node updates the state of the migration resource object to be a "migration completion state"; if the destination node obtains the state of the migrated resource object based on the migration information as a "migration complete state" and the recovery of the destination Pod service is successful, including modifying an ID, a name, an IP (Internet Protocol) address, and the like of the destination Pod, synchronizing the current flow to the master node, so that the master node updates the state of the migrated resource object as the "migration successful state"; if the destination node acquires that the state of the migrated resource object is a migration completion state based on the migration information and the recovery of the destination Pod service fails, including modifying the ID, the name, the IP address and the like of the destination Pod, synchronizing the current flow to the master node so that the master node updates the state of the migrated resource object to be a migration failure state.

A more complete embodiment is described below in more detail about a specific implementation flow of the resource migration method provided in the embodiment of the present application.

Referring to fig. 4, a resource migration method is provided for the embodiment of the present application, and a specific implementation flow thereof is as follows:

step 401: a source node acquires migration information of a migration resource object;

specifically, the migration information of the migration resource object includes: the method comprises the steps of (name of) a Pod to be migrated, (type of container memory migration), source node, destination node, current state of a migration resource object, and container information of each container in the Pod to be migrated, wherein the container information comprises container ID of each container in the Pod to be migrated and current state of each container.

In the embodiment of the application, the source node can periodically acquire the migration information of the migration resource object from the host node, and the synchronization of the migration resources among the nodes in the container arrangement system can be ensured by periodically acquiring the migration information, so that the automatic control of the migration process is realized.

Step 402: after the migration information is obtained, collecting container mirror image information in the Pod to be migrated, and sending the container mirror image information to the destination node;

after the source node acquires the migration information, if the source node in the migration information is matched with the source node, the migration process is started, in response to that the current state of the migration resource object in the migration information is a standby state, container mirror image information in the Pod to be migrated is collected, the container mirror image information is sent to the destination node, and Pod static information, container file system information and the like can be sent to the destination node.

Further, if the container mirror image information is sent to the destination node, the current state of the migrated resource object is modified to the starting state, so that the destination node creates a destination Pod and deploys the container mirror image information on the destination Pod, that is, the destination node executes the reconstruction work of the destination Pod, and after the reconstruction work of the destination Pod is completed, the current state of the migrated resource object is modified to the migration state.

Step 403: responding to the receiving of the container mirror image information and the creation of a destination Pod by the destination node, and determining that a migration rule corresponding to the container memory migration type is an online migration rule;

specifically, the container memory migration types at least include iterative migration and direct migration, the migration rule corresponding to the iterative migration is an online migration rule, and the migration rule corresponding to the direct migration is an offline migration rule.

In the embodiment of the present application, in order to meet the requirements of a wider migration scenario, for example, a load balancing scenario has a high requirement on sensing, and a fault-tolerant processing scenario has a high requirement on downtime, so that the embodiment of the present application mainly specifically describes an online migration rule, that is, determines that a migration rule corresponding to a container memory migration type is an online migration rule.

It should be noted that the present application is also applicable to the offline migration rule, and if the online migration rule is understood as an online iterative migration rule, the online migration rule may be equal to the offline migration rule when the online iteration number is 0, and details are not repeated below.

Step 404: and migrating the container memory information in the Pod to be migrated to the target Pod by adopting the online migration rule.

After the online migration rule is determined, in response to that the current state of the migration resource object is the migration state, adopting the online migration rule to migrate the container memory information in the Pod to be migrated to the target Pod, and if it is determined that all the container memory information in the Pod to be migrated is migrated to the target Pod, modifying the current state of the migration resource object to be the migration completion state.

In the embodiment of the present application, in order to maintain stability of a migration process, increase a success rate of a migration result, and improve migration efficiency of online migration, the embodiment of the present application provides an online migration rule based on time series prediction.

Specifically, as shown in fig. 5, all Memory pages in the Pod to be migrated are first used as data for first online transmission, then multiple rounds of online iterative transmission are performed on dirty pages generated in the process, an online migration rule based on time series prediction is introduced to determine an end time of the multiple rounds of online iterative transmission, and in response to a trigger end time, a halt copy is performed, where first, a halt copy is performed on the dirty pages in the last round, and then device state information such as Memory (Memory), CPU (Central Processing Unit ), and tty (computer concept) needs to be migrated to the target Pod together. In addition, all the data transmitted in the foregoing is compressed, and a preset compression algorithm may be specifically adopted to perform corresponding compression processing on the migrated data, such as LZ4 (lossless compression algorithm).

According to the aspects, on one hand, in the online iterative migration process, the change trend of the dirty pages of the memory pages in the container page is predicted according to each iteration, the ending time of the online iteration is determined based on the prediction result, and the rest dirty pages are migrated offline in response to the triggering of the ending time, so that the problem that the migration time is too long due to the large amount of the memory data of the container is solved; on the other hand, in the process of transferring the dirty page, a preset compression algorithm is adopted to perform corresponding compression processing on the transferred data, such as LZ4 and the like, so that the accuracy of the transferred data is further ensured, the transfer efficiency is improved, and the transfer duration is shortened.

Further, for a better understanding of those skilled in the art, reference is made to the following detailed description taken in conjunction with the accompanying drawings.

Referring to fig. 6, a specific implementation flow of the online migration rule based on time series prediction provided in the embodiment of the present application is as follows:

step 601: compressing and transmitting all memory pages in the Pod to be migrated to the target Pod;

specifically, the compression transmission may use a preset compression algorithm, such as LZ 4.

Step 602: after all memory pages are compressed and transmitted, N rounds of transmission are carried out on dirty pages generated in the summary of the transmission process;

in addition, in the embodiment of the application, the actual dirty page amount and the actual transmission time length corresponding to the N rounds of transmission can be counted.

Step 603: judging whether the actual dirty page amount corresponding to the N-round transmission is smaller than a preset dirty page amount or not;

in the embodiment of the application, the dirty pages generated in the summary of the transmission process are subjected to actual dirty page amount counted by N-round transmission, and whether the actual dirty page amount corresponding to the N-round transmission is less than the preset dirty page amount is judged: if yes, go to step 606; if not, go to step 604.

In detail, the number of the dirty pages in the transmission process shows a certain regular increasing and decreasing trend, and if the actual dirty page amount corresponding to the N-round transmission is lower than the preset dirty page amount, the data amount required to be transmitted currently is represented to be relatively small, in other words, the migration time required for performing the halt copy on a small amount of data is also shorter, so that the purpose of being insensitive to a user is achieved, and further, the data can be compressed and transmitted, so that the migration time is shortened. Similarly, if the actual dirty page amount corresponding to the N-round transmission is greater than or equal to the preset dirty page amount, the feature that the migration time required for performing the stop copy is long, which will also affect the normal operation of the container application and bring a strong poor perception to the user.

Step 604: determining to carry out multi-round iterative transmission on dirty pages generated in the transmission process;

specifically, the multi-round iterative transmission is an online transmission based on time sequence prediction, and in addition, an LZ4 algorithm can be adopted to perform data compression processing in the multi-round iterative transmission process, so that the purposes of shortening the transmission time and improving the transmission efficiency are achieved.

It should be noted that the iterative transmission rounds are different from the N-round transmission, which is an online transmission but not an online transmission based on time series prediction.

In the embodiment of the application, for multiple rounds of iterative transmission, the actual dirty page amount and the actual transmission time length corresponding to each round of iterative transmission need to be obtained, and the ratio between the actual dirty page amount and the actual transmission time length is used as the actual dirty page rate corresponding to each round of iterative transmission, that is, the number of rewritten memories in unit time. Specifically, the initial predicted dirty page rate corresponding to the M +1 th round may be obtained based on an actual dirty page rate set corresponding to the M-th round, where the actual dirty page rate set corresponding to the M-th round is composed of actual sub-dirty page rates sorted according to a time sequence, that is, the actual dirty page rate set M = { v = for the M-th round ₁ ，v ₂ ，v ₃ ，…，v _M 8230, wherein, the actual transmission duration corresponding to the M-th round is divided into a plurality of time slices, and time slice sequences are formed according to the time of the time slices, so that v ₁ Actual sub-dirty pages corresponding to the 1 st time sliceThe same applies to other actual sub-dirty page rates in the actual dirty page rate set M, and details are not repeated here.

Actual dirty page rate set M = { v } based on Mth round correspondence ₁ ，v ₂ ，v ₃ ，…，v _M \8230 }, the initial predicted dirty page rate corresponding to the M +1 th round can be obtained, and the specific calculation method is shown in the following formula:

wherein t is more than or equal to N; m _Mw Transmitting the corresponding initial prediction dirty page rate for the M +1 th iteration; w is a ₁ 、w ₂ 、...、w _N For the weighting coefficients, the weighting coefficients characterize their respective corresponding v _M 、v _M+1 、...、v _t-N+1 The degree of importance of.

Optionally, a preset prediction model may also be adopted, and the actual dirty page amount, the actual transmission duration, and the actual dirty page rate corresponding to each iterative transmission in the multiple iterative transmissions are used as inputs of the preset prediction model, so that the initial predicted dirty page rate corresponding to each iterative transmission is obtained based on the output of the preset prediction model.

Further, the final predicted dirty page rate can be obtained only by correcting the initial predicted dirty page rate, for example, if the predicted dirty page rate corresponding to the M +1 th iteration transmission is calculated, for the previous M rounds of iteration transmission, an error value between the predicted dirty page rate corresponding to each round of iteration transmission and the actual dirty page rate is calculated, M error values corresponding to the previous M rounds of iteration transmission are obtained, then, a weighted average value of the M error values is calculated, the weighted average value is used as a target correction value corresponding to the M +1 th iteration transmission, the initial dirty page rate predicted by the M +1 th iteration transmission is determined, and the target correction value is used to correct the initial dirty page rate, so that the predicted dirty page rate corresponding to the M +1 th iteration transmission is obtained.

Then, the product of the predicted dirty page rate corresponding to the M +1 th iteration transmission and the transmission time spent on the M +1 th iteration transmission is calculated, and the calculation result is used as the predicted dirty page amount corresponding to the M +1 th iteration transmission.

Step 605: for multiple rounds of iterative transmission: judging whether the predicted dirty page amount corresponding to the M +1 th iteration transmission is larger than the dirty page amount generated by the M +1 th iteration transmission or judging whether the total time length of the previous M +1 iteration transmission is larger than a preset threshold value or not;

in the embodiment of the present application, for multiple rounds of iterative transmission, there are two cases of determination of iterative transmission a and B as follows.

A. Judging whether the predicted dirty page amount corresponding to the M +1 th iteration transmission is larger than the dirty page amount generated by the M +1 th iteration transmission;

B. and judging whether the total time length of the previous M +1 round of iterative transmission is greater than a preset threshold value or not.

The preset threshold value may be designed according to an actual container thermal migration scenario. For example, in a migration scenario of load balancing, since the requirement on perception of the migration process is relatively high, a preset threshold with a relatively large value may be designed to increase the total time of iterative transmission. For example, in a migration scenario of fault-tolerant processing, because the requirement on the time length of the shutdown copy is high, and the physical fault influence is increased as the time length of the iterative migration is increased, a preset threshold with a small value can be designed to shorten the total time length of the iterative transmission. In addition, if the preset threshold is set to 0, the method can be applied to an offline migration rule.

Note that, for the above determination a and the above determination B: if yes, go to step 606; if the determination result is negative, go to step 604.

Whether the current iterative transmission is terminated or not is judged through the judgment, on one hand, the quantity of the dirty pages generated by the M-th iterative transmission process is judged to be smaller than the predicted quantity of the dirty pages based on A, namely, the reduction trend of the quantity of the dirty pages generated by the M + 1-th iterative transmission process exceeds the expectation (such as the exponential reduction trend), in this case, the current iterative transmission is terminated, more dirty pages can be prevented from being generated, and the downtime (such as the time required for copying the rest of the dirty pages) can be effectively reduced; on the other hand, whether the total time length of the previous M +1 iteration transmission is greater than a preset threshold is determined based on B, and the method can be applied to a scenario of multiple container live migration, for example, a large preset threshold is designed for a migration scenario of load balancing, and a small preset threshold is designed for a migration scenario of fault-tolerant processing.

Step 606: and copying the dirty pages generated by the Pod to be migrated in the last iteration transmission to the destination Pod.

Specifically, copying the dirty pages generated by the Pod to be migrated in the last iteration transmission to the destination Pod is a shutdown migration process, and in addition, the device state information such as Memory, CPU, tty and the like is also shutdown migrated to the destination Pod.

In summary, the embodiments of the present application provide an online migration rule based on time series prediction, which predicts the change of dirty page data in a container application, and designs an end condition of iterative transmission to trigger the last stop copy in combination with the current transmission speed of a network bandwidth and an actual migration scenario, and in addition, reduces redundant iterative transmission in combination with a data compression technology, thereby shortening the transmission time.

It is worth to be noted that, through the online migration rule based on time series prediction, in practical application, the number of times of iterative migration can be effectively reduced, and particularly for migration of a memory-intensive container type, the total migration time and the downtime time can be greatly shortened.

Further, after the container memory information in the Pod to be migrated is migrated to the destination Pod, the current state of the migrated resource object needs to be modified, that is, the current state of the migrated resource object is modified from the migration state to the migration completed state.

And the target node responds to the situation that the current state of the migration resource object obtained from the main node is a migration completion state, and performs service recovery aiming at the target Pod, wherein the service recovery comprises the modification of the ID, the name, the IP address and the like of the target Pod, so that the following two situations exist for the service recovery of the target Pod.

The first condition is as follows: and in response to the successful service recovery of the target Pod, deleting the Pod to be migrated, starting the target Pod, and modifying the current state of the migrated resource object into a successful migration state.

And a second condition: and responding to the service recovery failure of the target Pod, recovering the service of the Pod to be migrated, deleting the target Pod, and modifying the current state of the migrated resource object into a migration failure state.

By the resource migration method, automatic migration control of containerized applications on a plurality of hosts in a cluster can be achieved, and the following technical effects can be achieved.

1. The embodiment of the application designs a definition of migration resources, designs a migration control module in a resource migration system, and further realizes the automatic control of the full life cycle of the migration process based on the control mode of the native resources in the native container arrangement system;

2. the embodiment of the application expands the functions of the Kubelet components in each working node (such as a source node and a destination node), and in the migration process, the Kubelet components are used for being responsible for processing work such as migration resource synchronization and migration of containers in a Pod, and the processing work includes: migration processing of Pod information to be migrated, a container file system, a storage volume and the like, and reconstruction processing of a target Pod;

3. according to the embodiment of the application, the runC is adopted during the design of a resource migration system, the runC container can realize the functions of dirty page storage, memory tracking, iterative migration and the like during operation, and the container can be subjected to snapshot;

4. the embodiment of the application provides an online migration rule based on time sequence prediction, changes of dirty page data in a container application are predicted, termination conditions of iterative transmission are designed by combining the current transmission speed of network bandwidth and an actual migration scene to trigger the last stop copy, and in addition, redundant iterative transmission is reduced by combining a data compression technology, so that the transmission time is shortened.

Based on the same inventive concept, the present application further provides a device for resource migration, so as to implement automated migration control of containerized applications on multiple hosts in a cluster, and with reference to fig. 7, the device includes:

an obtaining module 701, a source node obtaining migration information of a migration resource object; wherein the migration information at least comprises: the Pod to be migrated, the container memory migration type, the source node and the destination node;

a sending module 702, configured to collect container mirror information in the Pod to be migrated after the migration information is obtained, and send the container mirror information to the destination node;

a determining module 703, configured to determine, in response to the destination node receiving the container mirror information and creating a destination Pod, that the migration rule corresponding to the container memory migration type is an online migration rule;

and the migration module 704 migrates the container memory information in the Pod to be migrated to the destination Pod by using the online migration rule.

after the migration information is obtained, the container mirror image information in the Pod to be migrated is collected, and the container mirror image information is sent to the destination node, where the sending module 702 is specifically configured to:

the migration module 704 is specifically configured to migrate the container memory information in the Pod to be migrated to the destination Pod according to the online migration rule, and to:

and if it is determined that all the container memory information in the Pod to be migrated is migrated to the target Pod, modifying the current state of the migrated resource object to be a migration completion state.

In a possible design, the migrating module 704 is specifically configured to, by using the online migration rule, migrate the container memory information in the Pod to be migrated to the destination Pod:

calculating a predicted dirty page rate corresponding to the M +1 th round of iterative transmission based on a predicted dirty page rate and an actual dirty page rate corresponding to each previous M round of iterative transmission, and taking the product of the predicted dirty page rate and the transmission time spent in the M +1 th round of iterative transmission as a predicted dirty page amount corresponding to the M +1 th round of iterative transmission;

In a possible design, after compressing and transmitting all the memory pages, performing multiple rounds of iterative transmission on the dirty pages generated in the transmission process, where the migration module 704 is specifically configured to:

judging whether the actual dirty page amount corresponding to the N-round transmission is smaller than a preset dirty page amount or not;

if yes, copying dirty pages generated by the Pod to be migrated in the Nth round of iterative transmission into the target Pod;

In a possible design, the predicted dirty page rate corresponding to the M +1 th iteration transmission is calculated based on the predicted dirty page rate and the actual dirty page rate corresponding to each previous M iteration transmissions, and the migration module 704 is specifically configured to:

aiming at the previous M-round iterative transmission, calculating an error value between a predicted dirty page rate and an actual dirty page rate corresponding to each round of iterative transmission to obtain M error values corresponding to the previous M-round iterative transmission;

In a possible design, after the container memory information in the Pod to be migrated is migrated to the destination Pod, the migration module 704 is further configured to:

after the Pod to be migrated is deleted in response to the successful service restoration of the destination Pod, the migration module 704 is further configured to:

after the service of the Pod to be migrated is recovered in response to the failure of the service recovery of the destination Pod, the migration module 704 is further configured to:

Based on the device, the migration resource definition designed based on the application is combined with a unified asynchronous message processing mechanism of the container arrangement system, so that the device is suitable for the existing container arrangement system, and further the automatic migration control of the container application is realized.

Based on the same inventive concept, an embodiment of the present application further provides an electronic device, where the electronic device may implement the function of the foregoing device/system for resource migration, and with reference to fig. 8, the electronic device includes:

at least one processor 801 and a memory 802 connected to the at least one processor 801, in this embodiment, a specific connection medium between the processor 801 and the memory 802 is not limited in this application, and fig. 8 illustrates an example in which the processor 801 and the memory 802 are connected by a bus 800. The bus 800 is shown in fig. 8 by a thick line, and the connection between other components is merely illustrative and not intended to be limiting. The bus 800 may be divided into an address bus, a data bus, a control bus, etc., and is shown in fig. 8 with only one thick line for ease of illustration, but does not represent only one bus or type of bus. Alternatively, the processor 801 may also be referred to as a controller, without limitation to name a few.

In the embodiment of the present application, the memory 802 stores instructions executable by the at least one processor 801, and the at least one processor 801 may execute the resource migration method discussed above by executing the instructions stored in the memory 802. The processor 801 may implement the functions of the various modules in the apparatus/system shown in fig. 7.

The processor 801 is a control center of the apparatus/system, and may connect various parts of the entire control device by using various interfaces and lines, and perform various functions and process data of the apparatus/system by operating or executing instructions stored in the memory 802 and calling up data stored in the memory 802, thereby performing overall monitoring of the apparatus/system.

In one possible design, the processor 801 may include one or more processing units and the processor 801 may integrate an application processor that handles primarily the operating system, user interfaces, application programs, etc. and a modem processor that handles primarily wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 801. In some embodiments, the processor 801 and the memory 802 may be implemented on the same chip, or in some embodiments, they may be implemented separately on separate chips.

The processor 801 may be a general-purpose processor, such as a Central Processing Unit (CPU), digital signal processor, application specific integrated circuit, field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or the like, that implements or performs the methods, steps, and logic blocks disclosed in embodiments of the present application. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of the resource migration method disclosed in the embodiments of the present application may be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in the processor.

The memory 802, which is a non-volatile computer-readable storage medium, may be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. The Memory 802 may include at least one type of storage medium, and may include, for example, a flash Memory, a hard disk, a multimedia card, a card-type Memory, a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Programmable Read Only Memory (PROM), a Read Only Memory (ROM), a charge Erasable Programmable Read Only Memory (EEPROM), a magnetic Memory, a magnetic disk, an optical disk, and so on. The memory 802 is any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to such. The memory 802 in the embodiments of the present application may also be circuitry or any other device/system capable of performing a storage function for storing program instructions and/or data.

By programming the processor 801, the code corresponding to the resource migration method described in the foregoing embodiment may be solidified into a chip, so that the chip can execute the steps of the resource migration method of the embodiment shown in fig. 4 when running. How to program the processor 801 is well known to those skilled in the art and will not be described in detail herein.

Based on the same inventive concept, the present application also provides a storage medium storing computer instructions, which when executed on a computer, cause the computer to perform the resource migration method discussed above.

In some possible embodiments, the various aspects of the resource migration method provided by the present application may also be implemented in the form of a program product, which includes program code for causing the control apparatus to perform the steps in the resource migration method according to various exemplary embodiments of the present application described above in this specification, when the program product is run on a device.

It should be apparent to one skilled in the art that embodiments of the present application may be provided as a method, apparatus/system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.

Claims

1. A method of resource migration, the method comprising:

2. The method of claim 1, wherein the migration information further comprises: the current state of the migration resource object and the container information of each container in the Pod to be migrated, where the container information at least includes a container attribute and the current state, then:

3. The method of claim 1, wherein the migration information further comprises: the current state of the migration resource object and the container information of each container in the Pod to be migrated, where the container information at least includes a container attribute and the current state, then:

4. The method according to claim 1, wherein the migrating the container memory information in the Pod to be migrated to the destination Pod by using the online migration rule includes:

5. The method according to claim 4, wherein after the compressing and transmitting all the memory pages, performing multiple rounds of iterative transmission on the dirty pages generated in the transmission process includes:

6. The method according to any one of claims 4-5, wherein the calculating the predicted dirty page rate corresponding to the M +1 th iteration transmission based on the predicted dirty page rate and the actual dirty page rate corresponding to each of the previous M iteration transmissions comprises:

7. The method according to claim 1, wherein after the migrating the container memory information in the Pod to be migrated to the destination Pod, further comprising:

8. The method of claim 7, wherein the migration information further comprises: the current state of the migration resource object and the container information of each container in the Pod to be migrated, where the container information at least includes a container attribute and the current state, then:

9. An apparatus for resource migration, the apparatus comprising:

the sending module is used for collecting container mirror image information in the Pod to be migrated after the migration information is obtained, and sending the container mirror image information to the destination node;

the determining module is used for responding to the receiving of the container mirror image information by the destination node and creating a destination Pod and determining that the migration rule corresponding to the container memory migration type is an online migration rule;

10. An electronic device, comprising:

a memory for storing a computer program;

a processor for implementing the method steps of any one of claims 1-8 when executing the computer program stored on the memory.

11. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, which computer program, when being executed by a processor, carries out the method steps of any one of claims 1 to 8.