CN107491347B - Method and equipment for virtual machine live migration - Google Patents
Method and equipment for virtual machine live migration Download PDFInfo
- Publication number
- CN107491347B CN107491347B CN201610407506.3A CN201610407506A CN107491347B CN 107491347 B CN107491347 B CN 107491347B CN 201610407506 A CN201610407506 A CN 201610407506A CN 107491347 B CN107491347 B CN 107491347B
- Authority
- CN
- China
- Prior art keywords
- task
- executed
- asynchronous
- executing
- live migration
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/485—Task life-cycle, e.g. stopping, restarting, resuming execution
- G06F9/4856—Task life-cycle, e.g. stopping, restarting, resuming execution resumption being on a different machine, e.g. task migration, virtual machine migration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Retry When Errors Occur (AREA)
- Hardware Redundancy (AREA)
Abstract
The application aims to provide a method and equipment for realizing virtual machine live migration. Specifically, determining a task type of a task to be executed in a live migration process of a virtual machine, wherein the task type comprises a task which is prohibited from failing and a task which can be tolerated by failing; executing the task to be executed; and when the task to be executed fails to be executed and the task to be executed is a task which is prohibited from failing, executing the rollback operation corresponding to the task to be executed. Compared with the prior art, the method and the device have the advantages that the task type of the task to be executed in the live migration process of the virtual machine is determined, the task to be executed is executed according to the task type, when the task to be executed fails to be executed, and the corresponding rollback operation is executed for the task to be prohibited, so that the execution and judgment processes of different classification tasks are decoupled, the rollback entrance of each type of task is increased, the influence of the task which is lower in dependence on the live migration main flow is weakened, and the live migration process is stable and the efficiency is improved.
Description
Technical Field
The present application relates to the field of computers, and in particular, to a technique for implementing virtual machine live migration.
Background
With the development of big data, the application of virtual machines is increasingly wide, so that a virtual machine live migration technology which is not perceived by a user in a migration process is also developed greatly, in the prior art, live migration operation is divided into pre-migration and post-migration, resource preparation of a migration target physical machine is mainly completed before migration, cleaning of resources of a source physical machine is guaranteed after migration, a large number of task execution operations are available in two logics which are split in the live migration process and are before migration and after migration, and whether rollback is needed or not is determined according to scenes after each task execution operation fails.
However, in the prior art, the rollback after the task execution operation fails is divided into two entries according to two logics before and after the migration: if the resource is abnormally rolled back before the migration, no matter which step of task is rolled back caused by executing operation, even if the main process of the hot migration is still failed, the resources related to the virtual machine on the physical machine can be cleaned; if the rollback occurs after the migration failure, the rollback is mainly used for recycling the previously opened network and related resources in the target object. Therefore, in the whole process of the existing virtual machine live migration, the number of rollback entries is small, only two rollback entries are needed before and after migration, and the rollback strategy of each step of task execution operation cannot be subdivided, so that all task execution operation logics are difficult to subdivide, and a large number of data base transactions and single rollback entries are relied on, and the live migration issuing interface has hidden troubles from the aspects of stability and efficiency.
Disclosure of Invention
An object of the present application is to provide a method and an apparatus for implementing virtual machine live migration, so as to solve the problems of low efficiency and insufficient stability in the live migration process due to fewer rollback entries in the virtual machine live migration process.
In order to achieve the above object, the present application provides a method for implementing virtual machine live migration, which solves the problems of low efficiency and insufficient stability in the live migration process caused by fewer rollback entries in the virtual machine live migration process, and includes:
determining a task type of a task to be executed in a live migration process of a virtual machine, wherein the task type comprises a task which is prohibited from failing and a task which can be tolerated by failing;
executing the task to be executed;
and when the task to be executed fails to be executed and the task to be executed is a task which is prohibited from failing, executing the rollback operation corresponding to the task to be executed.
The application provides a device for realizing virtual machine live migration, and the device solves the problems of low efficiency and insufficient stability in the live migration process caused by fewer rollback entries in the virtual machine live migration process, and comprises:
the task type determining device is used for determining the task type of a task to be executed in the live migration process of the virtual machine, wherein the task type comprises a task which is prohibited from failing and a task which can tolerate the failure;
the task execution device is used for executing the task to be executed;
and the execution result processing device is used for executing the rollback operation corresponding to the task to be executed when the task to be executed fails to be executed and the task to be executed is the task prohibited from failing.
Compared with the prior art, the method and the device for determining the task type of the task to be executed in the live migration process of the virtual machine comprise forbidding a failure task and a failure tolerable task, executing the task to be executed according to different task types, when the task to be executed fails to execute, and executing the rollback operation corresponding to the task to be executed for forbidding the failure task, so that the execution and judgment processes of different classification tasks are decoupled, the rollback entrance of each type of task is increased, the influence of the task which depends on a lower hot migration process with the live migration main flow is weakened, and the live migration process is stable and the efficiency is improved.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:
FIG. 1 illustrates a flow diagram of a method for implementing virtual machine live migration in accordance with an aspect of the subject application;
FIG. 2 illustrates a flow diagram of a method for implementing virtual machine live migration in accordance with another aspect of the subject application;
FIG. 3 illustrates a schematic diagram of an apparatus for implementing virtual machine live migration in accordance with an aspect of the subject application;
FIG. 4 illustrates a schematic diagram of an apparatus for implementing virtual machine live migration in accordance with another aspect of the subject application;
FIG. 5 is a schematic diagram illustrating task type determination according to another preferred embodiment of the present application;
FIG. 6 illustrates a main flow diagram of virtual machine live migration according to another preferred embodiment of the present application.
The same or similar reference numbers in the drawings identify the same or similar elements.
Detailed Description
The present application is described in further detail below with reference to the attached figures.
In a typical configuration of the present application, the terminal, the device serving the network, and the trusted party each include one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory computer readable media (transient media), such as modulated data signals and carrier waves.
FIG. 1 illustrates a flow diagram of a method for implementing virtual machine live migration in accordance with an aspect of the subject application. The method comprises steps S1, S2 and S3.
In step S1, the device 1 determines a task type of a task to be executed in the live migration process of the virtual machine, where the task type includes a task prohibited from failing and a task tolerable to failure; the device 1 executes the task to be executed in step S2; in step S3, when the task to be executed fails to be executed and the task to be executed is a task prohibited from failing, the device 1 executes a rollback operation corresponding to the task to be executed.
Specifically, in step S1, device 1 determines a task type of a task to be executed during the live migration of the virtual machine, where the task type includes a task prohibited from failing and a task tolerable to failure. The tasks to be executed refer to all tasks to be executed in the virtual machine live migration process, such as physical machine resource configuration, firewall delivery, cache configuration, and the like. The task type refers to a type to which a task divided according to the strength of the dependency relationship between the task to be executed and the main live migration process belongs, as shown in fig. 5, the dependency relationship between the task to be executed and the main live migration process is divided into two categories, namely "no rollback required" and "no rollback required", according to whether the main live migration process is required to wait for a task execution result, and the two categories respectively correspond to the task prohibited from failing and the task tolerable to fail. The task forbidden to fail is a task type which must be successfully executed before the main process of the thermal migration is successfully completed, and a task on a key path pushed by the main process of the thermal migration is data, the main process of the thermal migration must acquire an execution result of the task in which the myopia fails, if the main process of the thermal migration cannot be successfully executed, the thermal migration of the virtual machine fails, if the task marked as 'needing rollback' in fig. 5 corresponds to the task forbidden to fail, because the task forbidden to fail is executed and can be correctly executed and pushed after the main process of the thermal migration is ensured by performing rollback operation after the task forbidden to fail is executed, the physical machine resource allocation exemplified above, and the task forbidden to fail issued by the firewall is the task forbidden. The failure-tolerant task refers to that the execution and the promotion of the main live migration process can be continued without waiting for the execution result of the main live migration process, and whether the execution result is successful or not has no influence on whether the main live migration process is finally successful, that is, the failure-tolerant task is not on the key path of the main live migration process promotion, that is, the failure-tolerant task corresponds to the "no rollback" marked in fig. 5, because the failure-tolerant task has no influence on the promotion and the successful completion of the main live migration process after the execution of the failure-tolerant task, the above-mentioned exemplified cache configuration, and the log dotting, the monitoring and other bypass logics all belong to the type of task.
Based on two task types of a task which is prohibited from failing and a task which can be tolerated from failing, the method includes but is not limited to the following dividing mode, after a main process of the virtual machine live migration starts, all tasks to be executed under the scene are classified and labeled every time a scene is advanced, or all tasks to be executed are classified before the main process of the virtual machine live migration starts, and the task types of all the tasks to be executed are divided after the main process of the virtual machine live migration starts. The task types are divided according to the strength of the dependence relationship between the tasks to be executed and the main live migration process, so that the task types of the tasks to be executed are determined, namely the tasks are determined on the critical path, different processing schemes can be adopted according to the task types, and the problems that the main live migration process is insufficient in stability and low in efficiency due to the fact that the tasks with tolerable failure are waited or rollback is caused by failure after the main live migration process is executed are avoided
It should be understood by those skilled in the art that the above-mentioned manner of dividing or determining the task type of the task to be performed is only an example, and other manners of dividing or determining the task type of the task to be performed, which may occur now or in the future, such as may be applicable to the present application, are also included in the scope of the present application and are hereby incorporated by reference.
Next, the device 1 executes the task to be executed in step S2. In the process of promoting the virtual machine live migration, the task to be executed is executed, for example, the execution end is a control system, the execution target can be an API, and the target machine is a remote service. And for the task which is forbidden to fail, directly executing the task and checking whether the execution result is successful or not. For a task with tolerable failure, the execution of the task is triggered without waiting or checking the execution result, and even if the failure is compensated or ignorable by other logic, for example, there is some configuration information, such as network configuration or cache configuration of a virtual machine, the information is cleared after the source-end related work in the main process of the thermal migration is completed, but the failure of clearing does not affect the success or failure of the whole main process of the thermal migration. According to different types of tasks to be executed, different processing is carried out in the execution process, so that the number of tasks on a key path influencing the success of a main process of the thermal migration is reduced, the thermal migration stability is improved, and the throughput rate of a task interface is improved.
It will be appreciated by those of ordinary skill in the art that the above-described manners of performing tasks are merely exemplary, and that other manners of performing tasks, whether presently existing or later to be performed, that are presently contemplated by the present application, are also encompassed within the scope of the present application and are hereby incorporated by reference.
Preferably, the disabling of the failed task comprises at least one of: a strongly consistent task; and finally, the task is consistent. The strong consistent task means that the dependence degree of the main process of the thermal migration is the strongest, namely the strong correlation with the promotion of the main process of the thermal migration is realized, the main process of the thermal migration needs to wait for the type of task until the task is successfully executed, and the promotion can be continued, and if the strong consistent task fails to be executed, the thermal migration will fail. As shown in fig. 5, the strongly consistent task must immediately take the result returned from the execution, the live migration main flow must synchronously wait for the completion of the execution of the type of task, wait for the result of the execution of the task, end the live migration main flow if the result fails, continue the execution if the result succeeds, and have no retry logic, for example, acquire the physical machine resource, and if the result cannot be acquired, the live migration cannot be performed without the resource. The final consistent task refers to a task with a strong dependency degree with a live migration main flow, which is issued by an interface of the live migration main flow during a process of advancing the live migration main flow, but an executed process is asynchronous with the live migration main flow, that is, for example, as shown in a live migration main flow diagram shown in fig. 6, after the final consistent task to be executed is checked, the final asynchronous task is inserted to trigger the execution of the final consistent task, and after feedback that an asynchronous task is successfully inserted is obtained, the live migration main flow continues to advance, preferably, the flow asynchronous task marked by a dotted line in fig. 6 is executed within a certain repeated execution condition, for example, within a certain time or execution time allowed range, and if the flow asynchronous task fails, the flow asynchronous task is repeatedly executed within the allowed range. For example, tasks that are somewhat time consuming, but not as indispensable as physical machine resources, can be treated as asynchronous tasks, such as firewall delivery only requiring migration at the virtualization level, executing in parallel with firewall asynchronous distribution, and not relying on each other. Preferably, as shown in fig. 6, the overall live migration completion status is queried in the whole live migration main flow, when all the prohibited failed tasks are completed, the live migration is considered to be completed, and the live migration is ended, and if the query times out, the prohibited failed tasks are not yet completed successfully, the migration is considered to be failed, but there is no strict sequential relationship between the strong consistent task and the final consistent task in the prohibited failed tasks during execution. The forbidden tasks are further classified, so that the processing granularity of the tasks is finer, and the tasks can be executed and processed more efficiently by better utilizing the characteristics of different types of tasks.
Therefore, when the prohibited failed task includes a strongly consistent task, it is preferable that the device 1 executes the task to be executed when the task to be executed is a strongly consistent task in step S2. That is, according to the further division of the task types, when the task to be executed is executed, for example, as shown in fig. 6, it is determined that a strongly consistent task is to be completed, the strongly consistent task is executed, and at the same time, the main live migration flow waits for and determines the execution result thereof. When the prohibited failed task includes the final consistent task, preferably in step S2, the device 1 inserts the task to be executed into the asynchronous task execution module of the live migration process when the task to be executed is the final consistent task. The asynchronous task execution module is a module for executing the asynchronous tasks included in the final consistent task and judging and processing an execution result. The execution of the final consistent task is to be understood as the task delivery or triggering of all asynchronous tasks that are involved in the final consistent task that needs to be executed so that they can be further processed. The strong consistent task and the final consistent task are executed in a distinguishing mode, so that the characteristics of the final consistent task are fully utilized, the execution result of the final consistent task does not need to be obtained immediately, the execution result is directly issued or triggered to be executed, the main process of the live migration does not need to be stopped integrally to wait for the execution result of the task, and the live migration efficiency of the virtual machine is improved.
Then, in step S3, when the to-be-executed task fails to be executed and the to-be-executed task is a task prohibited from failing, the apparatus 1 executes a rollback operation corresponding to the to-be-executed task. After the failed task is forbidden, the execution result is waited, if the execution result is failed, the forbidden failed task is rolled back, and the step to which the execution result needs to be rolled back in the main hot migration process is determined according to the specific situation of the forbidden failed task. For example, as shown in fig. 6, only two parts, namely a task requiring synchronous waiting for strong consistency and an asynchronous task insertion, belong to tasks requiring success, otherwise rollback is triggered, for example, a strong consistency task fails to execute and triggers rollback, and a final consistency task inserts an asynchronous task and also triggers rollback, but whether an asynchronous task is successfully executed and how to process the asynchronous task after the asynchronous task fails is not determined in the logic, and here, only whether the asynchronous task insertion of the final consistency task is successful or not is determined and processed. Whether rollback exists is judged according to the specific execution condition of the task, and only two rollback inlets before and after migration are not needed, so that the stability of the hot migration interface is greatly improved, the task of each interface is very clear, the interfaces are not tightly coupled together, and the rollback logic is clear.
Preferably, in step S3, when the task to be executed fails to be executed and the task to be executed is a task prohibited from failing, the device 1 executes a rollback operation corresponding to the task to be executed; otherwise, executing the next task to be executed until the hot migration process is finished. Compared with the case of prohibiting the failed task from executing failure as described above, the following execution cases are classified: when the execution of the task which is prohibited from failing is successful, the next task to be executed is continuously executed according to the propulsion of the main process of the live migration, wherein as shown in fig. 6, if the execution of the task which is strongly consistent is successful, the next task to be executed is continuously executed, and if the execution of the task which is finally consistent and inserted into the asynchronous task is successful, the next task to be executed is continuously executed; when the failure-tolerant task is executed, the main process of the live migration is continued no matter whether the execution is successful or not, and the execution condition of the hot migration does not need to be waited and considered. In fig. 6, the checking whether the execution of the strongly consistent task and the finally consistent task is completed is performed at the end of the main process, for example, the execution end of all the strongly consistent tasks and the finally consistent tasks is the control system, the execution target may be the API, and the target machine is a remote service. The strongly consistent task is that this API or remote service must succeed, otherwise roll back, and the request is a block wait, the caller must wait for the result; the final consistency is that the caller of the API or the remote service does not wait, only an asynchronous task is inserted to execute, and the task must be inserted successfully, after the hottest migration termination main flow is completed, a special call for inquiring whether the live migration task is executed or not is carried out, whether all final consistent tasks are executed or not is detected, and if the final consistent tasks are executed, the live migration is finished. The advantage of processing the execution result in this way is that it is no longer a uniform rollback entry in the prior art, but a rollback at task granularity, so that the stability of the live migration interface is greatly improved, the tasks of each interface are also very clear, are not tightly coupled together, and have a self-defined rollback logic.
It should be understood by those skilled in the art that the above-mentioned manner of performing rollback at task granularity according to task type is only an example, and other manners of performing rollback at task granularity according to task type, which are currently or later may occur, such as may be applicable to the present application, are also included in the scope of the present application and are hereby incorporated by reference.
FIG. 2 illustrates a flow diagram of a method for implementing virtual machine live migration in accordance with another aspect of the subject application. The method includes step S1, step S2, step S4, step S3.
In step S1, the device 1 determines a task type of a task to be executed in the live migration process of the virtual machine, where the task type includes a task prohibited from failing and a task tolerable to failure; the device 1 executes the task to be executed in step S2; device 1 executes an asynchronous task of said set of asynchronous tasks in step S4; in step S3, when the task to be executed fails to be executed and the task to be executed is a task prohibited from failing, the device 1 executes a rollback operation corresponding to the task to be executed.
Here, steps S1, S2, and S3 in fig. 2 are the same as or similar to steps S1, S2, and S3 in fig. 1, and are not repeated herein.
Specifically, the device 1 executes the asynchronous task in the asynchronous task execution module in step S4. After the final consistent task to which the asynchronous task belongs is obtained and successfully triggered or issued, the asynchronous task is executed, and meanwhile, the main process of the live migration continues to be advanced, for example, a caller of an API or a remote service does not wait, but only one task is separated to be issued and executed. The task can be completed later, namely the final success can be tolerated without waiting, and the task is issued and executed only and is completed later, so that the propulsion of the main process of the thermal migration is not influenced, and the thermal migration efficiency is improved.
Preferably, the device 1 executes the asynchronous task in the asynchronous task execution module in step S4; for the asynchronous task which fails to be executed, if the asynchronous task meets preset repeatable execution conditions, the asynchronous task is executed again; otherwise, executing the rollback operation corresponding to the task which is prohibited from failing. That is, in the process of executing the asynchronous task, because the execution result of the asynchronous task does not need to be obtained immediately, and the successful execution result of the asynchronous task does not need to be obtained immediately before the main process of the live migration is not completely completed, the execution retry of the failed asynchronous task can be carried out within the allowable range. The preset repeatable execution condition is a limit on an allowable range of the asynchronous task retry, for example, the repeatable execution condition may be to allow the retry within a certain time or to allow the retry within a certain number of execution times, for example, in the asynchronous task execution flow indicated by the dotted line in fig. 6, when the execution result is still not obtained after the timeout or the timeout, the asynchronous task is rolled back. The time for waiting the execution result is fully utilized by allowing the asynchronous task failed to execute to retry within a certain range, so that unnecessary rollback of the asynchronous task once the asynchronous task fails is avoided, and the stability of the hot migration is improved.
It should be understood by those skilled in the art that the above-described manner of repeatedly executing asynchronous tasks is merely an example, and other manners of repeatedly executing asynchronous tasks that may exist or may later be found in the asynchronous task execution process, such as may be applicable to the present application, are also included within the scope of the present application and are hereby incorporated by reference.
FIG. 3 illustrates a schematic diagram of an apparatus for implementing virtual machine live migration in accordance with an aspect of the subject application. The device 1 comprises a task type determining means 11, a task executing means 12 and an execution result processing means 13.
The task type determining device 11 determines a task type of a task to be executed in a live migration process of a virtual machine, wherein the task type includes a task which is prohibited from failing and a task which can tolerate failure; the task execution device 12 executes the task to be executed; when the to-be-executed task fails to be executed and the to-be-executed task is a task prohibited from failing, the execution result processing device 13 executes the rollback operation corresponding to the to-be-executed task.
Specifically, the task type determining device 11 determines the task type of the task to be executed in the live migration process of the virtual machine, where the task type includes a task that is prohibited from failing and a task that is tolerable to failure. The tasks to be executed refer to all tasks to be executed in the virtual machine live migration process, such as physical machine resource configuration, firewall delivery, cache configuration, and the like. The task type refers to a type to which a task divided according to the strength of the dependency relationship between the task to be executed and the main live migration process belongs, as shown in fig. 5, the dependency relationship between the task to be executed and the main live migration process is divided into two categories, namely "no rollback required" and "no rollback required", according to whether the main live migration process is required to wait for a task execution result, and the two categories respectively correspond to the task prohibited from failing and the task tolerable to fail. The task forbidden to fail is a task type which must be successfully executed before the main process of the thermal migration is successfully completed, and a task on a key path pushed by the main process of the thermal migration is data, the main process of the thermal migration must acquire an execution result of the task in which the myopia fails, if the main process of the thermal migration cannot be successfully executed, the thermal migration of the virtual machine fails, if the task marked as 'needing rollback' in fig. 5 corresponds to the task forbidden to fail, because the task forbidden to fail is executed and can be correctly executed and pushed after the main process of the thermal migration is ensured by performing rollback operation after the task forbidden to fail is executed, the physical machine resource allocation exemplified above, and the task forbidden to fail issued by the firewall is the task forbidden. The failure-tolerant task refers to that the execution and the promotion of the main live migration process can be continued without waiting for the execution result of the main live migration process, and whether the execution result is successful or not has no influence on whether the main live migration process is finally successful, that is, the failure-tolerant task is not on the key path of the main live migration process promotion, that is, the failure-tolerant task corresponds to the "no rollback" marked in fig. 5, because the failure-tolerant task has no influence on the promotion and the successful completion of the main live migration process after the execution of the failure-tolerant task, the above-mentioned exemplified cache configuration, and the log dotting, the monitoring and other bypass logics all belong to the type of task.
Based on two task types of a task which is prohibited from failing and a task which can be tolerated from failing, the method includes but is not limited to the following dividing mode, after a main process of the virtual machine live migration starts, all tasks to be executed under the scene are classified and labeled every time a scene is advanced, or all tasks to be executed are classified before the main process of the virtual machine live migration starts, and the task types of all the tasks to be executed are divided after the main process of the virtual machine live migration starts. The task types are divided according to the strength of the dependence relationship between the tasks to be executed and the main live migration process, so that the task types of the tasks to be executed are determined, namely the tasks are determined on the critical path, different processing schemes can be adopted according to the task types, and the problems that the main live migration process is insufficient in stability and low in efficiency due to the fact that the tasks with tolerable failure are waited or rollback is caused by failure after the main live migration process is executed are avoided
It should be understood by those skilled in the art that the above-mentioned manner of dividing or determining the task type of the task to be performed is only an example, and other manners of dividing or determining the task type of the task to be performed, which may occur now or in the future, such as may be applicable to the present application, are also included in the scope of the present application and are hereby incorporated by reference.
Then, the task performing device 12 performs the task to be performed. In the process of promoting the virtual machine live migration, the task to be executed is executed, for example, the execution end is a control system, the execution target can be an API, and the target machine is a remote service. And for the task which is forbidden to fail, directly executing the task and checking whether the execution result is successful or not. For a task with tolerable failure, the execution of the task is triggered without waiting or checking the execution result, and even if the failure is compensated or ignorable by other logic, for example, there is some configuration information, such as network configuration or cache configuration of a virtual machine, the information is cleared after the source-end related work in the main process of the thermal migration is completed, but the failure of clearing does not affect the success or failure of the whole main process of the thermal migration. According to different types of tasks to be executed, different processing is carried out in the execution process, so that the number of tasks on a key path influencing the success of a main process of the thermal migration is reduced, the thermal migration stability is improved, and the throughput rate of a task interface is improved.
It will be appreciated by those of ordinary skill in the art that the above-described manners of performing tasks are merely exemplary, and that other manners of performing tasks, whether presently existing or later to be performed, that are presently contemplated by the present application, are also encompassed within the scope of the present application and are hereby incorporated by reference.
Preferably, the disabling of the failed task comprises at least one of: a strongly consistent task; and finally, the task is consistent. The strong consistent task means that the dependence degree of the main process of the thermal migration is the strongest, namely the strong correlation with the promotion of the main process of the thermal migration is realized, the main process of the thermal migration needs to wait for the type of task until the task is successfully executed, and the promotion can be continued, and if the strong consistent task fails to be executed, the thermal migration will fail. As shown in fig. 5, the strongly consistent task must immediately take the result returned from the execution, the live migration main flow must synchronously wait for the completion of the execution of the type of task, wait for the result of the execution of the task, end the live migration main flow if the result fails, continue the execution if the result succeeds, and have no retry logic, for example, acquire the physical machine resource, and if the result cannot be acquired, the live migration cannot be performed without the resource. The final consistent task refers to a task with a strong dependency degree with a live migration main flow, which is issued by an interface of the live migration main flow during a process of advancing the live migration main flow, but an executed process is asynchronous with the live migration main flow, that is, for example, as shown in a live migration main flow diagram shown in fig. 6, after the final consistent task to be executed is checked, the final asynchronous task is inserted to trigger the execution of the final consistent task, and after feedback that an asynchronous task is successfully inserted is obtained, the live migration main flow continues to advance, preferably, the flow asynchronous task marked by a dotted line in fig. 6 is executed within a certain repeated execution condition, for example, within a certain time or execution time allowed range, and if the flow asynchronous task fails, the flow asynchronous task is repeatedly executed within the allowed range. For example, tasks that are somewhat time consuming, but not as indispensable as physical machine resources, can be treated as asynchronous tasks, such as firewall delivery only requiring migration at the virtualization level, executing in parallel with firewall asynchronous distribution, and not relying on each other. Preferably, as shown in fig. 6, the overall live migration completion status is queried in the whole live migration main flow, when all the prohibited failed tasks are completed, the live migration is considered to be completed, and the live migration is ended, and if the query times out, the prohibited failed tasks are not yet completed successfully, the migration is considered to be failed, but there is no strict sequential relationship between the strong consistent task and the final consistent task in the prohibited failed tasks during execution. The forbidden tasks are further classified, so that the processing granularity of the tasks is finer, and the tasks can be executed and processed more efficiently by better utilizing the characteristics of different types of tasks.
Therefore, when the prohibited failed task includes a strongly consistent task, it is preferable that the task performing device 12 performs the task to be performed when the task to be performed is a strongly consistent task. That is, according to the further division of the task types, when the task to be executed is executed, for example, as shown in fig. 6, it is determined that a strongly consistent task is to be completed, the strongly consistent task is executed, and at the same time, the main live migration flow waits for and determines the execution result thereof. When the prohibited failed task includes a final consistent task, preferably, the task execution device 12 inserts the to-be-executed task into the asynchronous task execution module of the live migration process when the to-be-executed task is the final consistent task. The asynchronous task execution module is a module for executing the asynchronous tasks included in the final consistent task and judging and processing an execution result. The execution of the final consistent task is to be understood as the task delivery or triggering of all asynchronous tasks that are involved in the final consistent task that needs to be executed so that they can be further processed. The strong consistent task and the final consistent task are executed in a distinguishing mode, so that the characteristics of the final consistent task are fully utilized, the execution result of the final consistent task does not need to be obtained immediately, the execution result is directly issued or triggered to be executed, the main process of the live migration does not need to be stopped integrally to wait for the execution result of the task, and the live migration efficiency of the virtual machine is improved.
Then, when the to-be-executed task fails to be executed and the to-be-executed task is a task prohibited from failing, the execution result processing device 13 executes the rollback operation corresponding to the to-be-executed task. After the failed task is forbidden, the execution result is waited, if the execution result is failed, the forbidden failed task is rolled back, and the step to which the execution result needs to be rolled back in the main hot migration process is determined according to the specific situation of the forbidden failed task. For example, as shown in fig. 6, only two parts, namely a task requiring synchronous waiting for strong consistency and an asynchronous task insertion, belong to tasks requiring success, otherwise rollback is triggered, for example, a strong consistency task fails to execute and triggers rollback, and a final consistency task inserts an asynchronous task and also triggers rollback, but whether an asynchronous task is successfully executed and how to process the asynchronous task after the asynchronous task fails is not determined in the logic, and here, only whether the asynchronous task insertion of the final consistency task is successful or not is determined and processed. Whether rollback exists is judged according to the specific execution condition of the task, and only two rollback inlets before and after migration are not needed, so that the stability of the hot migration interface is greatly improved, the task of each interface is very clear, the interfaces are not tightly coupled together, and the rollback logic is clear.
Preferably, when the to-be-executed task fails to be executed and the to-be-executed task is a task for which failure is prohibited, the execution result processing device 13 executes a rollback operation corresponding to the to-be-executed task; otherwise, executing the next task to be executed until the hot migration process is finished. Compared with the case of prohibiting the failed task from executing failure as described above, the following execution cases are classified: when the execution of the task which is prohibited from failing is successful, the next task to be executed is continuously executed according to the propulsion of the main process of the live migration, wherein as shown in fig. 6, if the execution of the task which is strongly consistent is successful, the next task to be executed is continuously executed, and if the execution of the task which is finally consistent and inserted into the asynchronous task is successful, the next task to be executed is continuously executed; when the failure-tolerant task is executed, the main process of the live migration is continued no matter whether the execution is successful or not, and the execution condition of the hot migration does not need to be waited and considered. In fig. 6, the checking whether the execution of the strongly consistent task and the finally consistent task is completed is performed at the end of the main process, for example, the execution end of all the strongly consistent tasks and the finally consistent tasks is the control system, the execution target may be the API, and the target machine is a remote service. The strongly consistent task is that this API or remote service must succeed, otherwise roll back, and the request is a block wait, the caller must wait for the result; the final consistency is that the caller of the API or the remote service does not wait, only an asynchronous task is inserted to execute, and the task must be inserted successfully, after the hottest migration termination main flow is completed, a special call for inquiring whether the live migration task is executed or not is carried out, whether all final consistent tasks are executed or not is detected, and if the final consistent tasks are executed, the live migration is finished. The advantage of processing the execution result in this way is that it is no longer a uniform rollback entry in the prior art, but a rollback at task granularity, so that the stability of the live migration interface is greatly improved, the tasks of each interface are also very clear, are not tightly coupled together, and have a self-defined rollback logic.
It should be understood by those skilled in the art that the above-mentioned manner of performing rollback at task granularity according to task type is only an example, and other manners of performing rollback at task granularity according to task type, which are currently or later may occur, such as may be applicable to the present application, are also included in the scope of the present application and are hereby incorporated by reference.
FIG. 4 illustrates a schematic diagram of an apparatus for implementing virtual machine live migration in accordance with another aspect of the subject application. The device 1 comprises a task type determining means 21, a task executing means 22, an asynchronous task executing means 24, and an execution result processing means 23.
The task type determining device 21 determines a task type of a task to be executed in a live migration process of the virtual machine, where the task type includes a task prohibited from failing and a task tolerable to failure; the task execution device 22 executes the task to be executed; the asynchronous task execution device 24 executes asynchronous tasks in the asynchronous task set; when the to-be-executed task fails to be executed and the to-be-executed task is a task prohibited from failing, the execution result processing device 23 executes the rollback operation corresponding to the to-be-executed task.
Here, the type determining device 21, the task executing device 22, and the execution result processing device 23 in fig. 4 are the same as or similar to the type determining device 11, the task executing device 12, and the execution result processing device 13 in fig. 3, and are not described again.
Specifically, the asynchronous task execution device 24 executes the asynchronous task in the asynchronous task execution module. After the final consistent task to which the asynchronous task belongs is obtained and successfully triggered or issued, the asynchronous task is executed, and meanwhile, the main process of the live migration continues to be advanced, for example, a caller of an API or a remote service does not wait, but only one task is separated to be issued and executed. The task can be completed later, namely the final success can be tolerated without waiting, and the task is issued and executed only and is completed later, so that the propulsion of the main process of the thermal migration is not influenced, and the thermal migration efficiency is improved.
Preferably, the asynchronous task execution device 24 executes an asynchronous task in the asynchronous task execution module; for the asynchronous task which fails to be executed, if the asynchronous task meets preset repeatable execution conditions, the asynchronous task is executed again; otherwise, executing the rollback operation corresponding to the task which is prohibited from failing. That is, in the process of executing the asynchronous task, because the execution result of the asynchronous task does not need to be obtained immediately, and the successful execution result of the asynchronous task does not need to be obtained immediately before the main process of the live migration is not completely completed, the execution retry of the failed asynchronous task can be carried out within the allowable range. The preset repeatable execution condition is a limit on an allowable range of the asynchronous task retry, for example, the repeatable execution condition may be to allow the retry within a certain time or to allow the retry within a certain number of execution times, for example, in the asynchronous task execution flow indicated by the dotted line in fig. 6, when the execution result is still not obtained after the timeout or the timeout, the asynchronous task is rolled back. The time for waiting the execution result is fully utilized by allowing the asynchronous task failed to execute to retry within a certain range, so that unnecessary rollback of the asynchronous task once the asynchronous task fails is avoided, and the stability of the hot migration is improved.
It should be understood by those skilled in the art that the above-described manner of repeatedly executing asynchronous tasks is merely an example, and other manners of repeatedly executing asynchronous tasks that may exist or may later be found in the asynchronous task execution process, such as may be applicable to the present application, are also included within the scope of the present application and are hereby incorporated by reference.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.
It should be noted that the present application may be implemented in software and/or a combination of software and hardware, for example, implemented using Application Specific Integrated Circuits (ASICs), general purpose computers or any other similar hardware devices. In one embodiment, the software programs of the present application may be executed by a processor to implement the steps or functions described above. Likewise, the software programs (including associated data structures) of the present application may be stored in a computer readable recording medium, such as RAM memory, magnetic or optical drive or diskette and the like. Additionally, some of the steps or functions of the present application may be implemented in hardware, for example, as circuitry that cooperates with the processor to perform various steps or functions.
In addition, some of the present application may be implemented as a computer program product, such as computer program instructions, which when executed by a computer, may invoke or provide methods and/or techniques in accordance with the present application through the operation of the computer. Program instructions which invoke the methods of the present application may be stored on a fixed or removable recording medium and/or transmitted via a data stream on a broadcast or other signal-bearing medium and/or stored within a working memory of a computer device operating in accordance with the program instructions. An embodiment according to the present application comprises an apparatus comprising a memory for storing computer program instructions and a processor for executing the program instructions, wherein the computer program instructions, when executed by the processor, trigger the apparatus to perform a method and/or a solution according to the aforementioned embodiments of the present application.
It will be evident to those skilled in the art that the present application is not limited to the details of the foregoing illustrative embodiments, and that the present application may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned. Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or means recited in the apparatus claims may also be implemented by one unit or means in software or hardware. The terms first, second, etc. are used to denote names, but not any particular order.
Claims (14)
1. A method for implementing virtual machine live migration, comprising:
determining a task type of a task to be executed in a live migration process of a virtual machine, wherein the task type comprises a task which is prohibited from failing and a task which can be tolerated by failing; the task types are divided according to the dependency relationship between the tasks to be executed and the main process of the thermal migration; dividing the dependency relationship between the tasks to be executed and the live migration main flow according to whether the live migration main flow is needed to wait for the execution result of the tasks; the failure-tolerant tasks include: the task type of the execution result of the main flow of the live migration is not required to be waited; the disabling of the failed task includes: the type of task that must be performed successfully before the main live migration process completes successfully;
executing the task to be executed;
when the task to be executed fails to be executed and the task to be executed is a task which is prohibited from failing, executing rollback operation corresponding to the task to be executed; and if the task to be executed is a failure tolerable task, no rollback operation is required to be executed.
2. The method of claim 1, wherein, when the task to be executed fails to be executed and the task to be executed is a task that is prohibited from failing, executing the rollback operation corresponding to the task to be executed comprises:
when the task to be executed fails to be executed and the task to be executed is a task which is prohibited from failing, executing rollback operation corresponding to the task to be executed; if not, then,
and executing the next task to be executed until the hot migration process is finished.
3. The method of claim 1, wherein the disabling of failed tasks comprises at least one of:
a strongly consistent task;
and finally, the task is consistent.
4. The method of claim 3, wherein the prohibited failed tasks include strongly consistent tasks;
wherein the executing the task to be executed comprises:
and when the task to be executed is a strong consistent task, executing the task to be executed.
5. The method of claim 3, wherein the prohibited failed tasks include a final consistent task;
wherein the executing the task to be executed comprises:
and when the task to be executed is the final consistent task, inserting the task to be executed into an asynchronous task execution module of the live migration process.
6. The method of claim 5, wherein the method further comprises:
and executing the asynchronous task in the asynchronous task execution module.
7. The method of claim 6, wherein the executing the asynchronous task in the asynchronous task execution module comprises:
executing the asynchronous task in the asynchronous task execution module;
for the asynchronous task which fails to be executed, if the asynchronous task meets preset repeatable execution conditions, the asynchronous task is executed again; otherwise, executing the rollback operation corresponding to the task which is prohibited from failing.
8. An apparatus for implementing virtual machine live migration, comprising:
the task type determining device is used for determining the task type of a task to be executed in the live migration process of the virtual machine, wherein the task type comprises a task which is prohibited from failing and a task which can tolerate the failure; the task types are divided according to the dependency relationship between the tasks to be executed and the main process of the thermal migration; dividing the dependency relationship between the tasks to be executed and the live migration main flow according to whether the live migration main flow is needed to wait for the execution result of the tasks; the failure-tolerant tasks include: the task type of the execution result of the main flow of the live migration is not required to be waited; the disabling of the failed task includes: the type of task that must be performed successfully before the main live migration process completes successfully;
the task execution device is used for executing the task to be executed;
the execution result processing device is used for executing the rollback operation corresponding to the task to be executed when the task to be executed fails to be executed and the task to be executed is a task which is prohibited from failing; and if the task to be executed is a failure tolerable task, no rollback operation is required to be executed.
9. The apparatus of claim 8, wherein the execution result processing means is to:
the task failure processing unit is used for executing the rollback operation corresponding to the task to be executed when the task to be executed fails to be executed and the task to be executed is the task failure processing unit;
and the other execution result processing units are used for executing the next task to be executed until the live migration process is finished if the other execution result processing units are not used for executing the next task to be executed.
10. The device of claim 8, wherein the disabling of failed tasks comprises at least one of:
a strongly consistent task;
and finally, the task is consistent.
11. The apparatus of claim 10, wherein the prohibited failed tasks include strongly consistent tasks;
wherein the task execution device includes:
and the strong consistent task execution unit is used for executing the task to be executed when the task to be executed is a strong consistent task.
12. The apparatus of claim 10, wherein the prohibited failed tasks include a final consistent task;
wherein the task execution device includes:
and the final consistent task execution unit is used for inserting the task to be executed into the asynchronous task execution module of the live migration process when the task to be executed is the final consistent task.
13. The apparatus of claim 12, wherein the apparatus further comprises:
and the asynchronous task execution device is used for executing the asynchronous task in the asynchronous task execution module.
14. The apparatus of claim 13, wherein the asynchronous task execution device comprises:
the asynchronous task execution unit is used for executing the asynchronous task in the asynchronous task execution module;
the asynchronous task execution result processing unit is used for re-executing the asynchronous task which fails to be executed if the asynchronous task meets preset repeatable execution conditions; otherwise, executing the rollback operation corresponding to the task which is prohibited from failing.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610407506.3A CN107491347B (en) | 2016-06-12 | 2016-06-12 | Method and equipment for virtual machine live migration |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610407506.3A CN107491347B (en) | 2016-06-12 | 2016-06-12 | Method and equipment for virtual machine live migration |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107491347A CN107491347A (en) | 2017-12-19 |
| CN107491347B true CN107491347B (en) | 2021-04-27 |
Family
ID=60642916
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610407506.3A Active CN107491347B (en) | 2016-06-12 | 2016-06-12 | Method and equipment for virtual machine live migration |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107491347B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110874264B (en) * | 2018-08-30 | 2023-05-02 | 阿里巴巴集团控股有限公司 | Instance thermomigration method and device, storage medium and processor |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103077083A (en) * | 2013-01-09 | 2013-05-01 | 苏州亿倍信息技术有限公司 | Method and system for processing tasks |
| CN103279385A (en) * | 2013-06-01 | 2013-09-04 | 北京华胜天成科技股份有限公司 | Method and system for scheduling cluster tasks in cloud computing environment |
| CN103336709A (en) * | 2013-06-01 | 2013-10-02 | 北京华胜天成科技股份有限公司 | Method and system for realizing virtual distributed unified management in cluster |
| CN103856548A (en) * | 2012-12-07 | 2014-06-11 | 华为技术有限公司 | Dynamic resource scheduling method and dynamic resource scheduler |
| CN105045659A (en) * | 2015-07-17 | 2015-11-11 | 中国人民解放军国防科学技术大学 | Task overlapping and virtual machine migration based cloud fault-tolerant task scheduling method |
-
2016
- 2016-06-12 CN CN201610407506.3A patent/CN107491347B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103856548A (en) * | 2012-12-07 | 2014-06-11 | 华为技术有限公司 | Dynamic resource scheduling method and dynamic resource scheduler |
| CN103077083A (en) * | 2013-01-09 | 2013-05-01 | 苏州亿倍信息技术有限公司 | Method and system for processing tasks |
| CN103279385A (en) * | 2013-06-01 | 2013-09-04 | 北京华胜天成科技股份有限公司 | Method and system for scheduling cluster tasks in cloud computing environment |
| CN103336709A (en) * | 2013-06-01 | 2013-10-02 | 北京华胜天成科技股份有限公司 | Method and system for realizing virtual distributed unified management in cluster |
| CN105045659A (en) * | 2015-07-17 | 2015-11-11 | 中国人民解放军国防科学技术大学 | Task overlapping and virtual machine migration based cloud fault-tolerant task scheduling method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107491347A (en) | 2017-12-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108648078B (en) | Transaction preprocessing method and device and electronic equipment | |
| CN107040585B (en) | Service checking method and device | |
| US10503629B2 (en) | Debugging method, multi-core processor, and debugging device | |
| CN110442560B (en) | Log replay method, device, server and storage medium | |
| US20190020729A1 (en) | Method, apparatus, and electronic device for processing consensus requests in a blockchain consensus network | |
| US10409709B2 (en) | Debugging method, multi-core processor and debugging device | |
| WO2017008675A1 (en) | Method and device for transmitting data in virtual environment | |
| CN108846749B (en) | Partitioned transaction execution system and method based on block chain technology | |
| EP3934163A1 (en) | Methods and consensus nodes for block generation | |
| CN103198122B (en) | Restart the method and apparatus of memory database | |
| CN110648136B (en) | Consensus and transaction synchronous parallel processing method and device and electronic equipment | |
| US9875057B2 (en) | Method of live migration | |
| CN111711623A (en) | Data verification method and device | |
| JP6859518B2 (en) | How to prevent attacks on servers and devices | |
| CN112231403B (en) | Consistency verification method, device, equipment and storage medium for data synchronization | |
| CN106815094B (en) | Method and equipment for realizing transaction submission in master-slave synchronization mode | |
| US20240095174A1 (en) | Method for detecting error of operating system kernel memory in real time | |
| CN107491347B (en) | Method and equipment for virtual machine live migration | |
| CN112887436B (en) | Consensus method, consensus node and block chain system of pipeline mode | |
| US20200371882A1 (en) | Method, Apparatus, Device and Medium for Starting Virtual Machine | |
| CN112711462A (en) | Cloud platform virtual CPU hot binding method and device and computer readable storage medium | |
| CN115760405A (en) | Transaction execution method, device, computer equipment and medium | |
| CN112379952B (en) | Method for implementing cross-process callback | |
| CN111459474B (en) | Templated data processing method and device | |
| US20210272646A1 (en) | Efficient resource sharing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20230530 Address after: Room 1-2-A06, Yungu Park, No. 1008 Dengcai Street, Sandun Town, Xihu District, Hangzhou City, Zhejiang Province Patentee after: Aliyun Computing Co.,Ltd. Address before: Box 847, four, Grand Cayman capital, Cayman Islands, UK Patentee before: ALIBABA GROUP HOLDING Ltd. |
|
| TR01 | Transfer of patent right |