CN107368355B - Dynamic scheduling method and device of virtual machine - Google Patents
Dynamic scheduling method and device of virtual machine Download PDFInfo
- Publication number
- CN107368355B CN107368355B CN201710705796.4A CN201710705796A CN107368355B CN 107368355 B CN107368355 B CN 107368355B CN 201710705796 A CN201710705796 A CN 201710705796A CN 107368355 B CN107368355 B CN 107368355B
- Authority
- CN
- China
- Prior art keywords
- physical machine
- sequence
- physical
- machine
- level
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45562—Creating, deleting, cloning virtual machine instances
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention provides a dynamic scheduling method and a dynamic scheduling device for a virtual machine, wherein the method comprises the following steps: acquiring position information of a target service application; determining a first-level physical machine sequence to be scheduled based on the position information, wherein physical machines contained in the first-level physical machine sequence can provide a hardware environment for a target virtual machine; determining a second-level physical machine sequence meeting the conditions through the use information of each physical machine in the first-level physical machine sequence; and screening the physical machines in the second-level physical machine sequence to obtain a third-level physical machine sequence, and determining a first target physical machine in the third-level physical machine sequence, so as to deploy the virtual machines according to the first target physical machine, wherein the position information of any two physical machines in the third-level physical machine sequence is different, thereby relieving the technical problem of single-machine bottleneck or bottleneck at the same position caused by scheduling to the same physical machine or physical machines at the same physical position when a plurality of virtual machines are created by single service application.
Description
Technical Field
The invention relates to the technical field of cloud computing, in particular to a dynamic scheduling method and device of a virtual machine.
Background
With the rapid development of cloud computing technology, resources of a data center can be fully utilized, and more enterprises begin to adopt the technology. The existing scheduling method for the virtual machine is mainly based on the following two aspects. In the first aspect, available physical resources which can meet the virtual machine resource amount are searched in a physical resource list, and then virtual machine resources in the searched available physical resources are allocated according to the current application of the strategy. In the second aspect, load data of each virtual machine periodically reported by a physical machine monitor is collected; then, estimating the future load of the virtual machine according to the data; and then, obtaining an adjustment list of the virtual machine by using a packing algorithm, converting the adjustment list into a migration instruction and issuing the migration instruction to each physical machine monitor. And the physical machine monitor executes virtual machine migration according to the migration instruction, and if the migration target is a physical server in a dormant state, the physical server is firstly awakened to complete a scheduling task.
However, when the two scheduling methods are adopted to schedule the physical machine, the following problems exist, that is, when a physical machine computing node is added in a cluster, because the resource of the physical machine computing node is relatively sufficient, a plurality of virtual machine creation requests of a service application can be all scheduled to the physical machine node, and when the physical machine node fails, the service application is caused to have a condition that the processing capability is greatly reduced, even normal service cannot be provided, and the high availability requirement of the service application is greatly influenced. Similarly, when multiple virtual machine creation requests of a service application are scheduled to be deployed on a physical machine computing node on the same cabinet, if the cabinet is powered off, the physical machine node cannot provide services, and a condition that one service application cannot provide services is also caused.
Disclosure of Invention
In view of the above, an object of the present invention is to provide a method and an apparatus for dynamically scheduling virtual machines, so as to alleviate a technical problem of a single application machine bottleneck or a bottleneck at a same location caused by scheduling to a same physical machine or a physical machine at a same physical location when a single service application creates multiple virtual machines.
According to an aspect of the present invention, a method for dynamically scheduling a virtual machine is provided, including: acquiring position information of a target service application, wherein the target service application is a service application requesting to create a target virtual machine; determining a first-level physical machine sequence to be scheduled based on the position information, wherein physical machines contained in the first-level physical machine sequence can provide a hardware environment for the target virtual machine; determining a second-level physical machine sequence meeting the conditions through the use information of each physical machine in the first-level physical machine sequence; and screening the physical machines in the second-level physical machine sequence to obtain a third-level physical machine sequence, and determining a first target physical machine in the third-level physical machine sequence so as to deploy a virtual machine according to the first target physical machine, wherein the position information of any two physical machines in the third-level physical machine sequence is different.
Further, determining a first-level physical machine sequence to be scheduled based on the location information comprises: determining a physical machine sequence to be deleted based on the position information, wherein the physical machine in the physical machine sequence to be deleted is a physical machine which is created with a virtual machine by the target service application; acquiring partition information of the target virtual machine created by the target service application; acquiring a physical machine sequence belonging to the partition information; and deleting the physical machine sequence to be deleted in the physical machine sequence in the partition information, thereby obtaining the first-stage physical machine sequence.
Further, determining a sequence of physical machines to be deleted based on the location information comprises: judging whether to create a virtual machine for the first time based on the position information; if the virtual machine is judged not to be created for the first time, acquiring the position information of the created virtual machine; and determining all physical machines positioned at the historical positions according to the historical positions contained in the position information of the created virtual machines, and taking a sequence formed by all the physical machines as the sequence of the physical machines to be deleted.
Further, determining whether to create the target virtual machine for the first time based on the location information comprises: judging whether the position information is a null value; if the position information is judged to be a null value, the virtual machine is judged to be established for the first time, and if the position information is judged not to be the null value, the virtual machine is judged not to be established for the first time.
Further, determining a second-level sequence of physical machines satisfying a condition from usage information of each physical machine in the first-level sequence of physical machines comprises: acquiring CPU use information, memory use information and disk use information of each physical machine in the first-stage physical machine sequence; determining a physical machine meeting a virtual machine creation condition in the first-level physical machine sequence based on the CPU usage information, the memory usage information and the disk usage information of each physical machine, and taking the physical machine meeting the virtual machine creation condition as the second-level physical machine sequence.
Further, screening the physical machines in the second-level physical machine sequence to obtain a third-level physical machine sequence includes: sequencing the physical machines in the second-level physical machine sequence according to the position information; screening the sorted second-level physical machine sequence to obtain at least one second target physical machine, wherein the second target physical machine is a physical machine with the largest weight in physical machines with the same position information; determining the third level physical machine sequence based on the at least one second target physical machine.
Further, determining a first target physical machine in the third level sequence of physical machines comprises: selecting a physical machine with the largest weight value in the third-level physical machine sequence; and taking the physical machine with the maximum weight value as the first target physical machine.
According to another aspect of the present invention, there is also provided a dynamic scheduling apparatus for a virtual machine, including: the system comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring the position information of a target service application, and the target service application is the service application requesting to create a target virtual machine; a first determining unit, configured to determine, based on the location information, a first-level physical machine sequence to be scheduled, where a physical machine included in the first-level physical machine sequence is capable of providing a hardware environment for the target virtual machine; a second determining unit, configured to determine, through the usage information of each physical machine in the first-level physical machine sequence, a second-level physical machine sequence that satisfies a condition; and the screening unit is used for screening the physical machines in the second-level physical machine sequence to obtain a third-level physical machine sequence, and determining a first target physical machine in the third-level physical machine sequence so as to deploy a virtual machine according to the first target physical machine, wherein the position information of any two physical machines in the third-level physical machine sequence is different.
Further, the first determination unit includes: a determining module, configured to determine a sequence of physical machines to be deleted based on the location information, where a physical machine in the sequence of physical machines to be deleted is a physical machine for which a virtual machine has been created by the target service application; the first acquisition module is used for acquiring the partition information of the target virtual machine created by the target service application; the second acquisition module is used for acquiring the physical machine sequence belonging to the partition information; and the deleting module is used for deleting the physical machine sequence to be deleted from the physical machine sequence in the partition information so as to obtain the first-stage physical machine sequence.
Further, the determination module is to: judging whether to create a virtual machine for the first time based on the position information; if the virtual machine is judged not to be created for the first time, acquiring the position information of the created virtual machine; and determining all physical machines positioned at the historical positions according to the historical positions contained in the position information of the created virtual machines, and taking a sequence formed by all the physical machines as the sequence of the physical machines to be deleted.
In the embodiment of the invention, the position information of the target service application is firstly obtained; then, determining a first-level physical machine sequence to be scheduled according to the position information of the target service application; next, determining a second-level physical machine sequence which meets the conditions through the use information of each physical machine in the first-level physical machine sequence; and finally, screening the physical machines in the second-level physical machine sequence to obtain a third-level physical machine sequence, and determining a first target physical machine in the third-level physical machine sequence, so as to deploy the virtual machine according to the first target physical machine. In the embodiment of the invention, the virtual machines are dynamically scheduled through the position information of the service application, so that the technical problem of single-machine bottleneck or bottleneck at the same position caused by scheduling to the same physical machine or scheduling to the physical machines at the same physical position when a plurality of virtual machines are established by a single service application is solved.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
In order to make the aforementioned and other objects, features and advantages of the present invention comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a flowchart of a method for dynamic scheduling of virtual machines according to an embodiment of the present invention;
FIG. 2 is a flow chart of another method for dynamic scheduling of virtual machines according to an embodiment of the invention;
fig. 3 is a schematic diagram of a dynamic scheduling apparatus of a virtual machine according to an embodiment of the present invention;
fig. 4 is a schematic diagram of another dynamic scheduling apparatus for a virtual machine according to an embodiment of the present invention.
Detailed Description
To make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
With the expansion of the scale of the business application, the access amount of the business application is larger and larger, and the resource of a single virtual machine cannot meet the requirement of the business application, so that the virtual machine resource of the business application needs to be expanded. This, in turn, results in more and more physical machine resources being consumed by the virtual machine. When the occupied virtual machine resources exceed a certain threshold, the service quality of the physical machine is increasingly poor. At this time, if a plurality of virtual machines of the same service application are deployed to the same physical machine, the response speed of the service application is greatly reduced. And as time goes on, the probability of the physical machine being broken is gradually increased, and if the physical machine is broken and cannot be maintained in time, service application cannot provide service. Also, power outages due to human factors can cause applications to fail to provide service.
For some very important business applications, the problem of the critical business applications caused by the unavailability of the same cabinet or the same row of cabinets due to power failure or other human factors is prevented. When a service application creates a plurality of virtual machines, the virtual machines can be created on physical machine nodes on different cabinets so as to relieve the pressure of the physical machines. The following describes in detail a method for dynamically scheduling a virtual machine according to an embodiment of the present invention with reference to the first embodiment and the second embodiment.
The first embodiment is as follows:
in accordance with an embodiment of the present invention, there is provided an embodiment of a method for dynamic scheduling of virtual machines, it should be noted that the steps illustrated in the flowchart of the accompanying drawings may be performed in a computer system such as a set of computer-executable instructions, and that while a logical order is illustrated in the flowchart, in some cases the steps illustrated or described may be performed in an order different than here.
Fig. 1 is a flowchart of a dynamic scheduling method of a virtual machine according to an embodiment of the present invention, as shown in fig. 1, the method includes the following steps:
step S102, position information of a target service application is obtained, wherein the target service application is a service application requesting to create a target virtual machine;
a virtual machine is a tightly isolated software container, and one or more virtual machines can be created on one physical machine. It can run its own operating system and applications as if it were a physical computer. The virtual machine behaves exactly like a physical computer, containing its own virtual (i.e., software-based) CPU, RAM, hard disk, and network card.
It should be noted that the business application may be a micro service, such as a scheduling service jobcenter and a gateway service paygate. The location information of the target service application may be location information of a certain server in the server cluster. Wherein a target business application can request the creation of at least one target virtual machine.
Step S104, determining a first-level physical machine sequence to be scheduled based on the position information, wherein the physical machines contained in the first-level physical machine sequence can provide a hardware environment for the target virtual machine;
in the embodiment of the invention, through the position information of the service application, which physical machines are already used for deploying the virtual machines can be determined, and which physical machines are not yet used for deploying the virtual machines can be determined. Through the setting mode, the reasonable scheduling of the physical machine can be realized, so that the problems that the physical machine runs slowly and the service quality of the physical machine is poor due to the fact that a certain service application creates a plurality of virtual machines are solved.
Step S106, determining a second-level physical machine sequence meeting the conditions through the use information of each physical machine in the first-level physical machine sequence;
in the embodiment of the invention, after the physical machine sequence is primarily screened through the position information of the service application to obtain the first-level physical machine sequence, the physical machines in the first-level physical machine sequence can be secondarily screened to obtain the second-level physical machine sequence, wherein the second-level physical machine sequence is the physical machine sequence with better performance in the first-level physical machine sequence.
And S108, screening the physical machines in the second-level physical machine sequence to obtain a third-level physical machine sequence, and determining a first target physical machine in the third-level physical machine sequence so as to deploy the virtual machine according to the first target physical machine, wherein the position information of any two physical machines in the third-level physical machine sequence is different.
In the embodiment of the invention, the position information of the target service application is firstly obtained; then, determining a first-level physical machine sequence to be scheduled according to the position information of the target service application; next, determining a second-level physical machine sequence which meets the conditions through the use information of each physical machine in the first-level physical machine sequence; and finally, screening the physical machines in the second-level physical machine sequence to obtain a third-level physical machine sequence, and determining a first target physical machine in the third-level physical machine sequence, so as to deploy the virtual machine according to the first target physical machine. In the embodiment of the invention, the virtual machines are dynamically scheduled through the position information of the service application, so that the technical problem of single-machine bottleneck or bottleneck at the same position caused by scheduling to the same physical machine or scheduling to the physical machines at the same physical position when a plurality of virtual machines are established by a single service application is solved.
In an optional implementation manner of the embodiment of the present invention, the step S104, namely, determining the first-level physical machine sequence to be scheduled based on the location information includes the following steps:
step S1041, determining a physical machine sequence to be deleted based on the position information, wherein the physical machine in the physical machine sequence to be deleted creates a physical machine of a virtual machine for the target service application;
step S1042, obtaining the partition information of the target virtual machine created by the target service application;
step S1043, acquiring a physical machine sequence in the partition information;
and step S1044, deleting the physical machine sequence to be deleted from the physical machine sequence in the partition information, thereby obtaining a first-level physical machine sequence.
Specifically, the requirement of a target virtual machine to be created by a target service application is determined, and a physical machine sequence to be excluded is obtained according to the position information of the target service application in the server cluster. And then, selecting information of an available area according to the created target service application, and further obtaining a physical machine sequence belonging to the available area, wherein the available area is partition information which can be used for creating a target virtual machine according to the target service application. Next, the sequence of physical machines to be excluded may be subtracted from the sequence of physical machines belonging to the partition information, so as to obtain the sequence of first-level physical machines, where the physical machines in the sequence of first-level physical machines have not been created by the target service application.
It should be noted that, in the embodiment of the present invention, a partition may be understood as a classified area of servers, for example, 100 servers, where 50 servers are high-performance servers, and then the 50 servers are classified into one partition, and another 50 servers are low-performance servers, and then the other 50 servers are classified into another partition.
In the embodiment of the present invention, after the physical machine sequence to be deleted is deleted from the physical machine sequence belonging to the partition information, the physical machine in the first-level physical machine sequence is the physical machine to which the virtual machine has not been deployed by the target service application. By the setting mode, the virtual machines can be prevented from being deployed in the same physical machine, so that the service quality of the physical machine is reduced.
In another optional implementation manner of the embodiment of the present invention, if the location information is location information of the target virtual machine, the step S1041 of determining, based on the location information, a physical machine sequence to be deleted includes the following steps:
step S11, determining whether to create a virtual machine for the first time based on the location information;
wherein determining whether to create the virtual machine for the first time based on the location information comprises: judging whether the position information is a null value, if so, judging that the virtual machine is established for the first time, and if not, judging that the virtual machine is not established for the first time;
step S12, if the virtual machine is not established for the first time, the position information of the established virtual machine is obtained;
step S13, determining all physical machines located at the historical positions according to the historical positions contained in the position information of the created virtual machines, and using the sequence formed by all the physical machines as the sequence of physical machines to be deleted.
In the embodiment of the present invention, if a virtual machine is created by a service application, the location information of the service application is the physical machine where the created virtual machine is located. If the business application has not created a virtual machine, then the location of the business application is null.
Specifically, first, whether a virtual machine (i.e., the target virtual machine) is created for the first time by the target service application is determined according to the location information of the target service application, wherein if it is determined that the virtual machine is created for the first time, the location information of the target service application is null, and then a sequence of physical machines to be excluded is null; if the virtual machine is not created for the first time, the location information of the target service application is not null, and at this time, the location information of the created virtual machine (that is, the physical machine where the created virtual machine is located) is queried. And then, obtaining a physical machine sequence to be excluded according to a historical position contained in the position information of the created virtual machine, wherein the process mainly comprises the step of judging that if the target service application creates a virtual machine at the historical position, the virtual machine does not need to be created again at the historical position.
It should be noted that, when the target service application creates the target virtual machine, the resource information of the target virtual machine that is needed is given, where the resource information may be expressed as the following expression: s ═ Ccpu,Mmem,Ddisk,Stag,Zzone]. Wherein, CcpuRepresenting CPU usage; mmemRepresenting the use condition of the memory; ddiskRepresenting the use condition of a magnetic disk; zzoneIndicating the available area condition; stagRepresenting location information of the target virtual machine. If the target service application is to create the target virtual machine for the first time, the location information of the target virtual machine is null (that is, the location information of the target service application is null), and if the target service application is not to create the target virtual machine for the first time, the location information of the created virtual machine is acquired from the system. And then, acquiring all physical machines in the same historical position according to the position information of the created virtual machine, thereby obtaining a physical machine sequence to be excluded.
It should be further noted that, if the number of the target virtual machines to be created is multiple, and at least one of the multiple target virtual machines has already been created by the target business application, at least one sequence of physical machines to be excluded is obtained. At this time, at least one physical machine sequence to be excluded needs to be deleted from the physical machine sequences belonging to the partition information, so as to obtain a first-level physical machine sequence.
It should be noted that, in the embodiment of the present invention, the methods described in the above steps S1041 to S1044 and the methods described in the above steps S11 to S13 may be implemented by using the following pseudo codes:
resource information S required when a target virtual machine is created; wherein, S ═ Ccpu,Mmem,Ddisk,Stag,Zzone]，CcpuTo representCPU resource status, MmemIndicating the memory resource situation, DdiskIndicating the condition of the disk, StagLocation information representing the target business application, ZzoneRepresenting available area information;
judging whether the target service application is deployed with the target virtual machine before according to the Stag information, and if so, determining a physical machine sequence except _ servers of a previous deployment position node (namely, the historical position):
exclude_servers＝exclude(Stag)；
loop checking 3 times, and 3 times all fail to stop creating virtual machines:
while nt<3；do
according to Z provided in the resource informationzoneInformation initializes the physical machine sequence serverall under the available area:
ServersAll＝zone[Server1,Server2,Server3…]
excluding previously deployed virtual machine sequences:
Servers＝ServerAll-exclude_server。
in another optional implementation manner of the embodiment of the present invention, the step S106 of determining, by using the usage information of each physical machine in the first-level physical machine sequence, a second-level physical machine sequence that satisfies the condition includes the following steps:
step S1061, obtaining CPU use information, memory use information and disk use information of each physical machine in the first-level physical machine sequence;
step S1062, determining the physical machine meeting the virtual machine creation condition in the first-level physical machine sequence based on the CPU usage information, the memory usage information and the disk usage information of each physical machine, and taking the physical machine meeting the virtual machine creation condition as the second-level physical machine sequence.
In the embodiment of the invention, firstly, the CPU use information C of each physical machine in the first-level physical machine sequence is obtainedcpuMemory usage information MmemDisk use information MmemAnd further, obtaining a second-level physical machine sequence according to the use information.
Specifically, each is judgedC of physical machinecpu，Mmem，DdiskWhether a virtual machine creating condition is met, wherein the virtual machine creating condition is as follows: c of physical machinecpuGreater than or equal to C in the resource information ScpuAnd M of the physical machinememGreater than or equal to M in the resource information SmemAnd D of the physical machinediskGreater than or equal to D in the resource information Sdisk。
For example, the target service application jobcenter needs to create 3 virtual machines, and the resource information of each virtual machine is: 2 cores of CPU, 4G memory and 80G hard disk. Then, when the physical machine is created in the virtual machine, the requirements of the 2-core CPU, the 4G memory, and the 80G hard disk are satisfied.
It should be noted that, when a target business application needs to create multiple virtual machines, the multiple virtual machines need to be created on different physical machine servers.
After the physical machines meeting the virtual machine creation condition are determined, the physical machines meeting the condition can be added into the Server _ List List, so as to obtain a second-level physical machine sequence.
As can be seen from the above description, in the embodiment of the present invention, the sequence of the physical machine to be deleted in the partition information is deleted, so as to obtain the physical machine that is not deployed with the target virtual machine, and further, the physical machine that meets the creation condition of the virtual machine is screened from the sequence of the first-stage physical machine, that is, the physical machine with better performance is screened from the sequence of the first-stage physical machine, so that the service quality of the physical machine is further ensured.
It should be noted that, in the embodiment of the present invention, the methods described in step S1061 and step S1062 may be implemented by using the following pseudo codes:
server _ List [ ]; initializing a scheduling physical machine sequence to be not null, wherein the initializing scheduling physical machine sequence is a first-level physical machine sequence;
for i in Servers; circularly traversing each physical machine node
Reading the CPU residual condition cpuLeft, the memory residual condition menLeft and the disk residual condition diskLeft of each physical machine node;
if cpuLeft<Scpu；
break；
if memLeft<Smem；
break；
if diskLeft<Sdisk；
break; if the CPU, the memory and the disk resource all accord with the creation condition of the virtual machine, adding the resources into a Server _ List queue;
Server_List.add(i)；
End for。
in another optional implementation manner of the embodiment of the present invention, in step S108, the screening the physical machines in the second-level physical machine sequence to obtain a third-level physical machine sequence includes the following steps:
step S1081, sorting the physical machines in the second-level physical machine sequence according to the position information;
step S1082, screening the sorted second-level physical machine sequence to obtain at least one second target physical machine, wherein the second target physical machine is a physical machine with the largest weight in the physical machines with the same position information;
step S1083, a third level physical machine sequence is determined based on the at least one second target physical machine.
In the embodiment of the present invention, first, the location information of each physical machine in the second-level physical machine sequence (i.e., the Server _ List) is obtained, and then, the physical machines in the Server _ List are sorted according to the location information, for example, the physical machines with the same location information are arranged in a queue.
And then, according to the position information of each physical machine, removing the physical machines with the same position information to obtain a third-level physical machine sequence. Specifically, in the process of removing the physical machines having the same location information, the physical machine having the largest weight among the physical machines having the same location information may be retained, and the third-level physical machine sequence may be formed based on the physical machine having the largest weight among the physical machines having the same location information (i.e., the second target physical machine described above). That is to say, in the third-level physical machine sequence, the position information of any two physical machines is different, and the physical machines in the third-level physical machine sequence are all the physical machines with the largest weight among the physical machines with the same position information in the second-level physical machine sequence.
It should be noted that the third-level physical machine sequence may be represented by a Server _ List _ New List.
In this embodiment of the present invention, in step S108, the determining the first target physical machine in the third-level physical machine sequence includes the following steps:
step S1084, selecting a physical machine with the largest weight value in the third-level physical machine sequence;
and taking the physical machine with the largest weight value as a first target physical machine.
And step S1085, for the physical machine sequence scheduled at the third level, according to the weight information, obtaining a physical machine node with the highest weight, creating a virtual machine on the physical machine node, and updating the service application deployment position information.
Specifically, after a third-level physical machine sequence (that is, a Server _ List _ New List) is obtained, the physical machines in the Server _ List _ New List may be sorted according to weight, and a physical machine with the largest weight is selected to create a target virtual machine, and the service application deployment virtual machine location information Ex is marked, where the largest weight may be represented as: max _ right ═ Sj}i＝Server_List_New。
It should be noted that, if the number of the largest physical machines in the Server _ List _ New List is multiple, one physical machine may be randomly selected from the multiple physical machines as the first target physical machine.
It should be noted that, in the embodiment of the present invention, the method described in step S1081 and step S1085 may be implemented by using the following pseudo code:
racks (); initializing physical machine location information sequences
for server in Server_List；
racks [ server. Adding physical machines with the same position information in a Server _ List into the same queue, wherein server.rack represents the position information of the physical machines;
endfor；
keys for j in racks; sorting the physical machines with the same position information according to the weight;
calculating the weight of racks [ j ], and adding the physical machine with the highest weight into a Server _ List _ New;
Endfor；
and calculating the physical machine node with the highest weight in the Server _ List _ New sequence to create a virtual machine and update the Stag position information.
In summary, the embodiment of the present invention provides a dynamic scheduling method for a virtual machine, where the scheduling method is a dynamic scheduling method for a virtual machine based on "service application location awareness". The specific process of the method is described as follows:
firstly, acquiring the requirement of a target service application for creating a target virtual machine; secondly, initializing the position information of the target service application by judging whether the target service application is a first-time created target virtual machine; and then a physical machine sequence to be excluded is obtained. And then, according to the partition information selected when the target service application is created, obtaining a physical machine sequence to which the partition information belongs, subtracting the physical machine sequence to be excluded from the physical machine sequence to obtain a first-stage physical machine sequence, and determining a second-stage physical machine sequence when the sequence is not empty.
And judging whether the first-level physical machine sequence meets the requirements or not according to CPU (Central processing Unit) use information, memory use information and disk use information, adding the first-level physical machine sequence into the second-level physical machine sequence meeting the requirements, and determining the third-level physical machine sequence when the sequence is not empty.
And in the second-stage physical machine sequence, removing the physical machines of the same node according to the position information of the physical machines to obtain a third-stage physical machine sequence, and entering a virtual machine deployment stage when the queue is not empty. And according to the third-level physical machine sequence, creating a target virtual machine by using the physical machine with the highest weight, and updating the position information of the service application deployment.
In the embodiment of the invention, the virtual machines are dynamically scheduled through the position information of the service application, so that the technical problem of single-machine bottleneck or bottleneck at the same position caused by scheduling to the same physical machine or scheduling to the physical machines at the same physical position when a plurality of virtual machines are established by a single service application is solved.
Fig. 2 is a flowchart of another dynamic scheduling method for a virtual machine according to an embodiment of the present invention, and as shown in fig. 2, the process of the method is described as follows:
acquiring position information of a target service application; judging whether to create a target virtual machine for the first time based on the position information; if yes, setting the physical machine sequence of the same position information to be null; otherwise, determining a physical machine with the same historical position as the target virtual machine; next, obtaining a physical machine sequence of the available region according to the available region information; deleting the physical machine with the same historical position as the target virtual machine in the available area physical machine sequence to obtain a first-stage physical machine sequence; then, judging whether the resources of the physical machines in the first-level physical machine sequence meet the virtual machine creation condition, wherein if the resources meet the virtual machine creation condition, determining a second-level physical machine sequence based on the physical machines meeting the virtual machine creation condition in the first-level physical machine sequence; otherwise, ending the flow. Next, sorting the physical machines with the same position information in the second-level physical machine sequence according to the weight; aiming at the physical machines with the same position information in the second-level physical machine sequence, taking the physical machine with the maximum weight, and constructing a third-level physical machine sequence based on the physical machine with the maximum weight; and finally, in the third-level physical machine sequence, according to the weight information, obtaining the physical machine with the maximum weight as a first target physical machine, deploying a target virtual machine in the first target physical machine, and updating the position information of the target service application.
Example two:
the embodiment of the present invention further provides a dynamic scheduling apparatus for a virtual machine, where the dynamic scheduling apparatus for a virtual machine is mainly used to execute the dynamic scheduling method for a virtual machine provided in the foregoing content of the embodiment of the present invention, and the following description specifically describes the dynamic scheduling apparatus for a virtual machine provided in the embodiment of the present invention.
Fig. 3 is a schematic diagram of a dynamic scheduling apparatus of a virtual machine according to an embodiment of the present invention, and as shown in fig. 3, the dynamic scheduling apparatus of the virtual machine mainly includes: an obtaining unit 31, a first determining unit 32, a second determining unit 33 and a screening unit 34, wherein:
an obtaining unit 31, configured to obtain location information of a target service application, where the target service application is a service application requesting to create a target virtual machine;
a first determining unit 32, configured to determine, based on the location information, a first-level physical machine sequence to be scheduled, where a physical machine included in the first-level physical machine sequence is capable of providing a hardware environment for a target virtual machine;
a second determining unit 33, configured to determine, through the usage information of each physical machine in the first-level physical machine sequence, a second-level physical machine sequence that satisfies the condition;
and the screening unit 34 is configured to screen the physical machines in the second-level physical machine sequence to obtain a third-level physical machine sequence, and determine a first target physical machine in the third-level physical machine sequence, so as to deploy the virtual machine according to the first target physical machine, where location information of any two physical machines in the third-level physical machine sequence is different.
Optionally, the first determination unit includes: the determining module is used for determining a physical machine sequence to be deleted based on the position information, wherein the physical machine in the physical machine sequence to be deleted creates a physical machine of a virtual machine for the target service application; the first acquisition module is used for acquiring the partition information of the target virtual machine established by the target service application; the second acquisition module is used for acquiring the physical machine sequence in the partition information; and the deleting module is used for deleting the physical machine sequence to be deleted in the physical machine sequence in the partition information so as to obtain a first-stage physical machine sequence.
Optionally, the determining module is configured to: judging whether to create a virtual machine for the first time based on the position information; if the virtual machine is judged not to be created for the first time, acquiring the position information of the created virtual machine; and determining all physical machines positioned at the historical positions according to the historical positions contained in the position information of the created virtual machines, and taking a sequence formed by all the physical machines as a physical machine sequence to be deleted.
Optionally, the determining module is further configured to: judging whether the position information is a null value; if the position information is judged to be a null value, the virtual machine is judged to be established for the first time, and if the position information is judged not to be the null value, the virtual machine is judged not to be established for the first time.
Optionally, the second determining unit is configured to: acquiring CPU use information, memory use information and disk use information of each physical machine in a first-stage physical machine sequence; and determining the physical machine meeting the virtual machine creation condition in the first-level physical machine sequence based on the CPU use information, the memory use information and the disk use information of each physical machine, and taking the physical machine meeting the virtual machine creation condition as a second-level physical machine sequence.
Optionally, the screening unit is configured to: screening the physical machines in the second-level physical machine sequence to obtain a third-level physical machine sequence, wherein the third-level physical machine sequence comprises the following steps: sequencing the physical machines in the second-level physical machine sequence according to the position information; screening the sorted second-level physical machine sequence to obtain at least one second target physical machine, wherein the second target physical machine is the physical machine with the largest weight in the physical machines with the same position information; a third level physical machine sequence is determined based on the at least one second target physical machine.
Optionally, the screening unit is further configured to: determining the first target physical machine in the third level physical machine sequence comprises: selecting a physical machine with the largest weight value in the third-level physical machine sequence; and taking the physical machine with the largest weight value as a first target physical machine.
Fig. 4 is a schematic diagram of another dynamic scheduling apparatus of a virtual machine according to an embodiment of the present invention, and as shown in fig. 4, the dynamic scheduling apparatus of the virtual machine mainly includes: a service application location awareness module 41, a first-level scheduling module 42, a second-level scheduling module 43, a third-level scheduling module 44 and an establishment module 45.
The service application location sensing module is mainly used for acquiring and setting location information of service application according to service application information to obtain a physical machine sequence of the service application location information (namely, the physical machine sequence to be deleted).
And the first-stage scheduling module is used for obtaining a physical machine sequence under the partition according to the partition information of the virtual machine to be created, and then subtracting the physical machine sequence obtained in the service application sensing module to obtain a first-stage physical machine sequence.
And the second-level scheduling module is used for judging whether the CPU use information, the memory use information and the disk use information of each physical machine in the first-level physical machine sequence meet the virtual machine creation condition or not aiming at the first-level physical machine sequence obtained by the first-level scheduling module, adding the physical machine nodes meeting the virtual machine creation condition into the second-level physical machine sequence, and finally obtaining the second-level physical machine sequence.
And the third-stage scheduling module is used for sequencing the second-stage physical machine sequence obtained by the second-stage scheduling module according to the position information of the physical machine, and subtracting the physical machine nodes at the same position to obtain a third-stage physical machine sequence.
And the establishing module is used for obtaining a physical machine node (namely, a first target physical machine) with the highest weight in the third-level physical machine sequence obtained by the third-level scheduling module according to the weight information of the physical machine, establishing a virtual machine according to the first target physical machine, and updating the position information of the service application.
In addition, in the description of the embodiments of the present invention, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art.
In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, and are only for convenience of description and simplicity of description, but do not indicate or imply that the device or element being referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus, should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
The computer program product for performing the method and the apparatus for dynamically scheduling a virtual machine according to the embodiments of the present invention includes a computer-readable storage medium storing a non-volatile program code executable by a processor, where instructions included in the program code may be used to execute the method described in the foregoing method embodiments, and specific implementation may refer to the method embodiments, and will not be described herein again.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present invention, which are used for illustrating the technical solutions of the present invention and not for limiting the same, and the protection scope of the present invention is not limited thereto, although the present invention is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present invention, and they should be construed as being included therein. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (8)
1. A dynamic scheduling method of a virtual machine is characterized by comprising the following steps:
acquiring position information of a target service application, wherein the target service application is a service application requesting to create a target virtual machine;
determining a first-level physical machine sequence to be scheduled based on the position information, wherein physical machines contained in the first-level physical machine sequence can provide a hardware environment for the target virtual machine;
determining a second-level physical machine sequence meeting the conditions through the use information of each physical machine in the first-level physical machine sequence;
screening the physical machines in the second-level physical machine sequence to obtain a third-level physical machine sequence, and determining a first target physical machine in the third-level physical machine sequence so as to deploy the target virtual machine according to the first target physical machine, wherein the position information of any two physical machines in the third-level physical machine sequence is different;
wherein determining a first-level physical machine sequence to be scheduled based on the location information comprises:
determining a physical machine sequence to be deleted based on the position information, wherein the physical machine in the physical machine sequence to be deleted is a physical machine which is created with a virtual machine by the target service application;
acquiring partition information of the target virtual machine created by the target service application;
acquiring a physical machine sequence belonging to the partition information;
and deleting the physical machine sequence to be deleted in the physical machine sequence in the partition information, thereby obtaining the first-stage physical machine sequence.
2. The method of claim 1, wherein determining the sequence of physical machines to delete based on the location information comprises:
judging whether to create a virtual machine for the first time based on the position information;
if the virtual machine is judged not to be created for the first time, acquiring the position information of the created virtual machine;
and determining all physical machines positioned at the historical positions according to the historical positions contained in the position information of the created virtual machines, and taking a sequence formed by all the physical machines as the sequence of the physical machines to be deleted.
3. The method of claim 2, wherein determining whether to create a virtual machine for the first time based on the location information comprises:
judging whether the position information is a null value;
if the position information is judged to be a null value, the virtual machine is judged to be established for the first time, and if the position information is judged not to be the null value, the virtual machine is judged not to be established for the first time.
4. The method of claim 1, wherein determining a second sequence of level physical machines that satisfies a condition from usage information of each physical machine in the first sequence of level physical machines comprises:
acquiring CPU use information, memory use information and disk use information of each physical machine in the first-stage physical machine sequence;
determining a physical machine meeting a virtual machine creation condition in the first-level physical machine sequence based on the CPU usage information, the memory usage information and the disk usage information of each physical machine, and taking the physical machine meeting the virtual machine creation condition as the second-level physical machine sequence.
5. The method of claim 1, wherein screening physical machines in the second-level sequence of physical machines to obtain a third-level sequence of physical machines comprises:
sequencing the physical machines in the second-level physical machine sequence according to the position information;
screening the sorted second-level physical machine sequence to obtain at least one second target physical machine, wherein the second target physical machine is a physical machine with the largest weight in physical machines with the same position information;
determining the third level physical machine sequence based on the at least one second target physical machine.
6. The method of claim 5, wherein determining a first target physical machine in the sequence of third-level physical machines comprises:
selecting a physical machine with the largest weight value in the third-level physical machine sequence;
and taking the physical machine with the maximum weight value as the first target physical machine.
7. A dynamic scheduling apparatus for a virtual machine, comprising:
the system comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring the position information of a target service application, and the target service application is the service application requesting to create a target virtual machine;
a first determining unit, configured to determine a first-level physical machine sequence to be scheduled based on the location information, where a physical machine included in the first-level physical machine sequence is capable of providing a hardware environment for the target virtual machine;
a second determining unit, configured to determine, through the usage information of each physical machine in the first-level physical machine sequence, a second-level physical machine sequence that satisfies a condition;
the screening unit is used for screening the physical machines in the second-level physical machine sequence to obtain a third-level physical machine sequence, and determining a first target physical machine in the third-level physical machine sequence so as to deploy the target virtual machine according to the first target physical machine, wherein the position information of any two physical machines in the third-level physical machine sequence is different;
wherein the first determination unit includes:
a determining module, configured to determine a sequence of physical machines to be deleted based on the location information, where a physical machine in the sequence of physical machines to be deleted is a physical machine for which a virtual machine has been created by the target service application;
the first acquisition module is used for acquiring the partition information of the target virtual machine created by the target service application;
the second acquisition module is used for acquiring the physical machine sequence belonging to the partition information;
and the deleting module is used for deleting the physical machine sequence to be deleted from the physical machine sequence in the partition information so as to obtain the first-stage physical machine sequence.
8. The apparatus of claim 7, wherein the determining module is configured to:
judging whether to create a virtual machine for the first time based on the position information;
if the virtual machine is judged not to be created for the first time, acquiring the position information of the created virtual machine;
and determining all physical machines positioned at the historical positions according to the historical positions contained in the position information of the created virtual machines, and taking a sequence formed by all the physical machines as the sequence of the physical machines to be deleted.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710705796.4A CN107368355B (en) | 2017-08-17 | 2017-08-17 | Dynamic scheduling method and device of virtual machine |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710705796.4A CN107368355B (en) | 2017-08-17 | 2017-08-17 | Dynamic scheduling method and device of virtual machine |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107368355A CN107368355A (en) | 2017-11-21 |
| CN107368355B true CN107368355B (en) | 2020-05-15 |
Family
ID=60310702
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710705796.4A Active CN107368355B (en) | 2017-08-17 | 2017-08-17 | Dynamic scheduling method and device of virtual machine |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107368355B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11537424B2 (en) * | 2019-05-10 | 2022-12-27 | Kontain Inc. | Scalable and secure containers |
| CN113900775B (en) * | 2021-12-08 | 2022-08-02 | 北京交研智慧科技有限公司 | Method and device for processing edge-side cloud computing cross-node virtual machine |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102609295A (en) * | 2011-10-18 | 2012-07-25 | 华中科技大学 | Dynamic operation scheduling system of virtual machine |
| CN102929689A (en) * | 2012-11-01 | 2013-02-13 | 曙光云计算技术有限公司 | Method for building virtual machine in cloud computing system |
| CN104793982A (en) * | 2014-01-20 | 2015-07-22 | 联想(北京)有限公司 | Method and device for establishing virtual machine |
| CN106020937A (en) * | 2016-07-07 | 2016-10-12 | 腾讯科技（深圳）有限公司 | Method, device and system for creating virtual machine |
-
2017
- 2017-08-17 CN CN201710705796.4A patent/CN107368355B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102609295A (en) * | 2011-10-18 | 2012-07-25 | 华中科技大学 | Dynamic operation scheduling system of virtual machine |
| CN102929689A (en) * | 2012-11-01 | 2013-02-13 | 曙光云计算技术有限公司 | Method for building virtual machine in cloud computing system |
| CN104793982A (en) * | 2014-01-20 | 2015-07-22 | 联想(北京)有限公司 | Method and device for establishing virtual machine |
| CN106020937A (en) * | 2016-07-07 | 2016-10-12 | 腾讯科技（深圳）有限公司 | Method, device and system for creating virtual machine |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107368355A (en) | 2017-11-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108009016B (en) | Resource load balancing control method and cluster scheduler | |
| CN112506444A (en) | Kubernetes cluster-based expansion and contraction capacity control method and device and electronic equipment | |
| KR101600129B1 (en) | Application efficiency engine | |
| JP4293011B2 (en) | Server determination method in case of failure | |
| CN111880936B (en) | Resource scheduling method, device, container cluster, computer equipment and storage medium | |
| CN108132837B (en) | Distributed cluster scheduling system and method | |
| CN101964820A (en) | Method and system for keeping data consistency | |
| CN111381928B (en) | Virtual machine migration method, cloud computing management platform and storage medium | |
| CN112579692B (en) | Data synchronization method, device, system, equipment and storage medium | |
| CN111966449B (en) | Virtual machine backup management method, system, terminal and storage medium | |
| CN108009004B (en) | Docker-based method for realizing measurement and monitoring of availability of service application | |
| CN107368355B (en) | Dynamic scheduling method and device of virtual machine | |
| WO2021262332A1 (en) | Rightsizing virtual machine deployments in a cloud computing environment | |
| CN111385359A (en) | Load processing method and device of object gateway | |
| CN107426012B (en) | Fault recovery method and device based on super-fusion architecture | |
| CN111352726A (en) | Streaming data processing method and device based on containerized micro-service | |
| JP6722345B2 (en) | Sign detection device and sign detection method | |
| CN112631680A (en) | Micro-service container scheduling system, method, device and computer equipment | |
| CN117369941A (en) | Pod scheduling method and system | |
| CN108604231A (en) | Mirror processing method and computing device | |
| CN114697213A (en) | Upgrading method and device | |
| CN114564314A (en) | Method and device for collecting monitoring data in cluster | |
| CN111143073B (en) | Virtualized resource management method, device and storage medium | |
| CN113672665A (en) | Data processing method, data acquisition system, electronic device and storage medium | |
| CN113157476A (en) | Processing method and device for display card fault in virtual cloud environment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20221103 Address after: 037, Floor 3, Building 1, No. 88, Nongda South Road, Haidian District, Beijing 100,089 Patentee after: Yunting Zhitong (Beijing) Information Technology Co.,Ltd. Address before: Room 301, floor 3, building 1, yard 88, Nongda South Road, Haidian District, Beijing 100084 Patentee before: BEIJING YUNZONG INFORMATION TECHNOLOGY CO.,LTD. |