CN111638940A - Container heat transfer method for Shenwei platform - Google Patents
Container heat transfer method for Shenwei platform Download PDFInfo
- Publication number
- CN111638940A CN111638940A CN202010424330.9A CN202010424330A CN111638940A CN 111638940 A CN111638940 A CN 111638940A CN 202010424330 A CN202010424330 A CN 202010424330A CN 111638940 A CN111638940 A CN 111638940A
- Authority
- CN
- China
- Prior art keywords
- container
- data
- migrated
- server
- recovering
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 104
- 238000012546 transfer Methods 0.000 title claims abstract description 4
- 238000013508 migration Methods 0.000 claims abstract description 20
- 230000005012 migration Effects 0.000 claims abstract description 20
- 230000003071 parasitic effect Effects 0.000 claims abstract description 12
- 230000008014 freezing Effects 0.000 claims abstract description 10
- 238000007710 freezing Methods 0.000 claims abstract description 10
- 230000005540 biological transmission Effects 0.000 claims description 21
- 238000011112 process operation Methods 0.000 claims description 4
- 125000004122 cyclic group Chemical group 0.000 claims description 3
- 238000013507 mapping Methods 0.000 claims description 3
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 230000008447 perception Effects 0.000 abstract description 2
- 238000004904 shortening Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000010354 integration Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Stored Programmes (AREA)
Abstract
The invention discloses a container heat transfer method facing a Shenwei platform, which comprises the following steps: defining a unique identification CID for each container, acquiring basic information of a subprocess and a thread of the container to be migrated from a system file of the container to be migrated according to the CID number, freezing the container process of the container to be migrated, then inserting a compiled binary parasitic code into the container process of the container to be migrated, acquiring process data of the container process by using a service provided by the binary parasitic code, finally unfreezing the container process of the container to be migrated, serializing the acquired process data of the container process, and transmitting the serialized process data to a temporary file of a destination server. The invention can realize the program interruption with minimum cost, is beneficial to shortening the migration time and ensures that the service can finish the rapid migration of the container under the condition of no perception.
Description
Technical Field
The invention relates to a container thermal migration method for a Shenwei platform, and belongs to the technical field of container process migration.
Background
The states of various objects such as hardware and software of the data center are extremely complex, the operating environment of a single container cannot be kept stable all the time, and various temporary emergencies can be encountered. The conditions of node operating system upgrading, security patching, hardware replacement, machine room power-off maintenance, single-node resource tightening and the like can cause that the container can not continuously run in the original environment.
For the container live migration technology, the traditional method is only to migrate the container from the source server to the destination server, so as to ensure the integrity and the security of data, but the problem of long container process freezing time exists. In order to ensure the robustness of container application, improve the overall resource integration capability and management efficiency of a platform, study the hot migration technology in a large-scale container network environment, enable the container to perform 'invisible' transmission of states such as mirror images, application data, network connection and the like among nodes, enable services to complete the rapid migration of the container under the condition of no perception, and become the research direction of technicians in the field.
Disclosure of Invention
The invention aims to provide a container thermal migration method for a Shenwei platform, which can realize program interruption with minimum cost, is beneficial to shortening migration time and enables services to complete rapid migration of containers under an imperceptible condition.
In order to achieve the purpose, the invention adopts the technical scheme that: a container thermal migration method facing a Shenwei platform comprises the following steps:
defining a unique identification CID for each container;
acquiring a CID number of a container to be migrated on a source server, and acquiring basic information of a subprocess and a thread of the container to be migrated from a system file of the container to be migrated according to the CID number;
freezing the container process of the container to be migrated, then inserting the compiled binary parasitic code into the container process of the container to be migrated, acquiring process data of the container process by using a service provided by the binary parasitic code, and finally unfreezing the container process of the container to be migrated;
serializing the acquired process data of the container process, and transmitting the serialized process data to a temporary file of a destination server;
in the data transmission process, firstly, transmitting the process data acquired when the first container is frozen to a destination server, and simultaneously keeping the container on the source server to be continuously executed;
after the first transmission is finished, continuing to perform second-time data acquisition on the container, subtracting the process data acquired for the second time from the process data acquired when the first container is frozen to acquire incremental data, and transmitting the incremental data to a target server;
if the data volume of the obtained incremental data is larger than the set threshold value, repeating the steps to carry out data iterative transmission until the data volume of the incremental data is smaller than the set threshold value or the maximum iteration number upper limit is reached;
after multiple times of cyclic iterative transmission, the data of the source server and the data of the destination server are basically consistent;
freezing the container process of the container to be migrated on the source server, transmitting all the remaining data to the destination server, and ending the container process of the source server;
and restoring the container process of the destination server.
The further improved scheme in the technical scheme is as follows:
in the foregoing solution, the specific step of recovering the container process of the destination server includes:
reading the stored temporary file and analyzing the shared resources, and recovering the resources shared by the processes preferentially, wherein other resources are recovered when needed later;
calling an interface for establishing a process and a thread for the system for multiple times to establish the process and restore the whole process tree;
recovering all basic task resources except memory mapping, timers, certificates and threads;
and recovering the context environment of the process operation, recovering the rest other resources, and continuing to operate the process.
Due to the application of the technical scheme, compared with the prior art, the invention has the following advantages:
the invention relates to a container thermal migration method facing a Shenwei platform, wherein a data transmission part adopts incremental iterative transmission, so that a container service is frozen for a short time without being perceived by a user, and finally, a frozen container process transmits a small amount of residual data to a target server, so that the program interruption with minimum cost can be realized, the migration time can be shortened, and the quick migration of a container can be completed under the condition that a service is not perceived.
Drawings
FIG. 1 is a schematic flow diagram of the present invention;
fig. 2 is a schematic diagram of data transmission in the present invention.
Detailed Description
Example (b): a container thermal migration method facing a Shenwei platform comprises the following steps:
defining a unique identification CID for each container;
acquiring a CID number of a container to be migrated on a source server, and acquiring basic information of a subprocess and a thread of the container to be migrated from a system file/proc/of the container to be migrated according to the CID number, namely acquiring resources of the process, such as a CPU (central processing unit), a memory, a file descriptor and a disk of each process;
freezing a container process of a container to be migrated, inserting a compiled binary parasitic code into the container process of the container to be migrated, acquiring process data of the container process by using a service provided by the binary parasitic code, and finally unfreezing the container process of the container to be migrated, wherein a Linux system provides an interface for injecting binary codes into another process, and the injected codes are called parasitic codes;
serializing the acquired process data of the container process, and transmitting the serialized process data to a temporary file of a destination server;
in the data transmission process, firstly, transmitting the process data acquired when the 'first container is frozen' to a destination server, wherein the data volume is large, the transmission needs to consume much time, and meanwhile, the containers on the source server are kept to be continuously executed;
after the first transmission is finished, continuously performing second-time data acquisition on the container, subtracting the process data acquired for the second time from the process data acquired when the first container is frozen to acquire incremental data, wherein the incremental data has small data volume and short transmission time, and then transmitting the incremental data to a target server;
if the data amount of the obtained incremental data is larger than the set threshold, the threshold and the upper limit are empirical values related to the environment: for example, if the network is good and the performance of the CPU is good, if the threshold value is possibly larger, the above steps are repeated to perform data iterative transmission until the data amount of the incremental data is smaller than the set threshold value or the maximum iteration number upper limit is reached;
after multiple times of cyclic iterative transmission, the data of the source server and the data of the destination server are basically consistent;
freezing the container process of the container to be migrated on the source server, transmitting all the remaining data to the destination server, and ending the container process of the source server;
and restoring the container process of the destination server.
The specific steps of recovering the container process of the destination server include:
reading the stored temporary file and analyzing the shared resources, and recovering the resources shared by the processes preferentially, wherein other resources are recovered when needed later;
calling an interface for establishing a process and a thread for the system for multiple times to establish the process and restore the whole process tree, wherein the thread is not restored at the moment;
recovering all basic tasks including process and thread resources except memory mapping, timers, certificates and threads, wherein the step mainly comprises opening a file, preparing namespace, establishing socket connection and the like;
and recovering the context environment of the process operation, recovering the rest other resources, and continuing to operate the process.
The examples are further explained below:
each container process has a unique identification CID, the CID number of a container to be migrated on a source server A is firstly obtained, then basic information such as a subprocess, a thread and the like is obtained from a system file/proc/for example, the CID number of the container to be migrated is 2342, and the subprocess and the thread of the container are obtained, including information such as a CPU, a memory, a disk and the like of each process;
in the container pre-migration stage, firstly freezing a container process, then inserting the compiled binary parasitic code into the process of the container with the CID number of 2342, acquiring private data of the container process by using a service provided by the parasitic code, and finally unfreezing the container process;
and the incremental iteration data is transmitted to a target server B, the acquired process data of the container with the CID number of 2342 is serialized according to the service provided by the parasitic code, and the process data is transmitted to a temporary file of the target server B. Repeating iterative transmission for multiple times until the data of the source server A and the data of the destination server B are basically consistent;
freezing the container (container CID is 2342) process on the source server A, transmitting the rest data to the destination server B, and ending the container (container CID is 2342) process of the source server A;
the container process on destination server B is resumed. The specific method for recovering the container process comprises the following steps:
(1) reading the stored temporary file and analyzing the shared resource, and recovering the resource shared by the processes preferentially, and recovering other resources when needed.
(2) Fork () is then called multiple times to create a process-restore whole process tree, at which point the thread is not restored.
(3) All basic task (including process and thread) resources are then restored, except for memory maps, timers, certificates, and threads. This step is mainly to open the file, prepare namespace, create socket connections, etc.
(4) And finally, restoring the context environment of the process operation, restoring the rest other resources and continuing to operate the process.
When the container thermal migration method facing the Shenwei platform is adopted, the data transmission part adopts incremental iterative transmission, so that the container service is frozen for a short time without being perceived by a user, and finally, the frozen container process transmits the residual small amount of data to the destination server, so that the program interruption with minimum cost can be realized, the migration time can be shortened, and the quick migration of the container can be completed under the condition that the service is not perceived.
The above embodiments are merely illustrative of the technical ideas and features of the present invention, and the purpose thereof is to enable those skilled in the art to understand the contents of the present invention and implement the present invention, and not to limit the protection scope of the present invention. All equivalent changes and modifications made according to the spirit of the present invention should be covered within the protection scope of the present invention.
Claims (2)
1. A container heat transfer method facing a Shenwei platform is characterized by comprising the following steps:
defining a unique identification CID for each container;
acquiring a CID number of a container to be migrated on a source server, and acquiring basic information of a subprocess and a thread of the container to be migrated from a system file of the container to be migrated according to the CID number;
freezing the container process of the container to be migrated, then inserting the compiled binary parasitic code into the container process of the container to be migrated, acquiring process data of the container process by using a service provided by the binary parasitic code, and finally unfreezing the container process of the container to be migrated;
serializing the acquired process data of the container process, and transmitting the serialized process data to a temporary file of a destination server;
in the data transmission process, firstly, transmitting the process data acquired when the first container is frozen to a destination server, and simultaneously keeping the container on the source server to be continuously executed;
after the first transmission is finished, continuing to perform second-time data acquisition on the container, subtracting the process data acquired for the second time from the process data acquired when the first container is frozen to acquire incremental data, and transmitting the incremental data to a target server;
if the data volume of the obtained incremental data is larger than the set threshold value, repeating the steps to carry out data iterative transmission until the data volume of the incremental data is smaller than the set threshold value or the maximum iteration number upper limit is reached;
after multiple times of cyclic iterative transmission, the data of the source server and the data of the destination server are basically consistent;
freezing the container process of the container to be migrated on the source server, transmitting all the remaining data to the destination server, and ending the container process of the source server;
and restoring the container process of the destination server.
2. The Shenwei platform-oriented incremental container live migration technology according to claim 1, wherein the specific step of recovering the container process of the destination server comprises:
reading the stored temporary file and analyzing the shared resources, and recovering the resources shared by the processes preferentially, wherein other resources are recovered when needed later;
calling an interface for establishing a process and a thread for the system for multiple times to establish the process and restore the whole process tree;
recovering all basic task resources except memory mapping, timers, certificates and threads;
and recovering the context environment of the process operation, recovering the rest other resources, and continuing to operate the process.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010424330.9A CN111638940A (en) | 2020-05-19 | 2020-05-19 | Container heat transfer method for Shenwei platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010424330.9A CN111638940A (en) | 2020-05-19 | 2020-05-19 | Container heat transfer method for Shenwei platform |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111638940A true CN111638940A (en) | 2020-09-08 |
Family
ID=72328491
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010424330.9A Pending CN111638940A (en) | 2020-05-19 | 2020-05-19 | Container heat transfer method for Shenwei platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111638940A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103577245A (en) * | 2013-10-29 | 2014-02-12 | 中国科学院计算技术研究所 | Lightweight class virtual machine migration method |
| CN108132949A (en) * | 2016-12-01 | 2018-06-08 | 腾讯科技（深圳）有限公司 | The method and device of Data Migration in data-base cluster |
| US20180232174A1 (en) * | 2017-02-15 | 2018-08-16 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Data Migration Between Cloud Storage Systems |
| CN108810071A (en) * | 2018-03-30 | 2018-11-13 | 广东睿江云计算股份有限公司 | A method of the Apache Server cluster load balance based on process migration |
-
2020
- 2020-05-19 CN CN202010424330.9A patent/CN111638940A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103577245A (en) * | 2013-10-29 | 2014-02-12 | 中国科学院计算技术研究所 | Lightweight class virtual machine migration method |
| CN108132949A (en) * | 2016-12-01 | 2018-06-08 | 腾讯科技（深圳）有限公司 | The method and device of Data Migration in data-base cluster |
| US20180232174A1 (en) * | 2017-02-15 | 2018-08-16 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Data Migration Between Cloud Storage Systems |
| CN108810071A (en) * | 2018-03-30 | 2018-11-13 | 广东睿江云计算股份有限公司 | A method of the Apache Server cluster load balance based on process migration |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105740418A (en) | File monitoring and message pushing based real-time synchronization system | |
| CN112422519B (en) | Electric power internet of things IP fusion terminal and communication method thereof | |
| CN113177034B (en) | Cross-platform unified distributed graph data processing method | |
| CN103023805A (en) | MapReduce system | |
| CN105656947B (en) | A kind of methods, devices and systems that transaction middleware is interacted with third party system | |
| CN103577245B (en) | Lightweight class virtual machine migration method | |
| CN104683472A (en) | Data transmission method capable of supporting large data volume | |
| CN113900810A (en) | Distributed graph processing method, system and storage medium | |
| CN112000649B (en) | Method and device for synchronizing incremental data based on map reduce | |
| CN114510344A (en) | Space-based resource virtualization management and control system based on micro-service agent | |
| CN108123826B (en) | Cross-region data interaction system and method | |
| CN101383814A (en) | Device and method implementing data access based on connection pool | |
| CN111638939B (en) | Management system and method for Kubernets container platform application life cycle | |
| CN107844566B (en) | Dump control method and system | |
| CN111638940A (en) | Container heat transfer method for Shenwei platform | |
| CN113721856A (en) | Digital community management data storage system | |
| CN109766131A (en) | The system and method for the intelligent automatic upgrading of software is realized based on multithreading | |
| CN113297218B (en) | Multi-system data interaction method, device and system | |
| CN115686497A (en) | Business development data management method, development engine, electronic device, and storage medium | |
| CN116185988A (en) | Data modular transfer method based on Spark technology, server and storage medium | |
| CN109271354A (en) | A kind of journal file handle update method and system | |
| CN109144892A (en) | A kind of buffering linked list data structure design method of managing internal memory medium-high frequency delta data | |
| CN109542841A (en) | The method and terminal device of data snapshot are created in cluster | |
| CN112083914B (en) | Method and system for realizing object model embedded operation system soft bus | |
| CN110868461B (en) | Data distribution method facing heterogeneous bandwidth between nodes in Gaia cluster |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200908 |
|
| RJ01 | Rejection of invention patent application after publication |