CN105786585B - The system and method for online migration internal storage data - Google Patents
The system and method for online migration internal storage data Download PDFInfo
- Publication number
- CN105786585B CN105786585B CN201410767628.4A CN201410767628A CN105786585B CN 105786585 B CN105786585 B CN 105786585B CN 201410767628 A CN201410767628 A CN 201410767628A CN 105786585 B CN105786585 B CN 105786585B
- Authority
- CN
- China
- Prior art keywords
- stream
- page
- data
- internal storage
- storage data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses the system and method for online migration internal storage data.It include: to send Stream to flow by page number, page data whole pages when migrating beginning, and generate page number, the global table of number, the corresponding number of page number is 1；Containing dirty pages are sent Stream to flow, and update global table by the memory pages that monitoring source host is modified in real time, and the number of corresponding containing dirty pages number adds 1；Page number and corresponding page data are obtained from Stream stream, checks that the corresponding number of page number is then abandoned this page data and do not handled, subtract 1 for number if it is greater than 1 in global table；If being equal to 1,0 is set by number, and compression processing is carried out to this page data；It discriminates whether to reach shutdown threshold condition, if it is, stopping the update of source host internal storage data, carries out internal storage data migration.The present invention can reduce the number of transmissions of source host memory containing dirty pages, improve internal storage data compression efficiency, accelerate pre-copy loop convergence speed.
Description
Technical field
The present invention relates to field of cloud calculation, it is related specifically to reduce internal storage data repeating transmission by stream calculation technology, in raising
Deposit data compression efficiency, the system and method for realizing online migration internal storage data.
Background technique
Virtual machine (VM) migrates (online migration) online, also known as real-time migration (live migration), is
Refer to guarantee virtual in-fight service operate normally while, virtual machine is migrated between different physical hosts, in order to
Virtual machine service is available in transition process, and transition process only has very of short duration downtime.
The online Migration tools of mainstream require greatly between physical machine using SAN (storage area at present
Network), the centralization of NAS (network-attached storage) etc shares external equipment, thus only in migration
Need to consider that operating system memory executes the migration of state, to obtain preferable migration performance.
There are mainly three types of the strategies of the basic virutal machine memory migration of domestic and international mainstream: being based on virtual machine system logout
With the method for playback, the method that the method for (post-copy) is copied after being based on and is based on pre-copy (pre-copy).
It is not mature enough relative to the method based on virtual machine system logout and playback and rear copy method is shut down
The problem of time length, the virtual machine monitor XEN and KVM of mainstream use pre-copy technology, i.e. source host VM first at present
Wheel sends all memory page datas to destination host VM, and following each round is write during cycling through last round of pre-copy by VM
The containing dirty pages memory crossed hangs up source host, no longer progress memory refreshing, last wheel containing dirty pages is transmitted to until reaching preset threshold value
Destination host VM completes internal memory migration.To reduce network transmission data amount, also introduced on the basis of basic pre-copy method interior
Deposit the technologies such as the dirty probabilistic forecasting of the page, internal storage data compression.
Although pre-copy technology can reduce downtime, but if source host VM memory refreshing is frequent, will lead to circulation
Number increases, and overall transit time is elongated.
Summary of the invention
The technical problem to be solved by the present invention is to memory frequent updating, to cause traditional pre-copy iterative cycles to be difficult to convergent
Problem.
According to an aspect of the present invention, the system of online migration internal storage data is proposed, comprising:
Flow data acquisition unit, for sending Stream to flow by page number, page data whole pages when migrating beginning, and
Page number, the global table of number are generated, the corresponding number of page number is 1；For starting flow data processing unit, monitoring containing dirty pages with
Containing dirty pages are sent Stream to flow by track unit in real time, and update global table, and the number of corresponding containing dirty pages number adds 1；
Dirty page tracking unit, the memory pages modified for monitoring source host；
Flow data processing unit checks in global table for obtaining page number and corresponding page data from Stream stream
The corresponding number of page number is then abandoned this page data and is not handled, subtract 1 for number if it is greater than 1；If being equal to 1, number is set
It is set to 0, and compression processing is carried out to this page data, send migration units；
Migration units reach shutdown threshold condition for discriminating whether, if it is, stopping source host internal storage data more
Newly, internal storage data migration is carried out.
Further, the shutdown threshold condition is time restriction, that is, judges to flow the timing time generating since Stream
Whether preset time value T is greater than, if it is, stopping the update of source host internal storage data；Or
The shutdown threshold condition is threshold byte number, that is, differentiates that (page number number is greater than the page of 0 record in global table
Number summation+containing dirty pages in do not collect Stream stream in number of pages) × every page of byte number≤threshold byte number when, then stop source host
Internal storage data memory refreshing.
Further, flow data acquisition unit full page internal storage data will be all generated in Stream stream for the first time, restart stream
Data processing unit；Or
The flow data processing unit prepares the delay time that opposite Stream stream generates, that is, when reaching delay time,
Start the flow data processing unit.
Further, the flow data acquisition unit and flow data processing unit are that multi-core parallel concurrent or multi-thread concurrent execute.
Further, source host memory is divided into multiple regions by the flow data processing unit, the data of different zones by
Multi-thread concurrent or multi-core parallel concurrent execute, then times of acquisition summation determines whether to shut down from multiple global tables.
Further, destination host receives the internal storage data that source host is sent, and unzips it.
According to an aspect of the present invention, the method for online migration internal storage data is proposed, comprising:
Send Stream to flow by page number, page data whole pages when migrating and starting, and generate page number, number it is complete
Office's table, the corresponding number of page number are 1；
Containing dirty pages are sent Stream to flow, and update global table by the memory pages that monitoring source host is modified in real time, corresponding containing dirty pages
The number of number adds 1；
Page number and corresponding page data are obtained from Stream stream, checks the corresponding number of page number in global table, such as
Fruit is greater than 1, then abandons this page data and do not handle, subtract 1 for number；If being equal to 1, set 0 for number, and to this page data into
Row compression processing；
It discriminates whether to reach shutdown threshold condition, if it is, stopping the update of source host internal storage data, carries out memory number
According to migration.
Further, the shutdown threshold condition is time restriction, that is, judges to flow the timing time generating since Stream
Whether preset time value T is greater than, if it is, stopping the update of source host internal storage data；
Shutdown threshold condition is threshold byte number, that is, differentiates that (page number number is total greater than the number of pages of 0 record in global table
With the number of pages in Stream stream is not collected in+containing dirty pages) × every page of byte number≤threshold byte number when, then stop source host memory
Datarams update.
Further, it full page internal storage data will all be generated in Stream stream for the first time, and restart flow data processing；Or
Prepare the delay time that opposite Stream stream generates, that is, when reaching delay time, stream data is handled.
Further, it is multi-thread concurrent that original internal storage data and containing dirty pages Data collecting conversion, which are the process of Stream stream,
Or multi-core parallel concurrent executes；
Source host memory is divided into multiple regions, the data of different zones are that multi-thread concurrent or multi-core parallel concurrent execute；
The compression of data and duplicate removal processing are that multi-thread concurrent or multi-core parallel concurrent execute in Stream stream；
And/or
Multiple compressed internal storage datas are subjected to stream polymerization, and the internal storage data after stream polymerization is migrated.
Further, source host memory is divided into multiple regions, the data of different zones by multi-thread concurrent or multicore simultaneously
Row executes, then times of acquisition summation determines whether to shut down from multiple global tables.
Further, destination host receives the internal storage data that source host is sent, and unzips it.
In the present invention, check that the corresponding number of page number is then abandoned this page data and do not handled if it is greater than 1 in global table；
If being equal to 1, compression processing is carried out to this page data and is migrated.That is, number, which is greater than 1, to be indicated to insert one in stream again below
The updated page data with number, therefore this page being first inserted into has not needed to pass to destination host again, it is only necessary to it will
It is exactly memory last state that, which is transmitted to destination host with the page data of number, below.That is, to no duplicate pages
The memory page data of number is compressed, and is had the memory page data of repetition page number to abandon and is not handled, avoids data re-transmitting.Cause
This, can reduce the number of transmissions of source host memory containing dirty pages, improve internal storage data compression efficiency, accelerate pre-copy loop convergence speed
Degree.
By referring to the drawings to the detailed description of exemplary embodiment of the present invention, other feature of the invention and its
Advantage will become apparent.
Detailed description of the invention
The attached drawing for constituting part of specification describes the embodiment of the present invention, and together with the description for solving
Release the principle of the present invention.
The present invention can be more clearly understood according to following detailed description referring to attached drawing, in which:
Fig. 1 is the structural schematic diagram of the system of online migration internal storage data.
Fig. 2 is the structural schematic diagram of the system embodiment of online migration virtual machine memory data.
Fig. 3 is the flow diagram of the embodiment of the method for online migration internal storage data.
Specific embodiment
Carry out the various exemplary embodiments of detailed description of the present invention now with reference to attached drawing.It should also be noted that unless in addition having
Body explanation, the unlimited system of component and the positioned opposite of step, numerical expression and the numerical value otherwise illustrated in these embodiments is originally
The range of invention.
Simultaneously, it should be appreciated that for ease of description, the size of various pieces shown in attached drawing is not according to reality
Proportionate relationship draw.
Be to the description only actually of at least one exemplary embodiment below it is illustrative, never as to the present invention
And its application or any restrictions used.
Technology, method and apparatus known to person of ordinary skill in the relevant may be not discussed in detail, but suitable
In the case of, the technology, method and apparatus should be considered as authorizing part of specification.
It is shown here and discuss all examples in, any occurrence should be construed as merely illustratively, without
It is as limitation.Therefore, the other examples of exemplary embodiment can have different values.
It should also be noted that similar label and letter indicate similar terms in following attached drawing, therefore, once a certain Xiang Yi
It is defined in a attached drawing, then in subsequent attached drawing does not need that it is further discussed.
To make the objectives, technical solutions, and advantages of the present invention clearer, below in conjunction with specific embodiment, and reference
Attached drawing, the present invention is described in more detail.
Fig. 1 is the structural schematic diagram of the system of online migration internal storage data.The system include: flow data acquisition unit 110,
Dirty page tracking unit 120, flow data processing unit 130 and migration units 140.The system is suitable for virutal machine memory state
Online migration.Certainly, it will be understood by those skilled in the art that being only intended to illustrate here, should not be construed as to the present invention
Limitation.For example, being equally applicable to the online migration of physical machine internal storage state.Wherein:
Flow data acquisition unit 110, for whole pages to be sent into generation by page number, page data when migrating beginning
Stream stream, and page number, the global table of number are generated, the corresponding number of page number is 1；It is single for starting flow data processing
Member monitors dirty page tracking unit, send Stream to flow containing dirty pages in real time, and update global table, and the number of corresponding containing dirty pages number adds 1.
Wherein, source host is handled full memory to be migrated by the tuple structure paging of page number, page data.Operation system
System is internally deposited using Paging system and is managed, and physical memory is divided into the memory block of same size by page (such as 4K size), often
One page number of a block, tuple structure here are, for example, to create a table, and the field of table is page number, page data, the record of table
It is exactly the page number of all physical memories and the data (i.e. page data) of corresponding memory block.
Dirty page tracking unit 120, the memory pages modified for monitoring source host.
Flow data processing unit 130 checks global table for obtaining page number and corresponding page data from Stream stream
The corresponding number of middle page number is then abandoned this page data and is not handled, subtract 1 for number if it is greater than 1；If being equal to 1, by number
It is set as 0, and compression processing is carried out to this page data, send migration units.
Migration units 140 reach shutdown threshold condition for discriminating whether, if it is, stopping source host internal storage data
Update, carry out internal storage data migration.
Wherein, flow data acquisition unit and flow data processing unit can multi-core parallel concurrent or multi-thread concurrent execution.That is, former
Beginning internal storage data and containing dirty pages Data collecting conversion be Stream stream process can multi-thread concurrent or multi-core parallel concurrent execute, with
It improves efficiency；The compression of data and duplicate removal processing can also multi-thread concurrent or multicores in the flow data processing unit Stream stream
It is parallel to execute, to improve efficiency；Multiple compressed internal storage datas are carried out stream polymerization by the flow data processing unit, and will stream
Internal storage data after polymerization send migration units.To, can will be multiple compressed before migrating data copies destination host to
Page carries out stream polymerization, and small data quantity frequent transmission is avoided to reduce data transmission efficiency.
In this embodiment, check that the corresponding number of page number if it is greater than 1 then abandons this page data not in global table
Processing；If being equal to 1, compression processing is carried out to this page data and is migrated.That is, number, which is greater than 1, to be indicated to be inserted into stream below again
One updated page data with number, therefore this page being first inserted into do not needed to pass to destination host again, only
It needs that is transmitted to destination host with the page data of number below to be exactly memory last state.That is, to not having
The memory page data for repeating page number is compressed, and is had the memory page data of repetition page number to abandon and is not handled, avoids data weight
Hair.Therefore, it can reduce the number of transmissions of source host memory containing dirty pages, improve internal storage data compression efficiency, accelerate pre-copy circulation
Convergence rate.
In an embodiment of the present invention, the migration units discriminate whether to reach shutdown threshold condition, if it is, stopping
The update of source host internal storage data carries out internal storage data migration, completes subsequent memory duplication and migration work.Otherwise, Ke Yiji
Continuous internal storage data updates.
Shutdown threshold condition, e.g., time restriction, then the migration units judge to flow the meter generating since Stream
When the time whether be greater than preset time value T, if it is, even if other conditions do not meet also, all stopping source host internal storage datas
Update.Internal storage data migration is carried out, subsequent memory duplication and migration work are completed.Otherwise, internal storage data update can be continued.
Shutdown threshold condition, in another example being threshold byte number, then the migration units inspection (for example, regular check) is complete
Office's table differentiates and (does not collect in Stream stream in number of pages summation+containing dirty pages of record of the page number number greater than 0 in global table
Number of pages) × every page of byte number≤threshold byte number when, then stop source host internal storage data memory refreshing, carry out internal storage data and move
It moves, completes subsequent memory duplication and migration work.Otherwise, internal storage data update can be continued.
Source host memory can be divided into multiple regions by flow data processing unit, and the data of different zones are by different processes
Or thread carries out stream process, to realize that multi-thread concurrent or multi-core parallel concurrent execute, at this time the method for discrimination of shutdown threshold be from
Times of acquisition summations determines in multiple global tables.
It needs to stop source host when migrating internal storage data, then by source host memory data copy to destination host, restarts mesh
Host.Thus, it is ensured that the internal storage data transmitted after shutting down is minimum.The embodiment of the invention passes through setting threshold byte
Number passes data to allow after shutdown, shortens downtime less as far as possible so that the internal storage data amount of migration is less than threshold byte number.
In an embodiment of the present invention, it can generate and handle in the following way Stream stream:
Flow data acquisition unit full page internal storage data will be all generated in Stream stream for the first time, then restart flow data
Processing unit, data take out compression transmission processing in convection current, this mode generate and handle between have a time interval t, be raw
At time of whole pages into stream.This kind of mode duplicate removal effect is good, because data are more in initial flow, in subsequent containing dirty pages and stream
The case where Data duplication, probability was big.
Or
The flow data processing unit prepares the delay time that opposite Stream stream generates, that is, when reaching delay time,
Start the flow data processing unit.For example, setting delay time voluntarily as 0.5t, start at this time when generating half Stream stream
Stream stream processing unit carries out compression transmission processing.Delay time can be tested in practical application, it is optimal to find
Configuration, with reach within the time as few as possible shut down after transmit least data, i.e., reach the smallest mesh of downtime as early as possible
Mark.
The processing of Stream stream can be carried out before full page Stream for the first time is flowed and generated, can voluntarily be prepared opposite
The delay time that Stream stream generates reduces the EMS memory occupation of Stream stream to improve execution efficiency.That is, for the first time will be complete
Portion's page can not be shut down before being all generated in stream.It needs to be arranged a global variable at this time, identifies full memory for the first time
Whether page Stream stream generates and finishes, and such as 0 indicates not generated, and 1 indicates to have generated, for not generated state, not into
Row shutdown threshold condition criterion.This kind of mode such as shortens at the time of stream to be generated, and total virtual machine (vm) migration time may also can
Shorten.
In an embodiment of the present invention, which can also include destination host, wherein destination host receives source host hair
The internal storage data sent, and unzip it.Restore destination host VM internal storage state, complete other migration operations, it is such as shared external
Storage switching, starting virtual machine etc..
Below in conjunction with specific embodiment, system of the invention is described further.
Fig. 2 is the structural schematic diagram of the system embodiment of online migration virtual machine memory data.The system may include stream
Data acquisition unit, dirty page tracking unit, flow data processing unit, migration units and destination host.Wherein:
The page that dirty page tracking unit detection source host is modified；
Whole pages are pressed [page number, page data] when migrating and starting and Stream are sent to flow by flow data acquisition unit, and raw
At the global table of one [page number, number], the corresponding number of page number is 1.Containing dirty pages are obtained from dirty page tracking unit, in real time will
Containing dirty pages send Stream to flow, and update global table, the number that same memory page number occurs in mark Stream stream；
Flow data processing unit obtains page number and corresponding memory page data from Stream stream, according to page number complete
Number in office's table carries out duplicate removal judgement, if number is greater than 1, illustrates in Stream stream there are also identical memory page data, with
Subject to occurring afterwards, this page data is abandoned, avoids repeating to transmit, and subtract 1 for number；If number is equal to 1, this page is carried out
Compression, can immediately or polymerization it is a plurality of after send the migration units of source host, and set 0 for number；
Data after the compression that the migration units receiving stream-oriented data processing unit of source host is sent, send destination host, also, source
The migration units of host also need to differentiate (not to be acquired in number of pages summation+containing dirty pages of record of the page number number greater than 0 in global table
To the number of pages in Stream stream) × every page of byte number≤threshold byte number when, then stop source host VM memory refreshing, complete subsequent
Memory duplication and migration work；
Destination host receives the internal storage data that the migration units of source host are sent, and unzips it, and restores purpose master
Machine VM memory.
Fig. 3 is the flow diagram of the embodiment of the method for online migration internal storage data.Method includes the following steps:
It in step 310, send Stream to flow by page number, page data whole pages when migrating beginning, and generates page volume
Number, the global table of number, the corresponding number of page number is 1.
Wherein, source host is handled full memory to be migrated by the tuple structure paging of page number, page data.Operation system
System is internally deposited using Paging system and is managed, and physical memory is divided into the memory block of same size by page (such as 4K size), often
One page number of a block, tuple structure here are, for example, to create a table, and the field of table is page number, page data, the record of table
It is exactly the page number of all physical memories and the data (i.e. page data) of corresponding memory block.
In step 320, the memory pages that source host is modified are monitored, send Stream to flow containing dirty pages in real time, and update the overall situation
The number of table, corresponding containing dirty pages number adds 1.
In step 330, page number and corresponding page data are obtained from Stream stream, checks that page number is corresponding in global table
Number, judge whether to be greater than 1, if it is greater than 1, execute step 340, if being equal to 1, execute step 350.
In step 340, number is greater than 1, abandons this page data and do not handle, subtract 1 for number.
In step 350, number is equal to 1, sets 0 for number, and carry out compression processing to this page data, continues to execute step
Rapid 360.
It in step 360, discriminates whether to reach shutdown threshold condition, if so, executing step 370, otherwise, executes step
320。
In step 370, stop the update of source host internal storage data, carries out internal storage data migration.
This method is suitable for the online migration of virutal machine memory state.Certainly, it will be understood by those skilled in the art that
Here it is only intended to illustrate, should not be construed as limiting the invention.For example, being equally applicable to the online of physical machine internal storage state
Migration.
Wherein, original internal storage data and containing dirty pages Data collecting conversion be Stream stream process be multi-thread concurrent or
Multi-core parallel concurrent executes, and efficiency can be improved；
Source host memory is divided into multiple regions, the data of different zones are that multi-thread concurrent or multi-core parallel concurrent execute,
Efficiency can be improved；
The compression of data and duplicate removal processing are that multi-thread concurrent or multi-core parallel concurrent execute in Stream stream, and effect can be improved
Rate；
And/or
Multiple compressed internal storage datas are subjected to stream polymerization, and the internal storage data after stream polymerization is migrated, are avoided
Small data quantity frequent transmission reduces data transmission efficiency.
In this embodiment, check that the corresponding number of page number if it is greater than 1 then abandons this page data not in global table
Processing；If being equal to 1, compression processing is carried out to this page data and is migrated.That is, number, which is greater than 1, to be indicated to be inserted into stream below again
One updated page data with number, therefore this page being first inserted into do not needed to pass to destination host again, only
It needs that is transmitted to destination host with the page data of number below to be exactly memory last state.That is, to not having
The memory page data for repeating page number is compressed, and is had the memory page data of repetition page number to abandon and is not handled, avoids data weight
Hair.Therefore, it can reduce the number of transmissions of source host memory containing dirty pages, improve internal storage data compression efficiency, accelerate pre-copy circulation
Convergence rate.
In an embodiment of the present invention, it also discriminates whether to reach shutdown threshold condition, if it is, stopping source host memory
The update of data carries out internal storage data migration, completes subsequent memory duplication and migration work.Otherwise, internal storage data can be continued
It updates.
Shutdown threshold condition, e.g., time restriction then judges whether the timing time since Stream flows generation is big
In preset time value T, if it is, all stopping the update of source host internal storage data even if other conditions do not meet also.Progress in
Subsequent memory duplication and migration work are completed in deposit data migration.Otherwise, internal storage data update can be continued.
Shutdown threshold condition then checks (for example, regular check) global table in another example being threshold byte number, differentiates (complete
Do not collect the number of pages in Stream stream in office's table in number of pages summation+containing dirty pages of record of the page number number greater than 0) × every page of word
When joint number≤threshold byte number, then stop source host internal storage data memory refreshing, carry out internal storage data migration, completes subsequent memory
Duplication and migration work.Otherwise, internal storage data update can be continued.
In an embodiment of the present invention, source host memory can be divided into multiple regions, the data of different zones are not by
Stream process is carried out with process or thread, to realize that multi-thread concurrent or multi-core parallel concurrent execute, the differentiation side of shutdown threshold at this time
Method is that times of acquisition summations determines from multiple global tables.
It needs to stop source host when migrating internal storage data, then by source host memory data copy to destination host, restarts mesh
Host.Thus, it is ensured that the internal storage data transmitted after shutting down is minimum.The embodiment of the invention passes through setting threshold byte
Number passes data to allow after shutdown, shortens downtime less as far as possible so that the internal storage data amount of migration is less than threshold byte number.
In an embodiment of the present invention, it can generate and handle in the following way Stream stream:
It full page internal storage data will all be generated in Stream stream for the first time, and then restart flow data processing, number in convection current
According to take out compression transmission processing, this mode generate and handle between have a time interval t, be generation full memory page into stream
Time.This kind of mode duplicate removal effect is good, because data are more in initial flow, probability the case where Data duplication in subsequent containing dirty pages and stream
Greatly.
Or
Prepare the delay time that opposite Stream stream generates, that is, when reaching delay time, stream data is handled.
For example, setting delay time voluntarily as 0.5t, starts Stream stream process when generating half Stream stream at this time, carry out compression biography
It send.Delay time can be tested in practical application, to find allocation optimum, be stopped with reaching within the time as few as possible
Least data are transmitted after machine, i.e., reach the smallest target of downtime as early as possible.
The processing of Stream stream can be carried out before full page Stream for the first time is flowed and generated, can voluntarily be prepared opposite
The delay time that Stream stream generates reduces the EMS memory occupation of Stream stream to improve execution efficiency.That is, for the first time will be complete
Portion's page can not be shut down before being all generated in stream.It needs to be arranged a global variable at this time, identifies full memory for the first time
Whether page Stream stream generates and finishes, and such as 0 indicates not generated, and 1 indicates to have generated, for not generated state, not into
Row shutdown threshold condition criterion.This kind of mode such as shortens at the time of stream to be generated, and total virtual machine (vm) migration time may also can
Shorten.
It in an embodiment of the present invention, can be the following steps are included: destination host receives the memory number that source host is sent
According to, and unzip it.Restore destination host VM internal storage state, complete other migration operations, such as shared external storage switching is opened
Dynamic virtual machine etc..
So far, the present invention is described in detail.In order to avoid covering design of the invention, it is public that this field institute is not described
The some details known.Those skilled in the art as described above, completely it can be appreciated how implementing technology disclosed herein
Scheme.
Method and device of the invention may be achieved in many ways.For example, can by software, hardware, firmware or
Person's software, hardware, firmware any combination realize method and device of the invention.The step of for the method it is above-mentioned
Sequence is merely to be illustrated, and the step of method of the invention is not limited to sequence described in detail above, unless with other sides
Formula illustrates.In addition, in some embodiments, the present invention can be also embodied as recording program in the recording medium, these
Program includes for realizing machine readable instructions according to the method for the present invention.Thus, the present invention also covers storage for executing
The recording medium of program according to the method for the present invention.
Although some specific embodiments of the invention are described in detail by example, the skill of this field
Art personnel it should be understood that above example merely to being illustrated, the range being not intended to be limiting of the invention.The skill of this field
Art personnel are it should be understood that can without departing from the scope and spirit of the present invention modify to above embodiments.This hair
Bright range is defined by the following claims.
Claims (12)
1. the system of migration internal storage data online, comprising:
Flow data acquisition unit for sending Stream to flow by page number, page data whole pages when migrating beginning, and generates
Page number, the global table of number, the corresponding number of page number are 1；For starting flow data processing unit, dirty page tracking list is monitored
Containing dirty pages are sent Stream to flow by member in real time, and update global table, and the number of corresponding containing dirty pages number adds 1；
Dirty page tracking unit, the memory pages modified for monitoring source host；
Flow data processing unit checks that page is compiled in global table for obtaining page number and corresponding page data from Stream stream
Number corresponding number is then abandoned this page data and is not handled, subtract 1 for number if it is greater than 1；If being equal to 1, set number to
0, and compression processing is carried out to this page data, send migration units；
Migration units reach shutdown threshold condition for discriminating whether, if it is, stop the update of source host internal storage data,
Carry out internal storage data migration.
2. migrating the system of internal storage data online according to claim 1, comprising:
The shutdown threshold condition is time restriction, that is, it is default to judge whether the timing time since Stream flows generation is greater than
Time value T, if it is, stopping the update of source host internal storage data；Or
The shutdown threshold condition is threshold byte number, that is, differentiates that (page number number is total greater than the number of pages of 0 record in global table
With the number of pages in Stream stream is not collected in+containing dirty pages) × every page of byte number≤threshold byte number when, then stop source host memory
Datarams update.
3. migrating the system of internal storage data online according to claim 1, comprising:
Flow data acquisition unit full page internal storage data will be all generated in Stream stream for the first time, and it is single to restart flow data processing
Member；Or
The flow data processing unit prepares the delay time that opposite Stream stream generates, that is, when reaching delay time, starting
The flow data processing unit.
4. according to claim 1 to the system of 3 any online migration internal storage datas, comprising:
The flow data acquisition unit and flow data processing unit are that multi-core parallel concurrent or multi-thread concurrent execute.
5. according to claim 1 or 3 it is any it is described it is online migration internal storage datas systems, comprising:
Source host memory is divided into multiple regions by the flow data processing unit, the data of different zones by multi-thread concurrent or
Multi-core parallel concurrent executes, then times of acquisition summation determines whether to shut down from multiple global tables.
6. according to claim 1 to the system of 3 any online migration internal storage datas, comprising:
Destination host receives the internal storage data that source host is sent, and unzips it.
7. the method for migration internal storage data online, comprising:
It send Stream to flow by page number, page data whole pages when migrating beginning, and generates page number, the overall situation of number
Table, the corresponding number of page number are 1；
Containing dirty pages are sent Stream to flow, and update global table by the memory pages that monitoring source host is modified in real time, corresponding containing dirty pages number
Number add 1；
Page number and corresponding page data are obtained from Stream stream, the corresponding number of page number in global table is checked, if greatly
In 1, then abandons this page data and do not handle, number is subtracted 1；If being equal to 1,0 is set by number, and press this page data
Contracting processing；
It discriminates whether to reach shutdown threshold condition, if it is, stopping the update of source host internal storage data, carries out internal storage data and move
It moves.
8. migrating the method for internal storage data online according to claim 7, comprising:
The shutdown threshold condition is time restriction, that is, it is default to judge whether the timing time since Stream flows generation is greater than
Time value T, if it is, stopping the update of source host internal storage data；Or
Shutdown threshold condition be threshold byte number, that is, differentiate (in global table page number number greater than 0 record number of pages summation+
Do not collected in containing dirty pages Stream stream in number of pages) × every page of byte number≤threshold byte number when, then stop source host memory number
According to memory refreshing.
9. migrating the method for internal storage data online according to claim 7, comprising:
It full page internal storage data will all be generated in Stream stream for the first time, and restart flow data processing；Or
Prepare the delay time that opposite Stream stream generates, that is, when reaching delay time, stream data is handled.
10. according to the method for any online migration internal storage data of claim 7 to 9, comprising:
Original internal storage data and containing dirty pages Data collecting conversion are that the process of Stream stream is that multi-thread concurrent or multi-core parallel concurrent are held
Row；
Source host memory is divided into multiple regions, the data of different zones are that multi-thread concurrent or multi-core parallel concurrent execute；
The compression of data and duplicate removal processing are that multi-thread concurrent or multi-core parallel concurrent execute in Stream stream；
And/or
Multiple compressed internal storage datas are subjected to stream polymerization, and the internal storage data after stream polymerization is migrated.
11. according to the method for any online migration internal storage data of claim 7 or 9, comprising:
Source host memory is divided into multiple regions, the data of different zones are executed by multi-thread concurrent or multi-core parallel concurrent, then from
Times of acquisition summation determines whether to shut down in multiple global tables.
12. according to the method for any online migration internal storage data of claim 7 to 9, comprising:
Destination host receives the internal storage data that source host is sent, and unzips it.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410767628.4A CN105786585B (en) | 2014-12-15 | 2014-12-15 | The system and method for online migration internal storage data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410767628.4A CN105786585B (en) | 2014-12-15 | 2014-12-15 | The system and method for online migration internal storage data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105786585A CN105786585A (en) | 2016-07-20 |
| CN105786585B true CN105786585B (en) | 2019-01-29 |
Family
ID=56373749
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410767628.4A Active CN105786585B (en) | 2014-12-15 | 2014-12-15 | The system and method for online migration internal storage data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105786585B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112559116B (en) * | 2019-09-25 | 2024-05-07 | 阿里巴巴集团控股有限公司 | Memory migration method and device and computing equipment |
| CN113076175B (en) * | 2021-03-26 | 2022-09-30 | 北京汇钧科技有限公司 | Memory sharing method and device for virtual machine |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102103524A (en) * | 2010-12-23 | 2011-06-22 | 北京航空航天大学 | Memory redundancy oriented virtual machine migration device and method |
| CN102932409A (en) * | 2012-09-21 | 2013-02-13 | 中国电信股份有限公司云计算分公司 | Online migrating method and system of virtual machine |
| CN103365704A (en) * | 2012-03-26 | 2013-10-23 | 中国移动通信集团公司 | Memory pre-copying method in virtual machine migration, device executing memory pre-copying method and system |
| CN103473368A (en) * | 2013-09-27 | 2013-12-25 | 中国科学院信息工程研究所 | Virtual machine real-time migration method and system based on counting rank ordering |
| CN103581331A (en) * | 2013-11-13 | 2014-02-12 | 中国科学院计算技术研究所 | Virtual machine on-line transfer method and system |
| CN103631633A (en) * | 2012-08-20 | 2014-03-12 | 中国电信股份有限公司 | Virtual machine total-system online migration method, device and system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP6123388B2 (en) * | 2013-03-15 | 2017-05-10 | 日本電気株式会社 | Fault tolerant server |
-
2014
- 2014-12-15 CN CN201410767628.4A patent/CN105786585B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102103524A (en) * | 2010-12-23 | 2011-06-22 | 北京航空航天大学 | Memory redundancy oriented virtual machine migration device and method |
| CN103365704A (en) * | 2012-03-26 | 2013-10-23 | 中国移动通信集团公司 | Memory pre-copying method in virtual machine migration, device executing memory pre-copying method and system |
| CN103631633A (en) * | 2012-08-20 | 2014-03-12 | 中国电信股份有限公司 | Virtual machine total-system online migration method, device and system |
| CN102932409A (en) * | 2012-09-21 | 2013-02-13 | 中国电信股份有限公司云计算分公司 | Online migrating method and system of virtual machine |
| CN103473368A (en) * | 2013-09-27 | 2013-12-25 | 中国科学院信息工程研究所 | Virtual machine real-time migration method and system based on counting rank ordering |
| CN103581331A (en) * | 2013-11-13 | 2014-02-12 | 中国科学院计算技术研究所 | Virtual machine on-line transfer method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105786585A (en) | 2016-07-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102464845B1 (en) | Distributed hardware tracing | |
| JP6219512B2 (en) | Virtual hadoop manager | |
| CN103098014B (en) | Storage system | |
| US20120144123A1 (en) | Read-ahead processing in networked client-server architecture | |
| JP2017504874A5 (en) | ||
| CN103064733A (en) | Cloud computing virtual machine live migration technology | |
| CN104270409B (en) | Centrally stored method is efficiently utilized in a kind of cloud platform | |
| EP2425349A2 (en) | Application efficiency engine | |
| US20150046751A1 (en) | Managing traces to capture data for memory regions in a memory | |
| CN104268003A (en) | Memory state migration method applicable to dynamic migration of virtual machine | |
| CN106469085A (en) | The online migration method, apparatus and system of virtual machine | |
| US9984139B1 (en) | Publish session framework for datastore operation records | |
| KR20180086791A (en) | Cloud system for surpporting big data process and managing method thereof | |
| WO2014020959A1 (en) | Distributed processing device and distributed processing system as well as distributed processing method | |
| JPWO2016166844A1 (en) | Distributed processing system, task processing method, storage medium | |
| Desai et al. | Efficient virtual machine migration in cloud computing | |
| KR101264895B1 (en) | Virtual machine migration processing apparatus and method thereof | |
| CN105786585B (en) | The system and method for online migration internal storage data | |
| WO2018148322A1 (en) | Resource allocation method and apparatus | |
| Fu et al. | FARMS: Efficient mapreduce speculation for failure recovery in short jobs | |
| Dai et al. | Two-choice randomized dynamic I/O scheduler for object storage systems | |
| CN104410698B (en) | Initiation node abnormality eliminating method and device under a kind of share nothing cluster | |
| JPWO2011077533A1 (en) | Performance data collection method, performance data collection device, and performance data management system | |
| US20100269119A1 (en) | Event-based dynamic resource provisioning | |
| Yin et al. | Live virtual machine migration with optimized three-stage memory copy |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |