CN108874828A - The method and apparatus of Data Migration - Google Patents
The method and apparatus of Data Migration Download PDFInfo
- Publication number
- CN108874828A CN108874828A CN201710334373.6A CN201710334373A CN108874828A CN 108874828 A CN108874828 A CN 108874828A CN 201710334373 A CN201710334373 A CN 201710334373A CN 108874828 A CN108874828 A CN 108874828A
- Authority
- CN
- China
- Prior art keywords
- data
- database
- migration
- source database
- full dose
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention relates to a kind of method and apparatus of Data Migration, and the method includes the synchronous processes of full dose, and furthermore may also include the increment synchronization process based on binlog mechanism.The migration without loss of database data can be realized by this method, and improves the performance of migration.This method includes：Data full dose in source database is synchronized to new database；During the full dose is synchronous, the alteration of data in the source database is monitored, if the data in the source database change, has realized that synchronous data are updated for corresponding in the new database；According to user's dimension computation migration rate, when the mobility of the data of certain user reaches 100%, the corresponding database of certain described user is switched into new database from source database.
Description
Technical field
The present invention relates to computer fields, more particularly to a kind of method and apparatus of Data Migration.
Background technique
It is continued to bring out along with Internet-related business, some business have obtained quick swift and violent development, original data
Library storage and table structure are no longer satisfied the demand of business.It is deposited to meet normal development and the data of business
The problem of storage, table structure, needs to establish a set of completely new database, can accommodate mass data storage, and needs to there is design
The table structure of defect is redesigned.The newly-designed database table structure is referred to as isomery knot relative to original database structure
Structure.For this heterogeneous database, needs a kind of method and move to user data in new database from original database.
Existing data migration method is synchronous based on poll, that is to say, that for each record in database table
Modification time is polled synchronization, if modification time is later than the time recorded when last synchronization, synchronizes.But it will
Modification time with it is last synchronous when time for recording be compared and judge whether data synchronized and will lead to much together
The Data duplication of step is synchronous, to generate many invalid syncs, it reduce net synchronization capabilities.
It is higher for the precision and coherence request of time according to the synchronization scheme of time.If the time precision (second or
Person's millisecond) different or two machine clocks are asynchronous, then will lead to the loss of data.Clock is synchronous and precision is in major part
In the case of it is difficult to ensure that it is consistent precisely, thus in the synchronization scheme according to the time, loss of data it is very risky, be not able to satisfy
The demand of Internet service migration.
Summary of the invention
In view of this, the embodiment of the present invention provides a kind of method and device of Data Migration, data can not occur
Data Migration is carried out in the case where loss, data delay etc., user will not be allowed to perceive, the experience of user will not be influenced
Property.
To achieve the above object, according to an aspect of an embodiment of the present invention, a kind of method of Data Migration is provided.
A kind of method of Data Migration of the embodiment of the present invention includes：Data full dose in source database is synchronized to new number
According to library；During the full dose is synchronous, the alteration of data in the source database is monitored, if in the source database
Data change, then realized that synchronous data are updated for corresponding in the new database；According to user's dimension
Computation migration rate incites somebody to action the corresponding database of certain user from source data when the mobility of the data of certain user reaches 100%
Library switches to new database.
Optionally, before the data full dose in source database is synchronized to new database, the method also includes：According to
Data in source database are carried out structure conversion by the design structure of table in the new database.
Optionally, the data full dose in source database is synchronized to new database includes：Using from major key ID is increased, sequence is selected
The data for selecting the default number of branches in the source database, are synchronized to new database, until it is synchronous to complete full dose.
Optionally, the method also includes：Judged using ACK mechanism pre- in the source database of each sequential selection
If whether the data of item number are completed synchronization.
Optionally, after the data of the default number of branches in the source database by each sequential selection are completed to synchronize,
The method also includes：By all recording from increasing major key ID into migration database of record for the data of the default number of branches, and
Using the last item data of the data of the default number of branches from increasing major key ID as the starting displacement of next sequential selection data
ID is recorded in migration database of record.
Optionally, the alteration for monitoring data in the source database includes：Based on binlog mechanism, the source is monitored
The alteration of data in database.
Optionally, by the new database it is corresponding realized synchronous data be updated including：Judgement becomes
Data more whether there is in migration database of record from increasing major key ID, and if it exists, then to corresponding in the new database
Data be updated, do not update then if it does not exist.Invalid update is filtered by the judgement, thus raising property
Energy.
Optionally, the method also includes：Using visualization interface, real-time monitoring data migrates progress.
According to another aspect of an embodiment of the present invention, a kind of device of Data Migration is provided.
A kind of device of Data Migration of the embodiment of the present invention includes：Full dose synchronization module, for will be in source database
Data full dose is synchronized to new database；Increment synchronization module, for monitoring the source data during full dose is synchronous
The alteration of data in library, if the data in the source database change, by the new database it is corresponding
Realize that synchronous data are updated；Switching module, for according to user's dimension computation migration rate, when the data of certain user are moved
When shifting rate reaches 100%, the corresponding database of certain described user is switched into new database from source database.
Optionally, the full dose synchronization module is also used to：According to the design structure of table in the new database, by source data
Data in library carry out structure conversion.
Optionally, the full dose synchronization module is also used to：Using from increasing major key ID, in source database described in sequential selection
The data of default number of branches, are synchronized to new database, until it is synchronous to complete full dose.
Optionally, the full dose synchronization module is also used to：The source data of each sequential selection is judged using ACK mechanism
Whether the data of the default number of branches in library are completed synchronization.
Optionally, described device further includes：Migrate logging modle, for by the data of the default number of branches all from increasing
Major key ID is recorded into migration database of record, and the last item data of the data of the default number of branches are increased major key ID certainly
Starting displacement ID as next sequential selection data is recorded in migration database of record.
Optionally, the increment synchronization module is also used to：Based on binlog mechanism, data in the source database are monitored
Alteration.
Optionally, the increment synchronization module is also used to：Judge whether there is from increasing major key ID for the data changed
In migration database of record, and if it exists, be then updated to corresponding data in the new database, if it does not exist then not more
Newly.
Optionally, described device further includes：Visualization model, for utilizing visualization interface, real-time monitoring data migration
Progress.
In accordance with a further aspect of the present invention, a kind of electronic equipment terminal is provided, including：One or more processors；With
In the storage device for storing one or more programs, when one or more of programs are held by one or more of processors
Row, so that the method that one or more of processors realize provided Data Migration according to the present invention.
In accordance with a further aspect of the present invention, a kind of computer-readable medium is provided, computer program is stored thereon with, institute
State the method that Data Migration provided by the present invention is realized when program is executed by processor.
One embodiment in foregoing invention has the following advantages that or beneficial effect：By the process of the above method, thus
The seamless migration that data in database may be implemented guarantees that corresponding operation system is not withdrawn, and no data loses and delay, and
It can accomplish to switch user's unaware, promote the user experience of operation system；By the way that the data in source database are counted according to new
Table structure conversion is carried out according to the design structure of table in library, so as to realize the Data Migration between heterogeneous database；Due to this
Full dose synchronizing process carries out sequence synchronization from major key is increased according to database, so there is no the loss of data；Pass through ACK machine
System, so as to further guarantee data without loss；By recording the increasing major key ID certainly of all data of having moved, so as to
Easily it is based on user's dimension computation migration rate；It is displaced ID by record start, so as to facilitate accurate sequential selection source
Data in database；By being based on binlog mechanism, so as to monitor number in source database during data are synchronous
According to variation, achieve the effect that data change real-time synchronization；By the way that whether depositing from increasing major key ID for the data changed judged
It is to migrate in database of record, so as to filter invalid update, only retains and effectively update, improve net synchronization capability；Pass through
The visualization interface controls migration progress so as to the Data Migration rate of real time inspection current business system.
Further effect possessed by above-mentioned non-usual optional way adds hereinafter in conjunction with specific embodiment
With explanation.
Detailed description of the invention
Fig. 1 is that the embodiment of the present invention can be applied to exemplary system architecture figure therein；
Fig. 2 is the schematic diagram of the key step of the method for Data Migration according to an embodiment of the present invention；
Fig. 3 is the schematic diagram of the system framework of the method for the Data Migration for realizing the embodiment of the present invention；
Fig. 4 is the schematic diagram of the detailed process of the method for Data Migration according to an embodiment of the present invention；
Fig. 5 is the schematic diagram of the main modular of the device of Data Migration according to an embodiment of the present invention；With
Fig. 6 is adapted for the structural schematic diagram for the computer system for realizing the electronic device terminal of the embodiment of the present application.
Specific embodiment
Present invention is further described in detail in the following with reference to the drawings and specific embodiments, but not as to limit of the invention
It is fixed.
Fig. 1 is shown can be using the exemplary of the device of the method or Data Migration of the Data Migration of the embodiment of the present invention
System architecture 100.
As shown in Figure 1, system architecture 100 may include terminal device 101,102,103, network 104 and server 105.
Network 104 between terminal device 101,102,103 and server 105 to provide the medium of communication link.Network 104 can be with
Including various connection types, such as wired, wireless communication link or fiber optic cables etc..
User can be used terminal device 101,102,103 and be interacted by network 104 with server 105, to receive or send out
Send message etc..Various telecommunication customer end applications, such as the application of shopping class, net can be installed on terminal device 101,102,103
(merely illustrative) such as the application of page browsing device, searching class application, instant messaging tools, mailbox client, social platform softwares.
Terminal device 101,102,103 can be the various electronic equipments with display screen and supported web page browsing, packet
Include but be not limited to smart phone, tablet computer, pocket computer on knee and desktop computer etc..
Server 105 can be to provide the server of various services, such as utilize terminal device 101,102,103 to user
The shopping class website browsed provides the back-stage management server (merely illustrative) supported.Back-stage management server can be to reception
To the data such as information query request analyze etc. processing, and by processing result (such as target push information, product letter
Breath -- merely illustrative) feed back to terminal device.
It should be noted that the method for Data Migration provided by the embodiment of the present invention is generally executed by server 105, phase
Ying Di, the device of Data Migration are generally positioned in server 105.
It should be understood that the number of terminal device, network and server in Fig. 1 is only schematical.According to realization need
It wants, can have any number of terminal device, network and server.
Fig. 2 is the schematic diagram of the key step of the method for Data Migration according to an embodiment of the present invention.
As shown in Fig. 2, the method for the Data Migration of the embodiment of the present invention mainly includes the following steps：
Step S201：Data full dose in source database is synchronized to new database.In the embodiment of the present invention, source database
Refer to database to be upgraded, also referred to as old database, because of business development, amount of storage even original table structure cannot
Enough meet business development.And new database is then the number planned again for the operation system Future Services Development in practical application
According to storage, such as realize that table is divided in a point library.Meanwhile new database is potentially based on business demand and has redesigned table structure.This new number
Isomeric data structure is formed according to library and source database.On this basis, it is further included in source data in the embodiment of the present invention
Data full dose in library is synchronized to before new database, will be in source database according to the design structure of table in the new database
Data carry out structure conversion.
Wherein, it the data full dose in source database is synchronized to new database can be, but not limited to and include：It is led using from increasing
Key ID, the data of the default number of branches in source database described in sequential selection, is synchronized to new database, until it is synchronous to complete full dose.
In addition, the method for the Data Migration of the embodiment of the present invention can also include：Using ACK mechanism, (ACK is
The abbreviation of Acknowledgement, i.e. confirmation character, in data communication, receiving station issues a kind of transmission class control of dispatching station
Character processed.Indicate that the data sent have confirmed that reception is errorless.In the embodiment of the present invention, retried based on its success-confirmation, failure-
Mechanism guarantees that data transmission is not lost) judge default number of branches in the source database of each sequential selection data whether
Synchronization is completed.
In addition, after the data of the default number of branches in the source database by each sequential selection are completed to synchronize, this
The method of the Data Migration of inventive embodiments can also include：By the data of the default number of branches all from increasing major key ID record
Increase major key ID certainly as next suitable into migration database of record, and using the last item data of the data of the default number of branches
The starting displacement ID of sequence selection data is recorded in migration database of record.By the increasing major key certainly for recording all data of having moved
ID, so as to easily be based on user's dimension computation migration rate；It is displaced ID by record start, so as to convenient accurate
Data in sequential selection source database.
After synchronizing by the full dose of step S201, increment synchronization is carried out since step S202.
Step S202：During the full dose is synchronous, the alteration of data in the source database is monitored, if institute
The data stated in source database change, then have realized that synchronous data are updated for corresponding in the new database.
In the embodiment of the present invention, can be, but not limited to based on binlog mechanism (binlog be for database of record data change two
System log is the core technology for database master-slave synchronisation), monitor the alteration of data in the source database.
In the embodiment of the present invention, realizes that synchronous data are updated by corresponding in the new database and may include
Following process：Judge whether there is in migration database of record from increasing major key ID for the data changed, and if it exists, then right
Corresponding data are updated in the new database, are not updated then if it does not exist.By judge change data oneself
Increasing major key ID whether there is in migration database of record, so as to filter invalid update, only retains and effectively updates, improves
Net synchronization capability.
Step S203：It, will when the mobility of the data of certain user reaches 100% according to user's dimension computation migration rate
The corresponding database of described certain user switches to new database from source database.
In addition, the method for the Data Migration of the embodiment of the present invention can also include：Using visualization interface, number is monitored in real time
According to migration progress.
Fig. 3 is the schematic diagram of the system framework of the method for the Data Migration for realizing the embodiment of the present invention.
As shown in figure 3, can be used for realizing in the embodiment of the present invention in the system framework of the method for Data Migration and include：
Source database：Database to be upgraded.Because of business development, amount of storage and original table structure it is impossible to meet
Business development；
New database：For the data storage that Future Services Development is planned again, such as table is divided in a point library.Meanwhile it redesigning
Table structure.This new database and source database form isomeric data structure；
Migrate database of record：For recording the data generated in transition process；
Full dose synchronization module：Data in sequence synchronization source database table are carried out from major key is increased by database.Based on ACK machine
The major key ID value of increasing certainly that the last item of the lot data records after synchronizing batch of data, is renewed back to migration and records data by system
Library.All data synchronized of the batch are migrated into database of record from increasing major key ID insertion simultaneously；
Increment synchronization module：For after full dose synchronizes, the changed data of state carry out increment synchronization, use
Binlog carries out actively triggering and synchronizes；
Gray scale center (i.e. switching module)：It is the control centre of system switching.There is provided migration progress (can be, but not limited to is
Mobility) page presentation, mobility according to it is synchronous when data calculating in migration database of record gets.Migration is provided to open
Close configuration and inquiry ability, i.e., gray scale center exposure switch judge interface to upgrade operation system carry out using；
Operation system：Call gray scale center switch interface, judge active user be using source database data or
The data of new database.
Fig. 4 is the schematic diagram of the detailed process of the method for Data Migration according to an embodiment of the present invention.As shown in figure 4, whole
Volume data migration process is made of four sub-processes.The respectively synchronous process of full dose, increment synchronization process, gray scale central flow with
And operation system process.
1) process of full dose synchronization module is as follows：
Starting displacement is obtained from migration database of record first.It such as originates displacement to be not present in migration database of record, rise
Begin to be displaced is 0.
The beginning is removed from start bit, obtains batch batchSize data (in the embodiment of the present invention, in batches from source database
BatchSize can customize configuration, such as can be 1000, represent every batch of conversion, synchronous data are 1000), it carries out new
The conversion of male cousin's structure.In the data insertion new database converted.
It records this and takes turns recording in data Zi increasing major key ID value to migration for the last item record in the batch, as next time
Starting displacement, i.e., new starting displacement ID=epicycle lot data the last item data from increasing major key ID.It should be noted that
It only synchronizes and successfully just will be updated new starting displacement.Failure then originates displacement and last round of identical, and it is same to repeat last round of data
Step.Judge whether to synchronize at just ACK confirmation carries out next batch processing for success, and failure is then using ACK mechanism in the embodiment of the present invention
Retry mechanism is carried out without ACK.
All data of minute book wheel batch Zi increase major key ID to migration database of record, as gray scale center calculation migration
The basic data of rate.
In the embodiment of the present invention, because being by database from increasing major key progress sequence synchronization, Gu there is no transition process
The omission of middle data is lost.
2) increment synchronization block process is as follows：
The binlog for monitoring source database triggers increment synchronization module when source database has data change in real time.It uses
Binlog mechanism based on master-slave synchronisation can achieve the effect of data change real-time synchronization.
Show that the data changed, increment synchronization module judge the increasing major key ID certainly of the change data for binlog
With the presence or absence of in migration database of record.More new data does not update if being not present into new database if existing.Pass through this
Judgement only retains and effectively updates so as to filter invalid update, improves performance.
3) gray scale central flow is as follows：
Gray scale center carries out mobility calculating by user's dimension of business, and mobility is shown on the interface of foreground, with
Just real time inspection current system mobility controls migration progress.Here user's dimension refers to the unique identification of user, such as
The User ID that can be in e-commerce field can be the pet name etc. of user in the instant communications field.The embodiment of the present invention
In, the data for belonging to same user in source database are associated storage, such as electronics quotient by the unique identification of user
In business field, it may include that user browses commodity SKU, user adds shopping cart information, user and places an order in the data of same user
Information etc. when each information among these is as a data record storage, can all correspond to one from major key ID is increased, together with this
When, above-mentioned data store when all with its belonging to User ID be associated.By the association, so as to which which is easily identified
Data belong to same user.
In the embodiment of the present invention, the calculation formula of mobility is as follows：
Mobility=user is in migration database of record synchronized data item number/user's number of data all in legacy system
× 100%
In operation system after needing to move to certain user upgrading, i.e., when being switched to new database for database,
Only migration switch need to be opened at gray scale center.But if mobility is not equal to 100%, migration switch can not be opened.
As long as meaning that the data of the user are fully synchronized in this way, migration switch can be opened.When user moves to new database
Afterwards, because mobility has reached 100%, therefore user's unaware postpones to loss of data and data, the loss of no user experience.
4) operation system process is as follows：
Before the interface of the whole calling service new business system of operation system in practical applications, need first to open full dose
Synchronization module and increment synchronization module realize that data are synchronous.Meanwhile migration database of record is written into synchronous recording related data
In and be reported to gray scale center.
Administration interface at gray scale center can check the migration progress under active user's dimension.As no user migrates into
Degree reaches 100% and continues waiting for.Otherwise, the user that synchronous progress is 100% is chosen in batches and opens migration switch.
The migration switch at operation system real-time calling gray scale center carries out routing judgement, when the migration for judging certain user switchs
It has been switched on, then using the operation system after upgrading, otherwise using the operation system before upgrading.
In the embodiment of the present invention, the grade realized using binlog is synchronous and opens preceding 100% progress of migration switch
Strong verification, can achieve the beneficial effect of user's unaware.
Technical solution according to an embodiment of the present invention, by the process of the above method, so as to realize number in database
According to seamless migration, guarantee that corresponding operation system is not withdrawn, and no data lose and delay, and can accomplish switch user without
Perception, promotes the user experience of operation system；Pass through the design structure by the data in source database according to table in new database
Table structure conversion is carried out, so as to realize the Data Migration between heterogeneous database；Since the full dose synchronizing process is according to number
Sequence synchronization is carried out from major key is increased according to library, so there is no the loss of data；By ACK mechanism, so as to further
Guarantee data without loss；By recording the increasing major key ID certainly of all data of having moved, so as to easily be based on user's dimension
Computation migration rate；It is displaced ID by record start, so as to facilitate the data in accurate sequential selection source database；Pass through
Reach data change based on binlog mechanism so as to monitor the variation of data in source database during data are synchronous
The effect of more real-time synchronization；It whether there is by the major key ID that increases certainly for the data for judging to change in migration database of record
In, so as to filter invalid update, only retain and effectively update, improves net synchronization capability；By the visualization interface, so as to
With the Data Migration rate of real time inspection current business system, migration progress is controlled.
Fig. 5 is the schematic diagram of the main modular of the device of Data Migration according to an embodiment of the present invention.As shown in figure 5, this
A kind of device 500 of Data Migration of inventive embodiments mainly includes following module：Full dose synchronization module 501, increment synchronization mould
Block 402 and switching module 503.
Wherein, full dose synchronization module 501, for the data full dose in source database to be synchronized to new database；Increment is same
Module 502 is walked, for the alteration of data in the source database being monitored, if described during the full dose is synchronous
Data in source database change, then have realized that synchronous data are updated for corresponding in the new database；It cuts
Block 503 is changed the mold, is used for according to user's dimension computation migration rate, it, will be described when the mobility of the data of certain user reaches 100%
The corresponding database of certain user switches to new database from source database.
Wherein, full dose synchronization module 501 can also be used in：According to the design structure of table in the new database, by source data
Data in library carry out structure conversion.Also, also increase major key ID using oneself, the default item in source database described in sequential selection
Several data, are synchronized to new database, until it is synchronous to complete full dose.
In addition, full dose synchronization module 501 can also be used in：The source data of each sequential selection is judged using ACK mechanism
Whether the data of the default number of branches in library are completed synchronization.
In the embodiment of the present invention, device 500 may also include：Logging modle (not shown) is migrated, being used for will be described pre-
If the data of item number are all recorded from increasing major key ID into migration database of record, and most by the data of the default number of branches
Being recorded in from increasing major key ID as the starting displacement ID of next sequential selection data for latter data migrates in database of record.
In addition, increment synchronization module 502 can also be used in：Based on binlog mechanism, data in the source database are monitored
Alteration.And, it may also be used for the data that judgement is changed whether there is from increasing major key ID in migration database of record
In, and if it exists, then corresponding data in the new database are updated, are not updated then if it does not exist.
In addition, device 500 may also include：Visualization model (not shown) is supervised in real time for utilizing visualization interface
Control Data Migration progress.
From the above, it can be seen that the main modular of the device of Data Migration through the invention, so as to realize number
According to the seamless migration of data in library, guarantee that corresponding operation system is not withdrawn, and no data loses and delay, and can accomplish to cut
It uses family unaware instead, promotes the user experience of operation system；By by the data in source database according to table in new database
Design structure carries out table structure conversion, so as to realize the Data Migration between heterogeneous database；Since the full dose synchronized
Journey carries out sequence synchronization from major key is increased according to database, so there is no the loss of data；By ACK mechanism, so as to
Further guarantee data without loss；By recording the increasing major key ID certainly of all data of having moved, so as to easily be based on
User's dimension computation migration rate；It is displaced ID by record start, so as to facilitate in accurate sequential selection source database
Data；By being based on binlog mechanism, so as to monitor the variation of data in source database during data are synchronous,
Achieve the effect that data change real-time synchronization；Remembered by whether there is from increasing major key ID for data for judging to change in migration
It records in database, so as to filter invalid update, only retains and effectively update, improve net synchronization capability；Pass through visualization circle
Migration progress is controlled so as to the Data Migration rate of real time inspection current business system in face.
Below with reference to Fig. 6, it illustrates the departments of computer science for the electronic device terminal for being suitable for being used to realize the embodiment of the present application
The structural schematic diagram of system 600.Electronic device terminal shown in Fig. 6 is only an example, should not be to the function of the embodiment of the present application
Any restrictions can be brought with use scope.
As shown in fig. 6, computer system 600 includes central processing unit (CPU) 601, it can be read-only according to being stored in
Program in memory (ROM) 602 or be loaded into the program in random access storage device (RAM) 603 from storage section 608 and
Execute various movements appropriate and processing.In RAM 603, also it is stored with system 600 and operates required various programs and data.
CPU 601, ROM 602 and RAM 603 are connected with each other by bus 604.Input/output (I/O) interface 605 is also connected to always
Line 604.
I/O interface 605 is connected to lower component：Importation 606 including keyboard, mouse etc.；It is penetrated including such as cathode
The output par, c 607 of spool (CRT), liquid crystal display (LCD) etc. and loudspeaker etc.；Storage section 608 including hard disk etc.；
And the communications portion 609 of the network interface card including LAN card, modem etc..Communications portion 609 via such as because
The network of spy's net executes communication process.Driver 610 is also connected to I/O interface 605 as needed.Detachable media 611, such as
Disk, CD, magneto-optic disk, semiconductor memory etc. are mounted on as needed on driver 610, in order to read from thereon
Computer program be mounted into storage section 608 as needed.
Particularly, disclosed embodiment, the process described above with reference to flow chart may be implemented as counting according to the present invention
Calculation machine software program.For example, embodiment disclosed by the invention includes a kind of computer program product comprising be carried on computer
Computer program on readable medium, the computer program include the program code for method shown in execution flow chart.?
In such embodiment, which can be downloaded and installed from network by communications portion 609, and/or from can
Medium 611 is dismantled to be mounted.When the computer program is executed by central processing unit (CPU) 601, the system that executes the application
The above-mentioned function of middle restriction.
It should be noted that computer-readable medium shown in the application can be computer-readable signal media or meter
Calculation machine readable storage medium storing program for executing either the two any combination.Computer readable storage medium for example can be --- but not
Be limited to --- electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor system, device or device, or any above combination.Meter
The more specific example of calculation machine readable storage medium storing program for executing can include but is not limited to：Electrical connection with one or more conducting wires, just
Taking formula computer disk, hard disk, random access storage device (RAM), read-only memory (ROM), erasable type may be programmed read-only storage
Device (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), light storage device, magnetic memory device,
Or above-mentioned any appropriate combination.In this application, computer readable storage medium can be it is any include or storage journey
The tangible medium of sequence, the program can be commanded execution system, device or device use or in connection.And at this
In application, computer-readable signal media may include in a base band or as carrier wave a part propagate data-signal,
Wherein carry computer-readable program code.The data-signal of this propagation can take various forms, including but unlimited
In electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media can also be that computer can
Any computer-readable medium other than storage medium is read, which can send, propagates or transmit and be used for
By the use of instruction execution system, device or device or program in connection.Include on computer-readable medium
Program code can transmit with any suitable medium, including but not limited to：Wirelessly, electric wire, optical cable, RF etc. or above-mentioned
Any appropriate combination.
Flow chart and block diagram in attached drawing are illustrated according to the system of the various embodiments of the application, method and computer journey
The architecture, function and operation in the cards of sequence product.In this regard, each box in flowchart or block diagram can generation
A part of one module, program segment or code of table, a part of above-mentioned module, program segment or code include one or more
Executable instruction for implementing the specified logical function.It should also be noted that in some implementations as replacements, institute in box
The function of mark can also occur in a different order than that indicated in the drawings.For example, two boxes succeedingly indicated are practical
On can be basically executed in parallel, they can also be executed in the opposite order sometimes, and this depends on the function involved.Also it wants
It is noted that the combination of each box in block diagram or flow chart and the box in block diagram or flow chart, can use and execute rule
The dedicated hardware based systems of fixed functions or operations is realized, or can use the group of specialized hardware and computer instruction
It closes to realize.
Being described in module involved in the embodiment of the present application can be realized by way of software, can also be by hard
The mode of part is realized.Described module also can be set in the processor, for example, can be described as：A kind of processor packet
Include full dose synchronization module, increment synchronization module and switching module.Wherein, the title of these modules not structure under certain conditions
The restriction of the pairs of module itself, for example, full dose synchronization module is also described as " for the data in source database are complete
Amount is synchronized to the module of new database ".
As on the other hand, present invention also provides a kind of computer-readable medium, which be can be
Included in electronic device terminal described in above-described embodiment；It is also possible to individualism, and is set without the electronics is incorporated
In standby terminal.Above-mentioned computer-readable medium carries one or more program, when said one or multiple programs are by one
When a electronic device terminal executes, so that the electronic device terminal includes：Full dose synchronizes process, and the synchronous process of full dose includes such as
Lower step：Data full dose in source database is synchronized to new database；During the full dose is synchronous, the source is monitored
The alteration of data in database will be corresponding in the new database if the data in the source database change
Realized that synchronous data are updated；According to user's dimension computation migration rate, when the mobility of the data of certain user reaches
When 100%, the corresponding database of certain described user is switched into new database from source database.
Technical solution according to an embodiment of the present invention, it can be achieved that database data the migration without loss, and improve
The performance of migration guarantees that corresponding operation system is not withdrawn, and no data loses and delay, and can accomplish that switching user is noninductive
Know, promotes the user experience of operation system；By by the data in source database according to the design structure of table in new database into
Row table structure conversion, so as to realize the Data Migration between heterogeneous database；Since the full dose synchronizing process is according to data
Library carries out sequence synchronization from major key is increased, so there is no the loss of data；By ACK mechanism, so as to further protect
Data are demonstrate,proved without loss；By record all data of having moved from major key ID is increased, so as to easily based on user's dimension
Calculate mobility；It is displaced ID by record start, so as to facilitate the data in accurate sequential selection source database；Pass through base
Reach data change in binlog mechanism so as to monitor the variation of data in source database during data are synchronous
The effect of real-time synchronization；It whether there is in migration database of record by the major key ID that increases certainly for the data for judging to change,
So as to filter invalid update, only retain and effectively update, improves net synchronization capability；By the visualization interface, so as to
The Data Migration rate of real time inspection current business system controls migration progress.
Method provided by the embodiment of the present invention can be performed in the said goods, has the corresponding functional module of execution method and has
Beneficial effect.The not technical detail of detailed description in the present embodiment, reference can be made to method provided by the embodiment of the present invention.
Although the embodiment of the present invention is described in detail in conjunction with attached drawing, embodiment is only used for the explanation and illustration present invention,
Rather than for limiting the present invention.The scope of the present invention is defined by the claims, change, replacement of some of them element etc.
It is obvious.
Claims (18)
1. a kind of method of Data Migration, which is characterized in that including：
Data full dose in source database is synchronized to new database；
During the full dose is synchronous, the alteration of data in the source database is monitored, if in the source database
Data change, then realized that synchronous data are updated for corresponding in the new database；
According to user's dimension computation migration rate, when the mobility of the data of certain user reaches 100%, certain described user is corresponded to
Database switch to new database from source database.
2. the method according to claim 1, wherein the data full dose in source database is synchronized to new data
Before library, the method also includes：According to the design structure of table in the new database, the data in source database are tied
Structure conversion.
3. the method according to claim 1, wherein the data full dose in source database is synchronized to new database
Including：Using from major key ID is increased, the data of the default number of branches in source database described in sequential selection are synchronized to new database, directly
It is synchronous to full dose is completed.
4. according to the method described in claim 3, it is characterized in that, the method also includes：Judged using ACK mechanism suitable every time
Whether the data of the default number of branches in the source database of sequence selection are completed synchronization.
5. according to the method described in claim 3, it is characterized in that, pre- in the source database by each sequential selection
If the data of item number are completed after synchronizing, the method also includes：By the data of the default number of branches all from increasing major key ID
Record increases major key ID as under into migration database of record, and using the last item data of the data of the default number of branches certainly
The starting displacement ID of secondary sequential selection data is recorded in migration database of record.
6. the method according to claim 1, wherein monitoring the alteration packet of data in the source database
It includes：Based on binlog mechanism, the alteration of data in the source database is monitored.
7. according to the method described in claim 5, it is characterized in that, having realized synchronous number for corresponding in the new database
According to be updated including：Whether there is in migration database of record from increasing major key ID for the data changed is judged, if depositing
Then corresponding data in the new database are being updated, are not being updated then if it does not exist.
8. the method according to claim 1, wherein the method also includes：Using visualization interface, supervise in real time
Control Data Migration progress.
9. a kind of device of Data Migration, which is characterized in that including：
Full dose synchronization module, for the data full dose in source database to be synchronized to new database；
Increment synchronization module, for monitoring the alteration of data in the source database during full dose is synchronous,
If the data in the source database change, realize that synchronous data carry out more for corresponding in the new database
Newly；
Switching module is used for according to user's dimension computation migration rate, will when the mobility of the data of certain user reaches 100%
The corresponding database of described certain user switches to new database from source database.
10. device according to claim 9, which is characterized in that the full dose synchronization module is also used to：According to the new number
According to the design structure of table in library, the data in source database are subjected to structure conversion.
11. device according to claim 9, which is characterized in that the full dose synchronization module is also used to：Utilize increasing major key certainly
ID, the data of the default number of branches in source database described in sequential selection, is synchronized to new database, until it is synchronous to complete full dose.
12. device according to claim 11, which is characterized in that the full dose synchronization module is also used to：Utilize ACK mechanism
Judge whether the data of the default number of branches in the source database of each sequential selection are completed synchronization.
13. device according to claim 11, which is characterized in that described device further includes：Logging modle is migrated, being used for will
The data of the default number of branches are all recorded into migration database of record from increasing major key ID, and by the number of the default number of branches
According to the last item data record number from increasing starting displacement ID of the major key ID as next sequential selection data and be recorded in migration
According in library.
14. device according to claim 9, which is characterized in that the increment synchronization module is also used to：Based on binlog machine
System, monitors the alteration of data in the source database.
15. device according to claim 13, which is characterized in that the increment synchronization module is also used to：Judgement becomes
Data more whether there is in migration database of record from increasing major key ID, and if it exists, then to corresponding in the new database
Data be updated, do not update then if it does not exist.
16. device according to claim 9, which is characterized in that described device further includes：Visualization model, for utilizing
Visualization interface, real-time monitoring data migrate progress.
17. a kind of electronic equipment terminal, which is characterized in that including：
One or more processors；
Storage device, for storing one or more programs,
When one or more of programs are executed by one or more of processors, so that one or more of processors are real
Now such as method described in any one of claims 1-8.
18. a kind of computer-readable medium, is stored thereon with computer program, which is characterized in that described program is held by processor
Such as method described in any one of claims 1-8 is realized when row.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710334373.6A CN108874828A (en) | 2017-05-12 | 2017-05-12 | The method and apparatus of Data Migration |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710334373.6A CN108874828A (en) | 2017-05-12 | 2017-05-12 | The method and apparatus of Data Migration |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108874828A true CN108874828A (en) | 2018-11-23 |
Family
ID=64320053
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710334373.6A Pending CN108874828A (en) | 2017-05-12 | 2017-05-12 | The method and apparatus of Data Migration |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108874828A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111291023A (en) * | 2020-02-09 | 2020-06-16 | 苏州浪潮智能科技有限公司 | Data migration method, system, device and medium |
| CN111324592A (en) * | 2020-01-21 | 2020-06-23 | 腾讯科技（深圳）有限公司 | Application data migration method and device |
| CN111367892A (en) * | 2020-03-31 | 2020-07-03 | 中国建设银行股份有限公司 | Data migration method and device |
| CN111666286A (en) * | 2020-05-21 | 2020-09-15 | 微民保险代理有限公司 | Method and device for detecting sub-warehouse and sub-table, computer equipment and storage medium |
| CN113190531A (en) * | 2021-05-10 | 2021-07-30 | 挂号网（杭州）科技有限公司 | Database migration method, device, equipment and storage medium |
| CN113468150A (en) * | 2021-09-03 | 2021-10-01 | 四川新网银行股份有限公司 | Horizontal segmentation capacity expansion and migration method for payment subscription data |
| CN113485980A (en) * | 2021-07-27 | 2021-10-08 | 拉扎斯网络科技（上海）有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN113704270A (en) * | 2021-09-03 | 2021-11-26 | 携程金融科技(上海)有限公司 | Method, system, equipment and medium for expanding capacity of self-increment key of SQL Server database |
| CN113779000A (en) * | 2021-11-15 | 2021-12-10 | 北京金堤科技有限公司 | Database updating method and system, storage medium and electronic device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102096684A (en) * | 2009-12-11 | 2011-06-15 | 华大天元(北京)电力科技有限公司 | Grid real-time data integrating and sharing platform |
| CN103729204A (en) * | 2012-10-16 | 2014-04-16 | 阿里巴巴集团控股有限公司 | Data on-line migration method and device for updating applications in network platform |
| CN104486373A (en) * | 2014-11-21 | 2015-04-01 | 华为技术有限公司 | Lock resource migration method, nodes and distributed system |
| CN105718570A (en) * | 2016-01-20 | 2016-06-29 | 北京京东尚科信息技术有限公司 | Data migration method and device used for database |
-
2017
- 2017-05-12 CN CN201710334373.6A patent/CN108874828A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102096684A (en) * | 2009-12-11 | 2011-06-15 | 华大天元(北京)电力科技有限公司 | Grid real-time data integrating and sharing platform |
| CN103729204A (en) * | 2012-10-16 | 2014-04-16 | 阿里巴巴集团控股有限公司 | Data on-line migration method and device for updating applications in network platform |
| CN104486373A (en) * | 2014-11-21 | 2015-04-01 | 华为技术有限公司 | Lock resource migration method, nodes and distributed system |
| CN105718570A (en) * | 2016-01-20 | 2016-06-29 | 北京京东尚科信息技术有限公司 | Data migration method and device used for database |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111324592B (en) * | 2020-01-21 | 2022-02-25 | 腾讯科技（深圳）有限公司 | Application data migration method and device |
| CN111324592A (en) * | 2020-01-21 | 2020-06-23 | 腾讯科技（深圳）有限公司 | Application data migration method and device |
| CN111291023A (en) * | 2020-02-09 | 2020-06-16 | 苏州浪潮智能科技有限公司 | Data migration method, system, device and medium |
| CN111367892A (en) * | 2020-03-31 | 2020-07-03 | 中国建设银行股份有限公司 | Data migration method and device |
| CN111367892B (en) * | 2020-03-31 | 2024-01-19 | 中国建设银行股份有限公司 | Data migration method and device |
| CN111666286A (en) * | 2020-05-21 | 2020-09-15 | 微民保险代理有限公司 | Method and device for detecting sub-warehouse and sub-table, computer equipment and storage medium |
| CN111666286B (en) * | 2020-05-21 | 2023-06-30 | 微民保险代理有限公司 | Method, device, computer equipment and storage medium for detecting database and table |
| CN113190531A (en) * | 2021-05-10 | 2021-07-30 | 挂号网（杭州）科技有限公司 | Database migration method, device, equipment and storage medium |
| CN113485980A (en) * | 2021-07-27 | 2021-10-08 | 拉扎斯网络科技（上海）有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN113704270A (en) * | 2021-09-03 | 2021-11-26 | 携程金融科技(上海)有限公司 | Method, system, equipment and medium for expanding capacity of self-increment key of SQL Server database |
| CN113468150A (en) * | 2021-09-03 | 2021-10-01 | 四川新网银行股份有限公司 | Horizontal segmentation capacity expansion and migration method for payment subscription data |
| CN113704270B (en) * | 2021-09-03 | 2023-10-17 | 携程金融科技(上海)有限公司 | Automajor key capacity expansion method of SQL Server database and related equipment |
| CN113779000A (en) * | 2021-11-15 | 2021-12-10 | 北京金堤科技有限公司 | Database updating method and system, storage medium and electronic device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108874828A (en) | The method and apparatus of Data Migration | |
| KR101750429B1 (en) | Document editing system and method for collaborative editing | |
| CN110083660A (en) | A kind of method, apparatus of synchrodata, medium and electronic equipment | |
| CN109413127A (en) | A kind of method of data synchronization and device | |
| CN108965355A (en) | Method, apparatus and computer readable storage medium for data transmission | |
| CN109905286A (en) | A kind of method and system of monitoring device operating status | |
| CN109144785A (en) | Method and apparatus for Backup Data | |
| CN105847336A (en) | Address book synchronization method and device | |
| CN110119386A (en) | Data processing method, data processing equipment, medium and calculating equipment | |
| CN110019263A (en) | Information storage means and device | |
| CN109492200A (en) | Cooperated documents restoring method, device, storage medium and electronic equipment | |
| CN110019539A (en) | A kind of method and apparatus that the data of data warehouse are synchronous | |
| CN109597699A (en) | Data processing method, device, electronic equipment and computer-readable medium | |
| CN110019346A (en) | A kind of data processing method and device based on double primary databases | |
| CN110399253A (en) | Delay machine treating method and apparatus | |
| CN109885593A (en) | Method and apparatus for handling information | |
| CN109165261A (en) | Method of data synchronization, device, server and computer storage medium | |
| CN109976919A (en) | A kind of transmission method and device of message request | |
| CN109960212A (en) | Task sending method and device | |
| CN113076186B (en) | Task processing method, device, electronic equipment and storage medium | |
| CN110188969A (en) | A kind of method and apparatus that control task issues | |
| CN107526838A (en) | The method and apparatus of data-base cluster dilatation | |
| CN110443034A (en) | The recognition methods of risk program file, calculates equipment and medium at device | |
| CN114697285B (en) | Information display and information transmission methods, devices, equipment and media | |
| CN111984686A (en) | Data processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181123 |
|
| RJ01 | Rejection of invention patent application after publication |