CN106909595A - A kind of data migration method and device - Google Patents
A kind of data migration method and device Download PDFInfo
- Publication number
- CN106909595A CN106909595A CN201610445610.1A CN201610445610A CN106909595A CN 106909595 A CN106909595 A CN 106909595A CN 201610445610 A CN201610445610 A CN 201610445610A CN 106909595 A CN106909595 A CN 106909595A
- Authority
- CN
- China
- Prior art keywords
- file
- migrated
- database
- data
- identification information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/214—Database migration support
Abstract
The application is related to database technical field, more particularly to a kind of data migration method and device, is used to solve the problems, such as that storage in the prior art for binary file and access have impact on the data transmission bauds and stability of database.The embodiment of the present application provides a kind of data migration method to be included：Data record to be migrated in based on the first database, determines the first file to be migrated；By the described first file storage to be migrated in the file system for setting, and by the identification information storage of the described first file to be migrated in the second database；Wherein, the identification information of the described first file to be migrated is the information for identifying storage location of described first file to be migrated in the file system.
Description
Technical field
The application is related to database technical field, more particularly to a kind of data migration method and device.
Background technology
Computer documents is essentially divided into two kinds：Binary file and ASCII (also referred to as plain text text files), figure text
The computer program such as part and word processing program belongs to binary file.These files contain special form and computer generation
Code.ASCII can be the simple text file read with any word processing program.
In some cases, such as when the business of carrying out fractionation, current database can not meet binary file (binary system text
The memory space that part is typically constituted from is larger) storage demand etc. when, we may require that current database (hereinafter referred to as old storehouse)
In Data Migration in another database (hereinafter referred to as new storehouse), and after Data Migration is completed, by business on line from old
Storehouse is moved in new storehouse.During Data Migration is carried out, still use old storehouse on line business data, services are provided.
During Data Migration is carried out, having new data needs to be stored.In order to not influence business on line
It is normally carried out, the general migration that new data is carried out using synchronous double mechanism write.Specifically, new data are first written to old storehouse
Pending queue in, then by pending queue data syn-chronization write-in Jiu Ku and new storehouse in, and by old storehouse from the beginning of
Perform Data Migration before the synchronous double time points write to new storehouse.
At present, for binary file data transmission bauds and stability of the storage and access as influence database
Bottleneck.When Data Migration is carried out, if still performing the migration of binary file using existing mode, it will have a strong impact on data and move
The data access performance moved after efficiency, and migration.
The content of the invention
The embodiment of the present application provides a kind of data migration method and device, is used to solve in the prior art for binary system text
The storage and access of part have impact on the data transmission bauds of database and the problem of stability.
The embodiment of the present application provides a kind of data migration method, including：
Data record to be migrated in based on the first database, determines the first file to be migrated；
By the described first file to be migrated storage in the file system for setting, and by the mark of the described first file to be migrated
Knowledge information Store is in the second database；
Wherein, the identification information of the described first file to be migrated is for identifying the described first file to be migrated in the text
The information of the storage location in part system.
Alternatively, the file system is distributed file system.
Alternatively, it is in the file system for setting and to be migrated by described first by the described first file storage to be migrated
After the identification information storage of file is in the second database, also include：
When receive client for the described first file to be migrated access request when, based in second database
The identification information of first file to be migrated, obtains the described first file to be migrated stored in the file system, and return
Back to the client.
Alternatively, based on the first database in data record to be migrated, determine the first file to be migrated, including：
From the data record to be migrated, binary file is extracted；
Both text text files being converted to by the binary file extracted and based on the binary file
In at least one be defined as the described first file to be migrated.
Alternatively, if first file to be migrated includes the binary file, by binary file storage
In the file system for setting, and by the identification information storage of the binary file in the second database after, also include：
When receive client for the binary file download request when, described in second database
The identification information of binary file, obtains the binary file, and the binary system that will be obtained from the file system
File cache is in content distributing network CDN；
Reference address of the binary file in the CDN is returned to the client.
Alternatively, if the data record to be migrated is the new data in the queue to be stored of first database
Record, then methods described also includes：
The text files that will be converted to based on the binary file are stored in first database.
Alternatively, methods described also includes：
From the data record to be migrated, the second file to be migrated is extracted, second file to be migrated enters for non-two
File processed；
If the data record to be migrated is the new data records in the queue to be stored of first database,
By the described second file storage to be migrated in first database and the second database；
If the data record to be migrated is the historgraphic data recording stored in list positioned at first database,
Then by the described second file storage to be migrated in second database.
Alternatively, methods described also includes：
The identification information for having stored each bar historgraphic data recording in list of first database is write into present count
According in table；Wherein, every identification information of historgraphic data recording is used to identify this historgraphic data recording in first data
Storage location in storehouse；
Based on the identification information of each bar historgraphic data recording in the preset data table, respectively from first database
Every untreated historgraphic data recording is extracted as the data record to be migrated, and record should in the preset data table
The treatment state of bar historgraphic data recording.
The embodiment of the present application provides a kind of data migration device, including：
Determining module, for the data record to be migrated in based on the first database, determines the first file to be migrated；
Memory module, for by the described first file to be migrated storage in the file system for setting, and by described first
The identification information storage of file to be migrated is in the second database；
Wherein, the identification information of the described first file to be migrated is for identifying the described first file to be migrated in the text
The information of the storage location in part system.
The embodiment of the present application determines first when needing to carry out moving storehouse from the data record to be migrated in the first database
File to be migrated, by first file to be migrated storage in the file system for setting, and by the mark of first file to be migrated
Knowledge information Store is in the second database.So, believe due to only storing the mark of first file to be migrated in new database
Breath, therefore the efficiency to migrating data in new database can be greatly enhanced, in addition, this document system can be specifically distribution
Formula file system, because distributed file system can be the outside storage access that provides highly reliable and high concurrent, therefore can be with
Raising carries out the data access performance after Data Migration.So, the embodiment of the present application can be improved to the first file to be migrated, than
Storage and access ability such as binary file.
In addition, the application preferred embodiment in, carried out when being stored to binary file stylistic
Conversion, that is, extract text text contents therein and stored, after so being changed, receive client for this two
After the accessing page request of binary file, being based on the text text contents of the binary file, quickly to carry out page rendering anti-
Feedback, the mode of row format conversion is not entered compared to existing technology, improves response speed.
Brief description of the drawings
Fig. 1 carries out the schematic diagram of Data Migration for the embodiment of the present application；
Fig. 2 is the data migration method flow chart that the embodiment of the present application one is provided；
Fig. 3 is the data migration method flow chart that the embodiment of the present application two is provided；
Fig. 4 is the data migration method flow chart that the embodiment of the present application three is provided；
The data migration device structural representation that Fig. 5 is provided for the embodiment of the present application.
Specific embodiment
As shown in figure 1, carrying out the schematic diagram of Data Migration for the embodiment of the present application.In the embodiment of the present application, for true
Surely carry out moving the new data (after receiving new data, first in the pending queue of write-in old database) received behind storehouse, to it
In binary file and nonbinary file carry out performing after different treatment synchronous double writing respectively and (namely be respectively written into new number
According to storehouse and old database).Wherein, when binary file is write in new database, by the first file to be migrated, (including two enter
File processed and/or the text text files being converted to based on the binary file) store in distributed file system, and will
The identification information storage of storage location of first file to be migrated is identified in new database.For storing in old database
Historical data, be similarly divided into binary file and nonbinary file, and deposit after carrying out different treatment respectively
Store up new database.
The embodiment of the present application is described in further detail with reference to Figure of description.
Embodiment one
As shown in Fig. 2 being the data migration method flow chart of the offer of the embodiment of the present application one, comprise the following steps：
S201：Data record to be migrated in based on the first database, determines the first file to be migrated.
Here, there may be a plurality of data record to be migrated in the first database, for each of which data to be migrated
Record, determines the therein first file to be migrated.
In the embodiment of the present application, it is contemplated that the memory space that binary file takes is larger, and in data record to be migrated
Nonbinary file, namely text files take memory space it is all smaller, therefore can only to binary file therein
Perform the treatment of step S202.
Specifically, binary file can be extracted from data record to be migrated；The binary file and base that will be extracted
At least one in both text text files that the binary file is converted to is defined as the described first file to be migrated.
S202：By the described first file to be migrated storage in the file system for setting, and by the described first text to be migrated
The identification information storage of part is in the second database；Wherein, the identification information of the described first file to be migrated is for identifying
State the information of storage location of first file to be migrated in the file system.
Binary file in the embodiment of the present application includes graphic file and word processing program etc., is non-textual text texts
Part.Because the memory space that binary file takes is larger, therefore, it can put forward binary file from data record to be migrated
Take out, its corresponding identification information is only stored in database, and by binary file storage in file system.
Preferably, this document system can refer to distributed file system.Distributed file system (Distributed File
System a Filesystem Network) is constituted by numerous nodes.Distributed file system has high-performance, high reliability and can
The characteristics of autgmentability is strong, can support the access of high concurrent, large buffer memory, big handling capacity, and can have by increasing server
Effect avoids Single Point of Faliure.
S203：When receive client for the described first file to be migrated access request when, based on described second number
According to the identification information of the described in storehouse first file to be migrated, the described first text to be migrated stored in the file system is obtained
Part, and return to the client.
In specific implementation, if client needs to access the first file to be migrated, based on first file to be migrated
Identification information, obtains corresponding first file to be migrated, and return to client from distributed file system.
Embodiment two
It is below binary file with the first file to be migrated, as a example by the second file to be migrated is nonbinary file, makees
It is further described.
As shown in figure 3, being the data migration method flow chart of the offer of the embodiment of the present application two, comprise the following steps：
S301：Binary file in data record to be migrated in first database and nonbinary file are distinguished
Open, for binary file, perform following S302, for nonbinary file, perform following S303.
Here, data record to be migrated can be after it is determined that needing to carry out moving storehouse, to write the first database (old storehouse)
New data (new to receive, be not put in storage formally also) in pending queue, or stored the history in the first database
Data.
For historical data, can be by the mark for having stored each bar historgraphic data recording in the list letter of the first database
(tables of data is the interim table for creating to breath (Identity, ID) write-in preset data table, can be with after historical data migration is completed
Delete) in；Wherein, every identification information of historgraphic data recording is used to identify this historgraphic data recording in first data
In storehouse storage location (identification information can be storage address, or be in the first database record to should bar go through
The mark of history data record)；Based on the identification information of each bar historgraphic data recording in the preset data table, respectively from described
Every untreated historgraphic data recording is extracted in one database as the data record to be migrated, and in the preset data
The treatment state of this historgraphic data recording is recorded in table.Here it is possible to set up timed task, based on preset data table in it is each
The identification information of bar historgraphic data recording, periodically takes out in the first database each untreated (not writing the second database)
Historgraphic data recording processed (namely write-in second database).Record treatment state is marked by the preset data, can be with
Realize not repeating treatment to historgraphic data recording, namely support idempotence.
For each new data records or historgraphic data recording, all by binary file therein (such as PDF
Binary file) and nonbinary file distinguish and be respectively processed.
S302：Text text files are converted to based on the binary file；After binary file and conversion
Text files are stored in distributed file system, and will identify storage position of the binary file in distributed file system
The identification information storage of storage location of the identification information and text files put in distributed file system is in the second data
In storehouse；If data record to be migrated is new data, also the text files after conversion are stored in the first database.
In S302, the text files being converted to by binary file and based on binary file are stored in distribution
In file system, and the identification information (mark letter of storage location of the binary file in distributed file system will be identified
Breath can be storage address, or be in distributed file system record to should binary file mark), Yi Jibiao
Know the identification information storage of storage location of the text files in the second database.
So, (database of data, services from the first database migration to the second data is provided the user completing to move storehouse
Storehouse) after, when receive client for the binary file download request when, can be based on the second database in store should
The identification information of binary file, corresponding binary file is obtained in distributed file system and feeds back to client.In addition,
In order to further improve the speed of download of binary file, in the identification information based on binary file from distributed file system
It is middle obtain corresponding binary file after, the binary file of acquisition can be buffered in content distributing network (Content
Delivery Network, CDN) in, return to reference address of the binary file in the CDN to client.
Here, downloaded for binary file, after being cached using CDN, return to the client binary file in CDN
In corresponding reference address.CDN system can in real time according to network traffics, and each node connection, load state, and arrive
The integrated information such as the distance of client and response time, the service nearest from client is re-directed to by the access request of client
On node.So, client can nearby obtain access content, it is to avoid the situation of network congestion, improve user and access website
Response speed.
In addition, after completing to move storehouse, if after receiving client for the accessing page request of the binary file, can
With the identification information based on the text files, corresponding text files are obtained in distributed file system, and based on this
Text files carry out rendering feedback to accession page.
If in addition, the data record to be migrated be new data, due to binary file take memory space it is generally large,
The embodiment of the present application does not store binary file in the first database, namely binary system text is abandoned in the first database
Part, but in order to not influence being normally carried out for business on line, the text contents that will be extracted from binary file are stored first
In database (namely synchronizing double writing).So, it is literary for the binary system when client is received before completing to move storehouse
After the accessing page request of part, the text files that can be based in the first database carry out rendering feedback to accession page.
In specific implementation, when file of the binary file for PDF, it is possible to use Pdfbox instruments, from PDF texts
Text contents are extracted in part.
The embodiment of the present application has carried out stylistic conversion when being stored to binary file, that is, extract text therein
This text contents are stored, and after so being changed, are receiving access request of the client for the binary file
Afterwards, content that can be quickly to the binary file carries out rendering feedback, does not enter the mode of row format conversion compared to existing technology,
Improve response speed.
S303：If data record to be migrated is the new data records in the queue to be stored of the first database, will
Nonbinary file is stored in the first database and the second database；If data record to be migrated is positioned at the first database
The historgraphic data recording in list is stored, then by the storage of nonbinary file in the second database.
Here, for the nonbinary file in data record to be migrated, performing model conversation treatment (such as carries out data
The fractionation of table) store afterwards in the second database, in the case where data record to be migrated is new data, can also be by nonbinary
File is stored in the first database.
S304：By on line business provide the database of service from the first database migration to the second database.
Specifically, by line business the database interface of service be provided switch from the access interface of first database
To the access interface of second database.
Here, the database for providing clients with data, services is switched to the second database, namely uses the second data
Database service interface provides data, services, but the interface of client is constant.
The embodiment of the present application need carry out moving storehouse when, by the binary file in data record to be migrated and conversion after
Text files are stored in distributed file system, then will be used to identify that the binary file in distributed file system
The identification information of storage location and identify the identification information storage of the storage location of the text files in distributed file system
In new database.So, due to only stored in new database the binary file and conversion after text files mark
Information, can be greatly enhanced the efficiency to migrating data in new database.Further, since distributed file system can be outer
The storage that portion provides highly reliable and high concurrent is accessed, therefore can improve the data access performance after carrying out Data Migration.So,
The embodiment of the present application can improve the storage and access ability to binary file.The embodiment of the present application is entered to binary file
Stylistic conversion is carried out during row storage, that is, has been extracted text text contents therein and is stored, after so being changed,
After accessing page request of the client for the binary file is received, content that can be quickly to the binary file is entered
Row page rendering feeds back, and does not enter the mode of row format conversion compared to existing technology, improves response speed.
Embodiment two
Data Migration schematic diagram shown in Figure 1, the embodiment of the present application two gives a specific implementation process：
For it is determined that the new data received after moving storehouse, performs following S401a~S404a and S405~S407, for
It is determined that storing the historical data in the first database before moving storehouse, following S401b~S404b, and S405~S407 are performed.
S401a：The each new data records that will be received are write in the pending queue of the first database.
S402a：Take out each new data records successively from the pending queue of the first database, it is new for each
Data record, the binary file in this new data records and nonbinary file is distinguished, for binary system therein
File, performs following S403a, for nonbinary file therein, performs following S404a.
S403a：Text files are converted to based on the binary file, the text files and binary file are stored
In distributed file system, and by the text files and the corresponding mark of binary file difference in distributed file system
In storage location identification information storage in the second database；By text file synchronizations storage in the first database
In.
S404a：After the nonbinary file is carried out into model conversation, synchronous storage is in the first database and the second data
In storehouse.
S401b：By in the identification information write-in preset data table of each bar historgraphic data recording in the first database.
S402b：Based on the identification information of each bar historgraphic data recording in preset data table, carried from the first database respectively
Every untreated historgraphic data recording is taken, for each historgraphic data recording for extracting, by this historgraphic data recording
Binary file and nonbinary file distinguish, for binary file therein, following S403b are performed, for wherein
Nonbinary file, perform following S404b.And the treatment state of this historgraphic data recording is recorded in preset data table.
S403b：The text texts in binary file are extracted, by the text texts and binary file storage in distribution
In formula file system, and the mark of the storage location of the text texts and binary file in distributed file system will be identified
Knowledge information Store is in the second database.
S404b：After the nonbinary file is carried out into model conversation, store in the second database.
S405：After above-mentioned Data Migration is completed, by line business the database for servicing be provided moved from the first database
Move on to the second database.
S406：When client is received for the accessing page request for specifying binary file, based on the second database
In the corresponding text files of the specified binary file identification information, obtain the text texts stored in distributed file system
Part, carries out rendering feedback based on the text files to accession page.
S407：When receive client be directed to specify binary file download request when, based in the second database should
The identification information of binary file is specified, the specified binary file stored in distributed file system is obtained, and be buffered in
In CDN, the reference address of corresponding CDN is returned into client.
A kind of data corresponding with data migration method are additionally provided based on same inventive concept, in the embodiment of the present application to move
Moving device, because the principle of the device solve problem is similar to the embodiment of the present application data migration method, therefore the device reality
The implementation of the method for may refer to is applied, part is repeated and is repeated no more.
As shown in figure 5, the data migration device structure chart provided for the embodiment of the present application, including：
Determining module 51, for the data record to be migrated in based on the first database, determines the first file to be migrated；
Memory module 52, for by the described first file to be migrated storage in the file system for setting, and by described the
The identification information storage of one file to be migrated is in the second database；
Wherein, the identification information of the described first file to be migrated is for identifying the described first file to be migrated in the text
The information of the storage location in part system.
Alternatively, described device also includes：
First acquisition module 53, for when receive client for the described first file to be migrated access request when,
Based on the identification information of the first file to be migrated described in second database, what is stored in the acquisition file system is described
First file to be migrated, and return to the client.
Alternatively, the determining module 51 specifically for：
From the data record to be migrated in the first database, binary file is extracted；The binary system text that will be extracted
At least one in both part and the text text files that are converted to based on the binary file is defined as described first and treated
Migrated file.
Alternatively, described device also includes：
Second acquisition module 54, for when receive client for the binary file download request when, be based on
The identification information of binary file, obtains the binary file from the file system described in second database,
And the binary file for obtaining is buffered in content distributing network CDN；The binary system text is returned to the client
Reference address of the part in the CDN.
Alternatively, if the data record to be migrated is the new data in the queue to be stored of first database
Record, then the memory module 52 is additionally operable to：
The text files that will be converted to based on the binary file are stored in first database.
Alternatively, the determining module 51 is additionally operable to：
From the data record to be migrated in the first database, the second file to be migrated, second text to be migrated are extracted
Part is nonbinary file；
The memory module 52 is additionally operable to：If the data record to be migrated is positioned at the to be stored of first database
New data records in queue, then by the described second file storage to be migrated in first database and the second database；
If the data record to be migrated is the historgraphic data recording stored in list positioned at first database, will be described
Second file storage to be migrated is in second database.
Alternatively, described device also includes：
Logging modle 55, for by the mark for having stored each bar historgraphic data recording in list of first database
In information write-in preset data table；Wherein, every identification information of historgraphic data recording is used to identify this historgraphic data recording
Storage location in first database；Based on the identification information of each bar historgraphic data recording in the preset data table,
Every untreated historgraphic data recording is extracted from first database respectively as the data record to be migrated, and
The treatment state of this historgraphic data recording is recorded in the preset data table.
It should be understood by those skilled in the art that, embodiments herein can be provided as method, system or computer program
Product.Therefore, the application can be using the reality in terms of complete hardware embodiment, complete software embodiment or combination software and hardware
Apply the form of example.And, the application can be used and wherein include the computer of computer usable program code at one or more
The computer program implemented in usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.) is produced
The form of product.
The application is the flow with reference to method, device (system) and computer program product according to the embodiment of the present application
Figure and/or block diagram are described.It should be understood that every first-class during flow chart and/or block diagram can be realized by computer program instructions
The combination of flow and/or square frame in journey and/or square frame and flow chart and/or block diagram.These computer programs can be provided
The processor of all-purpose computer, special-purpose computer, Embedded Processor or other programmable data processing devices is instructed to produce
A raw machine so that produced for reality by the instruction of computer or the computing device of other programmable data processing devices
The device of the function of being specified in present one flow of flow chart or multiple one square frame of flow and/or block diagram or multiple square frames.
These computer program instructions may be alternatively stored in can guide computer or other programmable data processing devices with spy
In determining the computer-readable memory that mode works so that instruction of the storage in the computer-readable memory is produced and include finger
Make the manufacture of device, the command device realize in one flow of flow chart or multiple one square frame of flow and/or block diagram or
The function of being specified in multiple square frames.
These computer program instructions can be also loaded into computer or other programmable data processing devices so that in meter
Series of operation steps is performed on calculation machine or other programmable devices to produce computer implemented treatment, so as in computer or
The instruction performed on other programmable devices is provided for realizing in one flow of flow chart or multiple flows and/or block diagram one
The step of function of being specified in individual square frame or multiple square frames.
Although having been described for the preferred embodiment of the application, those skilled in the art once know basic creation
Property concept, then can make other change and modification to these embodiments.So, appended claims are intended to be construed to include excellent
Select embodiment and fall into having altered and changing for the application scope.
Obviously, those skilled in the art can carry out the essence of various changes and modification without deviating from the application to the application
God and scope.So, if these modifications of the application and modification belong to the scope of the application claim and its equivalent technologies
Within, then the application is also intended to comprising these changes and modification.
Claims (15)
1. a kind of data migration method, it is characterised in that the method includes：
Data record to be migrated in based on the first database, determines the first file to be migrated；
By the described first file storage to be migrated in the file system for setting, and the mark of the described first file to be migrated is believed
Breath storage is in the second database；
Wherein, the identification information of the described first file to be migrated is for identifying the described first file to be migrated in the file system
The information of the storage location in system.
2. the method for claim 1, it is characterised in that the file system is distributed file system.
3. the method for claim 1, it is characterised in that by the described first file storage to be migrated in the file system for setting
In system, and by the identification information storage of the described first file to be migrated in the second database after, also include：
When receive client for the described first file to be migrated access request when, described in second database
The identification information of the first file to be migrated, obtains the described first file to be migrated stored in the file system, and return to
The client.
4. the method for claim 1, it is characterised in that the data record to be migrated in based on the first database, it is determined that
First file to be migrated, including：
From the data record to be migrated, binary file is extracted；
In both text text files being converted to by the binary file extracted and based on the binary file
At least one is defined as the described first file to be migrated.
5. method as claimed in claim 4, it is characterised in that if first file to be migrated includes binary system text
Part, then by binary file storage in the file system for setting, and by the identification information storage of the binary file
After in the second database, also include：
When receive client for the binary file download request when, enter based on described in second database two
The identification information of file processed, obtains the binary file, and the binary file that will be obtained from the file system
It is buffered in content distributing network CDN；
Reference address of the binary file in the CDN is returned to the client.
6. method as claimed in claim 4, it is characterised in that if the data record to be migrated is positioned at first data
New data records in the queue to be stored in storehouse, then methods described also include：
The text files that will be converted to based on the binary file are stored in first database.
7. method as claimed in claim 4, it is characterised in that methods described also includes：
From the data record to be migrated, the second file to be migrated is extracted, second file to be migrated is nonbinary text
Part；
If the data record to be migrated is the new data records in the queue to be stored of first database, by institute
The second file to be migrated is stated to store in first database and the second database；
If the data record to be migrated is the historgraphic data recording stored in list positioned at first database, will
The second file storage to be migrated is in second database.
8. the method for claim 1, it is characterised in that methods described also includes：
The identification information for having stored each bar historgraphic data recording in list of first database is write into preset data table
In；Wherein, every identification information of historgraphic data recording is used to identify this historgraphic data recording in first database
Storage location；
Based on the identification information of each bar historgraphic data recording in the preset data table, extracted from first database respectively
Every untreated historgraphic data recording records this and goes through as the data record to be migrated in the preset data table
The treatment state of history data record.
9. a kind of data migration device, it is characterised in that the device includes：
Determining module, for the data record to be migrated in based on the first database, determines the first file to be migrated；
Memory module, for the described first file to be migrated to be stored in the file system for setting, and described first is waited to move
The identification information storage of text move part is in the second database；
Wherein, the identification information of the described first file to be migrated is for identifying the described first file to be migrated in the file system
The information of the storage location in system.
10. device as claimed in claim 9, it is characterised in that described device also includes：
First acquisition module, for when receive client for the described first file to be migrated access request when, based on institute
The identification information of the first file to be migrated described in the second database is stated, obtain storage in the file system described first is treated
Migrated file, and return to the client.
11. devices as claimed in claim 9, it is characterised in that the determining module specifically for：
From the data record to be migrated, binary file is extracted；By the binary file extracted and based on described two
At least one in both text text files that binary file is converted to is defined as the described first file to be migrated.
12. devices as claimed in claim 11, it is characterised in that described device also includes：
Second acquisition module, for when receive client for the binary file download request when, based on described the
The identification information of binary file described in two databases, obtains the binary file, and will obtain from the file system
The binary file for taking is buffered in content distributing network CDN；The binary file is returned in institute to the client
State the reference address in CDN.
13. devices as claimed in claim 11, it is characterised in that if the data record to be migrated is positioned at the described first number
According to the new data records in the queue to be stored in storehouse, then the memory module is additionally operable to：
The text files that will be converted to based on the binary file are stored in first database.
14. devices as claimed in claim 11, it is characterised in that the determining module is additionally operable to：
From the data record to be migrated, the second file to be migrated is extracted, second file to be migrated is nonbinary text
Part；
The memory module is additionally operable to：If the data record to be migrated is in the queue to be stored of first database
New data records, then by the described second file to be migrated storage in first database and the second database；If described
Data record to be migrated is the historgraphic data recording stored in list positioned at first database, then treated described second
Migrated file is stored in second database.
15. devices as claimed in claim 9, it is characterised in that described device also includes：
Logging modle, for the identification information for having stored each bar historgraphic data recording in list of first database to be write
In entering preset data table；Wherein, every identification information of historgraphic data recording is used to identify this historgraphic data recording described
Storage location in first database；Based on the identification information of each bar historgraphic data recording in the preset data table, respectively from
Every untreated historgraphic data recording is extracted in first database as the data record to be migrated, and described pre-
If recording the treatment state of this historgraphic data recording in tables of data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610445610.1A CN106909595B (en) | 2016-06-20 | 2016-06-20 | Data migration method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610445610.1A CN106909595B (en) | 2016-06-20 | 2016-06-20 | Data migration method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106909595A true CN106909595A (en) | 2017-06-30 |
| CN106909595B CN106909595B (en) | 2020-12-29 |
Family
ID=59206721
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610445610.1A Active CN106909595B (en) | 2016-06-20 | 2016-06-20 | Data migration method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106909595B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107704633A (en) * | 2017-11-01 | 2018-02-16 | 郑州云海信息技术有限公司 | A kind of method and system of file migration |
| CN107920136A (en) * | 2017-12-29 | 2018-04-17 | 广东欧珀移动通信有限公司 | data synchronization control method, device and server |
| CN108921065A (en) * | 2018-06-21 | 2018-11-30 | 北京陌上花科技有限公司 | The method and apparatus for establishing property data base |
| CN110019133A (en) * | 2017-12-21 | 2019-07-16 | 北京京东尚科信息技术有限公司 | Online data moving method and device |
| CN111475467A (en) * | 2020-03-27 | 2020-07-31 | 平安科技（深圳）有限公司 | File management method, cloud file management system and terminal |
| CN112527775A (en) * | 2020-12-18 | 2021-03-19 | 福建天晴数码有限公司 | Database expansion method and device based on double writing |
| CN112631991A (en) * | 2020-12-26 | 2021-04-09 | 中国农业银行股份有限公司 | File migration method and device |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080077605A1 (en) * | 2006-09-25 | 2008-03-27 | Microsoft Corporation | Automatic discovery of application settings' storage locations |
| US7620665B1 (en) * | 2000-11-21 | 2009-11-17 | International Business Machines Corporation | Method and system for a generic metadata-based mechanism to migrate relational data between databases |
| US20110060719A1 (en) * | 2009-09-05 | 2011-03-10 | Vivek Kapoor | Method for Transforming Setup Data in Business Applications |
| CN103631907A (en) * | 2013-11-26 | 2014-03-12 | 中国科学院信息工程研究所 | Method and system for migrating relational data to HBbase |
| CN104572842A (en) * | 2014-12-04 | 2015-04-29 | 长沙数析信息科技有限公司 | Striped roll expansion data migration method in distributed file system |
| CN104598459A (en) * | 2013-10-30 | 2015-05-06 | 阿里巴巴集团控股有限公司 | Database processing method and system and data access method and system |
| CN105354250A (en) * | 2015-10-16 | 2016-02-24 | 浪潮(北京)电子信息产业有限公司 | Data storage method and device for cloud storage |
| CN105530273A (en) * | 2014-09-29 | 2016-04-27 | 国际商业机器公司 | Method and device for data migration |
-
2016
- 2016-06-20 CN CN201610445610.1A patent/CN106909595B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7620665B1 (en) * | 2000-11-21 | 2009-11-17 | International Business Machines Corporation | Method and system for a generic metadata-based mechanism to migrate relational data between databases |
| US20080077605A1 (en) * | 2006-09-25 | 2008-03-27 | Microsoft Corporation | Automatic discovery of application settings' storage locations |
| US20110060719A1 (en) * | 2009-09-05 | 2011-03-10 | Vivek Kapoor | Method for Transforming Setup Data in Business Applications |
| CN104598459A (en) * | 2013-10-30 | 2015-05-06 | 阿里巴巴集团控股有限公司 | Database processing method and system and data access method and system |
| CN103631907A (en) * | 2013-11-26 | 2014-03-12 | 中国科学院信息工程研究所 | Method and system for migrating relational data to HBbase |
| CN105530273A (en) * | 2014-09-29 | 2016-04-27 | 国际商业机器公司 | Method and device for data migration |
| CN104572842A (en) * | 2014-12-04 | 2015-04-29 | 长沙数析信息科技有限公司 | Striped roll expansion data migration method in distributed file system |
| CN105354250A (en) * | 2015-10-16 | 2016-02-24 | 浪潮(北京)电子信息产业有限公司 | Data storage method and device for cloud storage |
Non-Patent Citations (4)
| Title |
|---|
| NOREEN NEYSA: "Data migration lessons learned: One facility relates hard-learned best practices after migrating to a new EHR", 《JOURNAL OF AHIMA》 * |
| 万里鹏: "非结构化到结构化数据转换的研究与实现", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
| 于海涛: "基于Java的异构数据库间数据迁移", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
| 姚书怀 等: "大型数据中心海量数据存储解决方案的设计", 《四川工业学院学报》 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107704633A (en) * | 2017-11-01 | 2018-02-16 | 郑州云海信息技术有限公司 | A kind of method and system of file migration |
| CN110019133A (en) * | 2017-12-21 | 2019-07-16 | 北京京东尚科信息技术有限公司 | Online data moving method and device |
| CN110019133B (en) * | 2017-12-21 | 2021-07-13 | 北京京东尚科信息技术有限公司 | Data online migration method and device |
| CN107920136A (en) * | 2017-12-29 | 2018-04-17 | 广东欧珀移动通信有限公司 | data synchronization control method, device and server |
| CN107920136B (en) * | 2017-12-29 | 2019-09-17 | Oppo广东移动通信有限公司 | Data synchronization control method, device and server |
| CN108921065A (en) * | 2018-06-21 | 2018-11-30 | 北京陌上花科技有限公司 | The method and apparatus for establishing property data base |
| CN111475467A (en) * | 2020-03-27 | 2020-07-31 | 平安科技（深圳）有限公司 | File management method, cloud file management system and terminal |
| CN112527775A (en) * | 2020-12-18 | 2021-03-19 | 福建天晴数码有限公司 | Database expansion method and device based on double writing |
| CN112631991A (en) * | 2020-12-26 | 2021-04-09 | 中国农业银行股份有限公司 | File migration method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106909595B (en) | 2020-12-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106909595A (en) | A kind of data migration method and device | |
| CN109254733B (en) | Method, device and system for storing data | |
| CN103136243B (en) | File system duplicate removal method based on cloud storage and device | |
| CN104881466B (en) | The processing of data fragmentation and the delet method of garbage files and device | |
| WO2015106711A1 (en) | Method and device for constructing nosql database index for semi-structured data | |
| US10831747B2 (en) | Multi stage aggregation using digest order after a first stage of aggregation | |
| CN107436725A (en) | A kind of data are write, read method, apparatus and distributed objects storage cluster | |
| CN104133867A (en) | DOT in-fragment secondary index method and DOT in-fragment secondary index system | |
| CN107391758A (en) | Database switching method, device and equipment | |
| US10255234B2 (en) | Method for storing data elements in a database | |
| CN104104717A (en) | Inputting channel data statistical method and device | |
| CN104731896A (en) | Data processing method and system | |
| CN107704202A (en) | A kind of method and apparatus of data fast reading and writing | |
| CN105677904B (en) | Small documents storage method and device based on distributed file system | |
| US20170053019A1 (en) | System to organize search and display unstructured data | |
| CN107402950A (en) | Divide the document handling method and device of table based on point storehouse | |
| CN105912698A (en) | Deletion method and system of data file in disk | |
| CN104657435A (en) | Storage management method for application data and network management system | |
| CN109343962A (en) | Data processing method, device and distribution service | |
| US20200349081A1 (en) | Method, apparatus and computer program product for managing metadata | |
| CN110633261A (en) | Picture storage method, picture query method and device | |
| CN105677579B (en) | Data access method in caching system and system | |
| CN107493309A (en) | File wiring method and device in a kind of distributed system | |
| CN112035428A (en) | Distributed storage system, method, apparatus, electronic device, and storage medium | |
| CN105279166B (en) | File management method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20200924 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant after: Innovative advanced technology Co.,Ltd. Address before: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant before: Advanced innovation technology Co.,Ltd. Effective date of registration: 20200924 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant after: Advanced innovation technology Co.,Ltd. Address before: Cayman Islands Grand Cayman capital building, a four storey No. 847 mailbox Applicant before: Alibaba Group Holding Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |