CN109359085A - File migration method, apparatus, computer equipment and storage medium - Google Patents
File migration method, apparatus, computer equipment and storage medium Download PDFInfo
- Publication number
- CN109359085A CN109359085A CN201810965515.3A CN201810965515A CN109359085A CN 109359085 A CN109359085 A CN 109359085A CN 201810965515 A CN201810965515 A CN 201810965515A CN 109359085 A CN109359085 A CN 109359085A
- Authority
- CN
- China
- Prior art keywords
- file
- migrated
- xml
- migration
- database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013508 migration Methods 0.000 title claims abstract description 166
- 230000005012 migration Effects 0.000 title claims abstract description 166
- 238000000034 method Methods 0.000 title claims abstract description 51
- 238000003860 storage Methods 0.000 title claims abstract description 19
- 238000004590 computer program Methods 0.000 claims description 18
- 238000012163 sequencing technique Methods 0.000 claims description 7
- 230000006870 function Effects 0.000 description 10
- 238000010586 diagram Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 2
- 238000013509 system migration Methods 0.000 description 2
- 101000695861 Arabidopsis thaliana Brefeldin A-inhibited guanine nucleotide-exchange protein 5 Proteins 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000010485 coping Effects 0.000 description 1
- 238000005520 cutting process Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000001727 in vivo Methods 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000002618 waking effect Effects 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Abstract
The invention discloses a kind of file migration methods, device, computer equipment and storage medium, this method comprises: to be migrated file of the file content there are preset keyword of a file destination is determined from multiple file destinations got automatically, therefore the file to be migrated for manually determining to need to migrate from the file of magnanimity is not needed, simultaneously, due to using a file to be migrated as current file to be migrated, and it automatically will be in current the second database of file migration to be migrated, if migrating successfully, then continue to execute migration movement, if migration failure, the reason of then being failed according to the migration of record updates current file to be migrated, so that the updated current file to be migrated is successfully migrated, realize migration failure file be updated after can Successful migration to the second database, until all files to be migrated all move to In second database, to improve the efficiency of migrated file.
Description
Technical field
The present invention relates to big data processing field more particularly to a kind of file migration method, apparatus, computer equipment and deposit
Storage media.
Background technique
Currently, with the progress of social electronics technology, in insurance industry, insurance company is in order to save Business Information or number
According to etc., it is provided with multiple and different e commerce transactions systems.
In general, the Business Information or data of insurance company exist with document form, such as XML file is each due to business
A part of file is needed from an e commerce transactions system migration to another e commerce transactions system in file, and another part file
Do not need to migrate, thus by each file to be migrated from an e commerce transactions system migration to another e commerce transactions system
Before, it is necessary first to manually determine each file for needing to migrate from the file of magnanimity, namely determine each to be migrated
File, then just by the way that each file to be migrated only is manually entered into another from an e commerce transactions system one
E commerce transactions system, such as by each complaint declaration form (namely the corresponding declaration form of insurance complained) to be migrated from Claims Resolution system
One, which only is entered into official communication, tells system, and " screening-typing " this process requires a great deal of time, simultaneously because format is not inconsistent
Etc. reasons be easy to cause typing to fail, so as to cause the inefficiency of migrated file.
Summary of the invention
The embodiment of the present invention provides a kind of file migration method, apparatus of efficiency that migrated file can be improved, computer
Equipment and storage medium, to solve the problems, such as the inefficiency of legacy migration file.
A kind of file migration method, comprising:
Obtain multiple file destinations in first database；
By file content in the file destination, there are the file destinations of preset keyword to be determined as file to be migrated；
A file to be migrated in the file to be migrated is determined as current file to be migrated；
In a manner of parallel multithread by the current file migration to be migrated into the second database；
If by the current migration results of the file migration to be migrated into the second database be migrate successfully, until
Before all files to be migrated are successfully migrated in second database in the first database, by first number
It is determined as new current file to be migrated according to next file to be migrated in library；
If being migration failure by the current migration results of the file migration to be migrated into the second database, record is moved
The reason of moving failure updates the current file to be migrated according to the reason of migration failure, updated works as so that described
Preceding file to be migrated is successfully migrated in second database, until files to be migrated all in the first database
Before being successfully migrated in second database, next file to be migrated in the first database is determined as
New current file to be migrated.
A kind of file migration device, comprising:
Module is obtained, for obtaining multiple file destinations in first database；
First determining module, for there are the file destination of preset keyword is true by file content in the file destination
It is set to file to be migrated；
Second determining module, for a file to be migrated in the file to be migrated to be determined as current text to be migrated
Part；
Transferring module is used for the current file migration to be migrated in a manner of parallel multithread into the second database；
Third determining module, if for being by the current migration results of the file migration to be migrated into the second database
It migrates successfully, is then successfully migrated in second database until files to be migrated all in the first database
Before, next file to be migrated in the first database is determined as to new current file to be migrated；
Update module, if for being migration by the current migration results of the file migration to be migrated into the second database
Failure then records the reason of migration fails, and the current file to be migrated is updated according to the reason of migration failure, so that institute
It states updated current file to be migrated to be successfully migrated in second database, until institute in the first database
Need before migrated file is successfully migrated in second database, it will be next wait move in the first database
Text move part is determined as new current file to be migrated.
A kind of computer equipment, including memory, processor and storage are in the memory and can be in the processing
The computer program run on device, the processor realize the step of above-mentioned file migration method when executing the computer program
Suddenly.
A kind of computer readable storage medium, the computer-readable recording medium storage have computer program, the meter
The step of calculation machine program realizes above-mentioned file migration method when being executed by processor.
Above-mentioned file migration method, apparatus, computer equipment and storage medium, because automatically from multiple targets got
To be migrated file of the file content there are preset keyword of a file destination, the acquisition mesh are determined in file
Mark file is that file destination is obtained from first database, so not needing manually to determine each needs from the file of magnanimity
The file to be migrated of migration, simultaneously as using a file to be migrated as current file to be migrated, and automatically will be currently wait move
Move the second database of file migration in, if migrate successfully, until all files to be migrated are successfully migrated will under
One file to be migrated is determined as new current file to be migrated, continues to execute migration movement, if migration failure, according to record
Migration failure the reason of update current file to be migrated so that the updated current file to be migrated is successfully migrated,
Next file to be migrated is being determined as to new current text to be migrated until all files to be migrated are successfully migrated
Part, therefore when migrating failure, current file to be migrated can be updated according to the reason of migration of record failure, realize migration
The file of failure be updated after can Successful migration to the second database, until all files to be migrated all move to the second number
According in library, without by manually by each file to be migrated from only a typing of e commerce transactions system one
To another e commerce transactions system, and then improve the efficiency of migrated file.
Detailed description of the invention
In order to illustrate the technical solution of the embodiments of the present invention more clearly, below by institute in the description to the embodiment of the present invention
Attached drawing to be used is needed to be briefly described, it should be apparent that, the accompanying drawings in the following description is only some implementations of the invention
Example, for those of ordinary skill in the art, without any creative labor, can also be according to these attached drawings
Obtain other attached drawings.
Fig. 1 is an application environment schematic diagram of file moving method in one embodiment of the invention；
Fig. 2 is a flow chart of file moving method in one embodiment of the invention；
Fig. 3 is the flow chart for determining file to be migrated in one embodiment of the invention in file moving method；
Fig. 4 is to migrate current file to be migrated in one embodiment of the invention in file moving method to the one of the second database
Flow chart；
Fig. 5 is the flow chart for reading XML subfile to caching in one embodiment of the invention in file moving method；
Fig. 6 is a schematic diagram of file moving apparatus in one embodiment of the invention；
Fig. 7 is a schematic diagram of computer equipment in one embodiment of the invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are some of the embodiments of the present invention, instead of all the embodiments.Based on this hair
Embodiment in bright, every other implementation obtained by those of ordinary skill in the art without making creative efforts
Example, shall fall within the protection scope of the present invention.
File migration method provided by the present application, can be applicable in the application environment such as Fig. 1, wherein computer equipment is logical
Network is crossed to be communicated with server.Server-side obtains each file destination in first database, and server-side is by each target
There are the file destinations of preset keyword to be determined as each file to be migrated for file content in file, and server-side is by each wait move
A file to be migrated is determined as current file to be migrated in text move part, and server-side counts current file migration to be migrated to second
According in library, if being to migrate successfully by current migration results of the file migration to be migrated into the second database, until first
Before each file to be migrated is successfully migrated in the second database in database, server-side will be under in first database
One file to be migrated is determined as new current file to be migrated, if by current file migration to be migrated into the second database
Migration results are migration failure, then the reason of server-side record migration failure, are updated according to the reason of migration failure currently wait move
Text move part, so that the updated current file to be migrated is successfully migrated in the second database, until the first data
Before each file to be migrated is successfully migrated in the second database in library, server-side will be next in first database
File to be migrated is determined as new current file to be migrated.Wherein, computer equipment can be, but not limited to various personal computers,
Laptop, smart phone, tablet computer and portable wearable device.Server can be with independent server either
The server cluster of multiple servers composition is realized.
In one embodiment, as shown in Fig. 2, providing a kind of file migration method, this document moving method is applied in finance
In industry, applies be illustrated for the server in Fig. 1 in this way, include the following steps:
S10: multiple file destinations in first database are obtained；
Specifically, obtain store path of each file destination in first database, then, according to the store path from
Each file destination is extracted in first database, there are multiple for the file destination.
It should be noted that first database can consult the database for system of telling for insurance industry, or insurance row
The database of industry other systems, first database can be sql database or oracle database, and first database may be
Other databases, the particular content of first database can be set, herein with no restrictions according to practical application.
S20: by file content in file destination, there are the file destinations of preset keyword to be determined as file to be migrated；
Specifically, there are the file destination of preset keyword is true for file content in each file destination extraction obtained
It is set to each file to be migrated.Wherein, preset keyword can be " Type=transmit class " or " Type=migrates class " etc., in advance
If keyword particular content, can be set according to practical application, herein with no restrictions.
S30: a file to be migrated in file to be migrated is determined as current file to be migrated；
Specifically, the file to be migrated determined in obtained each file to be migrated is determined as current text to be migrated
Part.
S40: in a manner of parallel multithread by current file migration to be migrated into the second database；
Specifically, obtained current file migration to be migrated will be determined using file migration tool in a manner of parallel multithread
Into the second database.
It should be noted that file migration tool can be SQLyog tool or Navicat Premium tool etc., second
Database can be sql database or oracle database, the particular content of file migration tool and the second database, Ke Yigen
It is set according to practical application, herein with no restrictions.
S50: judge whether current file migration to be migrated succeeds into the second database；
Specifically, judge whether current file migration to be migrated succeeds into the second database, if receive it is preset at
Function reminder message then represents current file Successful migration to be migrated into the second database, preset successfully mentions if not receiving
Awake message then represents migration failure.Wherein, preset success reminder message can be preset successfully to mention for short message or sound etc.
The particular content for message of waking up, can be set, herein with no restrictions according to actual needs.
S60: if by current migration results of the file migration to be migrated into the second database be migrate successfully, until
It, will be next in first database before all files to be migrated are successfully migrated in the second database in first database
A file to be migrated is determined as new current file to be migrated；
Specifically, when by current file migration to be migrated, into the second database, obtained migration results are to migrate successfully
When, until being successfully migrated in the second database until files to be migrated all in first database, by the first data
Next file to be migrated in library is determined as new current file to be migrated.
S70: if being migration failure by current migration results of the file migration to be migrated into the second database, record is moved
The reason of moving failure updates current file to be migrated according to the reason of migration failure, so that updated current file to be migrated
It is successfully migrated in the second database, is being successfully migrated to the second number until files to be migrated all in first database
Before in library, next file to be migrated in first database is determined as to new current file to be migrated.
In embodiments of the present invention, for format is not inconsistent, character set is not inconsistent or suffix name is not inconsistent the reason of migration failure.
Specifically, when being that migration is lost by current obtained migration results of the file migration to be migrated into the second database
When losing, record migrates the reason of failing, and then currently file to be migrated is updated according to the reason of migration failure, so that updated
Current file to be migrated is successfully migrated in the second database, until files to be migrated all in first database by
Before function moves in the second database, next file to be migrated in first database is determined as new current to be migrated
File.
In the corresponding embodiment of Fig. 2, S10 to step S70 through the above steps, because automatically from multiple mesh got
To be migrated file of the file content there are preset keyword of a file destination, the acquisition are determined in mark file
File destination is that file destination is obtained from first database, so not needing manually to determine to need from the file of magnanimity to move
The file to be migrated moved, simultaneously as using a file to be migrated as current file to be migrated, and automatically will be current to be migrated
In the second database of file migration, if migrating successfully, will be next until all files to be migrated are successfully migrated
A file to be migrated is determined as new current file to be migrated, continues to execute migration movement, if migration failure, according to record
The reason of migration failure, updates current file to be migrated, so that the updated current file to be migrated is successfully migrated,
Next file to be migrated is determined as to new current file to be migrated until all files to be migrated are successfully migrated,
Therefore when migrating failure, current file to be migrated can be updated according to the reason of migration of record failure, realizes migration and loses
The file lost be updated after can Successful migration to the second database, until all files to be migrated all move to the second data
In library, without by the way that manually each file to be migrated only is entered into from an e commerce transactions system one
Another e commerce transactions system, and then improve the efficiency of migrated file.
In one embodiment, this document moving method is applied in financial industry, and file destination is XML file, Yi Zhongwen
Current file to be migrated is updated according to the reason of migration failure in part moving method to specifically comprise the following steps:
S701: if the reason of migration failure is not inconsistent for format, by the lattice of the XML tag in current XML file to be migrated
Formula is updated to the reference format of XML tag in XML file；
Specifically, it when not being inconsistent for format the reason of migration failure, will determine in obtained current XML file to be migrated
The format of XML tag is updated to the reference format of XML tag in XML file, such as incite somebody to action "<p>this is a paragraph " is more
Newly for attribute tags "<p>this is a paragraph</p>".
S702: if the reason of migration failure is not inconsistent for character set, the character set in current XML file to be migrated is updated
To meet the standard character collection currently shown；
In the present embodiment, common character collection includes Unicode, ascii character-set, GB2312 character set, BIG5 character
Collection, GB18030 character set and Unicode character set etc..
Specifically, when not being inconsistent for character set the reason of migration failure, more by the character set in current XML file to be migrated
New is to meet the standard character collection currently shown.Such as, UTF-8 character set is updated to ascii character-set.
It should be noted that UTF-8, full name in English is 8-bit Unicode Transformation Format, Chinese
Entitled Unicode, ASCII, full name in English are American Standard Code for Information
Interchange, the entitled ASCII of Chinese.
S703: if the reason of migration failure is not inconsistent for suffix name, the suffix name of current XML file to be migrated is updated to
The standard suffix name of XML file.
Specifically, when not being inconsistent for suffix name the reason of migration failure, the suffix name of current XML file to be migrated is updated
For the standard suffix name of XML file.File name " vehicle insurance .x " is such as updated to file name " vehicle insurance .xml ".
In this embodiment, S701 to step S703 through the above steps, due to according to different migrations unsuccessfully
Current file to be migrated is updated, the updated current file to be migrated convenient for migration is obtained, it will be updated to facilitate
Current file migration to be migrated improves the convenience of file migration into the second database.
In one embodiment, this document moving method is applied in financial industry, and file destination is XML file, such as Fig. 3 institute
Flow chart of the step S20 under an application scenarios in a kind of file migration method in 2 corresponding embodiment of diagram, specifically include as
Lower step:
S201: the data of the predefined type in XML file, the XML file after obtaining the corresponding rejecting of XML file are rejected；
Specifically, the data for rejecting the predefined type in each XML file, after obtaining the corresponding rejecting of each XML file
XML file.
It should be noted that the data of predefined type can be space or escape character etc., the tool of the data of predefined type
Hold in vivo, can be set according to practical application, herein with no restrictions.
S202: the XML file after rejecting is parsed using the node analytical tool of preset XML file, is picked
The corresponding each XML node set of XML file after removing, wherein there are more than one XML nodes for XML node set；
Specifically, the XML file after obtained each rejecting is carried out using the node analytical tool of preset XML file
Parsing, the set of the corresponding each XML node of XML file after obtaining each rejecting, wherein there are one for XML node set
The above XML node, as root node "<bookstore>", simultaneously "<bookstore>" in there are four "<book>" node.
It should be noted that the node analytical tool of preset XML file can save for DOM node analytical tool or SAX
Point analytical tool, the particular content of the node analytical tool of preset XML file can be set according to practical application, this
Place is with no restrictions.
S203: whether more than one XML node in the corresponding each XML node set of XML file after judging rejecting
There are preset keywords, if so, S204 is thened follow the steps, if it is not, thening follow the steps S205；
Specifically, one in the set of the corresponding each XML node of XML file after each rejecting judged with
On XML node whether there is preset keyword.Wherein, preset keyword can for " XMLType=transmit class " or
" XMLType=migrates class " etc., the particular content of preset keyword can be set according to practical application, not done herein
Limitation.
S204: the XML file after rejecting is determined as each file to be migrated；
Specifically, the XML file after obtained each rejecting is determined as each file to be migrated, with to by each wait move
File migration is moved into the second database.
S205: the XML file after rejecting is determined as each backup file.
Specifically, the XML file after each rejecting is determined as each backup file.Each backup file is stored in
In standby database, it can call at any time.Wherein, standby database can be sql database or oracle database, spare number
According to the particular content in library, can be set according to actual needs, herein with no restrictions.
In the corresponding embodiment of Fig. 3, S201 to step S205 through the above steps, due to first using preset XML text
The node analytical tool of part accurately parses XML file, and XML node whether there is in the XML file that then judgement parsing obtains
Preset keyword, so as to accurately know that avoiding will be not present there are preset keyword in some XML file
The XML file of preset keyword is determined as file to be migrated, improves the accuracy for determining file to be migrated.
In one embodiment, this document moving method is applied in financial industry, and file destination is XML file, such as Fig. 4 institute
Flow chart of the step S40 under an application scenarios in a kind of file migration method in 2 corresponding embodiment of diagram, specifically include as
Lower step:
S401: current XML file to be migrated is cut according to the sequencing of the file content of current XML file to be migrated
For more than two current XML subfiles to be migrated；
Specifically, the file content of current XML file to be migrated is arranged according to sequencing first, then according to working as
Current XML file to be migrated is cut into current XML Ziwen to be migrated by the sequencing of the file content of preceding XML file to be migrated
Part, the current XML subfile to be migrated exist more than two.
S402: each migration task is created；
Specifically, obtained each current XML subfile to be migrated is generated into corresponding task, completed described in each migration
The creation of the migration task of current XML subfile to be migrated.
S403: each migration task is executed in a manner of parallel multithread.
Specifically, each migration task created is executed in a manner of parallel multithread, so that completing current to be migrated
File migration to the second database work.
In the corresponding embodiment of Fig. 4, S401 to step S403 through the above steps, due to according to current XML to be migrated
Current XML file to be migrated is cut into each current XML subfile to be migrated by the sequencing of the file content of file, then
According to each current XML subfile pair to be migrated of sequencing creation for each current XML subfile to be migrated that cutting obtains
The migration task answered finally executes each migration task, without executing one by one in a manner of parallel multithread
Migration task improves the efficiency for executing migration task.
In one embodiment, this document moving method is applied in financial industry, and file destination is XML file, in such as Fig. 5
Shown Fig. 2 into Fig. 4 corresponding embodiment in a kind of file migration method before step S20, this document moving method further include as
Lower step:
S801: the byte of each XML file is obtained；
Specifically, the byte-sized of each XML file is obtained by obtaining the tool of file size first, wherein obtain
The tool of file size can be FileChannel tool etc., obtain the particular content of the tool of file size, can be according to reality
Border application is set, herein with no restrictions.
S802: setting is greater than or equal to the caching of maximum byte in the byte of each XML file；
Specifically, setting is greater than or equal to the slow of maximum byte in the byte-sized of each XML file acquired
Region is deposited, so that guaranteeing that each XML file of coming in is read in enough memory space storages.
S803: each XML file is read to caching.
Specifically, each XML file is read to the buffer zone set.
In the corresponding embodiment of Fig. 5, S801 to step S803, is suitble to each XML text due to being arranged through the above steps
The buffer zone of the byte-sized of part, so as to not waste of resource in the case where each XML file is read to setting in an orderly manner
The buffer zone set, ensure that can steadily get buffer area when migrating each XML file to the second database
Each XML file in domain, improves the stability of migrated file.
It should be understood that the size of the serial number of each step is not meant that the order of the execution order in above-described embodiment, each process
Execution sequence should be determined by its function and internal logic, the implementation process without coping with the embodiment of the present invention constitutes any limit
It is fixed.
In one embodiment, a kind of file migration device is provided, this document moving apparatus is moved with file in above-described embodiment
Shifting method corresponds.As shown in fig. 6, this document moving apparatus is including obtaining module 61, the first determining module 62, second determines
Module 63, transferring module 64, third determining module 65 and update module 66.Detailed description are as follows for each functional module:
Module 61 is obtained, for obtaining each file destination in first database；
First determining module 62, for there are the file destinations of preset keyword by file content in each file destination
It is determined as each file to be migrated；
Second determining module 63, for a file to be migrated in each file to be migrated to be determined as current text to be migrated
Part；
Transferring module 64 is used for current file migration to be migrated into the second database；
Third determining module 65, if for being to move by current migration results of the file migration to be migrated into the second database
Function is moved into, then until being successfully migrated in the second database until file to be migrated each in first database, by
Next file to be migrated in one database is determined as new current file to be migrated；
Update module 66, if for being that migration is lost by current migration results of the file migration to be migrated into the second database
The reason of losing, then recording migration failure updates current file to be migrated according to the reason of migration failure, so that updated current
File to be migrated is successfully migrated in the second database, is successfully moved until file to be migrated each in first database
Before moving on in the second database, next file to be migrated in first database is determined as to new current text to be migrated
Part.
Further, file destination is XML file, updates current file packet to be migrated according to the reason of migration failure
It includes:
First replacement unit will be in current XML file to be migrated if for migrating failure the reason of is not inconsistent for format
The format of XML tag is updated to the reference format of XML tag in XML file；
Second replacement unit will be in current XML file to be migrated if not being inconsistent for migrating the reason of failing for character set
Character set be updated to meet the standard character collection currently shown；
Third replaces unit, if not being inconsistent for migrating the reason of failing for suffix name, by current XML file to be migrated
Suffix name is updated to the standard suffix name of XML file.
Further, file destination is XML file, and the first determining module 62 includes:
It is corresponding to obtain each XML file for rejecting the data of the predefined type in each XML file for culling unit
XML file after rejecting；
Resolution unit carries out the XML file after each rejecting for the node analytical tool using preset XML file
Parsing, the corresponding each XML node set of XML file after obtaining each rejecting, wherein XML node set there are one with
Upper XML node；
Judging unit, for judging more than one in the corresponding each XML node set of XML file after each rejecting
XML node whether there is preset keyword；
Unit is concluded, if for more than one in the corresponding each XML node set of the XML file after each rejecting
There are preset keywords for XML node, then the XML file after each rejecting are determined as each file to be migrated.
Further, file destination is XML file, and transferring module 64 includes:
Cutter unit, the sequencing for the file content according to current XML file to be migrated will current XML to be migrated
File is cut into more than two current XML subfiles to be migrated；
Creating unit, for creating each migration task, wherein migration task is to migrate current XML to be migrated
The task of file；
Execution unit, for executing each migration task in a manner of parallel multithread.
Further, file destination is XML file, the file migration device further include:
Obtaining unit, for obtaining the byte of each XML file；
Setting unit, for the caching of maximum byte in the byte for being greater than or equal to each XML file to be arranged；
Reading unit, for reading each XML file to caching.
Specific about file migration device limits the restriction that may refer to above for file migration method, herein not
It repeats again.Modules in above-mentioned file migration device can be realized fully or partially through software, hardware and combinations thereof.On
Stating each module can be embedded in the form of hardware or independently of in the processor in computer equipment, can also store in a software form
In memory in computer equipment, the corresponding operation of the above modules is executed in order to which processor calls.
In one embodiment, a kind of computer equipment is provided, which can be server, internal junction
Composition can be as shown in Figure 7.The computer equipment include by system bus connect processor, memory, network interface and
Database.Wherein, the processor of the computer equipment is for providing calculating and control ability.The memory packet of the computer equipment
Include non-volatile memory medium, built-in storage.The non-volatile memory medium is stored with operating system, computer program and data
Library.The built-in storage provides environment for the operation of operating system and computer program in non-volatile memory medium.The calculating
The database of machine equipment is for data involved by storage file moving method.The network interface of the computer equipment be used for
External terminal passes through network connection communication.To realize a kind of file migration method when the computer program is executed by processor.
In one embodiment, a kind of computer equipment is provided, including memory, processor and storage are on a memory
And the computer program that can be run on a processor, processor realize above-described embodiment file migration side when executing computer program
The step of method, such as step S10 shown in Fig. 2 to step S70.Alternatively, processor realizes above-mentioned reality when executing computer program
The function of each module/unit of file moving apparatus in example is applied, such as obtains the function of module 61 to update module 66 shown in Fig. 6
Energy.To avoid repeating, which is not described herein again.
In one embodiment, a kind of computer readable storage medium is provided, computer program is stored thereon with, is calculated
File moving method in above method embodiment is realized when machine program is executed by processor, alternatively, the computer program is processed
The function of each module/unit in file moving apparatus in above-mentioned apparatus embodiment is realized when device executes.To avoid repeating, here not
It repeats again.Those of ordinary skill in the art will appreciate that realizing all or part of the process in above-described embodiment method, being can be with
Relevant hardware is instructed to complete by computer program, the computer program can be stored in a non-volatile computer
In read/write memory medium, the computer program is when being executed, it may include such as the process of the embodiment of above-mentioned each method.Wherein,
To any reference of memory, storage, database or other media used in each embodiment provided herein,
Including non-volatile and/or volatile memory.Nonvolatile memory may include read-only memory (ROM), programming ROM
(PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM) or flash memory.Volatile memory may include
Random access memory (RAM) or external cache.By way of illustration and not limitation, RAM is available in many forms,
Such as static state RAM (SRAM), dynamic ram (DRAM), synchronous dram (SDRAM), double data rate sdram (DDRSDRAM), enhancing
Type SDRAM (ESDRAM), synchronization link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM
(RDRAM), direct memory bus dynamic ram (DRDRAM) and memory bus dynamic ram (RDRAM) etc..
It is apparent to those skilled in the art that for convenience of description and succinctly, only with above-mentioned each function
Can unit, module division progress for example, in practical application, can according to need and by above-mentioned function distribution by different
Functional unit, module are completed, i.e., the internal structure of described device is divided into different functional unit or module, more than completing
The all or part of function of description.
Embodiment described above is merely illustrative of the technical solution of the present invention, rather than its limitations；Although referring to aforementioned reality
Applying example, invention is explained in detail, those skilled in the art should understand that: it still can be to aforementioned each
Technical solution documented by embodiment is modified or equivalent replacement of some of the technical features；And these are modified
Or replacement, the spirit and scope for technical solution of various embodiments of the present invention that it does not separate the essence of the corresponding technical solution should all
It is included within protection scope of the present invention.
Claims (10)
1. a kind of file migration method, which is characterized in that the file migration method includes:
Obtain multiple file destinations in first database；
By file content in the file destination, there are the file destinations of preset keyword to be determined as file to be migrated；
A file to be migrated in the file to be migrated is determined as current file to be migrated；
In a manner of parallel multithread by the current file migration to be migrated into the second database；
If being to migrate successfully by the current migration results of the file migration to be migrated into the second database, until described
Before all files to be migrated are successfully migrated in second database in first database, by the first database
In next file to be migrated be determined as new current file to be migrated；
If being migration failure by the current migration results of the file migration to be migrated into the second database, records migration and lose
The reason of losing updates the current file to be migrated according to the reason of migration failure so that it is described it is updated currently to
Migrated file is successfully migrated in second database, until files to be migrated all in the first database quilt
Before Successful migration is into second database, next file to be migrated in the first database is determined as new
Current file to be migrated.
2. file migration method as described in claim 1, which is characterized in that the file destination is XML file, the basis
The reason of migration failure, updates the current file to be migrated
If the reason of migration failure is not inconsistent for format, by the format of the XML tag in the current XML file to be migrated
It is updated to the reference format of XML tag in XML file；
If the reason of migration failure is not inconsistent for character set, the character set in the current XML file to be migrated is updated
To meet the standard character collection currently shown；
If the reason of migration failure is not inconsistent for suffix name, the suffix name of the current XML file to be migrated is updated to
The standard suffix name of XML file.
3. file migration method as described in claim 1, which is characterized in that the file destination is XML file, described by institute
The file content for stating file destination is determined as file to be migrated there are the file destination of preset keyword and includes:
The data for rejecting the predefined type in the XML file, the XML file after obtaining the corresponding rejecting of the XML file；
The XML file after the rejecting is parsed using the node analytical tool of preset XML file, obtains the rejecting
The corresponding each XML node set of XML file afterwards, wherein there are more than one XML nodes for XML node set；
More than one XML node in the corresponding each XML node set of XML file after judging the rejecting is with the presence or absence of pre-
If keyword；
If there are preset passes for more than one XML node in the corresponding each XML node set of XML file after the rejecting
XML file after the rejecting is then determined as file to be migrated by key word.
4. file migration method as described in claim 1, which is characterized in that the file destination is XML file, described with simultaneously
The current file migration to be migrated includes: by row multithreading into the second database
The current XML file to be migrated is cut according to the sequencing of the file content of the current XML file to be migrated
For more than two current XML subfiles to be migrated；
Create each migration task, wherein the migration task is the task of the migration current XML subfile to be migrated；
Each migration task is executed in a manner of parallel multithread.
5. file migration method according to any one of claims 1 to 4, which is characterized in that the file destination is XML text
Part, it is described before file content in file destination is determined as file to be migrated there are the file destination of preset keyword
File migration method further include:
Obtain the byte of each XML file；
The caching of maximum byte in byte more than or equal to each XML file is set；
Each XML file is read to the caching.
6. a kind of file migration device, which is characterized in that the file migration device includes:
Module is obtained, for obtaining multiple file destinations in first database；
First determining module, for there are the file destinations of preset keyword to be determined as by file content in the file destination
File to be migrated；
Second determining module, for a file to be migrated in the file to be migrated to be determined as current file to be migrated；
Transferring module is used for the current file migration to be migrated in a manner of parallel multithread into the second database；
Third determining module, if for being migration by the current migration results of the file migration to be migrated into the second database
Success, then be successfully migrated to it in second database until files to be migrated all in the first database
Before, next file to be migrated in the first database is determined as to new current file to be migrated；
Update module, if for being that migration is lost by the current migration results of the file migration to be migrated into the second database
The reason of losing, then recording migration failure updates the current file to be migrated according to the reason of migration failure, so that described
Updated current file to be migrated is successfully migrated in second database, until owning in the first database
It, will be next to be migrated in the first database before file to be migrated is successfully migrated in second database
File is determined as new current file to be migrated.
7. file migration device as claimed in claim 6, which is characterized in that the file destination is XML file, the update
Module includes:
First replacement unit, if the reason of failing for the migration is not inconsistent for format, by the current XML file to be migrated
In the format of XML tag be updated to the reference format of XML tag in XML file；
Second replacement unit, if the reason of failing for the migration is not inconsistent for character set, by the current XML text to be migrated
Character set in part is updated to meet the standard character collection currently shown；
Third replaces unit, if the reason of failing for the migration is not inconsistent for suffix name, by the current XML text to be migrated
The suffix name of part is updated to the standard suffix name of XML file.
8. the file migration device as described in any one of claim 6 to 7, which is characterized in that the first determining module packet
It includes:
Culling unit obtains the corresponding rejecting of the XML file for rejecting the data of the predefined type in the XML file
XML file afterwards；
Resolution unit, for being solved using the node analytical tool of preset XML file to the XML file after the rejecting
Analysis, the corresponding each XML node set of XML file after obtaining the rejecting, wherein there are more than one for XML node set
XML node；
Judging unit, for judging more than one XML in the corresponding each XML node set of XML file after the rejecting
Node whether there is preset keyword；
Unit is concluded, if for more than one XML section in the corresponding each XML node set of the XML file after the rejecting
There are preset keywords for point, then the XML file after the rejecting are determined as file to be migrated.
9. a kind of computer equipment, including memory, processor and storage are in the memory and can be in the processor
The computer program of upper operation, which is characterized in that the processor realized when executing the computer program as claim 1 to
Described in any one of 5 the step of file migration method.
10. a kind of computer readable storage medium, the computer-readable recording medium storage has computer program, and feature exists
In the step of realization file migration method as described in any one of claims 1 to 5 when the computer program is executed by processor
Suddenly.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810965515.3A CN109359085A (en) | 2018-08-23 | 2018-08-23 | File migration method, apparatus, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810965515.3A CN109359085A (en) | 2018-08-23 | 2018-08-23 | File migration method, apparatus, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109359085A true CN109359085A (en) | 2019-02-19 |
Family
ID=65350263
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810965515.3A Pending CN109359085A (en) | 2018-08-23 | 2018-08-23 | File migration method, apparatus, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109359085A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111475467A (en) * | 2020-03-27 | 2020-07-31 | 平安科技（深圳）有限公司 | File management method, cloud file management system and terminal |
| CN111522792A (en) * | 2020-04-20 | 2020-08-11 | 中国银行股份有限公司 | File migration method and device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100077168A1 (en) * | 2008-09-24 | 2010-03-25 | Hitachi, Ltd. | System and method for controlling automated page-based tier management in storage systems |
| CN104065685A (en) * | 2013-03-22 | 2014-09-24 | 中国银联股份有限公司 | Data migration method in cloud computing environment-oriented layered storage system |
| WO2016045096A1 (en) * | 2014-09-26 | 2016-03-31 | 华为技术有限公司 | File migration method and apparatus and storage device |
| CN106446166A (en) * | 2016-09-23 | 2017-02-22 | 郑州云海信息技术有限公司 | Method and device for migrating data from DB2 database to K-DB database |
| CN107436912A (en) * | 2016-05-27 | 2017-12-05 | 三星Sds株式会社 | Data migration device and its method |
-
2018
- 2018-08-23 CN CN201810965515.3A patent/CN109359085A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100077168A1 (en) * | 2008-09-24 | 2010-03-25 | Hitachi, Ltd. | System and method for controlling automated page-based tier management in storage systems |
| CN104065685A (en) * | 2013-03-22 | 2014-09-24 | 中国银联股份有限公司 | Data migration method in cloud computing environment-oriented layered storage system |
| WO2016045096A1 (en) * | 2014-09-26 | 2016-03-31 | 华为技术有限公司 | File migration method and apparatus and storage device |
| CN107436912A (en) * | 2016-05-27 | 2017-12-05 | 三星Sds株式会社 | Data migration device and its method |
| CN106446166A (en) * | 2016-09-23 | 2017-02-22 | 郑州云海信息技术有限公司 | Method and device for migrating data from DB2 database to K-DB database |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111475467A (en) * | 2020-03-27 | 2020-07-31 | 平安科技（深圳）有限公司 | File management method, cloud file management system and terminal |
| CN111522792A (en) * | 2020-04-20 | 2020-08-11 | 中国银行股份有限公司 | File migration method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA2824922C (en) | Formatting data by example | |
| CN111339041A (en) | File parsing and warehousing and file generating method and device | |
| CN104133772A (en) | Automatic test data generation method | |
| CN106909595B (en) | Data migration method and device | |
| CN105677904B (en) | Small documents storage method and device based on distributed file system | |
| CN108536745B (en) | Shell-based data table extraction method, terminal, equipment and storage medium | |
| CN110019873B (en) | Face data processing method, device and equipment | |
| CN113568566A (en) | Method, host device and storage server for seamless migration of simple storage service by using index object | |
| CN112286934A (en) | Database table importing method, device, equipment and medium | |
| CN106021566A (en) | Method, device and system for improving concurrent processing capacity of single database | |
| CN113688288A (en) | Data association analysis method and device, computer equipment and storage medium | |
| CN109359085A (en) | File migration method, apparatus, computer equipment and storage medium | |
| CN110019169B (en) | Data processing method and device | |
| CN111984659B (en) | Data updating method, device, computer equipment and storage medium | |
| CN102073554A (en) | Method and device for recovering files closed abnormally | |
| EP3082050A1 (en) | Mass data fusion storage method and system | |
| CN112860412B (en) | Service data processing method and device, electronic equipment and storage medium | |
| CN116628042A (en) | Data processing method, device, equipment and medium | |
| CN112035486B (en) | Partition establishing method, device and equipment of partition table | |
| JP2008171225A (en) | Database automatic migration system | |
| CN103853577A (en) | Loading method and device of website header navigation bar | |
| CN105022743A (en) | Index management method and index management device | |
| CN113434359B (en) | Data traceability system construction method and device, electronic device and readable storage medium | |
| CN113051329B (en) | Data acquisition method, device, equipment and storage medium based on interface | |
| US11704094B2 (en) | Data integrity analysis tool |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |