CN106970920A - A kind of method and apparatus for database data migration - Google Patents
A kind of method and apparatus for database data migration Download PDFInfo
- Publication number
- CN106970920A CN106970920A CN201610023836.2A CN201610023836A CN106970920A CN 106970920 A CN106970920 A CN 106970920A CN 201610023836 A CN201610023836 A CN 201610023836A CN 106970920 A CN106970920 A CN 106970920A
- Authority
- CN
- China
- Prior art keywords
- database
- target
- data objects
- statement
- target data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/214—Database migration support
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The purpose of the application is to provide a kind of method and apparatus for database data migration.Specifically, the database migration submitted according to user is asked, and obtains the database statement that the user performs in corresponding source database instance；Pending target data objects are determined by parsing the database statement；By performing the inquiry operation on the target data objects in correspondence target database example, the target data objects are cached to the target database example.Compared with prior art, the application is asked according to database migration, obtain the database statement that user performs in corresponding source database instance, by the inquiry operation that identified target data objects after database statement parsing are performed in target database example, the target data objects are cached to the target database example, paused or stuck phenomenon in migration and handoff procedure using database so as to reduce user, the access performance of customer data base in lifting database migration.
Description
Technical field
The application is related to computer realm, more particularly to a kind of technology for database data migration.
Background technology
With the development of network, the use scale of database is progressively expanded, for database migration
Demand also expands therewith, such as database instance user is purchasing during database application under cloud computing environment
Across the physical machine migration demand in trigger data storehouse is understood when buying more multiple resource, was migrated in data across machine
Cheng Zhong, can be because of the caching that can not be read in target machine in original internal memory to accelerate data
Read-write, so that user occur accesses interim card or stuck.In order to avoid it is all access data will again from
Hard disk is loaded into the caching of upper strata, so as to cause in internal memory in the case that disk reading speed is certain greatly
Used time is excessive, user using being obstructed, prior art stops current database, by data cached export
And copy to the data cached file of startup target database example loading after target database.
However, prior art has to stop current database during data are preheated, it can just lead
Go out warmable data, it is impossible to used under the conditions of the cloud database of user's access is not interrupted.And database
Source instance and object instance, it is necessary to which specifications parameter is identical, the scheme of prior art are mainly used in example sheet
Preheating when restarting of body is used, under cloud computing environment, and database instance migration is often upgrading specification,
Therefore prior art can not be applicable.
The content of the invention
The purpose of the application is to provide a kind of method and apparatus for database data migration, is used to
Solve to service the problem of appearance pauses or be stuck during database migration.
To achieve the above object, should this application provides a kind of method for database data migration
Method services the problem of appearance pauses or be stuck during solving database migration, and this method includes：
The database migration request submitted according to user, obtains the user in corresponding source database instance
The database statement of execution；
Pending target data objects are determined by parsing the database statement；
By performing the inquiry operation on the target data objects in correspondence target database example,
The target data objects are cached to the target database example.
To achieve the above object, should this application provides a kind of equipment for database data migration
Equipment services the problem of appearance pauses or be stuck during solving database migration, and the equipment includes：
Database statement acquisition device, the database migration for being submitted according to user is asked, and obtains described
The database statement that user performs in corresponding source database instance；
Target data objects determining device, for determining pending mesh by parsing the database statement
Mark data object；
Target data objects buffer storage, for by being performed in correspondence target database example on institute
The inquiry operation of target data objects is stated, the target data objects are cached real to the target database
Example.
Compared with prior art, the database migration that the application is submitted according to user is asked, and obtains described use
Pending target data is determined after the database statement that family is performed in corresponding source database instance, parsing
Object, by performing the inquiry operation on the target data objects in correspondence target database example,
The target data objects are cached to the target database example, so that user can be in target
The high frequency obtained in database instance in caching uses data, reduces user and is being migrated and cut using database
Paused or stuck phenomenon during changing, improve the access performance of customer data base in transition process.
Brief description of the drawings
By reading the detailed description made to non-limiting example made with reference to the following drawings, this Shen
Other features, objects and advantages please will become more apparent upon：
Fig. 1 shows a kind of method flow for database data migration according to the application one side
Figure；
Fig. 2 shows the method flow diagram of the step S2 according to one preferred embodiment of the application；
Fig. 3 shows to be shown according to a kind of equipment for database data migration of the application other side
It is intended to；
Fig. 4 shows setting according to the target data objects determining device of the application another preferred embodiment
Standby schematic diagram；
Fig. 5 shows the database data migration process signal according to another preferred embodiment of the application
Figure.
Same or analogous reference represents same or analogous part in accompanying drawing.
Embodiment
The application is described in further detail below in conjunction with the accompanying drawings.
In one typical configuration of the application, terminal, the equipment of service network and trusted party include
One or more processors (CPU), input/output interface, network interface and internal memory.
Internal memory potentially includes the volatile memory in computer-readable medium, random access memory
And/or the form, such as read-only storage (ROM) or flash memory (flash such as Nonvolatile memory (RAM)
RAM).Internal memory is the example of computer-readable medium.
Computer-readable medium includes permanent and non-permanent, removable and non-removable media can be with
Realize that information is stored by any method or technique.Information can be computer-readable instruction, data knot
Structure, the module of program or other data.The example of the storage medium of computer includes, but are not limited to phase
Become internal memory (PRAM), static RAM (SRAM), dynamic random access memory
(DRAM), other kinds of random access memory (RAM), read-only storage (ROM), electricity
It is Erasable Programmable Read Only Memory EPROM (EEPROM), fast flash memory bank or other memory techniques, read-only
Compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storages,
Magnetic cassette tape, magnetic disk storage or other magnetic storage apparatus or any other non-transmission medium,
The information that can be accessed by a computing device available for storage.Defined according to herein, it is computer-readable
Medium not include non-temporary computer readable media (transitory media), such as modulation data-signal and
Carrier wave.
Fig. 1 shows a kind of method flow for database data migration according to the application one side
Figure.Methods described includes step S1, step S2 and step S3.
Wherein, the database migration that equipment 1 is submitted according to user in step sl is asked, and obtains described use
The database statement that family is performed in corresponding source database instance；Equipment 1 is by parsing in step s 2
State database statement and determine pending target data objects；Equipment 1 passes through in correspondence mesh in step s3
The inquiry operation on the target data objects is performed in mark database instance, by the target data pair
As caching to the target database example.
Specifically, the database migration that equipment 1 is submitted according to user in step sl is asked, and obtains described
The database statement that user performs in corresponding source database instance.Refer to that obtaining database migration asks it
Afterwards, that is, start to start the method for database data migration, obtain user's data in machine before migration
I.e. the database statement performed in the data source database example of storehouse, now, database is to maintain operation
, i.e., user's access is not interrupted.The database statement refers to for the sentence in the language in database,
Such as SQL statement, wherein SQL (Structured Query Language) abbreviation SQL.
The database statement that obtains is progress " SQL collections " for example shown in Fig. 5, specific method, including but
It is not limited to following two methods：First, server end is captured by the TCP layer of (SuSE) Linux OS to receive
The SQL bags arrived；2nd, by being intercepted in access link come mid logarithmic according to storehouse sentence.Here, Linux
It is a set of free class Unix operating systems using with Free propagation, is the stable multi-user's net of a performance
Network operating system, TCP (Transmission Control Protocol) agreement belongs to transport layer protocol, wherein
TCP provide reliable data transmission, it provide service include data stream, reliability, effectively stream control,
Full-duplex operation and multiplexing, by being sent towards connection, end-to-end and reliable packet.Because
The related data of the database statement performed is all deposited to slow from local persistent storage in the process of implementation
Depositing, therefore obtain the database statement causes user to obtain or can trace in source data example
It is data cached, so as to be migrated to data cached, and then remove from and directly moving not data cached
The service access for moving and causing needs to wait cache-time and cause to access interim card or stuck.
Those skilled in the art will be understood that the mode of the above-mentioned acquisition database statement is only for example,
Other modes for obtaining the database statement that are existing or being likely to occur from now on are such as applicable to this
Application, should also be included within the application protection domain, and be incorporated herein by reference herein.
Preferably, it is described to obtain the database statement that the user performs in corresponding source database instance
Including：Packet of the user in corresponding source database instance is captured, is carried from the packet
Corresponding candidate data storehouse sentence is taken, and stores candidate data storehouse sentence in the buffer；From described
Candidate data storehouse sentence is captured in caching to obtain corresponding database statement.
Wherein, what is generally corresponded in the packet in the corresponding source database instance is in original machine
Database instance the data statement that produces of history, specifically grasp at the method for the packet and for example lead to
Cross the SQL bags that the TCP layer crawl server end of (SuSE) Linux OS is received.Then, the extraction pair
The candidate data storehouse sentence answered refers to the packet of acquisition carrying out analytical decomposition, so as to obtain described
The SQL bags of database statement in packet, such as parsing obtain SQL statement, but extract data
The mode of sentence is not limited to parse packet in bag, other that the database language can be extracted from packet
The method of sentence is both contained in this.Wherein, candidate data storehouse sentence is to be used as subsequent treatment after extracting
Alternate data storehouse sentence, therefore, candidate data storehouse sentence not only include extracting from the packet
The sentence that produces of associated databases instance histories, in addition to service database that Interruption period between The following article is not produced newly
Sentence, for the acquisition methods of database statement newly produced include but is not limited to by access link come
Mid logarithmic is intercepted according to storehouse sentence.Then, acquired candidate data storehouse sentence is delayed
Deposit, in order to which candidate data storehouse sentence is further processed, while caching candidate's number
Also cause to run into the acquisition candidate data storehouse sentence mistake when handling or shifting database statement according to storehouse sentence
Have little time to handle in the case of fast and cause the performance issue after overstocking.
Those skilled in the art will be understood that the mode of database statement in said extracted packet is only
The mode of database statement such as may be used in citing, other existing or extraction packets for being likely to occur from now on
Suitable for the application, it should also be included within the application protection domain, and include by reference herein
In this.
Wherein, it is described that candidate data storehouse sentence is captured from the caching, refer to the caching
In candidate data storehouse sentence captured by streaming computing framework, but specific method is not limited
Carry out, can be selected according to the specific software frame of implementation, specifically in such as real-time streaming Computational frame
Implementation method it is a lot, oneself can also fully achieve a set of, but function is essentially identical meets to caching
In the function that is captured of candidate data storehouse sentence.It is preferred that recommending real-time streaming calculating
Frame J storm is implemented, wherein, Jstorm belongs to distributed computing engines in real time, is to be considered as
It is storm java enhancing versions, because it belongs to scheme the most ripe in such calculating, uses it
Carry out after crawl splicing, can be shorter and smaller, the storage of sentence is easy in refining, shifts and in number of targets
According to the execution on the example of storehouse, it is the corresponding database statement to play the spliced sentence of crawl.
It is highly preferred that the caching is Circular buffer.Candidate data storehouse sentence is stored in the buffer
Described in caching be preferably Circular buffer, that is, buffer circle because other caching numbers at present
According to structure speed relatively slowly and because of the limitation of machine memory size, the time is deposited using buffer circle
Database statement is selected preferably to adapt to the crawl under the sentence Writing condition of high-speed data storehouse.For example,
I.e. when being used cooperatively with real-time streaming Computational frame Jstorm, except will be from the packet
The candidate data storehouse sentence of middle extraction is put into outside Circular buffer, and the database statement newly produced also can
Constantly enter in Circular buffer, so all database statements are read all from Circular buffer.Because
Circular buffer is loop configuration, and the database statement newly produced constantly enters Circular buffer, when by Jstorm
After crawl, just it is removed from it playing a part of caching.It is that annular is slow using the purpose of Circular buffer
It is a round-robin queue to deposit, can be with the size of control system internal memory, it is to avoid produced or from described due to new
Database statement generation is extracted in packet too fast, and Jstorm has little time crawl and caused after overstocking,
Server memory is overflowed, and performance issue occurs.
Those skilled in the art will be understood that above-mentioned captures candidate data storehouse sentence from the caching
Mode be only for example, other it is existing or be likely to occur from now on the time is captured from the caching
Select the mode of database statement to be such as applicable to the application, also should be included in the application protection domain with
It is interior, and be incorporated herein by reference herein.
Then equipment 1 determines pending target data by parsing the database statement in step s 2
Object.Refer to the database statement by being captured described in lexical analyzer parsing from caching, lead to
Certain conditional filtering is crossed to go out to determine pending target data objects.Such as " filtering, sieve in Fig. 5
Choosing " function corresponds to this.Wherein, the target data objects are to be used to cache to the target data
The content of storehouse example, is specifically including but not limited to select query statements, and its be related to or comprising table name
Claim etc..I.e. the instrument lexical analyzer of the parsing include but is not limited to SQL parser, mixer and
The lexical analyzers such as mysqllex.The method for determining pending target data objects is included the number
Extracted according to the sentence of user's frequency of use height or focus in sentence, or the corresponding key message of sentence,
And the possibility of the frequency of use according to user in use is ranked up, so as to obtain the target
Data object, for example, be ranked up by using number of times or file size itself, wherein, access times
Many and file itself causes the database statement corresponding to it to be focus used in user compared with big city
Sentence.
Those skilled in the art will be understood that the mode of above-mentioned parsing database statement is only for example, other
The mode of parsing database statement that is existing or being likely to occur from now on is such as applicable to the application, also should
Within the application protection domain, and it is incorporated herein by reference herein.
Then, in step s3 equipment 1 by correspondence target database example in perform on the mesh
The inquiry operation of data object is marked, the target data objects are cached to the target database example.
Refer to after the destination object is determined, inquiry operation is performed to the destination object, i.e., for example schemed
" SQL playback " shown in 5 is in the stage to being pressed according to the table name in the set quoted after number of times sequence
Following sentences are performed according to from more to less or at most certain is several：Select count(*)from
Table_name, will perform the higher tables of data of number of times and be put into the caching of target database example.
By performing the inquiry operation of the target data objects in object instance, by object instance hard disc data
User's frequency of use will be had compared with higher number by being loaded into caching, the caching of such object instance
According to so that user's operating speed in database migration is unaffected, for example, it is preferred in mesh
The caching of the high table of frequency of use will be stored in the innodb_buffer_pool for marking database instance,
After user's request link switches, the SQL execution response time is hardly by any shadow
Ring.Wherein, innodb_buffer_pool is InnoDB cache pools, is present in calculator memory,
The data frequently accessed in mysql for caching, it both can also cache index with data cached,
Even other manage data, and such as metadata information, row level lock information uses MySQL orders：
Show variables like'innodb%pool%', can check the relevant parameter of InnoDB cache pools
Information.When user inquires about database information using query statement, presence therein can be preferentially inquired about
Data, can greatly improve system response time.
Those skilled in the art will be understood that the mode of above-mentioned caching of target data object is only for example, its
He is such as applicable to the application at the mode of caching of target data object that is existing or being likely to occur from now on,
Also it should be included within the application protection domain, and be incorporated herein by reference herein.
Step S4 (not shown) is additionally included in methods described, equipment 1 is by the source number in step s 4
The target database example is copied to according to corresponding binary log in the example of storehouse.By binlog in such as Fig. 5
Namely binary log belongs to database migration from database source example copies to object instance
A part, with the action such as " SQL collections " progress simultaneously after Data Migration is initiated, it is mainly composition
Another part of database migration, caches the target data objects to the target database with described
Example carries out completing the process of whole database migration together.
Fig. 2 shows the method flow diagram of the step S2 according to one preferred embodiment of the application.It is described
Step S2 includes step S21 and step S22.
Wherein, in the step s 21 equipment 1 parse the database statement with obtain processed one or
Multiple data objects；Equipment 1 is determined according to the object-related information of the data object in step S22
Pending target data objects.
Specifically, equipment 1 parses the database statement to obtain processed one in the step s 21
Or multiple data objects.Refer to the data by being captured described in lexical analyzer parsing from caching
Storehouse sentence, is easy to determine caching to the data object and its information of target data example so as to obtain, for example,
All select query statements in the database statement are divided by SQL parser lexical analyzers
Separate out be related to or comprising table name, and be put into a set.Wherein, the choosing in preferred pair SQL statement
Select query statement analyzed and find out its be related to or comprising table name because the caching in Data Migration
Migration mainly provides the performance speed-raising of maximum for selection query statement, so only entering to selection query statement
Row analysis, analyze and find out its be related to or comprising table name primarily to be easy to count, to using frequency
The hot spot table that rate is higher or itself is larger is cached.
Those skilled in the art will be understood that the mode of above-mentioned parsing database statement is only for example, other
The mode of parsing database statement that is existing or being likely to occur from now on is such as applicable to the application, also should
Within the application protection domain, and it is incorporated herein by reference herein.
Then, equipment 1 determines to wait to hold according to the object-related information of the data object in the step s 21
Capable target data objects.Refer to the object-related information according to the data object by sorting or comparing
Determined etc. method, such as all selection query statement and its be related to or comprising table name belong to the number
According to the object-related information of object, the execution number of times of table name is subjected to reference count, then according to reference
Count inverted order arrangement then according to from more to few order can by the table corresponding to table name in the top or
Sentence is defined as pending target data objects.Preferably in object-related information if quilt
Number of times is performed, preferentially according to execution number of times, the big first caching of number of times, the first sequence；If table has size
Data, then first cache larger table, the second sequence.The pending target data objects so determined
It is that user's frequency of use is high to enable to the database caches information moved in target machine caching, from
And cause user in the case where database continual service is migrated to the use of database not interim card or card
Extremely.
Preferably, the object-related information includes following at least any one：The data object is held
Row number information；The data size information of the data object.Wherein, the data object is performed
Number information, the database statement as parsed is performed number of times, such as exemplified above to be previously mentioned
Select query statement is performed number of times, and number information is performed according to described, and it performs number of times higher generation
The frequency of use of table user is higher, and its identified target data objects is got over can be in database migration to target
And make it that the probability that user inquires about or used in the buffer is higher thereafter.The data of the data object are big
Small information, is the size for the table that the database statement parsed is comprised or related to, and its table is bigger
The frequency that user's inquiry goods is used is higher, and its identified target data objects is got over and can arrived in database migration
Target and cause that user's probability for inquiring about or use in the buffer is higher thereafter.
Those skilled in the art, which will be understood that, above-mentioned determines pending number of targets according to object-related information
Be only for example according to the mode of object, other it is existing or be likely to occur from now on according to object-related information
Determine that the mode of pending target data objects is such as applicable to the application, should also be included in the application
Within protection domain, and it is incorporated herein by reference herein.
Preferably, the equipment 1 also includes the (not shown) of log processing parameter setting apparatus 15, institute
State log processing parameter setting apparatus 15 and the corresponding log processing parameter of the source database example is set
Information.Because acquisition database sentence and the process parsing database from the source database example
Sentence determines pending target data objects, by correspondence target database example in perform on
The inquiry operation of the target data objects, the target data objects are cached to the target data
Storehouse example.So, certain query performance loss is had to target database example in the process of implementation,
The speed of object instance application source instance binary log can be reduced.To solve problem above, then
Database relevant parameter can be changed in practical operation, such as concrete operations are as follows：
Sync_binlog=10000
Innodb_flush_log_at_trx_commit=2
By changing parameter, then object instance SQL can be accelerated to perform speed and be expected more than hundred times, because root
According to internal memory, the writing speed of disk is contrasted, it can be appreciated that disk is typically slower than memory speed tens of
Again to more than hundred times.
It is highly preferred that the log processing parameter information includes following at least any one：For submitting in batches
The daily record configuration information of daily record；For by daily record via file system again write storage device daily record configure
Information.The two are the key parameters of influence object instance performance main.It is described to be used for batch
The daily record configuration information such as Sync_binlog of daily record is submitted, if its value is set to 1, is represented per number
When being submitted according to storehouse affairs, binary log is all write into disk.If value is set to 0, represent not actively
Control refreshes binlog refreshing disk, is controlled by operating system oneself, is arranged to 10000 expressions and tires out
Binary log is write disk by property batch again after 10000 affairs of meter, so can be with after modification
Accelerate the submission speed of affairs, and then improve the overall write performance of database.With above-mentioned parameter information
It is similar, it is described be used for by daily record via file system again write storage device daily record configuration information for example
Innodb_flush_log_at_trx_commit, if its value is set to 1, represents that affairs are submitted all each time
Transaction journal is needed to write (flush) hard disk；After value is set to 2, when each affairs are submitted
Log buffer can be brushed in file system, do not brushed on disk also now, but every one
Second calls the i.e. flush buffers area of file system to operate such as " flush " will to be gone on cache flush to disk,
The effect of batch buffering to a certain degree can be provided by file system, writing for affairs has been also speeded up
Enter speed.
Those skilled in the art will be understood that at the corresponding daily record of the above-mentioned setting source database example
The mode of reason parameter information is only for example, other described source numbers of setting that are existing or being likely to occur from now on
The application is such as applicable to according to the mode of the corresponding log processing parameter information of storehouse example, should be also included in
Within the application protection domain, and it is incorporated herein by reference herein.
Fig. 3 shows to be shown according to a kind of equipment for database data migration of the application other side
It is intended to.The equipment 1 includes database statement acquisition device 11, target data objects determining device 12
And target data objects buffer storage 13.
Wherein, the database migration that database statement acquisition device 11 is submitted according to user is asked, and obtains institute
State the database statement that user performs in corresponding source database instance；Target data objects determining device 12
Pending target data objects are determined by parsing the database statement；Target data objects caching dress
13 are put by performing the inquiry operation on the target data objects in correspondence target database example,
The target data objects are cached to the target database example.
Specifically, the database migration that database statement acquisition device 11 is submitted according to user is asked, and is obtained
The database statement that the user performs in corresponding source database instance.Refer to that obtaining database migration asks
After asking, that is, start to start the method for database data migration, acquisition user is before migration in machine
I.e. the database statement performed in database data source database example, now, database is to maintain
Operation, i.e., user's access is not interrupted.The database statement refers to in the language in database
Sentence, such as SQL statement, wherein SQL (Structured Query Language) are referred to as
SQL.The database statement that obtains is progress " SQL collections " for example shown in Fig. 5, specific method,
Including but not limited to following two methods：First, captured and serviced by the TCP layer of (SuSE) Linux OS
The SQL bags that device end is received；2nd, by being intercepted in access link come mid logarithmic according to storehouse sentence.
This, Linux is a set of free class Unix operating systems using with Free propagation, is that a performance is stable
Multiple-user network operating system, TCP (Transmission Control Protocol) agreement belongs to transport layer
Agreement, wherein TCP provide reliable data transmission, it provide service include data stream, reliability,
Effectively stream control, full-duplex operation and multiplexing, by towards connection, end-to-end and reliable packet
Send.Because the related data of the database statement performed is all in the process of implementation from local persistence
Depositing to caching, therefore obtain the database statement causes user to obtain or can trace in source number
It is factually data cached in example, do not cached so as to migrate, and then remove to data cached
Data Direct Transfer and the service access that causes needs to wait cache-time and cause to access interim card or card
Extremely.
Those skilled in the art will be understood that the mode of the above-mentioned acquisition database statement is only for example,
Other modes for obtaining the database statement that are existing or being likely to occur from now on are such as applicable to this
Application, should also be included within the application protection domain, and be incorporated herein by reference herein.
Preferably, it is described to obtain the database statement that the user performs in corresponding source database instance
Including：Packet of the user in corresponding source database instance is captured, is carried from the packet
Corresponding candidate data storehouse sentence is taken, and stores candidate data storehouse sentence in the buffer；From described
Candidate data storehouse sentence is captured in caching to obtain corresponding database statement.
Wherein, what is generally corresponded in the packet in the corresponding source database instance is in original machine
Database instance the data statement that produces of history, specifically grasp at the method for the packet and for example lead to
Cross the SQL bags that the TCP layer crawl server end of (SuSE) Linux OS is received.Then, the extraction pair
The candidate data storehouse sentence answered refers to the packet of acquisition carrying out analytical decomposition, so as to obtain described
The SQL bags of database statement in packet, such as parsing obtain SQL statement, but extract data
The mode of sentence is not limited to parse packet in bag, other that the database language can be extracted from packet
The method of sentence is both contained in this.Wherein, candidate data storehouse sentence is to be used as subsequent treatment after extracting
Alternate data storehouse sentence, therefore, candidate data storehouse sentence not only include extracting from the packet
The sentence that produces of associated databases instance histories, in addition to service database that Interruption period between The following article is not produced newly
Sentence, for the acquisition methods of database statement newly produced include but is not limited to by access link come
Mid logarithmic is intercepted according to storehouse sentence.Then, acquired candidate data storehouse sentence is delayed
Deposit, in order to which candidate data storehouse sentence is further processed, while caching candidate's number
Also cause to run into the acquisition candidate data storehouse sentence mistake when handling or shifting database statement according to storehouse sentence
Have little time to handle in the case of fast and cause the performance issue after overstocking.
Those skilled in the art will be understood that the mode of database statement in said extracted packet is only
The mode of database statement such as may be used in citing, other existing or extraction packets for being likely to occur from now on
Suitable for the application, it should also be included within the application protection domain, and include by reference herein
In this.
Wherein, it is described that candidate data storehouse sentence is captured from the caching, refer to the caching
In candidate data storehouse sentence captured by streaming computing framework, but specific method is not limited
Carry out, can be selected according to the specific software frame of implementation, specifically in such as real-time streaming Computational frame
Implementation method it is a lot, oneself can also fully achieve a set of, but function is essentially identical meets to caching
In the function that is captured of candidate data storehouse sentence.It is preferred that recommending real-time streaming calculating
Frame J storm is implemented, wherein, Jstorm belongs to distributed computing engines in real time, is to be considered as
It is storm java enhancing versions, because it belongs to scheme the most ripe in such calculating, uses it
Carry out after crawl splicing, can be shorter and smaller, the storage of sentence is easy in refining, shifts and in number of targets
According to the execution on the example of storehouse, it is the corresponding database statement to play the spliced sentence of crawl.
It is highly preferred that the caching is Circular buffer.Candidate data storehouse sentence is stored in the buffer
Described in caching be preferably Circular buffer, that is, buffer circle because other caching numbers at present
According to structure speed relatively slowly and because of the limitation of machine memory size, the time is deposited using buffer circle
Database statement is selected preferably to adapt to the crawl under the sentence Writing condition of high-speed data storehouse.For example,
I.e. when being used cooperatively with real-time streaming Computational frame Jstorm, except will be from the packet
The candidate data storehouse sentence of middle extraction is put into outside Circular buffer, and the database statement newly produced also can
Constantly enter in Circular buffer, so all database statements are read all from Circular buffer.Because
Circular buffer is loop configuration, and the database statement newly produced constantly enters Circular buffer, when by Jstorm
After crawl, just it is removed from it playing a part of caching.It is that annular is slow using the purpose of Circular buffer
It is a round-robin queue to deposit, can be with the size of control system internal memory, it is to avoid produced or from described due to new
Database statement generation is extracted in packet too fast, and Jstorm has little time crawl and caused after overstocking,
Server memory is overflowed, and performance issue occurs.
Those skilled in the art will be understood that above-mentioned captures candidate data storehouse sentence from the caching
Mode be only for example, other it is existing or be likely to occur from now on the time is captured from the caching
Select the mode of database statement to be such as applicable to the application, also should be included in the application protection domain with
It is interior, and be incorporated herein by reference herein.
Then, target data objects determining device 12 by parse the database statement determine it is pending
Target data objects.Refer to the database by being captured described in lexical analyzer parsing from caching
Sentence, goes out to determine pending target data objects by certain conditional filtering.Such as " mistake in Fig. 5
Filter, screening " function corresponds to this.Wherein, the target data objects are to be used to cache to the mesh
Mark the content of database instance, be specifically including but not limited to select query statements, and its be related to or comprising
Table name etc..I.e. the instrument lexical analyzer of the parsing include but is not limited to SQL parser,
The lexical analyzers such as mixer and mysqllex.The method for determining pending target data objects includes
By the sentence of user's frequency of use height or focus in the data statement, or the corresponding key message of sentence
Extracted, and the possibility of the frequency of use according to user in use is ranked up, so as to obtain
The target data objects are taken, for example, are ranked up by using number of times or file size itself, wherein,
Access times are more and file itself causes the database statement corresponding to it to be that user is made compared with big city
Focus sentence.
Those skilled in the art will be understood that the mode of above-mentioned parsing database statement is only for example, other
The mode of parsing database statement that is existing or being likely to occur from now on is such as applicable to the application, also should
Within the application protection domain, and it is incorporated herein by reference herein.
Then, target data objects buffer storage 13 by correspondence target database example in perform on
The inquiry operation of the target data objects, the target data objects are cached to the target database
Example.Refer to after the destination object is determined, inquiry operation is performed to the destination object, i.e.,
" SQL playback " for example shown in Fig. 5 is in the stage to according to the table in the set quoted after number of times sequence
Title performs following sentences according to from more to less or at most certain is several：Select count(*)from
Table_name, will perform the higher tables of data of number of times and be put into the caching of target database example.
By performing the inquiry operation of the target data objects in object instance, by object instance hard disc data
User's frequency of use will be had compared with higher number by being loaded into caching, the caching of such object instance
According to so that user's operating speed in database migration is unaffected, for example, it is preferred in mesh
The caching of the high table of frequency of use will be stored in the innodb_buffer_pool for marking database instance,
After user's request link switches, the SQL execution response time is hardly by any shadow
Ring.Wherein, innodb_buffer_pool is InnoDB cache pools, is present in calculator memory,
The data frequently accessed in mysql for caching, it both can also cache index with data cached,
Even other manage data, and such as metadata information, row level lock information uses MySQL orders：
Show variables like'innodb%pool%', can check the relevant parameter of InnoDB cache pools
Information.When user inquires about database information using query statement, presence therein can be preferentially inquired about
Data, can greatly improve system response time.
Those skilled in the art will be understood that the mode of above-mentioned caching of target data object is only for example, its
He is such as applicable to the application at the mode of caching of target data object that is existing or being likely to occur from now on,
Also it should be included within the application protection domain, and be incorporated herein by reference herein.
Also include the (not shown) of daily record reproduction device 14, the daily record reproduction device 14 in the equipment 1
Corresponding binary log in the source database example is copied to the target database example.As schemed
I.e. binlog binary logs are belonged to from database source example copies to object instance in 5
A part for database migration, with the action such as " SQL collections " progress simultaneously after Data Migration is initiated,
It is mainly the another part for constituting database migration, caches the target data objects to institute with described
Target database example is stated to carry out completing the process of whole database migration together.
Fig. 4 shows setting according to the target data objects determining device of the application another preferred embodiment
Standby schematic diagram.The target data objects determining device 12 include database statement resolution unit 121 with
And target data objects determining unit 122.
Wherein, database statement resolution unit 121 parses the database statement to obtain processed one
Individual or multiple data objects；Target data objects determining unit 122 is according to the object phase of the data object
Close information and determine pending target data objects.
Specifically, to parse the database statement processed to obtain for database statement resolution unit 121
One or more data objects.Refer to by lexical analyzer parse described in captured from caching described in
Database statement, is easy to determine caching to the data object and its information of target data example so as to obtain,
For example, all select in the database statement are inquired about by SQL parser lexical analyzers
Sentence analysis go out be related to or comprising table name, and be put into a set.Wherein, preferred pair SQL statement
In selection query statement analyzed and find out its be related to or comprising table name because in Data Migration
Caching migration mainly for selection query statement provides maximum performance speed-raising, so only to select inquiry
Sentence is analyzed, analyze and find out its be related to or comprising table name primarily to be easy to count, it is right
The hot spot table that frequency of use is higher or itself is larger is cached.
Those skilled in the art will be understood that the mode of above-mentioned parsing database statement is only for example, other
The mode of parsing database statement that is existing or being likely to occur from now on is such as applicable to the application, also should
Within the application protection domain, and it is incorporated herein by reference herein.
Then, target data objects determining unit 122 is true according to the object-related information of the data object
Fixed pending target data objects.Refer to pass through sequence according to the object-related information of the data object
Or the method such as compare and determine, such as all selection query statement and its be related to or comprising table name belong to
The object-related information of the data object, carries out reference count by the execution number of times of table name, then presses
According to the arrangement of reference count inverted order then according to can be by corresponding to table name in the top from more to few order
Table or sentence be defined as pending target data objects.Preferably in object-related information such as
Fruit is performed number of times, preferentially according to execution number of times, the big first caching of number of times, the first sequence；If table
There is size data, then first cache larger table, the second sequence.The pending number of targets so determined
The database caches information moved in target machine caching is enabled to be that user's frequency of use is high according to object
So that user in the case where database continual service is migrated to the use of database not interim card
Or it is stuck.
Preferably, the object-related information includes following at least any one：The data object is held
Row number information；The data size information of the data object.Wherein, the data object is performed
Number information, the database statement as parsed is performed number of times, such as exemplified above to be previously mentioned
Select query statement is performed number of times, and number information is performed according to described, and it performs number of times higher generation
The frequency of use of table user is higher, and its identified target data objects is got over can be in database migration to target
And make it that the probability that user inquires about or used in the buffer is higher thereafter.The data of the data object are big
Small information, is the size for the table that the database statement parsed is comprised or related to, and its table is bigger
The frequency that user's inquiry goods is used is higher, and its identified target data objects is got over and can arrived in database migration
Target and cause that user's probability for inquiring about or use in the buffer is higher thereafter.
Those skilled in the art, which will be understood that, above-mentioned determines pending number of targets according to object-related information
Be only for example according to the mode of object, other it is existing or be likely to occur from now on according to object-related information
Determine that the mode of pending target data objects is such as applicable to the application, should also be included in the application
Within protection domain, and it is incorporated herein by reference herein.
Preferably, the equipment 1 also includes the (not shown) of log processing parameter setting apparatus 15, institute
State log processing parameter setting apparatus 15 and the corresponding log processing parameter of the source database example is set
Information.Because acquisition database sentence and the process parsing database from the source database example
Sentence determines pending target data objects, by correspondence target database example in perform on
The inquiry operation of the target data objects, the target data objects are cached to the target data
Storehouse example.So, certain query performance loss is had to target database example in the process of implementation,
The speed of object instance application source instance binary log can be reduced.To solve problem above, then
Database relevant parameter can be changed in practical operation, such as concrete operations are as follows：
Sync_binlog=10000
Innodb_flush_log_at_trx_commit=2
By changing parameter, then object instance SQL can be accelerated to perform speed and be expected more than hundred times, because root
According to internal memory, the writing speed of disk is contrasted, it can be appreciated that disk is typically slower than memory speed tens of
Again to more than hundred times.
It is highly preferred that the log processing parameter information includes following at least any one：For submitting in batches
The daily record configuration information of daily record；For by daily record via file system again write storage device daily record configure
Information.The two are the key parameters of influence object instance performance main.It is described to be used for batch
The daily record configuration information such as Sync_binlog of daily record is submitted, if its value is set to 1, is represented per number
When being submitted according to storehouse affairs, binary log is all write into disk.If value is set to 0, represent not actively
Control refreshes binlog refreshing disk, is controlled by operating system oneself, is arranged to 10000 expressions and tires out
Binary log is write disk by property batch again after 10000 affairs of meter, so can be with after modification
Accelerate the submission speed of affairs, and then improve the overall write performance of database.With above-mentioned parameter information
It is similar, it is described be used for by daily record via file system again write storage device daily record configuration information for example
Innodb_flush_log_at_trx_commit, if its value is set to 1, represents that affairs are submitted all each time
Transaction journal is needed to write (flush) hard disk；After value is set to 2, when each affairs are submitted
Log buffer can be brushed in file system, do not brushed on disk also now, but every one
Second calls the i.e. flush buffers area of file system to operate such as " flush " will to be gone on cache flush to disk,
The effect of batch buffering to a certain degree can be provided by file system, writing for affairs has been also speeded up
Enter speed.
Those skilled in the art will be understood that at the corresponding daily record of the above-mentioned setting source database example
The mode of reason parameter information is only for example, other described source numbers of setting that are existing or being likely to occur from now on
The application is such as applicable to according to the mode of the corresponding log processing parameter information of storehouse example, should be also included in
Within the application protection domain, and it is incorporated herein by reference herein.
Obviously, those skilled in the art can to the application carry out it is various change and modification without departing from
Spirit and scope.So, if these modifications and variations of the application belong to the application power
Within the scope of profit requirement and its equivalent technologies, then the application is also intended to exist comprising these changes and modification
It is interior.
It should be noted that the application can be carried out in the assembly of software and/or software and hardware,
For example, can be using application specific integrated circuit (ASIC), general purpose computer or any other is similar hard
Part equipment is realized.In one embodiment, the software program of the application can pass through computing device
To realize steps described above or function.Similarly, the software program of the application (includes the number of correlation
According to structure) it can be stored in computer readable recording medium storing program for performing, for example, RAM memory, magnetic
Or CD-ROM driver or floppy disc and similar devices.In addition, some steps or function of the application can be used
Hardware realizes, for example, as coordinating with processor so as to performing the circuit of each step or function.
In addition, the part of the application can be applied to computer program product, such as computer program
Instruction, when it is computer-executed, by the operation of the computer, can call or provide basis
The present processes and/or technical scheme.And the programmed instruction of the present processes is called, it may be deposited
Store up in fixed or moveable recording medium, and/or by broadcast or other signal bearing medias
Data flow and be transmitted, and/or be stored according to the computer equipment of described program instruction operation
In working storage.Here, including a device, the device bag according to one embodiment of the application
The memory for storing computer program instructions and the processor for execute program instructions are included, its
In, when the computer program instructions are by the computing device, trigger the plant running and be based on foregoing
According to the methods and/or techniques scheme of multiple embodiments of the application.
It is obvious to a person skilled in the art that the application is not limited to the thin of above-mentioned one exemplary embodiment
Section, and in the case of without departing substantially from spirit herein or essential characteristic, can be with other specific
Form realizes the application.Therefore, no matter from the point of view of which point, embodiment all should be regarded as exemplary
, and be nonrestrictive, scope of the present application is limited by appended claims rather than described above
It is fixed, it is intended that all changes fallen in the implication and scope of the equivalency of claim are included
In the application.The right that any reference in claim should not be considered as involved by limitation will
Ask.Furthermore, it is to be understood that the word of " comprising " one is not excluded for other units or step, odd number is not excluded for plural number.
The multiple units or device stated in device claim can also be passed through soft by a unit or device
Part or hardware are realized.The first, the second grade word is used for representing title, and is not offered as any spy
Fixed order.
Claims (16)
1. a kind of method for database data migration, wherein, this method includes：
The database migration request submitted according to user, obtains the user in corresponding source database instance
The database statement of execution；
Pending target data objects are determined by parsing the database statement；
By performing the inquiry operation on the target data objects in correspondence target database example,
The target data objects are cached to the target database example.
2. according to the method described in claim 1, wherein, methods described also includes：
Corresponding binary log in the source database example is copied to the target database example.
3. according to the method described in claim 1, wherein, it is described true by parsing the database statement
Fixed pending target data objects include：
The database statement is parsed to obtain processed one or more data objects；
Pending target data objects are determined according to the object-related information of the data object.
4. method according to claim 3, wherein, the object-related information include it is following at least
Any one：
The data object is performed number information；
The data size information of the data object.
5. method according to any one of claim 1 to 4, wherein, it is described to obtain the user
The database statement performed in corresponding source database instance includes：
Packet of the user in corresponding source database instance is captured, the extraction pair from the packet
The candidate data storehouse sentence answered, and candidate data storehouse sentence is stored in the buffer；
Candidate data storehouse sentence is captured from the caching to obtain corresponding database statement.
6. method according to claim 5, wherein, the caching is Circular buffer.
7. according to the method described in claim 1, wherein, methods described also includes：
The corresponding log processing parameter information of the source database example is set.
8. method according to claim 7, wherein, the log processing parameter information includes following
Any one of at least：
Daily record configuration information for submitting daily record in batches；
For by daily record configuration information of the daily record via file system write storage device again.
9. a kind of equipment for database data migration, wherein, the equipment includes：
Database statement acquisition device, the database migration for being submitted according to user is asked, and obtains described
The database statement that user performs in corresponding source database instance；
Target data objects determining device, for determining pending mesh by parsing the database statement
Mark data object；
Target data objects buffer storage, for by being performed in correspondence target database example on institute
The inquiry operation of target data objects is stated, the target data objects are cached real to the target database
Example.
10. equipment according to claim 9, wherein, the equipment also includes：
Daily record reproduction device, for corresponding binary log in the source database example to be copied into institute
State target database example.
11. equipment according to claim 9, wherein, the target data objects determining device bag
Include：
Database statement resolution unit, for parsing the database statement with obtain processed one or
Multiple data objects；
Target data objects determining unit, for being determined to treat according to the object-related information of the data object
The target data objects of execution.
12. equipment according to claim 11, wherein, the object-related information is included below extremely
It is any one of few：
The data object is performed number information；
The data size information of the data object.
13. the equipment according to any one of claim 9 to 12, wherein, it is described to obtain described use
The database statement that family is performed in corresponding source database instance includes：
Packet of the user in corresponding source database instance is captured, the extraction pair from the packet
The candidate data storehouse sentence answered, and candidate data storehouse sentence is stored in the buffer；
Candidate data storehouse sentence is captured from the caching to obtain corresponding database statement.
14. equipment according to claim 13, wherein, the caching is Circular buffer.
15. equipment according to claim 9, wherein, the equipment also includes：
Log processing parameter setting apparatus, for setting the corresponding log processing ginseng of the source database example
Number information.
16. equipment according to claim 15, wherein, the log processing parameter information include with
Under any one of at least：
Daily record configuration information for submitting daily record in batches；
For by daily record configuration information of the daily record via file system write storage device again.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610023836.2A CN106970920A (en) | 2016-01-14 | 2016-01-14 | A kind of method and apparatus for database data migration |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610023836.2A CN106970920A (en) | 2016-01-14 | 2016-01-14 | A kind of method and apparatus for database data migration |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106970920A true CN106970920A (en) | 2017-07-21 |
Family
ID=59334481
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610023836.2A Pending CN106970920A (en) | 2016-01-14 | 2016-01-14 | A kind of method and apparatus for database data migration |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106970920A (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108959354A (en) * | 2018-05-04 | 2018-12-07 | 北京小米移动软件有限公司 | Data processing method, device and server |
| CN110008284A (en) * | 2019-03-18 | 2019-07-12 | 武汉达梦数据库有限公司 | Method for synchronizing data of database and equipment based on data page preloading and rollback |
| CN110334075A (en) * | 2019-04-04 | 2019-10-15 | 平安科技（深圳）有限公司 | Data migration method and relevant device based on message-oriented middleware |
| CN110908974A (en) * | 2018-09-14 | 2020-03-24 | 阿里巴巴集团控股有限公司 | Database management method, device, equipment and storage medium |
| CN111090638A (en) * | 2019-12-25 | 2020-05-01 | 中国工商银行股份有限公司 | Method and device for comparing transaction functions in database migration |
| CN111241068A (en) * | 2020-01-14 | 2020-06-05 | 阿里巴巴集团控股有限公司 | Information processing method, device and equipment and computer readable storage medium |
| CN111367894A (en) * | 2020-03-31 | 2020-07-03 | 中国工商银行股份有限公司 | Data comparison method and device based on database migration |
| CN112559448A (en) * | 2021-02-19 | 2021-03-26 | 腾讯科技（深圳）有限公司 | Data state migration method and device, computer equipment and storage medium |
| CN113312331A (en) * | 2020-04-01 | 2021-08-27 | 阿里巴巴集团控股有限公司 | Data migration method, device, system, electronic equipment and computer readable medium |
| CN113360482A (en) * | 2021-08-10 | 2021-09-07 | 深圳市中科鼎创科技股份有限公司 | SQL database-based online migration method |
| WO2023065861A1 (en) * | 2021-10-18 | 2023-04-27 | Huawei Cloud Computing Technologies Co., Ltd. | Application migration assessment |
| CN116980475A (en) * | 2023-07-31 | 2023-10-31 | 深圳市亲邻科技有限公司 | Data pushing system based on binlog and double annular buffer areas |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040024771A1 (en) * | 2002-08-01 | 2004-02-05 | Oracle International Corporation | Buffered message queue architecture for database management systems with transactional enqueue support |
| CN101187888A (en) * | 2007-12-11 | 2008-05-28 | 浪潮电子信息产业股份有限公司 | Method for coping database data in heterogeneous environment |
| CN101615199A (en) * | 2009-07-31 | 2009-12-30 | 深圳市珍爱网信息技术有限公司 | Heterogeneous database synchronization method and system |
| CN102521260A (en) * | 2011-11-18 | 2012-06-27 | 华为技术有限公司 | Data preheating method and device |
| CN105183581A (en) * | 2015-07-23 | 2015-12-23 | 深圳市沃信科技有限公司 | Database disaster tolerance system |
-
2016
- 2016-01-14 CN CN201610023836.2A patent/CN106970920A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040024771A1 (en) * | 2002-08-01 | 2004-02-05 | Oracle International Corporation | Buffered message queue architecture for database management systems with transactional enqueue support |
| CN101187888A (en) * | 2007-12-11 | 2008-05-28 | 浪潮电子信息产业股份有限公司 | Method for coping database data in heterogeneous environment |
| CN101615199A (en) * | 2009-07-31 | 2009-12-30 | 深圳市珍爱网信息技术有限公司 | Heterogeneous database synchronization method and system |
| CN102521260A (en) * | 2011-11-18 | 2012-06-27 | 华为技术有限公司 | Data preheating method and device |
| CN105183581A (en) * | 2015-07-23 | 2015-12-23 | 深圳市沃信科技有限公司 | Database disaster tolerance system |
Non-Patent Citations (6)
| Title |
|---|
| （D.皮蒂斯）DAVID PITTS等: "《Red Hat Linux 6大全》", 31 May 2000, 机械工业出版社 * |
| CIYONGYUAN4432: ""通过sql profiler捕获sql"", 《HTTPS://BLOG.CSDN.NET/CIYONGYUAN4432/ARTICLE/DETAILS/100466161》 * |
| 于宏毅 等: "《无线传感器网络理论、技术与实现》", 30 September 2008, 国防工业出版社 * |
| 杨正洪 等: "《中文SQL Server 2000关系数据库系统管理和开发指南》", 30 September 2001, 机械工业出版社 * |
| 王老虎888: ""快速预热innodb buffer pool的方法"", 《360个人图书馆》 * |
| 程国建: "《迁移到云端 在云计算的新世界开发应用》", 30 June 2015, 国防工业出版社 * |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108959354A (en) * | 2018-05-04 | 2018-12-07 | 北京小米移动软件有限公司 | Data processing method, device and server |
| CN110908974A (en) * | 2018-09-14 | 2020-03-24 | 阿里巴巴集团控股有限公司 | Database management method, device, equipment and storage medium |
| CN110008284A (en) * | 2019-03-18 | 2019-07-12 | 武汉达梦数据库有限公司 | Method for synchronizing data of database and equipment based on data page preloading and rollback |
| CN110334075B (en) * | 2019-04-04 | 2023-06-20 | 平安科技（深圳）有限公司 | Data migration method based on message middleware and related equipment |
| CN110334075A (en) * | 2019-04-04 | 2019-10-15 | 平安科技（深圳）有限公司 | Data migration method and relevant device based on message-oriented middleware |
| CN111090638A (en) * | 2019-12-25 | 2020-05-01 | 中国工商银行股份有限公司 | Method and device for comparing transaction functions in database migration |
| CN111090638B (en) * | 2019-12-25 | 2023-07-04 | 中国工商银行股份有限公司 | Comparison method and device for transaction functions in database migration |
| CN111241068A (en) * | 2020-01-14 | 2020-06-05 | 阿里巴巴集团控股有限公司 | Information processing method, device and equipment and computer readable storage medium |
| CN111241068B (en) * | 2020-01-14 | 2023-04-07 | 阿里巴巴集团控股有限公司 | Information processing method, device and equipment and computer readable storage medium |
| CN111367894A (en) * | 2020-03-31 | 2020-07-03 | 中国工商银行股份有限公司 | Data comparison method and device based on database migration |
| CN111367894B (en) * | 2020-03-31 | 2023-08-18 | 中国工商银行股份有限公司 | Data comparison method and device based on database migration |
| CN113312331A (en) * | 2020-04-01 | 2021-08-27 | 阿里巴巴集团控股有限公司 | Data migration method, device, system, electronic equipment and computer readable medium |
| CN112559448B (en) * | 2021-02-19 | 2021-05-18 | 腾讯科技（深圳）有限公司 | Data state migration method and device, computer equipment and storage medium |
| CN112559448A (en) * | 2021-02-19 | 2021-03-26 | 腾讯科技（深圳）有限公司 | Data state migration method and device, computer equipment and storage medium |
| CN113360482A (en) * | 2021-08-10 | 2021-09-07 | 深圳市中科鼎创科技股份有限公司 | SQL database-based online migration method |
| WO2023065861A1 (en) * | 2021-10-18 | 2023-04-27 | Huawei Cloud Computing Technologies Co., Ltd. | Application migration assessment |
| CN116980475A (en) * | 2023-07-31 | 2023-10-31 | 深圳市亲邻科技有限公司 | Data pushing system based on binlog and double annular buffer areas |
| CN116980475B (en) * | 2023-07-31 | 2024-06-04 | 深圳市亲邻科技有限公司 | Data pushing system based on binlog and double annular buffer areas |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106970920A (en) | A kind of method and apparatus for database data migration | |
| US11379755B2 (en) | Feature processing tradeoff management | |
| US12118122B2 (en) | Masking sensitive information in records of filtered accesses to unstructured data | |
| US11100420B2 (en) | Input processing for machine learning | |
| US10339465B2 (en) | Optimized decision tree based models | |
| US11182691B1 (en) | Category-based sampling of machine learning data | |
| US10318882B2 (en) | Optimized training of linear machine learning models | |
| US10366053B1 (en) | Consistent randomized record-level splitting of machine learning data | |
| US10540606B2 (en) | Consistent filtering of machine learning data | |
| US10452992B2 (en) | Interactive interfaces for machine learning model evaluations | |
| CN109034993A (en) | Account checking method, equipment, system and computer readable storage medium | |
| US9817879B2 (en) | Asynchronous data replication using an external buffer table | |
| CN107368517A (en) | A kind of method and apparatus of high amount of traffic inquiry | |
| US10437837B2 (en) | Generating descriptive topic labels | |
| US20090249248A1 (en) | User directed refinement of search results while preserving the scope of the initial search | |
| US20210318935A1 (en) | Method of using statistical backup validation for predictable database restore time | |
| US10664508B1 (en) | Server-side filtering of unstructured data items at object storage services | |
| US10515102B2 (en) | Pre-processing of geo-spatial sensor data | |
| US20220277012A1 (en) | Workload-aware column inprints | |
| CN106407442A (en) | Massive text data processing method and apparatus | |
| US11526502B1 (en) | Parameterized disjoint samples of data sets | |
| US20240232722A1 (en) | Handling system-characteristics drift in machine learning applications | |
| US11693853B2 (en) | Capture and replay of user requests for performance analysis | |
| CN115809304A (en) | Method and device for analyzing field-level blood margin, computer equipment and storage medium | |
| US20070150449A1 (en) | Database program acceleration |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170721 |