CN109726243A - A kind of purpose data classifying method, apparatus and system - Google Patents
A kind of purpose data classifying method, apparatus and system Download PDFInfo
- Publication number
- CN109726243A CN109726243A CN201910004489.2A CN201910004489A CN109726243A CN 109726243 A CN109726243 A CN 109726243A CN 201910004489 A CN201910004489 A CN 201910004489A CN 109726243 A CN109726243 A CN 109726243A
- Authority
- CN
- China
- Prior art keywords
- collected
- data
- history
- classifying
- state
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
This application discloses a kind of purpose data classifying methods, apparatus and system, by the purpose data classifying state for obtaining history data to be collected；According to the purpose data classifying state of history data to be collected, the history data to be collected collected not successfully are determined from source database；According to the history data to be collected collected not successfully, data currently to be collected are determined；Currently data to be collected are collected to target database.Before carrying out purpose data classifying every time, the purpose data classifying state of history data to be collected can be all obtained；To the purpose data classifying state according to history data to be collected, the history data to be collected collected not successfully are determined from source database, and then when carrying out purpose data classifying, the history collected not successfully data to be collected are collected to target database again, ensure that the data to be collected in source database can be collected successfully in target database, ensure that the consistency of the data in the data and target database in source database.
Description
Technical field
The present invention relates to technical field of data processing, and in particular to a kind of purpose data classifying method, apparatus and system.
Background technique
Existing prevalent systems framework can generate a large amount of business datum due to the diversity of business transaction channel, and
These business datums will disperse in different source databases, these source databases may reside in the same database service
On device, it is also possible to be distributed on different database servers.In order to facilitate inquiry, statistics or these data are analyzed, are needed
It will be dispersed in the target database that the purpose data classifying in not genbank database is fixed to some.
The prior art is in the purpose data classifying that will be dispersed in not genbank database to some target database fixed
In the process, often due to network delay or server it is soft/hard fault or data volume it is excessive, processor load is overweight, can not
The reasons such as all data are disposably collected, partial data is caused to collect not successfully, lead to data and number of targets in source database
It is inconsistent according to the data in library.
Summary of the invention
In view of this, the embodiment of the present invention provides a kind of purpose data classifying method, apparatus and system, it is able to solve the prior art
The inconsistent problem of data in data and target database in middle source database.
A kind of purpose data classifying method, comprising:
Obtain the purpose data classifying state of history data to be collected；
According to the purpose data classifying state of history data to be collected, the history collected not successfully is determined from source database
Data to be collected；
According to the history collected not successfully the data to be collected, data currently to be collected are determined；
The data currently to be collected are collected to target database.
Preferably, the purpose data classifying state for obtaining history data to be collected includes:
In the purpose data classifying state table stored from target database, the purpose data classifying shape of query history data to be collected
State.
Preferably, the purpose data classifying state according to history data to be collected, from source database determine not at
The process of the history that function collects data to be collected includes:
According to the purpose data classifying state of the history data to be collected inquired, from the purpose data classifying state table, determine
Purpose data classifying state is the storage location of the history data to be collected collected not successfully；
From the storage location of the history data to be collected collected not successfully, the history number to be collected collected not successfully is extracted
According to.
Preferably, include: by the process that the data currently to be collected are collected to target database
The period is collected according to default, currently data to be collected are collected to target database；
Or；According to the storage location of the source database, currently data to be collected are collected to target database.
Preferably, after the data currently to be collected being collected to target database, further includes:
After detecting that currently data to be collected are collected to target database, the data currently to be collected are obtained
Purpose data classifying state；
According to the purpose data classifying state of the data currently to be collected, to the purpose data classifying in the purpose data classifying state table
State is updated.
A kind of purpose data classifying device, comprising:
First purpose data classifying state acquisition module, for obtaining the purpose data classifying state of history data to be collected；
History data determining module to be collected, for the purpose data classifying state according to history data to be collected, from source
The history data to be collected collected not successfully are determined in database；
Currently data determining module to be collected, for according to the history collected not successfully the data to be collected, determination to be worked as
Preceding data to be collected；
Purpose data classifying module, for collecting the data currently to be collected to target database.
Preferably, the first purpose data classifying state acquisition module includes:
First purpose data classifying state acquisition submodule, the purpose data classifying state table for being stored from target database
In, the purpose data classifying state of query history data to be collected.
Preferably, history data determining module to be collected includes:
Storage location determining module, for according to the purpose data classifying state of history data to be collected inquired, from described
In purpose data classifying state table, determine that purpose data classifying state is the storage location of the history data to be collected collected not successfully；
History data extraction module to be collected, for from the storage location of the history data to be collected collected not successfully,
Extract the history data to be collected collected not successfully.
Preferably, the purpose data classifying module includes:
First purpose data classifying submodule, for collecting the period according to default, by currently data to be collected to target database
It is collected；
Or；
Second purpose data classifying submodule, for the storage location according to the source database, by currently data to be collected to
Target database is collected.
Preferably, further includes:
Second purpose data classifying state acquisition module, be used for purpose data classifying state determining module, for detect currently to
It collects after data collect to target database, obtains the purpose data classifying state of the data currently to be collected；
Purpose data classifying state update module, for the purpose data classifying state according to the data currently to be collected, to described
Purpose data classifying state in purpose data classifying state table is updated.
A kind of purpose data classifying system, comprising: purpose data classifying device as described above.
Based on the above-mentioned technical proposal, the embodiment of the invention discloses a kind of purpose data classifying methods, apparatus and system, by obtaining
Take the purpose data classifying state of history data to be collected；According to the purpose data classifying state of history data to be collected, from source data
The history data to be collected collected not successfully are determined in library；According to the history collected not successfully the data to be collected, determination is worked as
Preceding data to be collected；The data currently to be collected are collected to target database.The embodiment of the present invention is carrying out every time
Before purpose data classifying, the purpose data classifying state of history data to be collected can be all obtained；To the number according to history data to be collected
According to collecting state, determine the history data to be collected collected not successfully from source database, so carry out purpose data classifying when
Wait, the history collected not successfully data to be collected are collected to target database again, it is ensured that in source database wait return
Collection data can be collected successfully in target database, ensure that the data in the data and target database in source database
Consistency.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
The embodiment of invention for those of ordinary skill in the art without creative efforts, can also basis
The attached drawing of offer obtains other attached drawings.
Fig. 1 is a kind of flow chart of purpose data classifying method provided in an embodiment of the present invention；
Fig. 2 is the flow chart of another purpose data classifying method provided in an embodiment of the present invention；
Fig. 3 is the flow chart of another purpose data classifying method provided in an embodiment of the present invention；
Fig. 4 is a kind of structural block diagram of purpose data classifying device provided in an embodiment of the present invention；
Fig. 5 is a kind of hardware block diagram of purpose data classifying device provided in an embodiment of the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment shall fall within the protection scope of the present invention.
Fig. 1 shows a kind of flow chart of purpose data classifying method, and this method can be realized by server, referring to Fig.1, the side
Method may include:
Step S100, the purpose data classifying state of history data to be collected is obtained；
It should be noted that the history data to be collected in the embodiment of the present invention refer to executing to target database
The data for collecting operation are crossed, but due to network delay or server failure etc., executed and returns to target database
The data of collection operation are possible to collect success, synchronize and are stored in target database；It is also possible to collect failure, can not synchronize and deposit
It stores up in target database, or since data volume is very big, to avoid causing server burden and process resource waste, only
Partial data can once be collected, remaining data need to be conducted batch-wise all or still and collect next, this collects operation, collects failure
Processing.
Based on this, the purpose data classifying state of history data to be collected is referred to number of targets in the embodiment of the present invention
It was executed according to library and collects the data of operation and whether succeeded from synchronizing the state collected into target database in source database.
Optionally, the purpose data classifying state of the history data to be collected in the embodiment of the present invention can be divided into: successfully collect
State and state is collected not successfully, wherein collect state again not successfully and can be divided into and do not collect state and collect abnormal shape
State.
Step S110, the purpose data classifying state according to history data to be collected determines failed from source database
The history collected data to be collected；
It by purpose data classifying state is that the data collected not successfully are determined as the history collected not successfully in the embodiment of the present invention
Data to be collected.
Step S120, according to the history collected not successfully the data to be collected, data currently to be collected are determined；
In the embodiment of the present invention, can using the history collected not successfully data to be collected as currently data to be collected,
The data collect that can be determined by the history collected not successfully data to be collected and currently are together as current
Data to be collected.
Step S130, the data currently to be collected are collected to target database.
Optionally, the embodiment of the present invention is in the process for collecting the data currently to be collected to target database
In, the period can be collected according to default, currently data to be collected are collected to target database；Or；According to the source
The storage location of database collects currently data to be collected to target database.
In the storage location according to the source database, by the mistake that currently data to be collected are collected to target database
Cheng Zhong, can be according to the storage location of the source database, and reference data collects the sequence recorded in state table, successively will place
It is collected in the current data to be collected of different location to target database.
It is described default to collect the period and be held for those skilled in the art according to the data volume and network of data to be collected
The time range that the information such as loading capability determine, for example, can for daily, every two days, weekly or every month etc., also, at each
Default collect sets preset time period in the period, carry out purpose data classifying, some time being specifically as follows in daily 24 hours
Section, in order to not influence the operation of operation system, it is 1 point to 6 points generally daily between any time period, the embodiment of the present invention do not do
It is specific to limit.
The storage location of source database can be different storage servers, different storage systems or different storages
Tables of data etc., the embodiment of the present invention is not specifically limited.
The purpose data classifying state that the embodiment of the present invention passes through acquisition history data to be collected；According to history number to be collected
According to purpose data classifying state, the history data to be collected that collect not successfully are determined from source database；Return not successfully according to described
The history of collection data to be collected determine data currently to be collected；The data currently to be collected are returned to target database
Collection.The embodiment of the present invention can all obtain the purpose data classifying state of history data to be collected before carrying out purpose data classifying every time；From
And the purpose data classifying state according to history data to be collected, the history number to be collected collected not successfully is determined from source database
According to, and then when carrying out purpose data classifying, the history collected not successfully data to be collected are carried out to target database again
It collects, it is ensured that the data to be collected in source database can be collected successfully in target database, ensure that in source database
The consistency of data in data and target database.
Optionally, Fig. 2 shows the flow chart of another purpose data classifying method, this method can be realized by server, reference
Fig. 2, the method may include:
Step S200, in the purpose data classifying state table stored from target database, query history data to be collected
Purpose data classifying state；
It should be noted that a purpose data classifying state table, this hair are stored in the embodiment of the present invention in target database
Can be from purpose data classifying state table in bright embodiment, the purpose data classifying state of query history data to be collected, the data are returned
Collect state table in other than being stored with the state of purpose data classifying success or not, at the beginning of being also stored with purpose data classifying and
End time, the information such as storage location of the data collected not successfully, the embodiment of the present invention are not specifically limited.
Step S210, the purpose data classifying state according to the history data to be collected inquired, from the purpose data classifying state
In table, determine that purpose data classifying state is the storage location of the history data to be collected collected not successfully；
Wherein, storage location is specifically as follows the history collected not successfully and waits collecting server or system where data
In specific location, such as: the position number etc. of place server address, source systems, data table name or data in table.
Since the data dispersion storage in source database is on a different server or in different system databases, this
By recording the storage location of the history collected not successfully data to be collected in purpose data classifying state table in inventive embodiments, from
And it can quickly navigate at the storage location of the history collected not successfully data to be collected, and then rapidly extracting to returning not successfully
The history of collection data to be collected.
Step S220, from the storage location of the history data to be collected collected not successfully, what extraction collected not successfully is gone through
History data to be collected；
Step S230, according to the history collected not successfully the data to be collected, data currently to be collected are determined；
Step S240, the data currently to be collected are collected to target database.
The embodiment of the present invention is by obtaining history number to be collected in the purpose data classifying state table that stores in target database
According to purpose data classifying state；According to the purpose data classifying state table that is stored in target database, determined from source database not at
The storage location of the history that function collects data to be collected；From the storage location of the history data to be collected collected not successfully, mention
Take the history collected not successfully data to be collected；According to the history collected not successfully the data to be collected, determine currently wait return
Collect data；The data currently to be collected are collected to target database.The embodiment of the present invention is returned in progress data every time
Before collection, the purpose data classifying state of history data to be collected can be all obtained；To the purpose data classifying according to history data to be collected
State determines the history data to be collected collected not successfully from source database, and then when carrying out purpose data classifying, will not
The history successfully collected data to be collected are collected to target database again, it is ensured that the data to be collected in source database are all
It can successfully collect in target database, ensure that the consistency of the data in the data and target database in source database.
Simultaneously as by having recorded the history collected not successfully data to be collected in purpose data classifying state table in the embodiment of the present invention
Storage location, so as to according to the storage location, quickly navigate to the history collected not successfully data to be collected, and then fast
Speed extracts the history collected not successfully data to be collected, and improves treatment effeciency, saves system resource.
Optionally, Fig. 3 shows the flow chart of another purpose data classifying method, and this method can be realized by server, reference
Fig. 3, the method may include:
Step S300, the purpose data classifying state of history data to be collected is obtained；
Step S310, the purpose data classifying state according to history data to be collected determines failed from source database
The history collected data to be collected；
Step S320, according to the history collected not successfully the data to be collected, data currently to be collected are determined；
Step S330, the data currently to be collected are collected to target database；
The specific implementation process of step S300- step S330 in the embodiment of the present invention please refers to the step in above-described embodiment
The specific implementation process of rapid S100- step S130.
After the embodiment of the present invention is again collected the data currently to be collected to target database, also to having returned
The data of collection carry out mark, which data that convenient processor distinguishes in source database, which have been done, collects processing, which data is not done
Processing is collected, and then is collected in period or next storage location in next preset, can quickly navigate to and not collect processing
The initial position of the data of (i.e. non-mark), continues data in the original position of the data for not collecting processing and returns
Collection, prevents the omission of data.
The embodiment of the present invention is according to the default storage location for collecting period or source database, by currently data to be collected
During being collected to target database, the period can be collected in current preset or is currently deposited by the way of poll
Storage space is set, and the data collected are judged whether there is, if it is present the data to non-mark execute purpose data classifying
Operation continues to judge, thus certainly if it does not exist, then jumping to next default collect in period or next storage location
It is dynamic to skip the not date of data or the not storage location of data, improve the treatment effeciency for collecting operation.
It should be noted that data to be collected for the history collected not successfully, even if being done during collecting for the first time
Mark, but since the partial data collects not successfully, it is still needed during this carries out purpose data classifying for the partial data
Carry out purpose data classifying.
Step S340, after detecting that currently data to be collected are collected to target database, obtain it is described currently to
Collect the purpose data classifying state of data；
It is finished it should be noted that detecting currently data to be collected in the embodiment of the present invention and whether being collected to target database
Mode can be with are as follows: it is each it is default collect in the period, whether detection current preset collects the data to be collected in the period to mesh
Mark database is collected and is finished, alternatively, in each storage location, whether detects data to be collected on current storage location to mesh
Mark database, which collects, to be finished.
Currently the purpose data classifying state of data to be collected can be successfully to collect state and collect state not successfully,
In, state is collected not successfully again can be divided into do not collect state and collect abnormality.
Step S350, the purpose data classifying state according to the data currently to be collected, in the purpose data classifying state table
Purpose data classifying state be updated.
In the embodiment of the present invention, after the purpose data classifying state for determining data currently to be collected, in addition to waiting returning to history
Except the purpose data classifying state of collection data is updated, also the state that collects of this data collected is added in number
According to collecting in state table, mainly according to currently wait collect the different storage locations where data, being collected with this
The states that collects of data the purpose data classifying state of the history of respective memory locations data to be collected is updated.
The embodiment of the present invention is after purpose data classifying, by the purpose data classifying state according to the data currently to be collected,
After being updated to the purpose data classifying state in the purpose data classifying state table, before carrying out purpose data classifying next time, lead to
Cross the purpose data classifying state for obtaining the history data to be collected recorded in updated purpose data classifying state table；And then according to described
The purpose data classifying state of history data to be collected determines the history data to be collected collected not successfully, in turn from source database
When carrying out purpose data classifying, the history collected not successfully data to be collected are collected to target database again, really
The data to be collected protected in source database can be collected successfully in target database, ensure that the data and mesh in source database
The consistency of the data in database is marked, in target database data are further processed, shown with operation provides
Accurate and comprehensive data basis.
Purpose data classifying device provided in an embodiment of the present invention is introduced below, purpose data classifying device described below,
Reference can be corresponded to each other with above-described purpose data classifying method.Purpose data classifying device described below, it is believed that be service
The device purpose data classifying method that embodiment provides to realize the present invention, the functional module framework of required setting.
Fig. 4 is the structural block diagram of purpose data classifying device provided in an embodiment of the present invention, which can be applied to server, is joined
According to Fig. 4, the apparatus may include:
First purpose data classifying state acquisition module 100, for obtaining the purpose data classifying state of history data to be collected；
It should be noted that the history data to be collected in the embodiment of the present invention refer to executing to target database
The data for collecting operation are crossed, but due to network delay or server failure etc., executed and returns to target database
The data of collection operation are possible to collect success, synchronize and are stored in target database；It is also possible to collect failure, can not synchronize and deposit
It stores up in target database, or since data volume is very big, to avoid causing server burden and process resource waste, only
Partial data can once be collected, remaining data need to be conducted batch-wise all or still and collect next, this collects operation, collects failure
Processing.
Based on this, the purpose data classifying state of history data to be collected is referred to number of targets in the embodiment of the present invention
It was executed according to library and collects the data of operation and whether succeeded from synchronizing the state collected into target database in source database.
History data determining module 110 to be collected, for the purpose data classifying state according to history data to be collected, from
The history data to be collected collected not successfully are determined in source database；
It by purpose data classifying state is that the data collected not successfully are determined as the history collected not successfully in the embodiment of the present invention
Data to be collected.
Currently data determining module 120 to be collected, for determining according to the history collected not successfully the data to be collected
Currently data to be collected；
In the embodiment of the present invention, can using the history collected not successfully data to be collected as currently data to be collected,
The data collect that can be determined by the history collected not successfully data to be collected and currently are together as current
Data to be collected.
Purpose data classifying module 130, for collecting the data currently to be collected to target database.
Optionally, the embodiment of the present invention is in the process for collecting the data currently to be collected to target database
In, the period can be collected according to default, currently data to be collected are collected to target database；Or；According to the source
The storage location of database collects currently data to be collected to target database.
Based on above-mentioned purpose data classifying Installation practice, the present invention can also be chosen proper modules from lower module and be formed newly
Purpose data classifying device, concrete composition mode can the associated description in corresponding method embodiment be determined, the present embodiment is no longer superfluous
It states.
The first purpose data classifying state acquisition module includes:
First purpose data classifying state acquisition submodule, the purpose data classifying state table for being stored from target database
In, the purpose data classifying state of query history data to be collected.
The history data determining module to be collected includes:
Storage location determining module, for according to the purpose data classifying state of history data to be collected inquired, from described
In purpose data classifying state table, determine that purpose data classifying state is the storage location of the history data to be collected collected not successfully；
History data extraction module to be collected, for from the storage location of the history data to be collected collected not successfully,
Extract the history data to be collected collected not successfully.
The purpose data classifying module includes:
First purpose data classifying submodule, for collecting the period according to default, by currently data to be collected to target database
It is collected；
Or；
Second purpose data classifying submodule, for the storage location according to the source database, by currently data to be collected to
Target database is collected.
Further include:
Second purpose data classifying state acquisition module, be used for purpose data classifying state determining module, for detect currently to
It collects after data collect to target database, obtains the purpose data classifying state of the data currently to be collected；
Purpose data classifying state update module, for the purpose data classifying state according to the data currently to be collected, to described
Purpose data classifying state in purpose data classifying state table is updated.
Optionally, purpose data classifying device can be hardware device, and above-described module can be set to be filled in purpose data classifying
Set interior functional module.Fig. 5 shows the hardware block diagram of purpose data classifying device, and referring to Fig. 5, purpose data classifying device be can wrap
It includes: processor 1, communication interface 2, memory 3 and communication bus 4；Wherein processor 1, communication interface 2, memory 3 pass through communication
Bus 4 completes mutual communication；Optionally, communication interface 2 can be the interface of communication module, such as the interface of gsm module；
Processor 1, for executing program；Memory 3, for storing program；Program may include program code, the journey
Sequence code includes computer operation instruction；
Processor 1 may be a central processor CPU or specific integrated circuit ASIC (Application
Specific Integrated Circuit), or be arranged to implement the integrated electricity of one or more of the embodiment of the present invention
Road；Memory 3 may include high speed RAM memory, it is also possible to further include nonvolatile memory (non-volatile
Memory), a for example, at least magnetic disk storage.
Wherein, program can be specifically used for:
Obtain the purpose data classifying state of history data to be collected；
According to the purpose data classifying state of history data to be collected, the history collected not successfully is determined from source database
Data to be collected；
According to the history collected not successfully the data to be collected, data currently to be collected are determined；
The data currently to be collected are collected to target database.
The embodiment of the invention also discloses a kind of purpose data classifying systems, comprising: purpose data classifying device as described above.
In summary:
The embodiment of the invention discloses a kind of purpose data classifying methods, apparatus and system, by obtaining history data to be collected
Purpose data classifying state；According to the purpose data classifying state of history data to be collected, determination is returned not successfully from source database
The history of collection data to be collected；According to the history collected not successfully the data to be collected, data currently to be collected are determined；By institute
Currently data to be collected are stated to be collected to target database.The embodiment of the present invention all can before carrying out purpose data classifying every time
Obtain the purpose data classifying state of history data to be collected；To the purpose data classifying state according to history data to be collected, from source number
According to the history data to be collected collected not successfully determining in library, and then when carrying out purpose data classifying, by what is collected not successfully
History data to be collected are collected to target database again, it is ensured that the data to be collected in source database can be collected successfully
Into target database, the consistency of the data in the data and target database in source database ensure that.
Each embodiment in this specification is described in a progressive manner, the highlights of each of the examples are with other
The difference of embodiment, the same or similar parts in each embodiment may refer to each other.For device disclosed in embodiment
For, since it is corresponded to the methods disclosed in the examples, so being described relatively simple, related place is said referring to method part
It is bright.
Professional further appreciates that, unit described in conjunction with the examples disclosed in the embodiments of the present disclosure
And algorithm steps, can be realized with electronic hardware, computer software, or a combination of the two, in order to clearly demonstrate hardware and
The interchangeability of software generally describes each exemplary composition and step according to function in the above description.These
Function is implemented in hardware or software actually, the specific application and design constraint depending on technical solution.Profession
Technical staff can use different methods to achieve the described function each specific application, but this realization is not answered
Think beyond the scope of this invention.
The step of method described in conjunction with the examples disclosed in this document or algorithm, can directly be held with hardware, processor
The combination of capable software module or the two is implemented.Software module can be placed in random access memory (RAM), memory, read-only deposit
Reservoir (ROM), electrically programmable ROM, electrically erasable ROM, register, hard disk, moveable magnetic disc, CD-ROM or technology
In any other form of storage medium well known in field.
The foregoing description of the disclosed embodiments enables those skilled in the art to implement or use the present invention.
Various modifications to these embodiments will be readily apparent to those skilled in the art, as defined herein
General Principle can be realized in other embodiments without departing from the spirit or scope of the present invention.Therefore, of the invention
It is not intended to be limited to the embodiments shown herein, and is to fit to and the principles and novel features disclosed herein phase one
The widest scope of cause.
Claims (11)
1. a kind of purpose data classifying method characterized by comprising
Obtain the purpose data classifying state of history data to be collected；
According to the purpose data classifying state of history data to be collected, determine that the history collected not successfully waits returning from source database
Collect data；
According to the history collected not successfully the data to be collected, data currently to be collected are determined；
The data currently to be collected are collected to target database.
2. the method according to claim 1, wherein the purpose data classifying state for obtaining history data to be collected
Include:
In the purpose data classifying state table stored from target database, the purpose data classifying state of query history data to be collected.
3. according to the method described in claim 2, it is characterized in that, the purpose data classifying according to history data to be collected
State determines that the process of the history data to be collected collected not successfully includes: from source database
Purpose data classifying state according to the history data to be collected inquired determines data from the purpose data classifying state table
Collect the storage location that state is the history data to be collected collected not successfully；
From the storage location of the history data to be collected collected not successfully, the history data to be collected collected not successfully are extracted.
4. the method according to claim 1, wherein the data currently to be collected are carried out to target database
The process collected includes:
The period is collected according to default, currently data to be collected are collected to target database；
Or；According to the storage location of the source database, currently data to be collected are collected to target database.
5. the method according to claim 1, wherein the data currently to be collected are carried out to target database
After collecting, further includes:
After detecting that currently data to be collected are collected to target database, the data of the data currently to be collected are obtained
Collect state；
According to the purpose data classifying state of the data currently to be collected, to the purpose data classifying state in the purpose data classifying state table
It is updated.
6. a kind of purpose data classifying device characterized by comprising
First purpose data classifying state acquisition module, for obtaining the purpose data classifying state of history data to be collected；
History data determining module to be collected, for the purpose data classifying state according to history data to be collected, from source data
The history data to be collected collected not successfully are determined in library；
Currently data determining module to be collected, for according to the history collected not successfully the data to be collected, determine currently to
Collect data；
Purpose data classifying module, for collecting the data currently to be collected to target database.
7. device according to claim 6, which is characterized in that the first purpose data classifying state acquisition module includes:
First purpose data classifying state acquisition submodule in the purpose data classifying state table for storing from target database, is looked into
Ask the purpose data classifying state of history data to be collected.
8. device according to claim 7, which is characterized in that the history data determining module to be collected includes:
Storage location determining module, for the purpose data classifying state according to the history data to be collected inquired, from the data
It collects in state table, determines that purpose data classifying state is the storage location of the history data to be collected collected not successfully；
History data extraction module to be collected, for extracting from the storage location of the history data to be collected collected not successfully
The history collected not successfully data to be collected.
9. device according to claim 6, which is characterized in that the purpose data classifying module includes:
First purpose data classifying submodule carries out currently data to be collected to target database for collecting the period according to default
It collects；
Or；
Second purpose data classifying submodule, for the storage location according to the source database, by currently data to be collected to target
Database is collected.
10. device according to claim 6, which is characterized in that further include:
Second purpose data classifying state acquisition module is used for purpose data classifying state determining module, for detecting currently wait collect
After data are collected to target database, the purpose data classifying state of the data currently to be collected is obtained；
Purpose data classifying state update module, for the purpose data classifying state according to the data currently to be collected, to the data
The purpose data classifying state collected in state table is updated.
11. a kind of purpose data classifying system characterized by comprising data described in claim 6-10 any one as above are returned
Acquisition means.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910004489.2A CN109726243A (en) | 2019-01-03 | 2019-01-03 | A kind of purpose data classifying method, apparatus and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910004489.2A CN109726243A (en) | 2019-01-03 | 2019-01-03 | A kind of purpose data classifying method, apparatus and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109726243A true CN109726243A (en) | 2019-05-07 |
Family
ID=66298088
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910004489.2A Pending CN109726243A (en) | 2019-01-03 | 2019-01-03 | A kind of purpose data classifying method, apparatus and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109726243A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113392078A (en) * | 2021-06-04 | 2021-09-14 | 上海浦东发展银行股份有限公司 | Method and device for monitoring operation change state, computer equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160292206A1 (en) * | 2015-03-31 | 2016-10-06 | Oracle International Corporation | Validating coherency between multiple data sets between database transfers |
| CN106021527A (en) * | 2016-05-24 | 2016-10-12 | 努比亚技术有限公司 | Data processing method and search server and synchronization server |
| CN106709066A (en) * | 2017-01-19 | 2017-05-24 | 杭州数梦工场科技有限公司 | Data synchronization method and device |
| CN108304553A (en) * | 2018-02-01 | 2018-07-20 | 平安普惠企业管理有限公司 | Data migration method, device, computer equipment and storage medium |
-
2019
- 2019-01-03 CN CN201910004489.2A patent/CN109726243A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160292206A1 (en) * | 2015-03-31 | 2016-10-06 | Oracle International Corporation | Validating coherency between multiple data sets between database transfers |
| CN106021527A (en) * | 2016-05-24 | 2016-10-12 | 努比亚技术有限公司 | Data processing method and search server and synchronization server |
| CN106709066A (en) * | 2017-01-19 | 2017-05-24 | 杭州数梦工场科技有限公司 | Data synchronization method and device |
| CN108304553A (en) * | 2018-02-01 | 2018-07-20 | 平安普惠企业管理有限公司 | Data migration method, device, computer equipment and storage medium |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113392078A (en) * | 2021-06-04 | 2021-09-14 | 上海浦东发展银行股份有限公司 | Method and device for monitoring operation change state, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20080046880A1 (en) | Method for managing internal software of terminal through device management server | |
| CN109471843B (en) | Metadata caching method, system and related device | |
| JP6870466B2 (en) | Control programs, control methods, controls, and database servers | |
| CN105989098B (en) | Icon package generation method and server, and icon processing method and system | |
| CN110968478A (en) | Log collection method, server and computer storage medium | |
| CN108881461A (en) | A kind of data transmission method, apparatus and system | |
| CN107798135B (en) | Paging query method and device and electronic equipment | |
| CN111222972A (en) | Account checking and clearing method and device | |
| CN111061610B (en) | Generation method and device of cluster system performance test report and computer equipment | |
| CN109726243A (en) | A kind of purpose data classifying method, apparatus and system | |
| CN109522305B (en) | Big data deduplication method and device | |
| CN103034636A (en) | Roll-back method, device and system of non relational database | |
| CN111371818B (en) | Data request verification method, device and equipment | |
| CN108491294A (en) | A kind of database backup method, apparatus and system | |
| CN110765125B (en) | Method and device for storing data | |
| CN117472535A (en) | Log playback method, device, equipment and medium | |
| CN117056428A (en) | Automatic expansion method, device and server of partition table | |
| CN106502696B (en) | A kind of generation method and relevant apparatus of upgrade installation package | |
| CN111061554B (en) | Intelligent task scheduling method and device, computer equipment and storage medium | |
| CN110908807B (en) | IO task processing method, device, equipment and readable storage medium | |
| CN110765144B (en) | Distributed heterogeneous database data processing method and device | |
| CN113923260A (en) | Method, device, terminal and storage medium for processing proxy environment | |
| CN110213055A (en) | Intelligence update method, apparatus, computer equipment and computer readable storage medium | |
| CN110674214A (en) | Big data synchronization method and device, computer equipment and storage medium | |
| CN111367750A (en) | Exception handling method, device and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190507 |
|
| RJ01 | Rejection of invention patent application after publication |