CN107085603A - A kind of data processing method and device - Google Patents
A kind of data processing method and device Download PDFInfo
- Publication number
- CN107085603A CN107085603A CN201710210430.XA CN201710210430A CN107085603A CN 107085603 A CN107085603 A CN 107085603A CN 201710210430 A CN201710210430 A CN 201710210430A CN 107085603 A CN107085603 A CN 107085603A
- Authority
- CN
- China
- Prior art keywords
- data
- version number
- target data
- database
- processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/219—Managing data history or versioning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiments of the invention provide a kind of data processing method and device, methods described includes：Target data in database is handled；After the processing to the target data is completed, whether the version number of the target data stored in the version number of the target data after contrast processing and current database is consistent, wherein, the version number of each data stored in database is that correspondence updates when each data update；When the version number of the target data stored in the version number of the target data after processing and current database is inconsistent, target data is handled again.The embodiment of the present invention can avoid error in data, it is ensured that the uniformity of data, improve the accuracy of the data preserved in database.
Description
Technical field
The present invention relates to field of computer technology, more particularly to a kind of data processing method and device.
Background technology
With the high speed development of computer technology and Internet technology, it can all produce daily on network and update substantial amounts of
Various data.For example, the index data amount of video search has reached 500,000,000 mass data at present in a video website.
Typically substantial amounts of data can all be preserved by various databases at present.And it is possible to as needed, to protecting
The data deposited are handled accordingly, and preserve the data after processing.For example, for the substantial amounts of data preserved in database,
Regularly the data in database can be carried out with the processing of batch.The processing of data needs can have many according to different business
The form of kind, such as, and the processing that the total data in database can be normalized, and the data after processing are protected
Deposit.Meanwhile, according to specific business needs, real-time data processing can also be carried out to individual data, it is possible to after processing
Data preserved.
In the prior art, when data processor process have read data from database, and line number is being entered
During according to processing, the data in database are possible to update.This will cause the data handled expired, so that at data
Reason program is completed after data processing, and the data after the processing preserved can be not corresponding with the data after being updated in database, and then
Data are caused to make a mistake.
The content of the invention
The purpose of the embodiment of the present invention is to provide a kind of data processing method and device, to improve what is preserved in database
The accuracy of data, it is to avoid occur error in data.Concrete technical scheme is as follows：
The embodiment of the invention discloses a kind of data processing method, including：
Target data in database is handled；
After completing to the processing of the target data, the version number of the target data after contrast processing and presently described data
Whether the version number of the target data stored in storehouse is consistent, wherein, the version number of each data stored in the database is each
Correspondence updates when data update；
The version number of target data after the processing and the version of the target data stored in presently described database
When number inconsistent, the target data is handled again.
Optionally, it is described completion to the processing of the target data after, contrast processing after target data version number with
The step whether version number of the target data stored in presently described database is consistent includes：
When there is multiple processes to handle the target data simultaneously, each process is completed to the target data
Processing after, the target data stored in version number and the presently described database of the target data after contrast processing
Whether version number is consistent.
Optionally, methods described also includes：
Monitor whether the data stored in the database update；
When the data update, the version number of data described in synchronized update.
Optionally, the database is Hbase databases, and whether the data stored in the monitoring database occur
The step of renewal, includes：
By the coprocessor mechanism of the Hbase databases, data in the Hbase databases are monitored in real time whether
Update；
It is described when the data update, include described in synchronized update the step of the version number of data：
When the data update, pass through the version number of data described in the coprocessor mechanism synchronized update.
Optionally, include described in the synchronized update the step of version number of data：
According to the current version number of the data, and default version number's ordering rule, the current version number is obtained
Next version number；
Acquired version number is defined as to the version number of the data.
The embodiment of the invention also discloses a kind of data processing equipment, including：
First processing module, for handling the target data in database；
Contrast module, for completing after the processing to the target data, the version number of the target data after contrast processing
Whether the version number of the target data with being stored in presently described database is consistent, wherein, each number stored in the database
According to version number be that correspondence updates when each data update；
Second processing module, the version number for the target data after processing in presently described database with storing
The target data version number it is inconsistent when, the target data is handled again.
Optionally, the contrast module, specifically for when having multiple processes while when handling the target data,
Each process complete to the processing of the target data after, contrast processing after target data version number with it is presently described
Whether the version number of the target data stored in database is consistent.
Optionally, described device also includes：
Monitoring modular, for monitoring whether the data stored in the database update；
Update module, for when the data update, the version number of data described in synchronized update.
Optionally, the database is Hbase databases, the monitoring modular, specifically for passing through the Hbase data
Whether the coprocessor mechanism in storehouse, the data monitored in real time in the Hbase databases update；
The update module, it is synchronous more by the coprocessor mechanism specifically for when the data update
The version number of the new data.
Optionally, the update module, specifically for the current version number according to the data, and default version number
Ordering rule, obtains next version number of the current version number；Acquired version number is defined as to the version of the data
Number.
A kind of data processing method provided in an embodiment of the present invention and device, can after data processing is completed to data
By the version number of data, judge whether the data are updated, when complete data processing after data version number with
When now the version number of the data in database is not consistent, then it represents that during handling the data, the data
Updated, in this case, this data handled can be abandoned, and reuse the data after updating carrying out
Data processing, and then can avoid error in data, it is ensured that the uniformity of data, improves the accurate of the data that are preserved in database
Property so that the renewal operation of individual data can be carried out simultaneously when carrying out large batch of data processing, without causing data
Make a mistake.Certainly, any product or method for implementing the present invention it is not absolutely required to while reaching all the above excellent
Point.
Brief description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing
There is the accompanying drawing used required in technology description to be briefly described, it should be apparent that, drawings in the following description are only this
Some embodiments of invention, for those of ordinary skill in the art, on the premise of not paying creative work, can be with
Other accompanying drawings are obtained according to these accompanying drawings.
A kind of flow chart for the data processing method that Fig. 1 provides for the bright embodiment of this law；
The structural representation for the data processing equipment that Fig. 2 provides for the bright embodiment of this law.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete
Site preparation is described, it is clear that described embodiment is only a part of embodiment of the invention, rather than whole embodiments.It is based on
Embodiment in the present invention, it is every other that those of ordinary skill in the art are obtained under the premise of creative work is not made
Embodiment, belongs to the scope of protection of the invention.
Referring to Fig. 1, a kind of flow chart for the data processing method that Fig. 1 provides for the bright embodiment of this law, including：
Step 101, the target data in database is handled.
Method provided in an embodiment of the present invention can apply to electronic equipment.Wherein, above-mentioned electronic equipment can be desk-top
Computer, portable computer, intelligent mobile terminal etc..
Database refer to storing in a certain way together, can for multiple users to share, with redundancy as small as possible
Degree and application program data acquisition system independent of each other.Database in the embodiment of the present invention can be current existing a variety of data
One kind in storehouse, such as Hbase databases, DB2 database, SQL Server databases, sybase database etc..
Target data refers to some or all data being stored in database, needing progress data processing.It is right
Data progress processing refers to will be stored in data in database according to different business needs, at the data for carrying out respective rule
The process of reason, for example, data can be normalized, removes the dimension that data are included, is allowed to be converted into pure values；
Or data are screened, remove heavy remainder evidence etc..
The need for according to specific difference, data can be carried out with the processing of various ways, what data were handled is specific
Method belongs to prior art, is no longer repeated herein.
In the embodiment of the present invention, the data processing that electronic equipment can be by various data processors or database in itself
Program is handled data, when carrying out the processing of data, can will first need the target data that be handled from database
In be read out.For example, when target data is stored in table form in database, data processor can be first from number
According to target data is read in the form in storehouse, the processing of target data is then carried out again., can be by after processing after the completion of processing
Data be stored in other databases, be either stored in another table of former database or be stored in former database
The form of former data simultaneously covers the former data in former database.
Step 102, after completing to the processing of target data, the version number of the target data after contrast processing and current data
Whether the version number of the target data stored in storehouse is consistent.Wherein, the version number of each data stored in database is each data
Correspondence updates during renewal.
In embodiments of the present invention, the data correspondence that electronic equipment can be directed in database stores its version number.Data
Version number can for corresponding data renewal each time, can reflect whether data there occurs by different version numbers
Update.
For different types of database, various forms of version numbers can be added for data.For example, in Hbase data
In storehouse, data are generally stored in table form, so as in existing data storage form, increase a row version newly
Number.Referring to table 1, table 1 is the data storage form in Hbase databases.
Table 1
| RowKey | A is arranged | B is arranged | C is arranged | Version number |
| RowKey1 | A12 | B20 | C500 | 1 |
| RowKey2 | A15 | B21 | C600 | 3 |
RowKey represents the row major key in Hbase databases, the inquiry for data in table 1.Every a line in table is represented
One group of data.After every a line, version number can be added, version number can be represented with numeral, whenever one or more columns per page in a line
When data are once updated, version number is also synchronously once updated.For example, the numeral in version number can add up one in advance
If numerical value, so as to realize the renewal of version number.
After electronic equipment completes the processing to the target data extracted, it, which may determine that, is carrying out target data
During processing, the original target data stored in database is either with or without updating.Because electronic equipment is carrying out target
During the reading of data, the version number of target data can be read simultaneously, so after the processing to target data is completed, Ke Yicong
The version number for the target data being currently stored in database, the version of the target data after contrast processing are read in database
Whether the version number of the target data number with being stored in current database is consistent.
Step 103, the version of the target data stored in the version number of target data after processing and current database
When number inconsistent, target data is handled again.
By comparing, if the version of target data stored in the version number of target data after processing and current database
This number inconsistent, then it represents that data processor is during handling target data, the number of targets stored in database
According to being updated, so as to result in the inconsistent of version number.
Now, because target data is updated, so the target data for passing through processing is just expired, do not have
Application value.So electronic equipment can give up the target data for passing through processing, read again from database by more
Target data after new, and re-start the processing of data.Certainly, can be with after the processing to target data is completed again
The comparison of version number is carried out again, to ensure the accuracy of the data after processing.
When the version number of target data of the version number with being stored in current database of the target data after processing is consistent,
Show that the target data in data processing, stored in database does not update.In this case, electronic equipment can by through
The target data crossed after processing is preserved.
Treated target data can will be stored in subsequent vessel by electronic equipment.Container is used in computer technology
The storage of data, in embodiments of the present invention, container can be other storage locations for the database for storing target data, also may be used
Be store target data database beyond other databases.
In the embodiment of the present invention, by adding version number, treated target data, and right to the data in database
Target data completes the comparison of the target data progress version number deposited in the database at the current time of processing, can find in time
Whether the target data for having completed processing is updated, and it is expired to result in the target data handled well, works as version
This number it is inconsistent when, then it represents that the target data handled well is expired, can be to the target data in database again
Handled, so as to exempt from error in data, it is ensured that the uniformity of data, improve the accuracy of the data preserved in database,
And enable when carrying out large batch of data processing while the renewal operation of individual data is carried out, without causing data
Make a mistake.
Optionally, in data processing method provided in an embodiment of the present invention, after completing to the processing of target data, at contrast
Whether consistent the version number of the target data stored in the version number of target data after reason and current database step be permissible
Including：
When having multiple processes while when handling target data, the completion of each process to the processing of target data it
Afterwards, whether the version number of the target data stored in the version number of the target data after contrast is handled and current database is consistent.
When the target data in database is handled, multiple processes of multiple data processors occur often
Situation about being handled simultaneously target data.For example, in being normally applied, daily all can timing to all in database
Target data carries out once the data processing operation of batch, and at any time can also be to the individual target number in database
According to the real-time data processing operation of progress.While all target datas are carried out with the processing of batch, to indivedual mesh therein
When marking data progress processing operation in real time, then same target data can be handled.
In this case, each process can carry out the reading of target data from database, and carry out independent
Processing.After handling in the completed, each progress can contrast version number and the current database of the target data after processing
Whether the version number of the target data of middle storage is consistent, when inconsistent, re-starts the processing of data.
In actual applications, multiple processes may carry out same processing to target data, for example, superincumbent example
In, the data processing operation for the batch that timing is carried out, and processing is operated in real time to individual target data therein progress daily
When be probably that identical processing is carried out to data, and be finally completed the data of processing and can be stored in same storage location.
In this case, when being updated during target data is processed, it there is also the problem of data are mutually covered.
For example, when batch processing process have read target data, and when proceeding by the processing of target data, number
Updated according to the target data stored in storehouse；Real-time treatment progress have read the target data that have updated, and start
The processing of progress data, and the predeterminated position being stored in the target data handled well after the completion of processing in database；Now
Batch processing process also completes the processing of target data, and the target data handled well is also stored in the default position in database
Put, and cover the target data for the completion processing that real-time treatment progress is preserved.
Then now the target data for the completion processing that batch processing process is preserved is according to without the number of targets before updating
According to obtained result, it is the data of mistake, and can also covers the correct result that real-time treatment progress is preserved.
After each process completes the processing of target data, all judge whether target data has been carried out more by version number
Newly, it is possible to avoid the appearance of above-mentioned situation.
For example, when batch processing process have read target data, and when proceeding by the processing of target data, number
Updated according to the target data stored in storehouse, and the version number of synchronized update target data；Real-time treatment progress have read
The target data that have updated, and the processing of data is proceeded by, after the completion of processing, by the mesh for judging to find to complete processing
The version number for marking data is with the version number of the target data in current database consistent, so as to will will be handled well after the completion of processing
Target data is stored in the predeterminated position in database；Batch processing process also completes the processing of target data, by judging
It was found that the version number of the target data in the version number and current database of the target data of completion processing is inconsistent, so as to abandon
This has completed the target data of processing, and re-starts the processing of target data, or can also abandon at this data
The operation of reason.So as to avoid the data for preserving mistake.
When multi-process is handled target data simultaneously, each process is completed after the processing of target data, is all led to
The version number for crossing target data judges whether target data is updated, and correct data is covered so as to avoid wrong data
Lid, further avoid error in data.
Optionally, in data processing method provided in an embodiment of the present invention, this method also includes：
Whether the data stored in the first step, monitor database update.
In embodiments of the present invention, electronic equipment can in real time be monitored to the data in database, Monitoring Data
Whether renewal is had.Specifically, having different modes to the monitoring of data in different databases, for example, it can pass through extra journey
Ordered pair database is monitored in real time, or by database plug-in unit in itself or subprogram, the data come in monitor database
Whether updated.
The renewal of data and data are carried out do not have between various data processings it is inevitable contact, be separate mistake
Journey, the renewal of data may be carried out at any time, so being carried out in real time for the monitoring that data update.
Second step, when data update, the version number of synchronized update data.
It is synchronous when monitoring that the data in database are updated, version number can be updated.Version number
Renewal can be carried out by default more new procedures, and specifically more new procedures can be by those skilled in the art for different
Type of database, is developed and obtained by various programming languages, is belonged to prior art, is no longer repeated herein.
,, can when data update by being monitored in real time to the data in database in the embodiment of the present invention
The version number timely updated the data, and the monitoring and the synchronous renewal to version number of data renewal can be passed through
Extra program is realized, is not present and couple with the process handled data, be conducive in database the management of data with
Processing to data, improves the efficiency handled data.
Optionally, in data processing method provided in an embodiment of the present invention, when database is Hbase databases, number is monitored
The step of whether being updated according to the data stored in storehouse can include：
By the coprocessor mechanism of Hbase databases, whether the data monitored in real time in Hbase databases occur more
Newly.
Hbase databases, are a high reliability, high-performance, towards row, telescopic distributed memory system, utilize
HBase technologies can erect large-scale structure storage cluster on low-cost server.So Hbase databases have been achieved for
It is widely applied.
The coprocessor mechanism of Hbase databases can allow the customized code of developer to be performed in server end, come
Into some specific functions.
By the coprocessor mechanism of Hbase databases, the renewal behaviour of data in Hbase databases can be monitored in real time
Make, when data are updated each time, be capable of the data for detecting to update of synchronization.
It can include when data update, the step of the version number of synchronized update data：
When data update, pass through the version number of coprocessor mechanism synchronized update data.
After the coprocessor mechanism of Hbase databases has monitored that data there occurs renewal, renewal number that can be synchronous
According to version number.For example, data of the coprocessor mechanism of Hbase databases in the data form of Hbase databases occur
During renewal, data corresponding version number in modification form that can be synchronous.
In the embodiment of the present invention, when using Hbase databases, realized by the coprocessor mechanism of Hbase databases
The monitoring that data in database are updated, and when data update, the synchronous version number updated the data.Pass through
The coprocessor mechanism of Hbase databases, can to couple reduction between data processor and database, be easy to form system
One specification.And problem is solved with Hbase self-characteristics, reduce development cost.
Optionally, can in data processing method provided in an embodiment of the present invention, the step of the version number of synchronized update data
With including：
The first step, according to the current version number of data, and default version number's ordering rule, obtains current version number
Next version number.
Version number can have a variety of representations, for example, version number can be numeral or letter.
When version number is numeral, default version number's ordering rule can be that cumulative one of current version number is default
Value, so as to obtain next version number of current version number.For example, during the firm write into Databasce of data, because data are not deposited before this
So initial version number can be 0, the data often occur once to update, and version number can add up 1, and the data are for the first time
When updating, next version number of current version number is 1.
When version number is letter, default version number's ordering rule can be the row of putting in order by English alphabet
Sequence, and 26 English alphabets are recycled, so as to obtain next version number of current version number.For example, data just write data
During storehouse, initial version number can be A, when the data update for the first time, can be with by next version number of current version number
For B.
Second step, acquired version number is defined as the version number of data.
By default version number's ordering rule, obtained according to current version number after next version number, by data correspondence
Version number be updated to the next version number obtained so that the renewal of complete paired data version number.
In the embodiment of the present invention, current version number is obtained according to current version number by default version number's ordering rule
Next version number, enable to the renewal of version number more orderly, and can be obtained by version number primary data renewal time
Number, is conducive to the management and application of data in database.
Referring to Fig. 2, the structure chart for the data processing equipment that Fig. 2 provides for the bright embodiment of this law, including：
First processing module 201, for handling the target data in database；
Contrast module 202, for completing after the processing to target data, contrast processing after target data version number with
Whether the version number of the target data stored in current database is consistent.Wherein, the version number of each data stored in database
It is that correspondence updates when each data update；
Second processing module 203, for when the target data stored in the version number of target data and current database
When version number is inconsistent, target data is handled again.
In the embodiment of the present invention, by adding version number, treated target data, and right to the data in database
Target data completes the comparison of the target data progress version number deposited in the database at the current time of processing, can find in time
Whether the target data for having completed processing is updated, and it is expired to result in the target data handled well, works as version
This number it is inconsistent when, then it represents that the target data handled well is expired, can be to the target data in database again
Handled, so as to exempt from error in data, it is ensured that the uniformity of data, improve the accuracy of the data preserved in database,
And enable when carrying out large batch of data processing while the renewal operation of individual data is carried out, without causing data
Make a mistake.
The device of the embodiment of the present invention is the institute of the device, then above-mentioned data processing method using above-mentioned data processing method
There is embodiment to be applied to the device, and can reach same or analogous beneficial effect.
Optionally, in data processing equipment provided in an embodiment of the present invention, contrast module 202 has multiple specifically for working as
Process is handled target data simultaneously when, after each process is completed to the processing of target data, after contrast processing
Whether the version number of the target data stored in the version number of target data and current database is consistent.
Optionally, in data processing equipment provided in an embodiment of the present invention, the device also includes：
Whether monitoring modular (not shown), the data for being stored in monitor database update；
Update module (not shown), for when data update, the version number of synchronized update data.
Optionally, in data processing equipment provided in an embodiment of the present invention, when database is Hbase databases, mould is monitored
Block, specifically for the coprocessor mechanism by Hbase databases, whether the data monitored in real time in Hbase databases occur
Update；
Update module, specifically for when data update, passing through the version of coprocessor mechanism synchronized update data
Number.
Optionally, in data processing equipment provided in an embodiment of the present invention, update module, specifically for working as according to data
Preceding version number, and default version number's ordering rule, obtain next version number of current version number.By acquired version number
It is defined as the version number of data.
It should be noted that herein, such as first and second or the like relational terms are used merely to a reality
Body or operation make a distinction with another entity or operation, and not necessarily require or imply these entities or deposited between operating
In any this actual relation or order.Moreover, term " comprising ", "comprising" or its any other variant are intended to
Nonexcludability is included, so that process, method, article or equipment including a series of key elements not only will including those
Element, but also other key elements including being not expressly set out, or also include being this process, method, article or equipment
Intrinsic key element.In the absence of more restrictions, the key element limited by sentence "including a ...", it is not excluded that
Also there is other identical element in process, method, article or equipment including the key element.
Each embodiment in this specification is described by the way of related, identical similar portion between each embodiment
Divide mutually referring to what each embodiment was stressed is the difference with other embodiment.It is real especially for system
Apply for example, because it is substantially similar to embodiment of the method, so description is fairly simple, related part is referring to embodiment of the method
Part explanation.
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the scope of the present invention.It is all
Any modification, equivalent substitution and improvements made within the spirit and principles in the present invention etc., are all contained in protection scope of the present invention
It is interior.
Claims (10)
1. a kind of data processing method, it is characterised in that including：
Target data in database is handled；
After completing to the processing of the target data, in the version number and presently described database of the target data after contrast processing
Whether the version number of the target data of storage is consistent, wherein, the version number of each data stored in the database is each data
Correspondence updates during renewal；
The version number of the target data stored in the version number of target data after the processing and presently described database is not
When consistent, the target data is handled again.
2. according to the method described in claim 1, it is characterised in that after the completion is to the processing of the target data, contrast
Whether the version number of the target data of the version number of the target data after processing with being stored in presently described database is consistent
The step of include：
When there is multiple processes to handle the target data simultaneously, each process completes the place to the target data
After reason, version number and the version of the target data stored in presently described database of the target data after contrast processing
It is number whether consistent.
3. according to the method described in claim 1, it is characterised in that methods described also includes：
Monitor whether the data stored in the database update；
When the data update, the version number of data described in synchronized update.
4. method according to claim 3, it is characterised in that the database is Hbase databases, described in the monitoring
The step of whether data stored in database update includes：
By the coprocessor mechanism of the Hbase databases, whether the data monitored in real time in the Hbase databases occur
Update；
It is described when the data update, include described in synchronized update the step of the version number of data：
When the data update, pass through the version number of data described in the coprocessor mechanism synchronized update.
5. method according to claim 3, it is characterised in that wrapped described in the synchronized update the step of version number of data
Include：
According to the current version number of the data, and default version number's ordering rule, obtain under the current version number
One version number；
Acquired version number is defined as to the version number of the data.
6. a kind of data processing equipment, it is characterised in that including：
First processing module, for handling the target data in database；
Contrast module, for completing after the processing to the target data, the version number of the target data after contrast processing is with working as
Whether the version number of the target data stored in the preceding database is consistent, wherein, each data stored in the database
Version number is that correspondence updates when each data update；
Second processing module, for the institute stored in the version number of the target data after processing and presently described database
State target data version number it is inconsistent when, the target data is handled again.
7. device according to claim 6, it is characterised in that the contrast module, has multiple processes same specifically for working as
When handling the target data, after each process is completed to the processing of the target data, after contrast processing
Target data the target data of the version number with being stored in presently described database version number it is whether consistent.
8. device according to claim 6, it is characterised in that described device also includes：
Monitoring modular, for monitoring whether the data stored in the database update；
Update module, for when the data update, the version number of data described in synchronized update.
9. device according to claim 8, it is characterised in that the database is Hbase databases, the monitoring mould
Block, specifically for the coprocessor mechanism by the Hbase databases, monitors the data in the Hbase databases in real time
Whether update；
The update module, specifically for when the data update, passing through the coprocessor mechanism synchronized update institute
State the version number of data.
10. device according to claim 8, it is characterised in that the update module, specifically for according to the data
Current version number, and default version number's ordering rule, obtain next version number of the current version number；Will be acquired
Version number is defined as the version number of the data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710210430.XA CN107085603B (en) | 2017-03-31 | 2017-03-31 | Data processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710210430.XA CN107085603B (en) | 2017-03-31 | 2017-03-31 | Data processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107085603A true CN107085603A (en) | 2017-08-22 |
| CN107085603B CN107085603B (en) | 2020-04-03 |
Family
ID=59615293
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710210430.XA Active CN107085603B (en) | 2017-03-31 | 2017-03-31 | Data processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107085603B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108388438A (en) * | 2018-02-07 | 2018-08-10 | 平安普惠企业管理有限公司 | System base table update method, device, computer equipment and storage medium |
| CN109299194A (en) * | 2018-09-25 | 2019-02-01 | 平安科技（深圳）有限公司 | Multi-edition data memory management method and device, electronic equipment, storage medium |
| CN110134670A (en) * | 2019-05-16 | 2019-08-16 | 拉扎斯网络科技（上海）有限公司 | Data storage method and device, electronic equipment and computer readable storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1522409A (en) * | 2001-06-09 | 2004-08-18 | 存储交易株式会社 | Cache-conscious concurrency control scheme for database systems |
| CN103744936A (en) * | 2013-12-31 | 2014-04-23 | 华为技术有限公司 | Multi-version concurrency control method in database and database system |
| CN104469474A (en) * | 2014-10-28 | 2015-03-25 | 深圳国微技术有限公司 | Channel list updating method and device, and terminal device |
| CN105095313A (en) * | 2014-05-22 | 2015-11-25 | 阿里巴巴集团控股有限公司 | Data access method and equipment |
| CN105843895A (en) * | 2016-03-22 | 2016-08-10 | 北京建飞无限科技有限公司 | EhCache-based data querying and synchronizing method, device and system |
| CN106294607A (en) * | 2016-07-29 | 2017-01-04 | 北京奇虎科技有限公司 | Data cached update method and updating device |
-
2017
- 2017-03-31 CN CN201710210430.XA patent/CN107085603B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1522409A (en) * | 2001-06-09 | 2004-08-18 | 存储交易株式会社 | Cache-conscious concurrency control scheme for database systems |
| CN103744936A (en) * | 2013-12-31 | 2014-04-23 | 华为技术有限公司 | Multi-version concurrency control method in database and database system |
| CN105095313A (en) * | 2014-05-22 | 2015-11-25 | 阿里巴巴集团控股有限公司 | Data access method and equipment |
| CN104469474A (en) * | 2014-10-28 | 2015-03-25 | 深圳国微技术有限公司 | Channel list updating method and device, and terminal device |
| CN105843895A (en) * | 2016-03-22 | 2016-08-10 | 北京建飞无限科技有限公司 | EhCache-based data querying and synchronizing method, device and system |
| CN106294607A (en) * | 2016-07-29 | 2017-01-04 | 北京奇虎科技有限公司 | Data cached update method and updating device |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108388438A (en) * | 2018-02-07 | 2018-08-10 | 平安普惠企业管理有限公司 | System base table update method, device, computer equipment and storage medium |
| CN108388438B (en) * | 2018-02-07 | 2021-04-23 | 平安普惠企业管理有限公司 | System base table updating method and device, computer equipment and storage medium |
| CN109299194A (en) * | 2018-09-25 | 2019-02-01 | 平安科技（深圳）有限公司 | Multi-edition data memory management method and device, electronic equipment, storage medium |
| CN110134670A (en) * | 2019-05-16 | 2019-08-16 | 拉扎斯网络科技（上海）有限公司 | Data storage method and device, electronic equipment and computer readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107085603B (en) | 2020-04-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103810224B (en) | information persistence and query method and device | |
| CN104794162B (en) | Real-time data memory and querying method | |
| EP3308303B1 (en) | Mechanisms for merging index structures in molap while preserving query consistency | |
| CN103377260B (en) | The analysis method and device of a kind of network log URL | |
| CN103902698A (en) | Data storage system and data storage method | |
| CN112115152B (en) | Data increment updating and inquiring method and device, electronic equipment and storage medium | |
| CN110109910A (en) | Data processing method and system, electronic equipment and computer readable storage medium | |
| US9323798B2 (en) | Storing a key value to a deleted row based on key range density | |
| CN107085603A (en) | A kind of data processing method and device | |
| CN113407785B (en) | Data processing method and system based on distributed storage system | |
| CN108763341B (en) | Electronic device, automatic table building method and storage medium | |
| US9489413B2 (en) | Asynchronous global index maintenance during partition maintenance | |
| CN109815240A (en) | For managing method, apparatus, equipment and the storage medium of index | |
| CN112052309A (en) | Text data retrieval method, related equipment and readable storage medium | |
| CN110647577A (en) | Data cube partitioning method and device, computer equipment and storage medium | |
| CN102831172A (en) | Method and device for processing data in enterprise patent databank | |
| EP3227796B1 (en) | Real time indexing | |
| CN111984673A (en) | Fuzzy retrieval method and device for tree structure of power grid electric energy metering system | |
| CN111752958A (en) | Intelligent associated label method, device, computer equipment and storage medium | |
| CN107515916A (en) | The performance optimization method and device of data query | |
| CN113505172B (en) | Data processing method, device, electronic equipment and readable storage medium | |
| CN110334055B (en) | Method for acquiring material calculation data | |
| CN106354829A (en) | Physic-chemical method and device of column storage database | |
| CN104424196B (en) | The sorting and storing method and device of inlay, the method and device for creating supplement character library | |
| US9158798B2 (en) | Database large object reorganization |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |