CN108897806A - Comparison of data consistency method, apparatus, storage medium and electronic equipment - Google Patents
Comparison of data consistency method, apparatus, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN108897806A CN108897806A CN201810623674.5A CN201810623674A CN108897806A CN 108897806 A CN108897806 A CN 108897806A CN 201810623674 A CN201810623674 A CN 201810623674A CN 108897806 A CN108897806 A CN 108897806A
- Authority
- CN
- China
- Prior art keywords
- data
- fragment
- identification information
- result
- fragmentation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Abstract
This disclosure relates to a kind of comparison of data consistency method, apparatus, storage medium and electronic equipment, for solving the technical problem that comparison of data consistency efficiency is lower in the prior art.This method includes：Data cutting message is sent to first end and second end, data cutting message includes data fragmentation strategy and verification data format；Obtain the data fragmentation result that first end and second end are sent, data fragmentation result include the check number of each fragment data accordingly and the identification information of each fragment data, wherein, fragment data is that first end and second end are divided according to the data that stripping strategy stores itself, and verification data are that first end and second end are generated according to verification data format for each fragment data；Compare the verification data in the data fragmentation result with same identification information, wherein if verification data are identical, show that the corresponding fragment data of first end and second end identification information is consistent.
Description
Technical field
This disclosure relates to technical field of data storage, and in particular, to a kind of comparison of data consistency method, apparatus is deposited
Storage media and electronic equipment.
Background technique
In the application field of database technology, it can be stored in equipment from source and read data, and data are passed through into centre
In storaging medium or network storage to target storage device, to generate data copy.However, due to network, server or soft
Part break down, may result in partial data content it is inconsistent or because be written failure due to lose.Therefore, in database
When data are replicated, needs to verify source storage equipment and whether the data in target storage device are consistent, to ensure data
Safety.
It in the related art, is to deposit source storage equipment with target when the data between database carry out uniformity comparison
The stripping strategy cutting that the initial data in equipment is formulated according to center-side is compared is stored up, obtains fragment data, and source storage is set
The standby same fragment data with target storage device, which is successively uploaded to, to be compared center-side and is compared.In this way, in mass data
Or under the limited application scenarios of network bandwidth, data upload can expend a large amount of network bandwidths and waiting time, forming properties bottle
Neck causes comparison of data consistency low efficiency.
Summary of the invention
Purpose of this disclosure is to provide a kind of comparison of data consistency method, apparatus, storage medium and electronic equipments, are used for
Solve the lower technical problem of comparison of data consistency efficiency in the prior art.
To achieve the goals above, disclosure first aspect provides a kind of comparison of data consistency method, is applied to compare
End, the method includes：
Send data cutting message to first end and second end, the data cutting message include data fragmentation strategy with
And verification data format；
The data fragmentation that the first end and the second end are sent is obtained as a result, the data fragmentation result includes every
The check number of one fragment data is accordingly and the identification information of each fragment data, wherein the fragment data is the first end
And the second end is divided according to the data that the stripping strategy stores itself, the verification data are institutes
State what first end and the second end were generated according to the verification data format for each fragment data；
Compare the verification data in the data fragmentation result with same identification information, wherein if the verification data phase
Together, then show that first end fragment data corresponding with identification information described in the second end is consistent.
Optionally, the method also includes：
After obtaining the data fragmentation result that the first end and the second end are sent, the first end is sent
Data fragmentation result store to first caching in, by the data fragmentation result that the second end is sent store to second caching
In；
The verification data compared in the data fragmentation result with same identification information, including：
Start multiple threads read respectively it is described first caching in and it is described second caching in have same identification information
Data fragmentation result in verification data；
Judge have the verification data in the data fragmentation result of same identification information whether identical.
Optionally, the method also includes：
If the verification data in the data fragmentation result with same identification information are identical, to the first end and institute
It states second end and sends caching clearance order；
Wherein, the caching clearance order is used to indicate the first end and the second end removes identical verification data
The caching of corresponding fragment data.
Optionally, the method also includes：
If the verification data in the data fragmentation result with a certain same identification information are not identical, to the first end
And the second end sends data echo request；
If a certain identification information does not exist simultaneously in first caching and second caching, to the mark
Corresponding data terminal is cached where information sends data echo request；
Wherein, the data echo request includes identification information, and it is corresponding to return to the identification information for request data end
Fragment data；
The fragment data of the first end loopback and the fragment data of the second end loopback are compared, data difference is obtained
Information.
Disclosure second aspect provides a kind of comparison of data consistency method, the method includes：
Receive the data cutting message sent than opposite end；
It is divided according to the data that the data fragmentation strategy in the data cutting message stores itself, obtains fragment
The identification information of data and each fragment data；And
According to the verification data format in the data cutting message, the verification data of each fragment data are generated；
Data fragmentation is sent as a result, the data fragmentation result includes the check number of each fragment data than opposite end to described
Accordingly and the identification information of each fragment data.
Optionally, the method also includes：
The caching clearance order sent than opposite end is received, the caching clearance order includes the mark letter of fragment data
Breath；
The caching of the corresponding fragment data of identification information of the fragment data is removed according to the caching clearance order.
Optionally, the method also includes：
The data echo request sent than opposite end is received, the data echo request includes the mark letter of fragment data
Breath；
According to the data echo request to the corresponding fragment data of identification information for sending the fragment data than opposite end.
Optionally, it is divided in the data stored according to the data fragmentation strategy in the data cutting message to itself
Before, the method also includes：
Judge whether the data volume of all data of itself storage is greater than preset data amount threshold value；
The data that the data fragmentation strategy according in the data cutting message stores itself divide, packet
It includes：
When the data volume is greater than the preset data amount threshold value, according to the data fragmentation in the data cutting message
The data that strategy stores itself divide；
The method also includes：
When the data volume is less than the preset data amount threshold value, the data are directly transmitted to than opposite end and carry out data
It compares.
The disclosure third aspect provides a kind of comparison of data consistency device, and described device includes：
Cutting message transmission module, for sending data cutting message, the data cutting to first end and second end
Message includes data fragmentation strategy and verification data format；
Fragment result obtains module, for obtaining data fragmentation that the first end and the second end are sent as a result,
The data fragmentation result include the check number of each fragment data accordingly and the identification information of each fragment data, wherein institute
Stating fragment data is that the first end and the second end are divided according to the data that the stripping strategy stores itself
It obtains, the verification data are the first end and the second end according to the verification data format for each fragment
What data generated；
Comparison module, for comparing the verification data in the data fragmentation result with same identification information, wherein if institute
It is identical to state verification data, then shows that first end fragment data corresponding with identification information described in the second end is consistent.
Optionally, described device further includes：
Memory module, for inciting somebody to action after obtaining the data fragmentation result that the first end and the second end are sent
The data fragmentation result that the first end is sent is stored into the first caching, and the data fragmentation result that the second end is sent is deposited
Storage is into the second caching；
The comparison module includes：
Reading data submodule, for starting, multiple threads are read respectively in first caching and described second caches
In with same identification information data fragmentation result in verification data；
Judging submodule, for judge the verification data in the data fragmentation result with same identification information whether phase
Together.
Optionally, described device further includes：
Clearance order sending module is identical for the verification data in the data fragmentation result with same identification information
When, caching clearance order is sent to the first end and the second end；
Wherein, the caching clearance order is used to indicate the first end and the second end removes identical verification data
The caching of corresponding fragment data.
Optionally, described device further includes：
Echo request sending module, not for the verification data in the data fragmentation result with a certain same identification information
When identical, data echo request is sent to the first end and the second end；
The echo request sending module is also used to a certain identification information in first caching and second caching
In when not existing simultaneously, cache corresponding data terminal where to the identification information and send data echo request；
Wherein, the data echo request includes identification information, and it is corresponding to return to the identification information for request data end
Fragment data；
The comparison module, for comparing the fragment data of the first end loopback and the fragment of the second end loopback
Data obtain data difference information.
Disclosure fourth aspect provides a kind of data bank management device, and described device includes：
Cutting message reception module, for receiving the data cutting message sent than opposite end；
Data cutting module, the data for being stored according to the data fragmentation strategy in the data cutting message to itself
It is divided, obtains the identification information of fragment data and each fragment data；And
According to the verification data format in the data cutting message, the verification data of each fragment data are generated；
Fragment result sending module, for sending data fragmentation as a result, the data fragmentation result packet than opposite end to described
Include the check number of each fragment data accordingly and the identification information of each fragment data.
Optionally, described device further includes：
Clearance order receiving module, for receiving the caching clearance order sent than opposite end, the caching removing refers to
Enable the identification information including fragment data；
Caching removes module, and the identification information for removing the fragment data according to the caching clearance order is corresponding
The caching of fragment data.
Optionally, described device further includes：
Echo request receiving module, for receiving the data echo request sent than opposite end, the data loopback is asked
Seek the identification information including fragment data；
Fragment data sending module, for according to the data echo request to the mark for sending the fragment data than opposite end
Know the corresponding fragment data of information.
Optionally, described device further includes：
Judgment module, for according to the data fragmentation strategy in the data cutting message to the data itself stored into
Before row divides, judge whether the data volume of all data of itself storage is greater than preset data amount threshold value；
The data cutting module is used for when the data volume is greater than the preset data amount threshold value, according to the number
It is divided according to the data that the data fragmentation strategy in cutting message stores itself；
The fragment data sending module is also used to when the data volume is less than the preset data amount threshold value, directly
It sends the data and carries out comparing to than opposite end.
The 5th aspect of the disclosure provides a kind of computer readable storage medium, is stored thereon with computer program, the program
The step of comparison of data consistency method that disclosure first aspect provides is realized when being executed by processor.
The 6th aspect of the disclosure provides a kind of computer readable storage medium, is stored thereon with computer program, the program
The step of comparison of data consistency method that disclosure second aspect provides is realized when being executed by processor.
The 7th aspect of the disclosure provides a kind of electronic equipment, including：
Memory is stored thereon with computer program；
Processor, for executing the computer program in the memory, to realize that disclosure first aspect provides
Comparison of data consistency method the step of.
Disclosure eighth aspect provides a kind of electronic equipment, including：
Memory is stored thereon with computer program；
Processor, for executing the computer program in the memory, to realize that disclosure second aspect provides
Comparison of data consistency method the step of.
According to the above technical scheme, send to first end and second end includes data fragmentation strategy and verification data lattice
The data cutting message of formula, and receive the data fragmentation result that first end and second end are sent.Wherein, data fragmentation result packet
It includes the check number of each fragment data accordingly and the identification information of each fragment data, fragment data is first end and second end
It is divided according to the data that stripping strategy stores itself, verification data are first end and second end according to verification
Data format is generated for each fragment data.In this way, passing through configuration data format, it is possible to reduce the occupancy of network bandwidth,
When whether comparison data consistent, can by compare same identification information data fragmentation result in verification data whether one
Cause to determine whether data are consistent, reduces the time expended because waiting a large amount of fragment datas to upload, and then improve data one
The efficiency that cause property compares.
Other feature and advantage of the disclosure will the following detailed description will be given in the detailed implementation section.
Detailed description of the invention
Attached drawing is and to constitute part of specification for providing further understanding of the disclosure, with following tool
Body embodiment is used to explain the disclosure together, but does not constitute the limitation to the disclosure.In the accompanying drawings：
Fig. 1 is a kind of flow chart for comparison of data consistency method that the embodiment of the present disclosure provides；
Fig. 2 is a kind of flow chart for comparison of data consistency method that the embodiment of the present disclosure provides；
Fig. 3 is a kind of flow chart for comparison of data consistency method that the embodiment of the present disclosure provides；
Fig. 4 is a kind of schematic diagram interacted than opposite end with data terminal that the embodiment of the present disclosure provides；
Fig. 5 is a kind of block diagram for comparison of data consistency device that the embodiment of the present disclosure provides；
Fig. 6 is a kind of block diagram for data bank management device that the embodiment of the present disclosure provides；
Fig. 7 is the block diagram for a kind of electronic equipment that the embodiment of the present disclosure provides；
Fig. 8 is the block diagram for a kind of electronic equipment that the embodiment of the present disclosure provides.
Specific embodiment
It is described in detail below in conjunction with specific embodiment of the attached drawing to the disclosure.It should be understood that this place is retouched
The specific embodiment stated is only used for describing and explaining the disclosure, is not limited to the disclosure.
Fig. 1 is a kind of flow chart for comparison of data consistency method that the embodiment of the present disclosure provides, and is applied to than opposite end, such as
Shown in Fig. 1, this approach includes the following steps：
S11, data cutting message is sent to first end and second end.
Wherein, data cutting message includes data fragmentation strategy and verification data format.Illustratively, data fragmentation strategy
Fragment value size including fragment data format information and fragment data, verification data format is fragment data according to preset
What algorithm was encrypted, for example, verification data format can be MD5 format or DES format.
S12, the data fragmentation result that first end and second end are sent is obtained.
After first end and second end receive the data cutting message sent than opposite end, according to data cutting message
In include data fragmentation strategy and verification data format data that itself is stored divide, and by data fragmentation result
It is sent to than opposite end, the data fragmentation result that first end and second end are sent then is obtained than opposite end.Wherein, data fragmentation result
Check number including each fragment data accordingly and the identification information of each fragment data, for example, the identification information of fragment data
For the end of identification of fragment or the ID of fragment data；Verification data are first end and second end according to verification data format needle
Each fragment data encryption conversion is generated, such as the verification data of fragment data 1 are MD5 format
8fe7bb8f022e9e9ceb1d6e9180103865。
S13, comparison have the verification data in the data fragmentation result of same identification information.
For the data fragmentation with same identification information as a result, verification data in comparison data fragment result whether phase
Together, if verification data are identical, show that the corresponding fragment data of first end and second end identification information is consistent.Illustratively,
Verification data are MD5 format 8fe7bb8f022e9e9ceb1d6e9180103866, second end in the fragment data 2 that one end is sent
It is MD5 format 8fe7bb8f022e9e9ceb1d6e9180103866 that data are verified in the fragment data 2 of transmission, then first end point
The content of fragment data corresponding to sheet data 2 and fragment data corresponding to second end fragment data 2 is consistent.
Using the above method, sending to first end and second end includes data fragmentation strategy and verification data format
Data cutting message, and the data fragmentation for receiving first end and second end transmission can subtract as a result, by configuration data format
The occupancy of few network bandwidth can pass through the data fragmentation result of comparison same identification information when whether comparison data is consistent
In verification data whether unanimously determine whether data consistent, reduce because wait that a large amount of fragment datas expend due tos upload when
Between, and then improve the efficiency of comparison of data consistency.
Fig. 2 is a kind of flow chart for comparison of data consistency method that the embodiment of the present disclosure provides, and is applied to than opposite end, such as
Shown in Fig. 2, this approach includes the following steps：
S21, data cutting message is sent to first end and second end.
S22, the data fragmentation result that first end and second end are sent is obtained.
S23, the data fragmentation result that first end is sent is stored into the first caching, the data fragmentation that second end is sent
As a result it stores into the second caching.
Than opposite end after obtaining the data fragmentation result that first end and second end are sent, data fragmentation result is distinguished
It stores into the first caching and the second caching, convenient for reading data fragmentation as a result, and to same identification information than opposite end
Verification data in data fragmentation result are compared.
S24, the multiple threads of starting read the number in the first caching and in the second caching with same identification information respectively
According to the verification data in fragment result.
Start multiple threads than opposite end and obtains data fragmentation from caching as a result, specifically, search has phase in the buffer
With the verification data in the data fragmentation result of identification information.
Whether S25, judgement have the verification data in the data fragmentation result of same identification information identical.
After reading the verification data in the data fragmentation result with same identification information, the two check number is judged
According to whether identical.If it is identical to verify data, S26 is thened follow the steps, it is no to then follow the steps S27 or step S28.
S26, caching clearance order is sent to first end and second end.
Illustratively, first end and second end have the caching for safeguarding fragment data, in the tool got than opposite end
There are verification data in the data fragmentation result of same identification information also in identical situation, is then sent out to first end and second end
Caching clearance order is sent, which is used to indicate first end and second end is removed corresponding to identical verification data
The caching of fragment data improves the access reading speed of data to discharge spatial cache.
If the verification data in S27, the data fragmentation result with a certain same identification information are not identical, to first end
And second end sends data echo request.
When verification data in the data fragmentation result with same identification information got than opposite end are not identical, than
Opposite end then sends data echo request to first end and second end, which includes identification information, for requesting
Data terminal returns to the corresponding fragment data of identification information.
If S28, a certain identification information do not exist simultaneously in the first caching and the second caching, where to identification information
It caches corresponding data terminal and sends data echo request.
In the case where alternatively possible, there are a certain identification information in the first caching, in the second caching there is no with
The identical identification information of the identification information then illustrates that the corresponding fragment data of the identification information exists only in first end.Similarly,
There are a certain identification informations in two cachings, and identification information identical with the identification information is not present in the first caching, then illustrates
The corresponding fragment data of the identification information exists only in second end.In this way, can be corresponded to caching where identification information than opposite end
Data terminal send data echo request, which includes identification information, for request the data terminal return mark
The corresponding fragment data of information.
The fragment data of S29, the fragment data for comparing first end loopback and second end loopback obtain data difference letter
Breath.
After executing step S27, the fragment data of first end loopback and the fragment data of second end loopback are compared, is obtained
To data difference information；After executing step S28, the fragment number of the corresponding data terminal loopback of caching where comparing identification information
According to obtaining data difference information.
It is worth noting that first end and second end may be collectively referred to as the provider for participating in the service of comparison data library
Data terminal compares the data for supporting that first end and second end provide, and can execute comparison operation.Wherein, when first end is
Source is compared, second end is then to compare target side；When first end is to compare target side, second end is then to compare source.
Using the above method, sending to first end and second end includes data fragmentation strategy and verification data format
Data cutting message, and the data fragmentation of first end and second end transmission is received as a result, reducing because waiting a large amount of fragment numbers
The time expended according to upload.Data fragmentation result is stored in the caching than opposite end, is compared using multithreading, monitoring caches
Data accelerate data access speed, and then improve the efficiency of comparison of data consistency.
Fig. 3 is a kind of flow chart for comparison of data consistency method that the embodiment of the present disclosure provides, as shown in figure 3, the party
Method includes the following steps：
The data cutting message that S31, reception are sent than opposite end.
S32, it is divided according to the data that the data fragmentation strategy in data cutting message stores itself, obtains fragment
The identification information of data and each fragment data；And according to the verification data format in data cutting message, each point is generated
The verification data of sheet data.
Specifically, data cutting message includes data fragmentation strategy and verification data format, first end and second end
After receiving the data cutting message sent than opposite end, cutting is carried out to initial data according to data fragmentation strategy.For example,
The specified verification data format of data fragmentation strategy is MD5 format, and fragment value is 16 bytes, then is byte long by initial data cutting
Degree is the fragment data of 16 bytes, and fragment data is converted to the verification data of MD5 format by preset algorithm.
S33, data fragmentation result is sent to than opposite end.
Specifically, data fragmentation result includes the check number of each fragment data accordingly and the mark of each fragment data is believed
Breath, first end and second end actively repeatedly send the data fragmentation results of all fragment datas to than opposite end, while by fragment
Data are stored into the fragment data caching of data terminal, are read from caching and loopback fragment data is to than opposite end convenient for subsequent,
It avoids repeatedly requesting to cause service performance loss larger.
Using the above method, data that first end and second end store itself according to the data cutting message received
It is divided, and generates the verification data of each fragment data according to the verification data format in data cutting message, by matching
Set data format, it is possible to reduce the occupancy of network bandwidth, and reduce and expend the time because transmitting a large amount of fragment datas, it is passed in data
The defeated stage accelerates comparing speed, and then improves the efficiency of comparison of data consistency.
Fig. 4 is a kind of schematic diagram interacted than opposite end with data terminal that the embodiment of the present disclosure provides, as shown in figure 4, the friendship
Mutual method includes the following steps：
S41, data cutting message is sent to data terminal than opposite end.
S42, data terminal receive the data cutting message sent than opposite end.
S43, data terminal are divided according to the data that the data fragmentation strategy in data cutting message stores itself, are obtained
To fragment data and the identification information of each fragment data；And according to the verification data format in data cutting message, generate
The verification data of each fragment data.
S44, data terminal send data fragmentation result to than opposite end.
S45, the data fragmentation result that data terminal is sent is obtained than opposite end.
S46, than opposite end compare have same identification information data fragmentation result in verification data.
During being interacted than opposite end and data terminal, firstly, data cutting message is sent to data terminal than opposite end,
The data cutting message includes data fragmentation strategy and verification data format.Data terminal receives the data cutting sent than opposite end
Message, and divided according to the data that the data fragmentation strategy in data cutting message stores itself, obtain fragment data
And the identification information of each fragment data generates each point further according to the verification data format in data cutting message
Data fragmentation result, is then sent to and compares opposite end by the verification data of sheet data.The data point that data terminal is sent are obtained than opposite end
Piece as a result, and compare have same identification information data fragmentation result in verification data it is whether identical.
In a kind of possible embodiment, what is stored according to the data fragmentation strategy in data cutting message to itself
Before data are divided, judge whether the data volume of all data of itself storage is greater than preset data amount threshold value.In data
When amount is greater than preset data amount threshold value, number that data terminal then stores itself according to the data fragmentation strategy in data cutting message
According to being divided；Data volume be less than preset data amount threshold value when, data terminal can directly transmit initial data to than opposite end into
Row comparing.For example, preset data amount threshold value is a byte, when data volume is greater than a byte, data terminal is according to data cutting
The data that data fragmentation strategy in message stores itself divide, and when data volume is less than a byte, then directly transmit original
Beginning data carry out comparing to than opposite end.
Optionally, the verification data in the data fragmentation result that comparison result shows with same identification information are also identical
When, then caching clearance order is sent to first end and second end.First end and second end receive the caching sent than opposite end
Clearance order, wherein the caching clearance order includes the identification information of fragment data, and first end and second end are then according to caching
Clearance order removes the caching of the corresponding fragment data of identification information of fragment data.
Optionally, the verification data in the data fragmentation result with a certain same identification information are shown not in comparison result
When identical, then data echo request is sent to first end and second end；Or show that a certain identification information exists in comparison result
It is not existed simultaneously in first caching and the second caching, then sends data loopback to the corresponding data terminal of caching where identification information
Request.First end and second end receive the data echo request sent than opposite end, wherein the data echo request includes fragment
The identification information of data, first end and second end are then believed according to data echo request to the mark for sending fragment data than opposite end
Cease corresponding fragment data.
Using the above method, drawn in the data stored according to the data fragmentation strategy in data cutting message to itself
/ preceding, judge whether the data volume of all data of itself storage is greater than preset data amount threshold value, according to data volume size
Select the higher alignments of comparison efficiency.By establishing fragment data caching mechanism in first end and second end, receiving
When to the data echo request sent than opposite end, it can be read from caching and loopback fragment data is to than opposite end, avoiding repeatedly
Request inquiry database causes service performance loss larger, not only improves the access reading speed of data, also improves data
The efficiency of uniformity comparison.
Fig. 5 is a kind of block diagram for comparison of data consistency device that the embodiment of the present disclosure provides, which can pass through
Software, hardware or both are implemented in combination with as some or all of of electronic equipment.Referring to Fig. 5, which includes：
Cutting message transmission module 510, for sending data cutting message to first end and second end, data cutting disappears
Breath includes data fragmentation strategy and verification data format；
Fragment result obtains module 520, for obtaining the data fragmentation of first end and second end transmission as a result, data point
Piece result include the check number of each fragment data accordingly and the identification information of each fragment data, wherein fragment data is
What one end and second end were divided according to the data that stripping strategy stores itself, verification data be first end and
Second end is generated according to verification data format for each fragment data；
Comparison module 530, for comparing the verification data in the data fragmentation result with same identification information, wherein
If it is identical to verify data, show that the corresponding fragment data of first end and second end identification information is consistent.
Using above-mentioned apparatus, sent to first end and second end including data point by cutting message transmission module 510
Piece strategy and verify data format data cutting message, and using fragment result obtain module 520 obtain first end and
The data fragmentation that second end is sent is as a result, by configuration data format, it is possible to reduce the occupancy of network bandwidth, in comparison module
When whether 530 comparison datas consistent, can by compare same identification information data fragmentation result in verification data whether
It is consistent to determine whether data are consistent, reduce the time expended because waiting a large amount of fragment datas to upload, and then improve data
The efficiency of uniformity comparison.
Optionally, device 500 further includes memory module, in the data fragmentation for obtaining first end and second end transmission
As a result after, the data fragmentation result that first end is sent is stored into the first caching, the data fragmentation knot that second end is sent
Fruit stores into the second caching；
Comparison module 530 includes：Reading data submodule, for start multiple threads read in the first caching respectively and
Verification data in second caching in the data fragmentation result with same identification information；Judging submodule, for judging have
Whether the verification data in the data fragmentation result of same identification information are identical.
Optionally, device 500 further includes clearance order sending module, for the data fragmentation knot with same identification information
When verification data in fruit are identical, caching clearance order is sent to first end and second end；Wherein, caching clearance order is used for
Indicate that first end and second end remove the caching of fragment data corresponding to identical verification data.
Optionally, device 500 further includes echo request sending module, for the data with a certain same identification information point
When verification data in piece result are not identical, data echo request is sent to first end and second end；
Echo request sending module is also used to a certain identification information and does not exist simultaneously in the first caching and the second caching
When, corresponding data terminal is cached where to identification information sends data echo request；Wherein, data echo request includes mark letter
Breath returns to the corresponding fragment data of identification information for request data end；
Comparison module 530 is obtained for comparing the fragment data of first end loopback and the fragment data of second end loopback
Data difference information.
Using above-mentioned apparatus, sending to first end and second end includes data fragmentation strategy and verification data format
Data cutting message, and the data fragmentation of first end and second end transmission is received as a result, reducing because waiting a large amount of fragment numbers
The time expended according to upload.Data fragmentation result is stored in the caching than opposite end, is compared using multithreading, monitoring caches
Data accelerate data access speed, and then improve the efficiency of comparison of data consistency.
Fig. 6 is a kind of block diagram for data bank management device that the embodiment of the present disclosure provides, which can be by soft
Part, hardware or both are implemented in combination with as some or all of of electronic equipment.Referring to Fig. 6, which includes：
Cutting message reception module 610, for receiving the data cutting message sent than opposite end；
Data cutting module 620, the data for being stored according to the data fragmentation strategy in data cutting message to itself
It is divided, obtains the identification information of fragment data and each fragment data；And according to the check number in data cutting message
According to format, the verification data of each fragment data are generated；
Fragment result sending module 630, for sending data fragmentation as a result, data fragmentation result includes each to than opposite end
The check number of fragment data is accordingly and the identification information of each fragment data.
Using above-mentioned apparatus, data cutting that first end and second end are received according to cutting message reception module 610
The data that message stores itself divide, and generate each fragment number according to the verification data format in data cutting message
According to verification data, then will include check number accordingly and the data fragmentation result of identification information is sent to and compares opposite end.By matching
Set data format, it is possible to reduce the occupancy of network bandwidth, and reduce and expend the time because transmitting a large amount of fragment datas, it is passed in data
The defeated stage accelerates comparing speed, and then improves the efficiency of comparison of data consistency.
Optionally, device 600 further includes：
Clearance order receiving module, for receiving the caching clearance order sent than opposite end, caching clearance order includes point
The identification information of sheet data；
Caching removes module, for removing the corresponding fragment data of identification information of fragment data according to caching clearance order
Caching.
Optionally, device 600 further includes：
Echo request receiving module, for receiving the data echo request sent than opposite end, data echo request includes point
The identification information of sheet data；
Fragment data sending module, for according to data echo request to than opposite end send fragment data identification information pair
The fragment data answered.
Optionally, device 600 further includes：
Judgment module, for being drawn in the data stored according to the data fragmentation strategy in data cutting message to itself
/ preceding, judge whether the data volume of all data of itself storage is greater than preset data amount threshold value；
Data cutting module 620 is used for when data volume is greater than preset data amount threshold value, according in data cutting message
The data that data fragmentation strategy stores itself divide；
Fragment data sending module, be also used to data volume be less than preset data amount threshold value when, directly transmit data to than
Opposite end carries out comparing.
Using above-mentioned apparatus, in data cutting module 620 according to the data fragmentation strategy in data cutting message to itself
Before the data of storage are divided, by judgment module judge itself storage all data data volume whether be greater than it is default
Data-quantity threshold, to select the higher alignments of comparison efficiency according to data volume size.By in first end and second end
Fragment data caching mechanism is established, when echo request receiving module receives the data echo request than opposite end transmission, Ke Yicong
It is read in caching and loopback fragment data is extremely than opposite end, avoid that inquiry database is repeatedly requested to cause service performance loss larger,
The access reading speed of data is not only improved, the efficiency of comparison of data consistency is also improved.
Fig. 7 is the block diagram for a kind of electronic equipment that the embodiment of the present disclosure provides, as shown in fig. 7, electronic equipment 700 includes place
Device 701 is managed, quantity can be one or more and memory 702, for storing the calculating that can be executed by processor 701
Machine program.The computer program stored in memory 702 may include it is one or more each correspond to one group refer to
The module of order.In addition, processor 701, which can be configured as, executes the computer program, to execute above-mentioned be applied to than opposite end
Comparison of data consistency method.
In addition, electronic equipment 700 can also include power supply module 703 and communication component 704, which can be with
It is configured as executing the power management of electronic equipment 700, which, which can be configured as, realizes electronic equipment 700
Communication, for example, wired or wireless communication.In addition, the electronic equipment 700 can also include input/output (I/O) interface 705.Electricity
Sub- equipment 700 can be operated based on the operating system for being stored in memory 702, such as Windows ServerTM, Mac OS
XTM, UnixTM, LinuxTM etc..
In a further exemplary embodiment, a kind of computer readable storage medium including program instruction is additionally provided, it should
The step of above-mentioned comparison of data consistency method being applied to than opposite end is realized when program instruction is executed by processor.For example, should
Computer readable storage medium can be the above-mentioned memory 702 including program instruction, and above procedure instruction can be by electronic equipment
700 processor 701 is executed to complete the above-mentioned comparison of data consistency method being applied to than opposite end.
Fig. 8 is the block diagram for a kind of electronic equipment that the embodiment of the present disclosure provides, as shown in figure 8, electronic equipment 800 includes place
Device 801 is managed, quantity can be one or more and memory 802, for storing the calculating that can be executed by processor 801
Machine program.The computer program stored in memory 802 may include it is one or more each correspond to one group refer to
The module of order.It is above-mentioned applied to data terminal to execute in addition, processor 801, which can be configured as, executes the computer program
Comparison of data consistency method.
In addition, electronic equipment 800 can also include power supply module 803 and communication component 804, which can be with
It is configured as executing the power management of electronic equipment 800, which, which can be configured as, realizes electronic equipment 800
Communication, for example, wired or wireless communication.In addition, the electronic equipment 800 can also include input/output (I/O) interface 805.Electricity
Sub- equipment 800 can be operated based on the operating system for being stored in memory 802, such as Windows ServerTM, Mac OS
XTM, UnixTM, LinuxTM etc..
In a further exemplary embodiment, a kind of computer readable storage medium including program instruction is additionally provided, it should
The step of above-mentioned comparison of data consistency method applied to data terminal is realized when program instruction is executed by processor.For example, should
Computer readable storage medium can be the above-mentioned memory 802 including program instruction, and above procedure instruction can be by electronic equipment
800 processor 801 is executed to complete the above-mentioned comparison of data consistency method applied to data terminal.
The preferred embodiment of the disclosure is described in detail in conjunction with attached drawing above, still, the disclosure is not limited to above-mentioned reality
The detail in mode is applied, in the range of the technology design of the disclosure, a variety of letters can be carried out to the technical solution of the disclosure
Monotropic type, these simple variants belong to the protection scope of the disclosure.
It is further to note that specific technical features described in the above specific embodiments, in not lance
In the case where shield, it can be combined in any appropriate way.In order to avoid unnecessary repetition, the disclosure to it is various can
No further explanation will be given for the combination of energy.
In addition, any combination can also be carried out between a variety of different embodiments of the disclosure, as long as it is without prejudice to originally
Disclosed thought equally should be considered as disclosure disclosure of that.
Claims (10)
1. a kind of comparison of data consistency method is applied to than opposite end, which is characterized in that the method includes：
Data cutting message is sent to first end and second end, the data cutting message includes data fragmentation strategy and school
Test data format；
The data fragmentation of the first end and second end transmission is obtained as a result, the data fragmentation result includes each point
The check number of sheet data is accordingly and the identification information of each fragment data, wherein the fragment data be the first end and
The second end is divided according to the data that the stripping strategy stores itself, and the verification data are described
What one end and the second end were generated according to the verification data format for each fragment data；
Compare the verification data in the data fragmentation result with same identification information, wherein if the verification data are identical,
Show that first end fragment data corresponding with identification information described in the second end is consistent.
2. the method according to claim 1, wherein the method also includes：
After obtaining the data fragmentation result that the first end and the second end are sent, number that the first end is sent
It stores according to fragment result into the first caching, the data fragmentation result that the second end is sent is stored into the second caching；
The verification data compared in the data fragmentation result with same identification information, including：
Start multiple threads and reads the number in first caching and in second caching with same identification information respectively
According to the verification data in fragment result；
Judge have the verification data in the data fragmentation result of same identification information whether identical.
3. a kind of comparison of data consistency method, which is characterized in that the method includes：
Receive the data cutting message sent than opposite end；
It is divided according to the data that the data fragmentation strategy in the data cutting message stores itself, obtains fragment data
And the identification information of each fragment data；And
According to the verification data format in the data cutting message, the verification data of each fragment data are generated；
To it is described than opposite end send data fragmentation as a result, the data fragmentation result include the check number of each fragment data accordingly
And the identification information of each fragment data.
4. according to the method described in claim 3, it is characterized in that, the method also includes：
The caching clearance order sent than opposite end is received, the caching clearance order includes the identification information of fragment data；
The caching of the corresponding fragment data of identification information of the fragment data is removed according to the caching clearance order.
5. a kind of comparison of data consistency device, which is characterized in that described device includes：
Cutting message transmission module, for sending data cutting message, the data cutting message to first end and second end
Including data fragmentation strategy and verification data format；
Fragment result obtains module, for obtaining the data fragmentation of the first end and second end transmission as a result, described
Data fragmentation result include the check number of each fragment data accordingly and the identification information of each fragment data, wherein described point
Sheet data is that the first end and the second end are divided to obtain according to the data that the stripping strategy stores itself
, the verification data are the first end and the second end according to the verification data format for each fragment data
It generates；
Comparison module, for comparing the verification data in the data fragmentation result with same identification information, wherein if the school
It is identical to test data, then shows that first end fragment data corresponding with identification information described in the second end is consistent.
6. a kind of data bank management device, which is characterized in that described device includes：
Cutting message reception module, for receiving the data cutting message sent than opposite end；
Data cutting module, for being carried out according to the data fragmentation strategy in the data cutting message to the data itself stored
It divides, obtains the identification information of fragment data and each fragment data；And
According to the verification data format in the data cutting message, the verification data of each fragment data are generated；
Fragment result sending module, for sending data fragmentation as a result, the data fragmentation result includes every than opposite end to described
The check number of one fragment data is accordingly and the identification information of each fragment data.
7. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the program is held by processor
The step of method as claimed in claim 1 or 2 is realized when row.
8. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the program is held by processor
The step of claim 3 or 4 the method are realized when row.
9. a kind of electronic equipment, which is characterized in that including：
Memory is stored thereon with computer program；
Processor, for executing the computer program in the memory, to realize method as claimed in claim 1 or 2
Step.
10. a kind of electronic equipment, which is characterized in that including：
Memory is stored thereon with computer program；
Processor, for executing the computer program in the memory, to realize claim 3 or 4 the methods
Step.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810623674.5A CN108897806A (en) | 2018-06-15 | 2018-06-15 | Comparison of data consistency method, apparatus, storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810623674.5A CN108897806A (en) | 2018-06-15 | 2018-06-15 | Comparison of data consistency method, apparatus, storage medium and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108897806A true CN108897806A (en) | 2018-11-27 |
Family
ID=64345085
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810623674.5A Pending CN108897806A (en) | 2018-06-15 | 2018-06-15 | Comparison of data consistency method, apparatus, storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108897806A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109766330A (en) * | 2018-12-29 | 2019-05-17 | 北京三快在线科技有限公司 | Data fragmentation method, device, electronic equipment and storage medium |
| CN109818748A (en) * | 2018-12-28 | 2019-05-28 | 北京奇安信科技有限公司 | Data transmission method, equipment, system and medium |
| CN109871294A (en) * | 2019-01-31 | 2019-06-11 | 北京百度网讯科技有限公司 | Method and apparatus for verifying data, processing request |
| CN109918379A (en) * | 2019-03-08 | 2019-06-21 | 北京马尔马拉科技有限公司 | A kind of processing system and processing method of the big data based on block chain |
| CN111221862A (en) * | 2019-12-31 | 2020-06-02 | 五八有限公司 | Request processing method and device |
| CN111314021A (en) * | 2020-02-07 | 2020-06-19 | 北京字节跳动网络技术有限公司 | Data transmission method and device, electronic equipment and computer readable storage medium |
| CN111368279A (en) * | 2020-02-25 | 2020-07-03 | 深圳市腾讯计算机系统有限公司 | Data processing method, data processing device, computer readable storage medium and computer equipment |
| CN112580062A (en) * | 2019-09-27 | 2021-03-30 | 厦门网宿有限公司 | Data consistency checking method and data uploading and downloading device |
| WO2022088861A1 (en) * | 2020-10-27 | 2022-05-05 | 网联清算有限公司 | Database fault handling method and apparatus |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1818878A (en) * | 2006-03-10 | 2006-08-16 | 四川大学 | Information system data consistency detection |
| CN106033427A (en) * | 2015-03-11 | 2016-10-19 | 阿里巴巴集团控股有限公司 | A sampling data verification method and device |
| CN106658017A (en) * | 2016-10-26 | 2017-05-10 | 北京集创北方科技股份有限公司 | Image compression and decompression methods, image processing method and related devices |
| US20170177489A1 (en) * | 2014-09-15 | 2017-06-22 | Huawei Technologies Co.,Ltd. | Data deduplication system and method in a storage array |
| CN107092444A (en) * | 2017-05-03 | 2017-08-25 | 郑州云海信息技术有限公司 | A kind of system and implementation method of the consistency verification of data based on rsync |
| CN107219997A (en) * | 2016-03-21 | 2017-09-29 | 阿里巴巴集团控股有限公司 | A kind of method and device for being used to verify data consistency |
-
2018
- 2018-06-15 CN CN201810623674.5A patent/CN108897806A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1818878A (en) * | 2006-03-10 | 2006-08-16 | 四川大学 | Information system data consistency detection |
| US20170177489A1 (en) * | 2014-09-15 | 2017-06-22 | Huawei Technologies Co.,Ltd. | Data deduplication system and method in a storage array |
| CN106033427A (en) * | 2015-03-11 | 2016-10-19 | 阿里巴巴集团控股有限公司 | A sampling data verification method and device |
| CN107219997A (en) * | 2016-03-21 | 2017-09-29 | 阿里巴巴集团控股有限公司 | A kind of method and device for being used to verify data consistency |
| CN106658017A (en) * | 2016-10-26 | 2017-05-10 | 北京集创北方科技股份有限公司 | Image compression and decompression methods, image processing method and related devices |
| CN107092444A (en) * | 2017-05-03 | 2017-08-25 | 郑州云海信息技术有限公司 | A kind of system and implementation method of the consistency verification of data based on rsync |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109818748A (en) * | 2018-12-28 | 2019-05-28 | 北京奇安信科技有限公司 | Data transmission method, equipment, system and medium |
| CN109766330A (en) * | 2018-12-29 | 2019-05-17 | 北京三快在线科技有限公司 | Data fragmentation method, device, electronic equipment and storage medium |
| CN109766330B (en) * | 2018-12-29 | 2020-02-14 | 北京三快在线科技有限公司 | Data slicing method and device, electronic equipment and storage medium |
| CN109871294A (en) * | 2019-01-31 | 2019-06-11 | 北京百度网讯科技有限公司 | Method and apparatus for verifying data, processing request |
| CN109918379A (en) * | 2019-03-08 | 2019-06-21 | 北京马尔马拉科技有限公司 | A kind of processing system and processing method of the big data based on block chain |
| CN109918379B (en) * | 2019-03-08 | 2020-02-28 | 新加坡贝尔信息科技有限公司 | Block chain-based big data processing system and processing method |
| CN112580062A (en) * | 2019-09-27 | 2021-03-30 | 厦门网宿有限公司 | Data consistency checking method and data uploading and downloading device |
| CN111221862A (en) * | 2019-12-31 | 2020-06-02 | 五八有限公司 | Request processing method and device |
| CN111221862B (en) * | 2019-12-31 | 2023-08-11 | 五八有限公司 | Request processing method and device |
| CN111314021A (en) * | 2020-02-07 | 2020-06-19 | 北京字节跳动网络技术有限公司 | Data transmission method and device, electronic equipment and computer readable storage medium |
| CN111368279A (en) * | 2020-02-25 | 2020-07-03 | 深圳市腾讯计算机系统有限公司 | Data processing method, data processing device, computer readable storage medium and computer equipment |
| CN111368279B (en) * | 2020-02-25 | 2023-04-18 | 深圳市腾讯计算机系统有限公司 | Data processing method, data processing device, computer readable storage medium and computer equipment |
| WO2022088861A1 (en) * | 2020-10-27 | 2022-05-05 | 网联清算有限公司 | Database fault handling method and apparatus |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108897806A (en) | Comparison of data consistency method, apparatus, storage medium and electronic equipment | |
| CN107770182B (en) | Data storage method of home gateway and home gateway | |
| US10341469B2 (en) | Data transmission method and apparatus and data receiving method and apparatus | |
| CN104010285A (en) | Short message filtering method and system, short message service center and terminal | |
| CN109951546B (en) | Transaction request processing method, device, equipment and medium based on intelligent contract | |
| CN110944228B (en) | Video stream protection method, device, computing device and readable medium | |
| CN109309655B (en) | Stateless communication security signature method, terminal and server | |
| CN105354505B (en) | A kind of image adaptation method and electronic equipment | |
| CN107911381A (en) | Access method, system, server-side and the client of application programming interface | |
| CN105379164A (en) | Methods and apparatuses for transmitting and receiving data and recording medium for executing the methods | |
| US9491625B2 (en) | Access point apparatus for configuring multiple security tunnel, and system having the same and method thereof | |
| CN108881233A (en) | anti-attack processing method, device, equipment and storage medium | |
| CN108243146A (en) | A kind of efficient information submits method | |
| KR102269594B1 (en) | Method and apparatus for transmitting and receiving information of servers in content delivery network system | |
| CN109818943A (en) | A kind of authentication method suitable for low orbit satellite Internet of Things | |
| CN110995662B (en) | Data transmission method and system based on multi-path network media | |
| TWI242713B (en) | Method and apparatus for updating security records in a cache | |
| CN110381060A (en) | Data processing method, device, system and storage medium | |
| CN115189913B (en) | Data message transmission method and device | |
| CN104933059B (en) | File prestige acquisition methods, gateway and file reputation server | |
| CN106899554A (en) | A kind of method and device for preventing ARP from cheating | |
| CN107592361B (en) | Data transmission method, device and equipment based on dual IB network | |
| CN109361712A (en) | A kind of information processing method and information processing unit | |
| CN113676314B (en) | Quantum key management method, comparison method, output method, device and system | |
| CN110392083A (en) | Control method, device and the terminal device that the file second passes |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181127 |
|
| RJ01 | Rejection of invention patent application after publication |