CN109086172A - A kind of method and relevant apparatus of data processing - Google Patents
A kind of method and relevant apparatus of data processing Download PDFInfo
- Publication number
- CN109086172A CN109086172A CN201811108304.4A CN201811108304A CN109086172A CN 109086172 A CN109086172 A CN 109086172A CN 201811108304 A CN201811108304 A CN 201811108304A CN 109086172 A CN109086172 A CN 109086172A
- Authority
- CN
- China
- Prior art keywords
- target data
- identifier
- data
- storage equipment
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1464—Management of the backup or restore process for networked environments
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the present application discloses a kind of method of data processing, comprising: the first storage equipment reads target data；First storage equipment detection target data whether there is corresponding first identifier and corresponding second identifier, wherein second identifier is for marking target data by compression processing；If there are corresponding first identifier and corresponding second identifier, the first storage equipment to send first object data packet to the second storage equipment for target data.The embodiment of the present application also discloses a kind of data processing equipment.The embodiment of the present application alleviates the burden of main storage device and backup storage device data processing.
Description
Technical field
This application involves field of data storage more particularly to the methods and relevant apparatus of a kind of data processing.
Background technique
Remote copy technology is a kind of remote data backup technology based on storage equipment, is generally divided into synchronous remote copy
And asynchronous remote copy.Synchronous remote copy cardinal principle is that data need while writing main storage device and backup storage device
On, the cardinal principle of asynchronous remote copy is then first to write data on main storage device, it is subsequent by data from main storage device
It copies on backup storage device.
In big data era, the data storage of magnanimity occupies a large amount of memory space.It is deleted at processing technique and compression again
Reason technology is can currently to reduce the core technology of data space, especially in the full flash memory storage battle array of memory space higher cost
It arranges (all flash array, AFA), deletes processing technique again and compression processing technology has become characteristic indispensable in AFA.
It is usual that processing technique is deleted again are as follows: the data being newly written are calculated into a cryptographic Hash, are then compared with stored cryptographic Hash, if
It was found that there is identical cryptographic Hash, then the position of identical cryptographic Hash corresponding data is recorded, current data is not written into storage equipment.
However, carrying out target data between main storage device and backup storage device in existing remote copy technology
Duplication when, not perceive target data whether deleted processing and compression processing again, cause data replicate when, even if
Target data has been carried out and deletes processing and compression processing again, and main storage device is sent after still unziping it to target data
To backup storage device, backup storage device is deleted processing and compression processing to the target data after decompression again again.By
This, has not only aggravated the burden of main storage device and backup storage device data processing, while main storage device and backup storage
The data volume transmitted between equipment is very big, causes to restore point target (recovery point when asynchronous remote copy
Objective, RPO) it is very high.
Summary of the invention
The embodiment of the present application provides a kind of method of data processing, for storing the remote data backup of equipment.
In view of this, the application first aspect provides a kind of method of data processing, comprising:
First storage equipment reads target data；
The first storage equipment detects the target data and whether there is corresponding first identifier and corresponding second identifier,
Wherein, the second identifier is for marking the target data by compression processing；
If there are the corresponding first identifier and the corresponding second identifier, the first storage equipment for the target data
First object data packet is sent to the second storage equipment, so that the second storage equipment is according to the first object data packet to this
Target data is handled；
Wherein, the target data, the first identifier and the second identifier are at least carried in the first object data packet, it should
First identifier is used to indicate the second storage equipment and is deleted processing again to the target data according to the first identifier, second mark
Knowledge is used to indicate the second storage equipment and carries out write-in processing to the target data.
In conjunction with the embodiment of the present application in a first aspect, in the first possible implementation of the first aspect, this first
After storage equipment detects the target data with the presence or absence of corresponding first identifier and corresponding second identifier, this method is also wrapped
It includes:
If the corresponding first identifier and the corresponding second identifier is not present in the target data, which is set
It is standby to send the second target packet to the second storage equipment, so that the second storage equipment is according to second target packet
The target data is handled；
Wherein, the target data and third mark are at least carried in second data packet, third mark is used to indicate
The second storage equipment carries out compression processing to the target data.
In conjunction with the first possible implementation of the first aspect of the embodiment of the present application, second in first aspect can
In the implementation of energy, after which reads the target data, this method further include:
If the target data is stored in the buffer zone of the first storage equipment, which second deposits to this
Storage equipment sends second target packet, so that the second storage equipment is according to second target packet to the number of targets
According to being handled；
Wherein, the target data and third mark are at least carried in second data packet.
In conjunction with the embodiment of the present application in a first aspect, in a third possible implementation of the first aspect, this first
After storage equipment detects the target data with the presence or absence of corresponding first identifier and corresponding second identifier, this method is also wrapped
It includes:
If the target data is there are the corresponding first identifier and the corresponding second identifier is not present, first storage
Equipment sends third target packet to the second storage equipment, so that the second storage equipment is according to the third target data
Packet handles the target data；
Wherein, the target data, the first identifier and third mark are at least carried in the third target packet.
In conjunction with the embodiment of the present application in a first aspect, in a fourth possible implementation of the first aspect, this first
After storage equipment detects the target data with the presence or absence of corresponding first identifier and corresponding second identifier, this method is also wrapped
It includes:
If the target data is there is no the corresponding first identifier and there are the corresponding second identifier, first storages
Equipment sends the 4th target packet to the second storage equipment, so that the second storage equipment is according to the 4th target data
Packet handles the target data；
Wherein, the target data and the second identifier are at least carried in the 4th target packet.
The application second aspect provides a kind of data processing equipment, which includes:
Read module, for reading target data；
Detection module whether there is corresponding first identifier and corresponding second identifier for detecting the target data,
Wherein, the second identifier is for marking the target data by compression processing；
Sending module, if for the target data there are the corresponding first identifier and the corresponding second identifier,
The sending module sends first object data packet to the second storage equipment, so that the second storage equipment is according to the first object
Data packet handles the target data；
Wherein, the target data, the first identifier and the second identifier are at least carried in the first object data packet, it should
First identifier is used to indicate the second storage equipment and is deleted processing again to the target data according to the first identifier, second mark
Knowledge is used to indicate the second storage equipment and carries out write-in processing to the target data.
In conjunction with the second aspect of the embodiment of the present application, in the first possible implementation of the second aspect, provide
A kind of data processing equipment, comprising:
The sending module, if being also used to the target data, there is no the corresponding first identifier and corresponding second marks
Know, then the sending module to this second storage equipment send the second target packet so that this second storage equipment according to be somebody's turn to do
Second target packet handles the target data；
Wherein, the target data and third mark are at least carried in second data packet, third mark is used to indicate
The second storage equipment carries out compression processing to the target data.
In conjunction with the first possible implementation of the second aspect of the embodiment of the present application, second in second aspect can
In the implementation of energy, a kind of data processing equipment is provided, comprising:
The sending module, if being also used to the buffer zone that the target data is stored in the first storage equipment, the transmission
Module sends second target packet to the second storage equipment, so that the second storage equipment is according to second number of targets
The target data is handled according to packet；
Wherein, the target data and third mark are at least carried in second data packet.
In conjunction with the second aspect of the embodiment of the present application, in the third possible implementation of the second aspect, provide
A kind of data processing equipment, comprising:
The sending module, if be also used to the target data there are the corresponding first identifier and there is no it is corresponding this second
Mark, then the sending module to this second storage equipment send third target packet so that this second storage equipment according to
The third target packet handles the target data；
Wherein, the target data, the first identifier and third mark are at least carried in the third target packet.
In conjunction with the second aspect of the embodiment of the present application, in the fourth possible implementation of the second aspect, provide
A kind of data processing equipment, comprising:
The sending module, if be also used to the target data there is no the corresponding first identifier and there are it is corresponding this second
Mark, then the sending module to this second storage equipment send the 4th target packet so that this second storage equipment according to
4th target packet handles the target data；
Wherein, the target data and the second identifier are at least carried in the 4th target packet.
As can be seen from the above technical solutions, the embodiment of the present application has the advantage that
The embodiment of the present application provides a kind of method of data processing, for storing the remote data backup of equipment.Mitigate
The burden of main storage device and backup storage device data processing, at the same reduce main storage device and backup storage device it
Between the data volume transmitted, reduce recovery point target when asynchronous remote copy.
Detailed description of the invention
Fig. 1 is the network frame schematic diagram that equipment is stored in the embodiment of the present application；
Fig. 2 is a flow diagram of data processing in the application scenarios of the application；
Fig. 3 is one embodiment schematic diagram of the method for data processing in the embodiment of the present application；
Fig. 4 is one embodiment schematic diagram of data processing equipment in the embodiment of the present application.
Specific embodiment
The embodiment of the present application provides a kind of method of data processing, for storing the remote data backup of equipment.Mitigate
The burden of main storage device and backup storage device data processing, at the same reduce main storage device and backup storage device it
Between the data volume transmitted, reduce recovery point target when asynchronous remote copy.
The description and claims of this application and term " first ", " second ", " third ", " in above-mentioned attached drawing
The (if present)s such as four " are to be used to distinguish similar objects, without being used to describe a particular order or precedence order.It should manage
The data that solution uses in this way are interchangeable under appropriate circumstances, so that the embodiments described herein can be in addition to illustrating herein
Or the sequence other than the content of description is implemented.In addition, term " includes " and " having " and their any deformation, it is intended that
Cover it is non-exclusive include, for example, containing the process, method, system, product or equipment of a series of steps or units need not limit
In step or unit those of is clearly listed, but may include be not clearly listed or for these process, methods, produce
The other step or units of product or equipment inherently.
Data processing equipment provided by the present application can be deployed in the number by main storage device and backup storage device foundation
According in backup network frame, in order to make it easy to understand, referring to Fig. 1, Fig. 1 is the network frame for storing equipment in the embodiment of the present application
Schematic diagram.Although in Fig. 1 including a main storage device and a backup storage device it should be appreciated that main memory
The type and quantity of the type and quantity and backup storage device of storing up equipment should all be determined according to actual scene, in practical application
In, the type and quantity of the type of main storage device and quantity and backup storage device are not defined, main storage device
And backup storage device is either individually storage equipment is also possible to the storage array of multiple storage equipment compositions, wherein
Equipment is stored either solid state hard disk (solid state drive, SSD) is also possible to hybrid hard disk (hybrid hard
Drive, HHD) it is also possible to mechanical hard disk (hard disk drive, HDD) and is also possible to CD server and tape library etc., when
When main storage device and backup storage device are the storage array of multiple storage equipment compositions, can by above-mentioned SSD, HHD,
One or more compositions of HDD, CD server and tape library, are not construed as limiting herein.Main storage device and backup storage device it
Between data communication can pass through transmission control protocol/Internet Protocol (transmission control
Protocol/internet protocol, TCP/IP) transmission.
The application can be applied to the remote copy technology of data, wherein the remote copy technology of data is generally divided into synchronization
Remote copy and asynchronous remote copy.Synchronous remote copy refers to through remote mirroring software, by the data of main storage device with
The mode of synchronous mirror copies to backup storage device, and input/output (in/out, I/O) affairs of each main storage device are equal
The completion confirmation message for needing to wait for remote copy, is just discharged.It is multiple that synchronous mirror require telecopy can with local
The content of system matches.When main storage device breaks down, after the application program of user is switched to backup storage device, by mirror
The remote copy of picture can guarantee that business is continued to execute without the loss of data.Asynchronous remote copy refers to guaranteeing updating
The basic I/O operation of main storage system is completed before backup storage device, the I/O operation of main storage device is not set by backup storage
Standby I/O operation influences.Long-range data duplication is carried out in a manner of background synchronization, this is subject to local system performance
Very little is influenced, transmission range is long (up to 1000 kilometers or more), small to network bandwidth requirement.
It is described in detail below from the angle of main storage device and backup storage device.In order to make it easy to understand, below
The application scenarios of a kind of method of data processing will be introduced in conjunction with Fig. 2, referring to Fig. 2, Fig. 2 is number in the application scenarios of the application
According to a flow diagram of processing, as shown, specifically:
In step S1, during the remote copy of data, when data need to carry out data synchronization job, main storage device
A difference bitmap can be generated, which, should for marking data different from backup storage device on main storage device
Data are usually the data being newly written on main storage device, need to be written the data at this time on backup storage device, complete number
According to synchronization job, main storage device first can find the logical volume address for needing the data replicated according to difference bitmap, in reality
In the application of border, it will include the target data that data, which are usually stored in storage equipment in the form of data block (data block),
Data block be known as target data block, since in software level, the processing to data is operated by logical volume address, because
This is in the method for the data processing that the application proposes, the logical volume address of target data block first in acquisition main storage device；
It is readable according to the address in getting main storage device behind the logical volume address of target data block in step S2
Get target data block, due to data through it is overweight delete processing when data can be calculated first, generate corresponding finger print information,
The finger print information be used to indicate main storage device using the finger print information search in the main storage device with the presence or absence of it is identical
The finger print information of storage then records the address of identical stored finger print information corresponding data if it exists, to be currently written
Data are not written into storage equipment, establish mapping relations with the address of the corresponding data of record, processing is deleted in completion again, and usually this refers to
Line information is stored in the build of the data block of the data.After data carry out compression processing, a mark can be generated, for identifying this
Data pass through compression processing, and the mark of the usual compression processing is stored in the build of the data block of the data.Getting main memory
In storage equipment behind the logical volume address of target data block, according to the address, target data block may be read into, by searching for number of targets
Compression is deleted again according to judging whether the data have been done with the presence or absence of the corresponding mark of finger print information and compression processing in block with this.If
Find in target data block there are finger print information and the corresponding mark of compression processing, judge the data done delete again processing with
And compression processing, judging result be it is yes, S4 is entered step, if finding in target data block there is no finger print information and compression
Handle corresponding mark, judge that the data are not done and delete processing and compression processing again, judging result be it is no, enter step S3.
In step S3, when finding in target data block there is no after finger print information and compression processing corresponding mark,
Data corresponding in target data block are directly sent to backup storage device by main storage device, and backup storage device is according to itself
The data received are further processed in business processing demand.
In step S4, when finding in target data block, there are finger print information and the corresponding mark of compression processing, judgements
The data have been done delete processing and compression processing again after, main storage device is by target data block address lookup target data block
The target data stored at build deletes finger print information again, if needing to enter step when the data block for replicating multiple batches simultaneously at this time
Rapid S5 enters step S6 if the data block for needing to replicate at this time is individual data block；
In step S5, after inquiring the corresponding finger print information of target data, when current needs while multiple batches are replicated
Data block when, the fingerprint, which can be used, in main storage device compares the finger print information of data block that is other while needing to replicate
Right, identical fingerprint, then retain a data block, and record the information of data block that is other while needing to replicate if it exists, into
Row duplicate removal processing.
In step S6, find the target data it is corresponding it is heavy delete finger print information after, due to the target data
Overcompression processing, therefore the target data corresponding with finger print information is deleted again read, for the target data compressed.
In step S7, after reading the target data compressed, main storage device is by the logical volume of target data block
Location, the compressed data deleting fingerprint again and reading are sent to backup storage device in the form of data packet, and backup storage is set
It is standby that the data received are further processed according to own service process demand.
In the present solution, main storage device, before sending target data to backup storage device, meeting is first to the target data institute
Target data block inquired, inquiry then will if it exists with the presence or absence of the mark of finger print information and compression processing is deleted again
This is heavy to delete the mark of finger print information and compression processing and is sent to backup in the form of data packet with the target data compressed and deposits
Equipment is stored up, backup storage device can be not repeated to calculate number of targets according to the mark for deleting finger print information and compression processing again
According to directly being deleted processing again using the heavy finger print information of deleting.Backup storage device can also be according to the compression processing received
Mark, judge that compression processing has been carried out in the target data being currently received, therefore do not need to carry out compression processing again,
It can write direct.The burden of main storage device and backup storage device data processing is alleviated, while reducing primary storage and setting
The standby data volume transmitted between backup storage device reduces recovery point target when asynchronous remote copy.
Referring to Fig. 3, Fig. 3 is one embodiment schematic diagram of the method for data processing in the embodiment of the present application, the application
One embodiment of the method for data processing includes: in embodiment
101, the first storage equipment reads target data；
In the present embodiment, the first storage equipment is different by obtaining record between the first storage equipment and the second storage equipment
The difference bitmap or differentiated identification of data determine target data, and according to the logical volume address where the target data, read
The target data, wherein logical volume address is a kind of position code, and the location information of data is indicated by Arabic numerals, such as:
The corresponding logical volume address of data 1 is 1, and the corresponding logical volume address of data 2 is 2, and so on.
102, the first storage equipment detection target data whether there is corresponding first identifier and corresponding second identifier；
In the present embodiment, when target data is deleted processing again, storage equipment can carry out Hash operation to target data
To generate corresponding cryptographic Hash (hash), which is known as the finger print information of the data, in the present embodiment the referred to as first mark
Know, carries out Hash operation and need using hash algorithm, the hash algorithm being applicable in the application may include: xxhash algorithm, MD
Hash algorithm, SHA-1 hash algorithm, SHA-2 hash algorithm, MD5 hash algorithm etc., are not construed as limiting herein.When target data into
After row compression processing, storage equipment can generate one and identify for marking the target data to have been carried out compression processing, the mark
It is known as second identifier in the present embodiment.The logical volume address of first identifier, second identifier and target data is target data
Metadata, metadata are normally stored in the build for the data block that target data is stored, also can setting according to different storage manufacturers
It sets and is stored in different regions, for example, metadata is stored in storage equipment in nonvolatile memory, be not construed as limiting herein.
First storage equipment detects in this storage equipment with the presence or absence of first identifier and second identifier.
If 103, target data is there are corresponding first identifier and corresponding second identifier, and the first storage equipment is to the
Two storage equipment send first object data packet；
In the present embodiment, there is the first mark corresponding with target data when the first storage equipment detects in this storage equipment
Know and second identifier corresponding with target data after, first storage equipment using get first identifier, second identifier with
And target data makes first object data packet, also includes the logical volume address of target data in first object data packet.The
First object data packet is sent to the second storage equipment by ICP/IP protocol by one storage equipment, and the second storage equipment is receiving
To after the first object data packet, the first identifier in the first object data packet can be used, detection second stores in equipment
With the presence or absence of finger print information identical with first identifier, and if it exists, then the target data in first object data packet is not written into, and
The logical volume address for recording the target data, and there are the addresses of the corresponding data of identical fingerprints information to establish mapping relations, complete
Processing is deleted again in the second storage equipment at target data.Second storage equipment is according to the second mark in first object data packet
Know, can determine that the target data being currently received is compressed data, therefore do not need again to carry out the target data
Compression processing.
In the embodiment of the present application, the first storage equipment, can be first to the mesh before sending target data to the second storage equipment
Target data block where mark data is detected, and detects whether there is the mark for deleting finger print information and compression processing again, if
In the presence of then the heavy mark for deleting finger print information and compression processing is sent in the form of data packet with the target data compressed
To the second storage equipment.Second storage equipment can be not repeated to count according to the mark for deleting finger print information and compression processing again
Target data is calculated, is directly deleted processing again using the heavy finger print information of deleting.Second storage equipment can also be according to receiving
The mark of compression processing judges that compression processing has been carried out in the target data being currently received, therefore does not need to carry out again
Compression processing can be write direct.The burden for alleviating the first storage equipment and the processing of the second storage device data, reduces simultaneously
The data volume transmitted between first storage equipment and the second storage equipment, reduces return contact mesh when asynchronous remote copy
Mark.
Optionally, on the basis of Fig. 3 corresponding embodiment, the side of second of data processing provided by the embodiments of the present application
In the embodiment of method, the first storage equipment detection target data whether there is corresponding first identifier and corresponding second identifier
Later, method further include:
If corresponding first identifier and corresponding second identifier is not present in target data, first stores equipment to second
It stores equipment and sends the second target packet, so that the second storage equipment carries out target data according to the second target packet
Processing；
Wherein, target data and third mark are at least carried in the second data packet, third mark is used to indicate second and deposits
It stores up equipment and compression processing is carried out to target data.
In the present embodiment, the first storage equipment detection target data is with the presence or absence of corresponding first identifier and corresponding the
Two mark after, if target data be not present corresponding first identifier and corresponding second identifier, i.e., the target data without
It is overweight delete processing and compression processing after, first storage equipment obtain third mark, and using third mark and number of targets
It also include the logical volume address of target data in the second target packet according to the second target packet is made.Wherein third mark
Know for the first storage equipment after detecting the uncompressed processing of the target data, the newly-built mark target data is without pressure
The mark of contracting processing data.Second storage equipment is identified according to the third carried in the second target packet received, is determined
Target data in second target packet is the data of uncompressed processing, simultaneously because not carrying in the second target packet
First identifier, therefore the second storage equipment can choose whether to need to be deleted processing again to the target data according to self-demand
And compression processing.
In the embodiment of the present application, when the first storage equipment detects target data, there is no first identifier and second identifiers
Later, the second target packet is sent to the second storage equipment, carries target data and third mark in the second target packet
Know.The method of data processing of the target data without deleting processing and compression processing again is provided, the realization spirit of scheme is improved
Activity.
Optionally, on the basis of the embodiment of the method for second of data processing provided by the embodiments of the present application, this Shen
Please embodiment provide the third data processing method embodiment in, first storage equipment read target data after, side
Method further include:
If target data is stored in the buffer zone of the first storage equipment, the first storage equipment is sent out to the second storage equipment
The second target packet is sent, so that the second storage equipment is handled target data according to the second target packet；
Wherein, target data and third mark are at least carried in the second data packet.
In the present embodiment, after the first storage equipment reads target data, the first storage equipment may determine that target
The currently stored position of data whether be the first storage equipment buffer zone (cache), if so, skip detection first store
The step of whether there is first identifier and second identifier in equipment sends the second target packet to the second storage equipment.The
Two storage equipment receive the process flow executed after the second target packet, similar second of number provided by the embodiments of the present application
According to the embodiment of the method for processing, details are not described herein again.
In the embodiment of the present application, when in the buffer zone that target data is stored in the first storage equipment, due to buffer area
Data in domain are to delete processing and compression processing without overweight, can directly judge the target data for without deleting processing again
And the data of compression processing, the first storage equipment send the second target packet to the second storage equipment.Provide a kind of mesh
Mark data are when being stored in buffer zone, the method for data processing, simplify the first storage equipment to the process flow of target data,
Improve the feasibility of scheme.
Optionally, on the basis of Fig. 3 corresponding embodiment, the side of the 4th kind of data processing provided by the embodiments of the present application
In the embodiment of method, the first storage equipment detection target data whether there is corresponding first identifier and corresponding second identifier
Later, method further include:
If target data is there are corresponding first identifier and corresponding second identifier is not present, the first storage equipment is to the
Two storage equipment send third target packets so that second storage equipment according to third target packet to target data into
Row processing；
Wherein, target data, first identifier and third mark are at least carried in third target packet.
In the present embodiment, when the first storage equipment detects target data there are corresponding first identifier and correspondence is not present
Second identifier when, i.e., target data through it is overweight delete processing and uncompressed processing, at this point, first storage equipment to second storage
Equipment sends third target packet, and target data, first identifier, third mark and mesh are carried in the third target packet
Mark the logical volume address of data.After second storage equipment receives third target packet, this is can be used in the second storage equipment
First identifier in first object data packet, detection second stores in equipment to be believed with the presence or absence of fingerprint identical with first identifier
Breath, and if it exists, then the target data in first object data packet is not written into, and records the logical volume address of the target data, with
There are the addresses of the corresponding data of identical fingerprints information to establish mapping relations, completes weight of the target data in the second storage equipment
Processing is deleted, the second storage equipment is identified according to the third carried in the third target packet received, determines third number of targets
It is the data of uncompressed processing according to the target data in packet, the second storage equipment can choose whether needs according to self-demand
Compression processing is carried out to the target data.
In the embodiment of the present application, providing target data is to delete processing and without the data processing of compression processing through overweight
Method, the second storage equipment deleted processing to target data according to the third target packet received again, and according to oneself
The demand of body chooses whether to carry out compression processing to target data, improves the realization flexibility of scheme.
Optionally, on the basis of Fig. 3 corresponding embodiment, the side of the 5th kind of data processing provided by the embodiments of the present application
In the embodiment of method, the first storage equipment detection target data whether there is corresponding first identifier and corresponding second identifier
Later, method further include:
If target data, there is no corresponding first identifier and there are corresponding second identifier, the first storage equipment is to the
Two storage equipment send the 4th target packets so that second storage equipment according to the 4th target packet to target data into
Row processing；
Wherein, target data and second identifier are at least carried in the 4th target packet.
In the present embodiment, when the first storage equipment detects target data there is no corresponding first identifier and there is correspondence
Second identifier when, i.e., target data deletes processing and through compression processing without overweight, and the first storage equipment is to the second storage equipment
The 4th target packet is sent, the logical volume of target data, second identifier and target data is carried in the 4th target packet
Address.Due to not carrying first identifier in the 4th target packet, the second storage equipment can be selected according to self-demand
Whether need to be deleted processing again to the target data.Second storage equipment according to the second identifier in the 4th target packet,
It can determine that the target data being currently received is compressed data, therefore not need again to compress the target data
Processing.
In the embodiment of the present application, providing target data is to delete processing and data processing Jing Guo compression processing without overweight
Method, second storage equipment is chosen whether to delete target data again processing according to the demand of itself, improve scheme
Realize flexibility.
Data processing equipment in the application is described in detail below, referring to Fig. 4, Fig. 4 is in the embodiment of the present application
One embodiment schematic diagram of data processing equipment, one embodiment of data processing equipment 20 provided by the embodiments of the present application
In, data processing equipment 20 includes:
Read module 201, for reading target data；
Detection module 202 whether there is corresponding first identifier and corresponding second identifier for detecting target data,
Wherein, second identifier is for marking target data by compression processing；
Sending module 203, if being sent for target data there are corresponding first identifier and corresponding second identifier
Module 203 sends first object data packet to the second storage equipment, so that the second storage equipment is according to first object data packet
Target data is handled.
In the present embodiment, read module 201 read target data, detection module 202 detect target data with the presence or absence of pair
The first identifier and corresponding second identifier answered, wherein second identifier is for marking target data by compression processing, if mesh
Marking data, there are corresponding first identifier and corresponding second identifiers, then are sent out by sending module 203 to the second storage equipment
First object data packet is sent, so that the second storage equipment is handled target data according to first object data packet.
In the embodiment of the present application, the first storage equipment, can be first to the mesh before sending target data to the second storage equipment
Target data block where mark data is detected, and detects whether there is the mark for deleting finger print information and compression processing again, if
In the presence of then the heavy mark for deleting finger print information and compression processing is sent in the form of data packet with the target data compressed
To the second storage equipment.Second storage equipment can be not repeated to count according to the mark for deleting finger print information and compression processing again
Target data is calculated, is directly deleted processing again using the heavy finger print information of deleting.Second storage equipment can also be according to receiving
The mark of compression processing judges that compression processing has been carried out in the target data being currently received, therefore does not need to carry out again
Compression processing can be write direct.The burden for alleviating the first storage equipment and the processing of the second storage device data, reduces simultaneously
The data volume transmitted between first storage equipment and the second storage equipment, reduces return contact mesh when asynchronous remote copy
Mark.
Optionally, on the basis of Fig. 4 corresponding embodiment, second of data processing equipment provided by the embodiments of the present application
Embodiment in,
Sending module 203, if being also used to target data is not present corresponding first identifier and corresponding second identifier,
Sending module 203 sends the second target packet to the second storage equipment, so that the second storage equipment is according to the second number of targets
Target data is handled according to packet；
Wherein, target data and third mark are at least carried in the second data packet, third mark is used to indicate second and deposits
It stores up equipment and compression processing is carried out to target data.
In the embodiment of the present application, when the first storage equipment detects target data, there is no first identifier and second identifiers
Later, the second target packet is sent to the second storage equipment, carries target data and third mark in the second target packet
Know.The method of data processing of the target data without deleting processing and compression processing again is provided, the realization spirit of scheme is improved
Activity.
Optionally, on the basis of the embodiment of second of data processing equipment provided by the embodiments of the present application, the application
In the embodiment for the third data processing equipment that embodiment provides,
Sending module 203, if being also used to the buffer zone that target data is stored in the first storage equipment, sending module
203 send the second target packet to the second storage equipment, so that the second storage equipment is according to the second target packet to mesh
Mark data are handled；
Wherein, target data and third mark are at least carried in the second data packet.
In the embodiment of the present application, when in the buffer zone that target data is stored in the first storage equipment, due to buffer area
Data in domain are to delete processing and compression processing without overweight, can directly judge the target data for without deleting processing again
And the data of compression processing, the first storage equipment send the second target packet to the second storage equipment.Provide a kind of mesh
Mark data are when being stored in buffer zone, the method for data processing, simplify the first storage equipment to the process flow of target data,
Improve the feasibility of scheme.
Optionally, on the basis of Fig. 4 corresponding embodiment, the 4th kind of data processing equipment provided by the embodiments of the present application
Embodiment in,
Sending module 203, if being also used to target data there are corresponding first identifier and corresponding second identifier being not present,
Then sending module 203 sends third target packet to the second storage equipment, so that the second storage equipment is according to third target
Data packet handles target data；
Wherein, target data, first identifier and third mark are at least carried in third target packet.
In the embodiment of the present application, providing target data is to delete processing and without the data processing of compression processing through overweight
Method, the second storage equipment deleted processing to target data according to the third target packet received again, and according to oneself
The demand of body chooses whether to carry out compression processing to target data, improves the realization flexibility of scheme.
Optionally, on the basis of Fig. 4 corresponding embodiment, the 5th kind of data processing equipment provided by the embodiments of the present application
Embodiment in,
Sending module 203, if being also used to target data there is no corresponding first identifier and there are corresponding second identifier,
Then sending module 203 sends the 4th target packet to the second storage equipment, so that the second storage equipment is according to the 4th target
Data packet handles target data；
Wherein, target data and second identifier are at least carried in the 4th target packet.
In the embodiment of the present application, providing target data is to delete processing and data processing Jing Guo compression processing without overweight
Method, second storage equipment is chosen whether to delete target data again processing according to the demand of itself, improve scheme
Realize flexibility.
It is apparent to those skilled in the art that for convenience and simplicity of description, the system of foregoing description,
The specific work process of device and unit, can refer to corresponding processes in the foregoing method embodiment, and details are not described herein.
In several embodiments provided herein, it should be understood that disclosed system, device and method can be with
It realizes by another way.For example, the apparatus embodiments described above are merely exemplary, for example, the unit
It divides, only a kind of logical function partition, there may be another division manner in actual implementation, such as multiple units or components
It can be combined or can be integrated into another system, or some features can be ignored or not executed.Another point, it is shown or
The mutual coupling, direct-coupling or communication connection discussed can be through some interfaces, the indirect coupling of device or unit
It closes or communicates to connect, can be electrical property, mechanical or other forms.
The unit as illustrated by the separation member may or may not be physically separated, aobvious as unit
The component shown may or may not be physical unit, it can and it is in one place, or may be distributed over multiple
In network unit.It can select some or all of unit therein according to the actual needs to realize the mesh of this embodiment scheme
's.
It, can also be in addition, each functional unit in each embodiment of the application can integrate in one processing unit
It is that each unit physically exists alone, can also be integrated in one unit with two or more units.Above-mentioned integrated list
Member both can take the form of hardware realization, can also realize in the form of software functional units.
If the integrated unit is realized in the form of SFU software functional unit and sells or use as independent product
When, it can store in a computer readable storage medium.Based on this understanding, the technical solution of the application is substantially
The all or part of the part that contributes to existing technology or the technical solution can be in the form of software products in other words
It embodies, which is stored in a storage medium, including some instructions are used so that a computer
Equipment (can be personal computer, server or the network equipment etc.) executes the complete of each embodiment the method for the application
Portion or part steps.And storage medium above-mentioned include: USB flash disk, mobile hard disk, read-only memory (read-only memory,
ROM), random access memory (random access memory, RAM), magnetic or disk etc. are various can store program
The medium of code.
The above, above embodiments are only to illustrate the technical solution of the application, rather than its limitations；Although referring to before
Embodiment is stated the application is described in detail, those skilled in the art should understand that: it still can be to preceding
Technical solution documented by each embodiment is stated to modify or equivalent replacement of some of the technical features；And these
It modifies or replaces, the spirit and scope of each embodiment technical solution of the application that it does not separate the essence of the corresponding technical solution.
Claims (10)
1. a kind of method of data processing characterized by comprising
First storage equipment reads target data；
The first storage equipment detects the target data and whether there is corresponding first identifier and corresponding second identifier,
Wherein, the second identifier is for marking the target data by compression processing；
If there are the corresponding first identifier and the corresponding second identifier, first storages for the target data
Equipment sends first object data packet to the second storage equipment, so that the second storage equipment is according to the first object number
The target data is handled according to packet；
Wherein, the target data, the first identifier and second mark are at least carried in the first object data packet
Know, the first identifier is used to indicate the second storage equipment and is deleted again according to the first identifier to the target data
Processing, the second identifier are used to indicate the second storage equipment and carry out write-in processing to the target data.
2. the method according to claim 1, wherein whether the first storage equipment detects the target data
There are after corresponding first identifier and corresponding second identifier, the method also includes:
If the corresponding first identifier and the corresponding second identifier is not present in the target data, described first is deposited
It stores up equipment and sends the second target packet to the second storage equipment, so that the second storage equipment is according to described second
Target packet handles the target data；
Wherein, the target data and third mark are at least carried in second data packet, the third mark is for referring to
Show that the second storage equipment carries out compression processing to the target data.
3. according to the method described in claim 2, it is characterized in that, the first storage equipment read the target data it
Afterwards, the method also includes:
If the target data is stored in the buffer zone of the first storage equipment, the first storage equipment is to described the
Two storage equipment send second target packet, so that the second storage equipment is according to second target packet
The target data is handled；
Wherein, the target data and third mark are at least carried in second data packet.
4. the method according to claim 1, wherein whether the first storage equipment detects the target data
There are after corresponding first identifier and corresponding second identifier, the method also includes:
If the target data is there are the corresponding first identifier and the corresponding second identifier is not present, described first
It stores equipment and sends third target packet to the second storage equipment, so that the second storage equipment is according to described the
Three target packets handle the target data；
Wherein, the target data, the first identifier and the third mark are at least carried in the third target packet
Know.
5. the method according to claim 1, wherein whether the first storage equipment detects the target data
There are after corresponding first identifier and corresponding second identifier, the method also includes:
If the target data is there is no the corresponding first identifier and there are the corresponding second identifier, described first
It stores equipment and sends the 4th target packet to the second storage equipment, so that the second storage equipment is according to described the
Four target packets handle the target data；
Wherein, the target data and the second identifier are at least carried in the 4th target packet.
6. a kind of data processing equipment characterized by comprising
Read module, for reading target data；
Detection module whether there is corresponding first identifier and corresponding second identifier for detecting the target data,
In, the second identifier is for marking the target data by compression processing；
Sending module, if for the target data there are the corresponding first identifier and the corresponding second identifier,
Then the sending module sends first object data packet to the second storage equipment, so that the second storage equipment is according to
First object data packet handles the target data；
Wherein, the target data, the first identifier and second mark are at least carried in the first object data packet
Know, the first identifier is used to indicate the second storage equipment and is deleted again according to the first identifier to the target data
Processing, the second identifier are used to indicate the second storage equipment and carry out write-in processing to the target data.
7. data processing equipment according to claim 6, which is characterized in that
The sending module, if being also used to the target data, there is no the corresponding first identifier and corresponding described the
Two marks, then the sending module sends the second target packet to the second storage equipment, so that second storage
Equipment is handled the target data according to second target packet；
Wherein, the target data and third mark are at least carried in second data packet, the third mark is for referring to
Show that the second storage equipment carries out compression processing to the target data.
8. data processing equipment according to claim 7, which is characterized in that
The sending module, it is described if being also used to the buffer zone that the target data is stored in the first storage equipment
Sending module sends second target packet to the second storage equipment, so that the second storage equipment is according to institute
The second target packet is stated to handle the target data；
Wherein, the target data and third mark are at least carried in second data packet.
9. data processing equipment according to claim 6, which is characterized in that
The sending module, if being also used to the target data there are the corresponding first identifier and being not present corresponding described
Second identifier, then the sending module sends third target packet to the second storage equipment, so that described second deposits
Storage equipment is handled the target data according to the third target packet；
Wherein, the target data, the first identifier and the third mark are at least carried in the third target packet
Know.
10. data processing equipment according to claim 6, which is characterized in that
The sending module, if being also used to, the corresponding first identifier is not present in the target data and there are corresponding described
Second identifier, then the sending module sends the 4th target packet to the second storage equipment, so that described second deposits
Storage equipment is handled the target data according to the 4th target packet；
Wherein, the target data and the second identifier are at least carried in the 4th target packet.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811108304.4A CN109086172B (en) | 2018-09-21 | 2018-09-21 | Data processing method and related device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811108304.4A CN109086172B (en) | 2018-09-21 | 2018-09-21 | Data processing method and related device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109086172A true CN109086172A (en) | 2018-12-25 |
| CN109086172B CN109086172B (en) | 2022-12-06 |
Family
ID=64842307
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811108304.4A Active CN109086172B (en) | 2018-09-21 | 2018-09-21 | Data processing method and related device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109086172B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107025289A (en) * | 2017-04-14 | 2017-08-08 | 腾讯科技（深圳）有限公司 | The method and relevant device of a kind of data processing |
| CN107179878A (en) * | 2016-03-11 | 2017-09-19 | 伊姆西公司 | The method and apparatus of data storage based on optimizing application |
| CN107193503A (en) * | 2017-05-27 | 2017-09-22 | 杭州宏杉科技股份有限公司 | A kind of data delete method and storage device again |
| CN107229420A (en) * | 2017-05-27 | 2017-10-03 | 郑州云海信息技术有限公司 | Date storage method, read method, delet method and data operation system |
| WO2018121455A1 (en) * | 2016-12-29 | 2018-07-05 | 华为技术有限公司 | Cached-data processing method and device, and storage controller |
| CN108268219A (en) * | 2018-02-01 | 2018-07-10 | 杭州宏杉科技股份有限公司 | A kind of method and device for handling I/O request |
-
2018
- 2018-09-21 CN CN201811108304.4A patent/CN109086172B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107179878A (en) * | 2016-03-11 | 2017-09-19 | 伊姆西公司 | The method and apparatus of data storage based on optimizing application |
| WO2018121455A1 (en) * | 2016-12-29 | 2018-07-05 | 华为技术有限公司 | Cached-data processing method and device, and storage controller |
| CN107025289A (en) * | 2017-04-14 | 2017-08-08 | 腾讯科技（深圳）有限公司 | The method and relevant device of a kind of data processing |
| CN107193503A (en) * | 2017-05-27 | 2017-09-22 | 杭州宏杉科技股份有限公司 | A kind of data delete method and storage device again |
| CN107229420A (en) * | 2017-05-27 | 2017-10-03 | 郑州云海信息技术有限公司 | Date storage method, read method, delet method and data operation system |
| CN108268219A (en) * | 2018-02-01 | 2018-07-10 | 杭州宏杉科技股份有限公司 | A kind of method and device for handling I/O request |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109086172B (en) | 2022-12-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20200150890A1 (en) | Data Deduplication Method and Apparatus | |
| US6397309B2 (en) | System and method for reconstructing data associated with protected storage volume stored in multiple modules of back-up mass data storage facility | |
| CN105339929B (en) | Select the storage for cancelling repeated data | |
| CN106201771B (en) | Data-storage system and data read-write method | |
| US7516286B1 (en) | Conversion between full-data and space-saving snapshots | |
| CN109327539A (en) | A kind of distributed block storage system and its data routing method | |
| US8458145B2 (en) | System and method of storage optimization | |
| US20120323864A1 (en) | Distributed de-duplication system and processing method thereof | |
| US8745744B2 (en) | Storage system and storage system management method | |
| CN106407040A (en) | Remote data copy method and system | |
| CN107038092B (en) | Data copying method and device | |
| CN106407224B (en) | The method and apparatus of file compacting in a kind of key assignments storage system | |
| CN103959256A (en) | Fingerprint-based data deduplication | |
| EP3862883A1 (en) | Data backup method and apparatus, and system | |
| US20100070724A1 (en) | Storage system and method for operating storage system | |
| CN107451013B (en) | Data recovery method, device and system based on distributed system | |
| US10572335B2 (en) | Metadata recovery method and apparatus | |
| CN107193503B (en) | Data deduplication method and storage device | |
| US20110282843A1 (en) | Method and system for data backup and replication | |
| CN109582245A (en) | Data processing method, device and equipment | |
| CN107203331A (en) | Write the method and device of data | |
| CN105824846A (en) | Data migration method and device | |
| CN104935469A (en) | Distributive storage method and system for log information | |
| CN104520802A (en) | Data sending method, data receiving method and storage device | |
| CN109753381B (en) | Continuous data protection method based on object storage |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |