CN103049390A - Applied metadata processing method and storing system - Google Patents
Applied metadata processing method and storing system Download PDFInfo
- Publication number
- CN103049390A CN103049390A CN2012105412314A CN201210541231A CN103049390A CN 103049390 A CN103049390 A CN 103049390A CN 2012105412314 A CN2012105412314 A CN 2012105412314A CN 201210541231 A CN201210541231 A CN 201210541231A CN 103049390 A CN103049390 A CN 103049390A
- Authority
- CN
- China
- Prior art keywords
- physical address
- mapping relations
- logical address
- address
- map section
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/06—Addressing a physical block of locations, e.g. base addressing, module addressing, memory dedication
Abstract
The invention relates to an applied metadata processing method and a storing system. Applied metadata comprise a mapping area and an indexing area, the mapping area is used for storing a mapping relation of a logic address and a physical address, and the indexing area is used for inquiring the position of the mapping relation of the logic address and the physical address in the mapping area. The method includes that the storage system receives a read or write request of data, and the read or write request contains the logic address; whether the mapping relation of the logic address and the physical address is preserved in the mapping area is inquired through indexing information in the indexing area according to the logic address; and if the mapping relation is preserved in the mapping area, the data is subjected to read or write processing on the physical address corresponding to the mapping relation.
Description
Technical field
The present invention relates to field of computer technology, relate in particular to a kind of data processing method and storage system of apply metadata.
Background technology
Along with the development of information science and technology, need the data volume of storage with incredible speed rapid growth, therefore Rational Classification and the utilization of storage system resource are also had higher requirement.On the one hand, storage system need to satisfy the demand of current business, just can manage the storage of mass data by simple configuration; On the other hand, in the process that business not very increases, storage system can expand, and the storage resources of distributing according to need along with the growth of business.Thus, thin provisioning arises at the historic moment.
Thin provisioning is directly to develop and next a kind of storage virtualization technology from traditional allocation of space mode, this technology is pooled to all storage spaces to simplify carries out unified management in the pond, create and to simplify logical block simplifying the pond, obtain amount of physical memory from simplifying the pond as required.The use of thin provisioning can be simplified the configuration management of storage resources, saves the physical store resource.
In thin provisioning, metadata is mainly used in the mapping relations of store logical addresses and physical address.The reliability of metadata is technology most crucial in the thin provisioning.And in the existing data processing method, the storage system utilization has the metadata of B tree or B tree variant (such as the B+ tree) structure data is read and write processing.In the node of B tree metadata, can store logical address and physical address map relation, utilize the access efficiency that the B tree construction can the Effective Raise metadata, have good extendability.But, in case a node of B tree metadata damages, can lose logical address and the physical address map relation of this node storage, can affect the lower level node that storage system is searched this damage node simultaneously, cause storage system on the logical address of the affected part storage of the metadata physical address corresponding with the physical address map relation, to process data.
Summary of the invention
In view of this, the embodiment of the invention provides a kind of data processing method and storage system of apply metadata, to realize when metadata is partly damaged, can not affect storage system and on the logical address of other part storages of the metadata physical address corresponding with the physical address map relation, data be processed.
In first aspect, the embodiment of the invention provides a kind of data processing method of apply metadata.Wherein, described metadata comprises: be used for the map section of store logical addresses and physical address map relation and be used for the query logic address and the physical address map pass ties up to the index area of the position of described map section; Described method comprises:
The request that reads or writes of storage system receive data, described reading or writing carried logical address in the request;
Inquire about the mapping relations of whether preserving in the described map section between described logical address and the physical address by the index information of described index area according to described logical address;
If preserve described mapping relations in the described map section, on physical address corresponding to described mapping relations, described data are read or write processing.
In the possible implementation of the first of first aspect, described map section comprises a plurality of zones, and each zone is reserved one section space and is used for store logical addresses and physical address map relation.
In conjunction with the possible implementation of the first of first aspect, in the possible implementation of the second, described method also comprises: if do not preserve described mapping relations in the described map section, and described reading or writing asks to be write request, the logical address allocated physical address of then carrying for described write request; On the physical address that distributes, described data are write processing; According to the physical address of described distribution, the described mapping relations that described write request is corresponding are stored in the map section; According to the logical address that described write request is carried, the position of in described index area, recording the mapping relations between the physical address of preserving described logical address and distribution in the map section.
In conjunction with the possible implementation of the first of first aspect, in the third possible implementation, described method also comprises: described inquire about the mapping relations of whether preserving described logical address and physical address in the described map section according to described logical address by the index information of described index area in, metadata is detected; Damage if preserve the zone of described mapping relations in the map section, and described reading or writing asks to be write request that the logical address of then carrying for described write request is redistributed physical address; On the physical address of redistributing, described data are write processing; According to the described physical address of redistributing, the described mapping relations that described write request is corresponding are stored in the map section; The logical address of carrying according to described write request, in described index area record, preserve described logical address in the map section and the physical address redistributed between the position of mapping relations.
The third possible implementation in conjunction with first aspect, in the 4th kind of possible implementation, described method also comprises: if described index area is local or all damages, recover described index area according to the logical address in described mapping relations of each zone preservation in the map section.
In second aspect, the embodiment of the invention provides a kind of storage system of apply metadata.Wherein, described metadata comprises: be used for the map section of store logical addresses and physical address map relation and be used for the query logic address and the physical address map pass ties up to the index area of the position of described map section; Described storage system comprises:
Receiving element is used for the request that reads or writes of receive data, and described reading or writing carried logical address in the request;
Query unit is used for inquiring about described map section according to described logical address by the index information of described index area and whether preserves mapping relations between described logical address and the physical address;
Processing unit is preserved described mapping relations if be used for described map section, on physical address corresponding to described mapping relations described data is read or write processing.
In the possible implementation of the first of second aspect, described map section comprises a plurality of zones, and each zone is reserved one section space and is used for store logical addresses and physical address map relation.
In conjunction with the possible implementation of the first of first aspect, in the possible implementation of the second, described storage system also comprises: allocation units, do not preserve described mapping relations if be used for described map section, and described reading or writing asks to be write request, the logical address allocated physical address of then carrying for described write request; Described processing unit also is used on the physical address that distributes described data being write processing;
Record cell, be used for the physical address according to described distribution, the described mapping relations that described write request is corresponding are stored in the map section, according to the logical address that described write request is carried, the position of in described index area, recording the mapping relations between the physical address of preserving described logical address and distribution in the map section.
In conjunction with the possible implementation of the first of second aspect, in the third possible implementation, described storage system also comprises: detecting unit, be used for when described query unit is inquired about described map section according to described logical address by the index information of described index area and whether preserved the mapping relations of described logical address and physical address, metadata being detected; Allocation units, if be used for the zone damage that map section is preserved described mapping relations, and described reading or writing asks to be write request that the logical address of then carrying for described write request is redistributed physical address; Described processing unit also is used on the physical address of redistributing described data being write processing; Record cell, be used for according to the described physical address of redistributing, the described mapping relations that described write request is corresponding are stored in the map section, according to the logical address that described write request is carried, the position of in described index area, recording the mapping relations between the physical address of preserving described logical address in the map section and redistributing.
The third possible implementation in conjunction with second aspect, in the 4th kind of possible implementation, described storage system also comprises: recovery unit, if be used for the local or all damages in described index area, recover described index area according to the logical address in each regional described mapping relations of preserving in the map section.
Pass through such scheme, because index area and the map section of metadata are independently, when storage system is searched the position of the logical address corresponding to the request that reads or writes of save data in the map section and physical address map relation by the index area, if the part of map section is damaged, only can lose logical address and the physical address map relation of the storage of broken parts, can not affect storage system and search logical address and the physical address map relation of the storage of map section other parts by the index area, on the logical address of the not broken parts preservation physical address corresponding with the physical address map relation, data not read or not write processing thereby can not affect storage system.
Description of drawings
Fig. 1 is the metadata structure synoptic diagram that the embodiment of the invention is used;
The process flow diagram of the data processing method of a kind of apply metadata that Fig. 2 provides for the embodiment of the invention one;
The structural representation of the storage system of a kind of apply metadata that Fig. 3 provides for the embodiment of the invention two;
The structural representation of the storage system of the another kind of apply metadata that Fig. 4 provides for the embodiment of the invention two;
The structural representation of a kind of storage system that Fig. 5 provides for the embodiment of the invention three.
Embodiment
In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention is described in further detail below in conjunction with accompanying drawing, and obviously, described embodiment only is a part of embodiment of the present invention, rather than whole embodiment.Based on the embodiment among the present invention, those of ordinary skills belong to the scope of protection of the invention not making all other embodiment that obtain under the creative work prerequisite.
At first, introduce the metadata structure that the embodiment of the invention is used as an example of Fig. 1 example.Fig. 1 is the metadata structure synoptic diagram that the embodiment of the invention is used.This metadata comprises map section (Mapping Partitn, MP is hereinafter to be referred as the MP district) and index area (Index Partitn, IP is hereinafter to be referred as the IP district).Wherein, the MP district is used for store logical addresses and physical address map relation (Logic address and Physicaladdress Mapping, LPM).The MP district comprises a plurality of zones.Concrete, the every 1G physical space on physical disk is reserved one section zone and is preserved LPM, as shown in Figure 1, every 1G is a zone, black part is divided into the LPM of storage in each zone, and each black square represents a LPM, and white portion is the data on the physical address that LPM is corresponding in this zone of storage.Need to prove that the present invention does not limit the size in each zone, MP district, the size in each zone, MP district is that 1G only is a kind of situation of the embodiment of the invention, but and MP district infinite expanding.The IP district is used for inquiring about LPM corresponding to this logical address in the position in MP district according to logical address.The IP district can be the B tree construction, but is not limited only to the B tree construction, and the IP district can be other structures that can realize the straw line function.
As seen from Figure 1, the IP district of this metadata and MP district are two independently zones, when the subregion in MP district is damaged, can not affect the IP district and search the not LPM of failure area of MP district.And, when the IP district is damaged, also can recover according to the MP district.Therefore, in storage system, use this metadata, when the part because of the MP district is damaged when causing part LPM to lose, can not affect storage system and utilize the IP district to search the LPM of metadata MP district other parts storage, not process in the enterprising row data read-write of physical address corresponding to the LPM of MP district other parts storage thereby can not affect storage system.When the IP district is damaged, can not affect whole storage system to the processing of data.
The below describes the data processing method of a kind of apply metadata that the embodiment of the invention one provides in detail as an example of Fig. 2 example, the process flow diagram of the data processing method of a kind of apply metadata that Fig. 2 provides for the embodiment of the invention one.The executive agent of the data processing method of this apply metadata is storage system, uses above-mentioned metadata in this storage system.As shown in Figure 2, the data processing method of this apply metadata may further comprise the steps:
Step S201, the request that reads or writes of storage system receive data.
The read request of each data is carried identical logical address with write request, storage system receive data read or write request after, can search corresponding LPM according to the logical address that the request that reads or writes is carried, thereby find physical address corresponding to logical address, on physical address, these data are carried out read or write.
Whether step S202 preserves reading or writing of these data and asks the logical address of carrying and the LPM of physical address in the index information inquiry MP district of the request that the reads or writes logical address of carrying by IP district according to data.
Whether storage system at first inquires the zone that reads or writes MP district corresponding to request of these data according to the request that the reads or writes logical address of carrying of data by the index information in IP district, then judge to preserve in this zone with reading or writing of these data and ask the relevant LPM of the logical address of carrying.If inquire the relevant LPM of logical address that carries with the request that reads or writes of these data, LPM corresponding to the request that reads or writes that preserves these data in the MP district then be described, execution in step S203.If do not inquire the relevant LPM of logical address that carries with the request that reads or writes of these data, LPM corresponding to the request that reads or writes that does not preserve these data in the MP district then be described, execution in step S204.
For example, the data structure in IP district can be Hash Hash table, B tree and the various structures such as variant, ordering chained list thereof.The below describes the method for searching physical address according to logical address as an example of the Hash table example, the Key value is logical address in this Hash table, and the Value value is that LPM is in the address in MP district.This key value of at first preserving in the Hash table according to logical address is inquired about this with the corresponding relation of Value value and is read or write LPM corresponding to request in the address in MP district, then LPM is read in the address in MP district and can obtain LPM corresponding to the request of reading or writing, thereby obtain corresponding physical address according to LPM.
Step S203 if preserve the LPM corresponding to the request that reads or writes of these data in the MP district, reads or writes processing to these data on physical address corresponding to this LPM.
If preserve the LPM corresponding to the request that reads or writes of these data in the MP district, then know the corresponding physical address of logical address that request is carried that reads or writes of these data according to LPM, then on this physical address, these data are read or write processing.
Step S204, if preserve the LPM corresponding to the request that reads or writes of these data in the MP district, and the request that reads or writes of these data is the write request of these data, the logical address allocated physical address of carrying for the write request of these data.
In addition, do not read or write LPM corresponding to request if preserve these data in the MP district, and the request that reads or writes of these data is the read request of these data, then returns the full 0 data, not preserve corresponding data in this storage system of equipment of informing reading out data.
Step S205 writes processing to these data on the physical address that distributes.
Storage system just can write this data at this physical address after the logical address allocated physical address of carrying for the write request of these data.After this physical address is finished data writing, after guaranteeing, can find this physical address by metadata, the data that write on this physical address are processed, need the LPM that the write request of these data is corresponding to be recorded in the metadata, so storage system is also needed to carry out following steps.
Step S206, according to the physical address that distributes, the LPM that the write request of these data is corresponding is stored in the MP district.
The physical address of the logical address of carrying in the write request and distribution is formed mapping relations LPM, LPM is kept at concrete which position in MP district, this patent does not limit, for example MP can be distinguished into the continuous physical space that 1G is unit, every 1G continuous space is designated as a zone, and each regional top fixed area is used for preserving LPM corresponding to all physical addresss of this zone.
Storage system at first determines to preserve the zone of this LPM in the MP district according to the physical address that distributes, then allocation space is preserved this LPM in this zone.
Step S207, the logical address of carrying according to the write request of these data, the position of LPM corresponding to the write request of these data of preservation in the record MP district in the IP district.
Table is as example take the data structure in IP district as Hash, and the Key value is logical address in this Hash table, and the Value value is that LPM is in the address in MP district.This step namely is to increase a record in the Hash table.So that storage system receives reading or writing when asking of these data again, inquire about the address that read or write LPM corresponding to request of in MP preserving this data by the Key value in the Hash table in IP district with the corresponding relation of Value value according to the logical address that the request that reads or writes is carried, thereby on the corresponding physical address of LPM corresponding to the request that reads or writes of these data, these data are read or write processing.
In the disposal route of the apply metadata that present embodiment provides, except above-mentioned storage system reads or writes data the method for processing, comprise also when metadata is damaged that storage system is to the disposal route of data and the method for recovery metadata.
Concrete, step S202, whether preserve in the index information inquiry MP district of logical address by IP district of carrying according to the request that reads or writes of data these data read or write the LPM that asks correspondence the time, storage system detects metadata.If metadata is not damaged, then carry out above-mentioned steps S203-S207.
If preserving the zone of the LPM corresponding to the request that reads or writes of these data in the MP district of metadata damages, and the request that reads or writes of these data is the write request of data, the logical address of then again carrying for the write request of these data is redistributed physical address, on the physical address of redistributing, these data are write processing, when guaranteeing to be damaged in subregion, MP district, these data can normally write.
For so that afterwards storage system can find the physical address that this is redistributed by metadata, the data that write on this physical address are processed, need the LPM that the write request of these data is corresponding to be recorded in the metadata, recording method does not repeat them here with step S206-S207.
In addition, damage if preserve the zone of the LPM corresponding to the request that reads or writes of these data in the MP district of metadata, and the request that reads or writes of these data is the read request of data, then returns error message to the equipment that reads these data.
If the IP district of metadata is damaged, then recover the IP district according to logical address among the LPM of each zone preservation in the MP district.Concrete, storage system is accessed the space that is used for storage LPM in the every 1G of the physical disk space successively, reads LPM.Because comprise the logical address that request is carried that reads or writes of data among the LPM, then storage system is rebuild the IP district according to the LPM that reads, and is concrete, can adopt the various structures of Hash Hash table, B tree and variant, ordering chained list etc. to rebuild the IP district according to logical address.After finishing recovery, the IP district can normally carry out index.
Utilize the disposal route of the apply metadata that the embodiment of the invention one provides, storage system receive data read or write request after, utilize the IP district of metadata to search the LPM of storage in the MP district, because IP district and MP district are independently, therefore, when damaging because of MP district part when causing partial data to be lost, can not affect storage system and utilize the IP district of metadata to search the not LPM of broken parts preservation of MP district, thereby can not affect storage system missing data is not is not read or write processing.And, when the IP district of metadata is damaged, can recover according to the LPM that preserves in the MP district, can not affect storage system and utilize IP district to inquire about LPM in the MP district, then data are read or write processing.
Accordingly, the embodiment of the invention two provides a kind of in order to realize the storage system of embodiment one described method.As shown in Figure 3, the structural representation of the storage system of its a kind of apply metadata that provides for the embodiment of the invention two.The metadata of using in this storage system comprises for the MP district of storage LPM and is used for inquiry LPM in the IP district of the position in described MP district.Wherein, the MP district comprises a plurality of zones, and each zone is reserved one section space and is used for storage LPM.This storage system comprises: receiving element 310, query unit 320 and processing unit 330.
Receiving element 310 is used for the request that reads or writes of receive data.Reading or writing of same data carries identical logical address in the request.
The logical address that query unit 320 is at first carried according to the request that reads or writes of data utilizes the IP district to inquire the zone that reads or writes MP district corresponding to request of these data, then judges whether preserve the relevant LPM of logical address that carries with the request that reads or writes of these data in this zone.If query unit 320 inquires the relevant LPM of logical address that carries with the request that reads or writes of these data, LPM corresponding to the request that reads or writes that preserves these data in the MP district is described then.If query unit 320 does not inquire the relevant LPM of logical address that carries with the request that reads or writes of these data, LPM corresponding to the request that reads or writes that does not preserve these data in the MP district is described then.
If processing unit 330 is used for LPM corresponding to the request that reads or writes that the MP district preserves these data, on the corresponding physical address of LPM corresponding to the request that reads or writes of these data, these data are read or write processing.
In addition, this storage system also comprises: allocation units 340 and record cell 350.As shown in Figure 4, the structural representation of the storage system of its another kind of apply metadata that provides for the embodiment of the invention two.
If allocation units 340 are used for LPM corresponding to the request that reads or writes that MP does not preserve these data, and the request that reads or writes of these data is the write request of these data, the logical address allocated physical address of carrying for the write request of these data.Then processing unit 330 also is used on the physical address that allocation units 340 distribute these data being write processing.
The physical address that record cell 350 is used for according to allocation units 340 distribution, the LPM that the write request of these data is corresponding is stored in the MP district, the logical address of carrying according to the write request of these data, the position of LPM corresponding to the write request of these data of preservation in record MP district, described IP district.After record was finished, query unit 320 just can find the physical address of this distribution by metadata according to the request that the reads or writes logical address of carrying of these data, so that the data that write on 330 pairs of these physical addresss of processing unit are processed.
This storage system can also comprise: detecting unit 360 and recovery unit 370.
Detecting unit 360 is used for the logical address that query unit 320 is carried according to the request that reads or writes of data is inquired about LPM corresponding to the request that reads or writes that whether preserves these data in the MP district by the index information in IP district when, metadata is detected, so that when metadata is damaged, data are done the metadata of corresponding processing and Recover from damaging.If then allocation units 340 zone that also is used for preserving the LPM corresponding to the request that reads or writes of these data in MP district is damaged, and reading or writing of these data ask to be the write request of these data, for the logical address that the write request of these data is carried is redistributed physical address.Accordingly, processing unit 330 also is used on the physical address of redistributing these data being write processing.And record cell 350 also is used for according to the physical address of redistributing, the LPM that the write request of these data is corresponding is stored in the MP district, the logical address of then carrying according to the write request that changes data, the position of LPM corresponding to the write request of these data of preservation in record MP district, IP district.
If recovery unit 370 is used for the local or all damages in IP district, the LPM that preserves according to each zone in the MP district recovers the IP district.Concrete, recovery unit 370 is accessed the space that is used for storage LPM in the every 1G of the physical disk space successively, reads LPM.Owing to comprise the logical address that request is carried that reads or writes of data among the LPM, so recovery unit 370 is rebuild the IP district according to the logical address among the LPM that reads.After finishing recovery, the IP district can normally carry out index.
Need to prove that the included unit of the storage system of a kind of apply metadata that present embodiment provides is just divided according to function logic, but is not limited to above-mentioned division, as long as can realize corresponding function; In addition, the concrete title of each functional unit also just for the ease of mutual differentiation, is not limited to protection scope of the present invention.
Utilize the storage system of the apply metadata that the embodiment of the invention two provides, storage system receive data read or write request after, utilize the IP district of metadata to search the LPM of storage in the MP district, because IP district and MP district are independently, therefore, when damaging because of MP district part when causing partial data to be lost, can not affect storage system and utilize the IP district of metadata to search the not LPM of broken parts preservation of MP district, thereby can not affect storage system missing data is not is not read or write processing.And, when the IP district of metadata is damaged, can recover according to the LPM that preserves in the MP district, can not affect storage system and utilize IP district to inquire about LPM in the MP district, then data are read or write processing.
On hardware was realized, above receiving element 310 can be receiver or transceiver.More than other unit except receiving element 310 can be embedded in or is independent of in the processor of storage system with example, in hardware, also can be stored in the storer of storage system with software form, carry out operation corresponding to above modules so that processor calls.This processor can be CPU (central processing unit) (CPU), microprocessor, single-chip microcomputer etc.
As shown in Figure 5, the structural representation of its a kind of storage system that provides for the embodiment of the invention three.This storage system comprises transceiver 510, storer 520 and is connected the processor 530 that is connected with storer with transceiver 510 respectively.Certainly, storage system can also comprise the universal components such as antenna, Base-Band Processing parts, middle radio frequency processing parts, input-output unit, and the embodiment of the invention is in this no longer any restriction.
Wherein, storage batch processing code in the storer 520, and processor 530 is used for carrying out following the operation for the program code that calls storer 520 storages:
The request that reads or writes of receive data, described reading or writing carried logical address in the request;
Inquire about the mapping relations of whether preserving in the described map section between described logical address and the physical address by the index information of described index area according to described logical address;
If preserve described mapping relations in the described map section, on physical address corresponding to described mapping relations, described data are read or write processing.
Further, storage batch processing code in the storer 520, and processor 530 can also be used for carrying out following the operation for the program code that calls storer 520 storages:
If do not preserve described mapping relations in the described map section, and described reading or writing asks to be write request, the logical address allocated physical address of then carrying for described write request;
On the physical address that distributes, described data are write processing;
According to the physical address of described distribution, the described mapping relations that described write request is corresponding are stored in the map section;
According to the logical address that described write request is carried, the position of in described index area, recording the mapping relations between the physical address of preserving described logical address and distribution in the map section.
Further, storage batch processing code in the storer 520, and processor 530 can also be used for carrying out following the operation for the program code that calls storer 520 storages:
Described inquire about the mapping relations of whether preserving described logical address and physical address in the described map section according to described logical address by the index information of described index area in, metadata is detected;
Damage if preserve the zone of described mapping relations in the map section, and described reading or writing asks to be write request that the logical address of then carrying for described write request is redistributed physical address;
On the physical address of redistributing, described data are write processing;
According to the described physical address of redistributing, the described mapping relations that described write request is corresponding are stored in the map section;
According to the logical address that described write request is carried, the position of in described index area, recording the mapping relations between the physical address of preserving described logical address in the map section and redistributing.
Further, storage batch processing code in the storer 520, and processor 530 can also be used for carrying out following the operation for the program code that calls storer 520 storages:
If described index area is local or all damages, recover described index area according to the logical address in described mapping relations of each zone preservation in the map section.
Need to prove that the storage system that provides among embodiment two and the embodiment three all can be used for realize the method that embodiment one provides, and about the description of metadata etc. with the description among the embodiment two, do not repeat them here.
The storage system of utilizing the embodiment of the invention three to provide, storage system receive data read or write request after, utilize the IP district of metadata to search the LPM of storage in the MP district, because IP district and MP district are independently, therefore, when damaging because of MP district part when causing partial data to be lost, can not affect storage system and utilize the IP district of metadata to search the not LPM of broken parts preservation of MP district, thereby can not affect storage system missing data is not is not read or write processing.And, when the IP district of metadata is damaged, can recover according to the LPM that preserves in the MP district, can not affect storage system and utilize IP district to inquire about LPM in the MP district, then data are read or write processing.
The professional should further recognize, unit and the algorithm steps of each example of describing in conjunction with embodiment disclosed herein, can realize with electronic hardware, computer software or the combination of the two, for the interchangeability of hardware and software clearly is described, composition and the step of each example described in general manner according to function in the above description.These functions are carried out with hardware or software mode actually, depend on application-specific and the design constraint of technical scheme.The professional and technical personnel can specifically should be used for realizing described function with distinct methods to each, but this realization should not thought and exceeds scope of the present invention.
The method of describing in conjunction with embodiment disclosed herein or the step of algorithm can use the software module of hardware, processor execution, and perhaps the combination of the two is implemented.Software module can place the storage medium of any other form known in random access memory (RAM), internal memory, ROM (read-only memory) (ROM), electrically programmable ROM, electrically erasable ROM, register, hard disk, moveable magnetic disc, CD-ROM or the technical field.
Above-described embodiment; purpose of the present invention, technical scheme and beneficial effect are further described; institute is understood that; the above only is the specific embodiment of the present invention; the protection domain that is not intended to limit the present invention; within the spirit and principles in the present invention all, any modification of making, be equal to replacement, improvement etc., all should be included within protection scope of the present invention.
Claims (10)
1. the data processing method of an apply metadata, it is characterized in that described metadata comprises: be used for the map section of store logical addresses and physical address map relation and be used for the query logic address and the physical address map pass ties up to the index area of the position of described map section; Described method comprises:
The request that reads or writes of storage system receive data, described reading or writing carried logical address in the request;
Inquire about the mapping relations of whether preserving in the described map section between described logical address and the physical address by the index information of described index area according to described logical address;
If preserve described mapping relations in the described map section, on physical address corresponding to described mapping relations, described data are read or write processing.
2. method according to claim 1 is characterized in that, described map section comprises a plurality of zones, and each zone is reserved one section space and is used for storing described mapping relations.
3. method according to claim 2 is characterized in that, described method also comprises:
If do not preserve described mapping relations in the described map section, and described reading or writing asks to be write request, the logical address allocated physical address of then carrying for described write request;
On the physical address that distributes, described data are write processing;
According to the physical address of described distribution, the described mapping relations that described write request is corresponding are stored in the map section;
According to the logical address that described write request is carried, the position of in described index area, recording the mapping relations between the physical address of preserving described logical address and distribution in the map section.
4. method according to claim 2 is characterized in that, described method also comprises:
Described inquire about the mapping relations of whether preserving described logical address and physical address in the described map section according to described logical address by the index information of described index area in, metadata is detected;
Damage if preserve the zone of described mapping relations in the map section, and described reading or writing asks to be write request that the logical address of then carrying for described write request is redistributed physical address;
On the physical address of redistributing, described data are write processing;
According to the described physical address of redistributing, the described mapping relations that described write request is corresponding are stored in the map section;
According to the logical address that described write request is carried, the position of in described index area, recording the mapping relations between the physical address of preserving described logical address in the map section and redistributing.
5. method according to claim 4 is characterized in that, described method also comprises:
If described index area is local or all damages, recover described index area according to the logical address in described mapping relations of each zone preservation in the map section.
6. the storage system of an apply metadata, it is characterized in that described metadata comprises: be used for the map section of store logical addresses and physical address map relation and be used for the query logic address and the physical address map pass ties up to the index area of the position of described map section; Described storage system comprises:
Receiving element is used for the request that reads or writes of receive data, and described reading or writing carried logical address in the request;
Query unit is used for inquiring about described map section according to described logical address by the index information of described index area and whether preserves mapping relations between described logical address and the physical address;
Processing unit is preserved described mapping relations if be used for described map section, on physical address corresponding to described mapping relations described data is read or write processing.
7. storage system according to claim 6 is characterized in that, described map section comprises a plurality of zones, and each zone is reserved one section space and is used for storing described mapping relations.
8. storage system according to claim 7 is characterized in that, described storage system also comprises:
Allocation units are not preserved described mapping relations if be used for described map section, and described reading or writing asks to be write request, the logical address allocated physical address of then carrying for described write request;
Described processing unit also is used on the physical address that distributes described data being write processing;
Record cell, be used for the physical address according to described distribution, the described mapping relations that described write request is corresponding are stored in the map section, the logical address of carrying according to described write request, in described index area, preserve in the record map section described logical address and distribution physical address between the position of mapping relations.
9. storage system according to claim 7 is characterized in that, described storage system also comprises:
Detecting unit is used for when described query unit is inquired about described map section according to described logical address by the index information of described index area and whether preserved the mapping relations of described logical address and physical address metadata being detected;
Allocation units, if be used for the zone damage that map section is preserved described mapping relations, and described reading or writing asks to be write request that the logical address of then carrying for described write request is redistributed physical address;
Described processing unit also is used on the physical address of redistributing described data being write processing;
Record cell, be used for according to the described physical address of redistributing, the described mapping relations that described write request is corresponding are stored in the map section, according to the logical address that described write request is carried, the position of in described index area, recording the mapping relations between the physical address of preserving described logical address in the map section and redistributing.
10. storage system according to claim 9 is characterized in that, described storage system also comprises:
Recovery unit if be used for the local or all damages in described index area, recovers described index area according to the logical address in each regional described mapping relations of preserving in the map section.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210541231.4A CN103049390B (en) | 2012-12-14 | 2012-12-14 | The data processing method of apply metadata and storage system |
| PCT/CN2013/080973 WO2014089981A1 (en) | 2012-12-14 | 2013-08-07 | Data processing method and storing system for applied metadata |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210541231.4A CN103049390B (en) | 2012-12-14 | 2012-12-14 | The data processing method of apply metadata and storage system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103049390A true CN103049390A (en) | 2013-04-17 |
| CN103049390B CN103049390B (en) | 2016-03-09 |
Family
ID=48062037
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210541231.4A Active CN103049390B (en) | 2012-12-14 | 2012-12-14 | The data processing method of apply metadata and storage system |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN103049390B (en) |
| WO (1) | WO2014089981A1 (en) |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2014089981A1 (en) * | 2012-12-14 | 2014-06-19 | 华为技术有限公司 | Data processing method and storing system for applied metadata |
| WO2016026433A1 (en) * | 2014-08-19 | 2016-02-25 | 华为技术有限公司 | Data processing method and data processing apparatus |
| WO2016045316A1 (en) * | 2014-09-22 | 2016-03-31 | 中兴通讯股份有限公司 | Method and apparatus for controlling soft-reset data recovery, and packet transport network device |
| CN106201923A (en) * | 2016-07-20 | 2016-12-07 | 杭州宏杉科技有限公司 | Method for reading and writing data and device |
| CN107329909A (en) * | 2017-06-27 | 2017-11-07 | 郑州云海信息技术有限公司 | A kind of data managing method and device |
| CN107480010A (en) * | 2017-08-21 | 2017-12-15 | 郑州云海信息技术有限公司 | A kind of method and device for recovering metadata |
| CN107632944A (en) * | 2017-09-22 | 2018-01-26 | 郑州云海信息技术有限公司 | A kind of method and device for reading data |
| CN107704208A (en) * | 2017-10-13 | 2018-02-16 | 郑州云海信息技术有限公司 | A kind of restorative procedure of metadata, device and medium |
| CN107885456A (en) * | 2016-09-29 | 2018-04-06 | 北京忆恒创源科技有限公司 | Reduce the conflict that I/O command accesses NVM |
| CN108089817A (en) * | 2016-11-23 | 2018-05-29 | 三星电子株式会社 | The method of storage system and its operating method and operating data processing system |
| CN109101194A (en) * | 2018-07-26 | 2018-12-28 | 郑州云海信息技术有限公司 | One kind writing with a brush dipped in Chinese ink performance optimization method and storage system |
| CN109508150A (en) * | 2018-11-13 | 2019-03-22 | 郑州云海信息技术有限公司 | A kind of method and apparatus of memory allocation |
| CN110019073A (en) * | 2017-12-29 | 2019-07-16 | 北京忆恒创源科技有限公司 | The method and its file system of file system are provided using KV storage equipment |
| CN111125447A (en) * | 2019-12-22 | 2020-05-08 | 北京浪潮数据技术有限公司 | Metadata access method, device and equipment and readable storage medium |
| CN111881064A (en) * | 2020-07-24 | 2020-11-03 | 北京浪潮数据技术有限公司 | Method, device and equipment for processing access request in full flash memory storage system |
| CN113282249A (en) * | 2021-07-19 | 2021-08-20 | 苏州浪潮智能科技有限公司 | Data processing method, system, device and medium |
| CN113568582A (en) * | 2021-07-30 | 2021-10-29 | 重庆紫光华山智安科技有限公司 | Data management method and device and storage equipment |
| CN115185723A (en) * | 2022-06-02 | 2022-10-14 | 广州市玄武无线科技股份有限公司 | Method, system, device and storage medium for repairing Region meta-information |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060064568A1 (en) * | 2004-09-21 | 2006-03-23 | Pak-Lung Seto | Integrated circuit capable of mapping logical block address data across multiple domains |
| CN101510144A (en) * | 2009-03-24 | 2009-08-19 | 中国科学院计算技术研究所 | Distributed cache system based on distributed virtual machine manager and working method thereof |
| CN102483714A (en) * | 2009-07-24 | 2012-05-30 | 苹果公司 | Restore Index Page |
| CN102629247A (en) * | 2011-12-31 | 2012-08-08 | 成都市华为赛门铁克科技有限公司 | Method, device and system for data processing |
| CN102651009A (en) * | 2011-02-28 | 2012-08-29 | 国际商业机器公司 | Method and equipment for retrieving data in storage system |
| CN102779080A (en) * | 2012-06-18 | 2012-11-14 | 上海爱数软件有限公司 | Method for generating snapshot, method and device for data recovery by using snapshot |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8005792B2 (en) * | 2004-05-14 | 2011-08-23 | Oracle International Corporation | System and method for managing versions of metadata |
| CN101656094B (en) * | 2009-09-25 | 2012-04-18 | 杭州华三通信技术有限公司 | Data storage method and storage device |
| US9430155B2 (en) * | 2010-03-25 | 2016-08-30 | International Business Machines Corporation | File index, metadata storage, and file system management for magnetic tape |
| CN101814044B (en) * | 2010-04-19 | 2013-03-20 | 中兴通讯股份有限公司 | Method and device for processing metadata |
| CN102024044B (en) * | 2010-12-08 | 2012-11-21 | 华为技术有限公司 | Distributed file system |
| CN103049390B (en) * | 2012-12-14 | 2016-03-09 | 华为技术有限公司 | The data processing method of apply metadata and storage system |
-
2012
- 2012-12-14 CN CN201210541231.4A patent/CN103049390B/en active Active
-
2013
- 2013-08-07 WO PCT/CN2013/080973 patent/WO2014089981A1/en active Application Filing
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060064568A1 (en) * | 2004-09-21 | 2006-03-23 | Pak-Lung Seto | Integrated circuit capable of mapping logical block address data across multiple domains |
| CN101510144A (en) * | 2009-03-24 | 2009-08-19 | 中国科学院计算技术研究所 | Distributed cache system based on distributed virtual machine manager and working method thereof |
| CN102483714A (en) * | 2009-07-24 | 2012-05-30 | 苹果公司 | Restore Index Page |
| CN102651009A (en) * | 2011-02-28 | 2012-08-29 | 国际商业机器公司 | Method and equipment for retrieving data in storage system |
| CN102629247A (en) * | 2011-12-31 | 2012-08-08 | 成都市华为赛门铁克科技有限公司 | Method, device and system for data processing |
| CN102779080A (en) * | 2012-06-18 | 2012-11-14 | 上海爱数软件有限公司 | Method for generating snapshot, method and device for data recovery by using snapshot |
Cited By (27)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2014089981A1 (en) * | 2012-12-14 | 2014-06-19 | 华为技术有限公司 | Data processing method and storing system for applied metadata |
| WO2016026433A1 (en) * | 2014-08-19 | 2016-02-25 | 华为技术有限公司 | Data processing method and data processing apparatus |
| WO2016045316A1 (en) * | 2014-09-22 | 2016-03-31 | 中兴通讯股份有限公司 | Method and apparatus for controlling soft-reset data recovery, and packet transport network device |
| CN106201923A (en) * | 2016-07-20 | 2016-12-07 | 杭州宏杉科技有限公司 | Method for reading and writing data and device |
| CN106201923B (en) * | 2016-07-20 | 2019-05-07 | 杭州宏杉科技股份有限公司 | Method for reading and writing data and device |
| CN107885456A (en) * | 2016-09-29 | 2018-04-06 | 北京忆恒创源科技有限公司 | Reduce the conflict that I/O command accesses NVM |
| CN107885456B (en) * | 2016-09-29 | 2022-09-16 | 北京忆恒创源科技股份有限公司 | Reducing conflicts for IO command access to NVM |
| CN108089817B (en) * | 2016-11-23 | 2023-12-19 | 三星电子株式会社 | Storage system, method of operating the same, and method of operating a data processing system |
| CN108089817A (en) * | 2016-11-23 | 2018-05-29 | 三星电子株式会社 | The method of storage system and its operating method and operating data processing system |
| US11644992B2 (en) | 2016-11-23 | 2023-05-09 | Samsung Electronics Co., Ltd. | Storage system performing data deduplication, method of operating storage system, and method of operating data processing system |
| CN107329909A (en) * | 2017-06-27 | 2017-11-07 | 郑州云海信息技术有限公司 | A kind of data managing method and device |
| CN107329909B (en) * | 2017-06-27 | 2020-07-07 | 郑州云海信息技术有限公司 | Data management method and device |
| CN107480010A (en) * | 2017-08-21 | 2017-12-15 | 郑州云海信息技术有限公司 | A kind of method and device for recovering metadata |
| CN107632944A (en) * | 2017-09-22 | 2018-01-26 | 郑州云海信息技术有限公司 | A kind of method and device for reading data |
| CN107632944B (en) * | 2017-09-22 | 2021-06-18 | 郑州云海信息技术有限公司 | Method and device for reading data |
| CN107704208A (en) * | 2017-10-13 | 2018-02-16 | 郑州云海信息技术有限公司 | A kind of restorative procedure of metadata, device and medium |
| CN110019073A (en) * | 2017-12-29 | 2019-07-16 | 北京忆恒创源科技有限公司 | The method and its file system of file system are provided using KV storage equipment |
| CN109101194A (en) * | 2018-07-26 | 2018-12-28 | 郑州云海信息技术有限公司 | One kind writing with a brush dipped in Chinese ink performance optimization method and storage system |
| CN109508150B (en) * | 2018-11-13 | 2022-02-18 | 郑州云海信息技术有限公司 | Method and device for allocating storage space |
| CN109508150A (en) * | 2018-11-13 | 2019-03-22 | 郑州云海信息技术有限公司 | A kind of method and apparatus of memory allocation |
| CN111125447A (en) * | 2019-12-22 | 2020-05-08 | 北京浪潮数据技术有限公司 | Metadata access method, device and equipment and readable storage medium |
| CN111881064A (en) * | 2020-07-24 | 2020-11-03 | 北京浪潮数据技术有限公司 | Method, device and equipment for processing access request in full flash memory storage system |
| CN113282249A (en) * | 2021-07-19 | 2021-08-20 | 苏州浪潮智能科技有限公司 | Data processing method, system, device and medium |
| CN113568582A (en) * | 2021-07-30 | 2021-10-29 | 重庆紫光华山智安科技有限公司 | Data management method and device and storage equipment |
| CN113568582B (en) * | 2021-07-30 | 2023-05-26 | 重庆紫光华山智安科技有限公司 | Data management method, device and storage equipment |
| CN115185723A (en) * | 2022-06-02 | 2022-10-14 | 广州市玄武无线科技股份有限公司 | Method, system, device and storage medium for repairing Region meta-information |
| CN115185723B (en) * | 2022-06-02 | 2023-06-02 | 广州市玄武无线科技股份有限公司 | Region meta-information restoration method, system, device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103049390B (en) | 2016-03-09 |
| WO2014089981A1 (en) | 2014-06-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103049390B (en) | The data processing method of apply metadata and storage system | |
| CN103019953B (en) | Construction system and construction method for metadata | |
| CN104346357B (en) | The file access method and system of a kind of built-in terminal | |
| US9298384B2 (en) | Method and device for storing data in a flash memory using address mapping for supporting various block sizes | |
| EP3617867B1 (en) | Fragment management method and fragment management apparatus | |
| CN103176916B (en) | The address conversion method of flash memory and flash memory | |
| CN105242881A (en) | Distributed storage system and data read-write method for same | |
| CN102779180A (en) | Operation processing method of data storage system and data storage system | |
| US20160364407A1 (en) | Method and Device for Responding to Request, and Distributed File System | |
| CN104156380A (en) | Distributed memory Hash indexing method and system | |
| CN104461390A (en) | Method and device for writing data into imbricate magnetic recording SMR hard disk | |
| CN106201355A (en) | Data processing method and device and storage system | |
| CN103942159A (en) | Data read-write method and device based on mixed storage device | |
| CN102682108B (en) | Row and line mixed database storage method | |
| CN104536903A (en) | Mixed storage method and system for conducting classified storage according to data attributes | |
| CN105786401A (en) | Data management method and device in server cluster system | |
| CN104008111A (en) | Data storage management method and device | |
| CN101916290A (en) | Managing method of internal memory database and device | |
| CN109407985B (en) | Data management method and related device | |
| CN103985393A (en) | Method and device for parallel management of multi-optical-disc data | |
| CN113094374A (en) | Distributed storage and retrieval method and device and computer equipment | |
| CN109669621A (en) | A kind of file management method, file management system, electronic equipment and storage medium | |
| CN103019963B (en) | The mapping method of a kind of high-speed cache and storage device | |
| CN104714898A (en) | Cache allocation method and device | |
| CN103490923A (en) | Log file reading and writing method, device and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |