CN103049390B - The data processing method of apply metadata and storage system - Google Patents
The data processing method of apply metadata and storage system Download PDFInfo
- Publication number
- CN103049390B CN103049390B CN201210541231.4A CN201210541231A CN103049390B CN 103049390 B CN103049390 B CN 103049390B CN 201210541231 A CN201210541231 A CN 201210541231A CN 103049390 B CN103049390 B CN 103049390B
- Authority
- CN
- China
- Prior art keywords
- mapping relations
- map section
- physical address
- logical address
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/06—Addressing a physical block of locations, e.g. base addressing, module addressing, memory dedication
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention relates to a kind of data processing method and storage system of apply metadata.Wherein, the metadata of application comprises: for store logical addresses and physical address map relation map section and close the index area of the position tied up in described map section for query logic address and physical address map; Described method comprises: what storage system received data reads or writes request, described in read or write in request and carry logical address; Inquired about in described map section the mapping relations of whether preserving between described logical address and physical address by the index information of described index area according to described logical address; If preserve described mapping relations in described MP district, the physical address that described mapping relations are corresponding reads or writes process to described data.
Description
Technical field
The present invention relates to field of computer technology, particularly relate to a kind of data processing method and storage system of apply metadata.
Background technology
Along with the development of information science and technology, the data volume stored is needed rapidly to increase with incredible speed, therefore to the Rational Classification of storage system resource with utilize and it is also proposed higher requirement.On the one hand, the demand of storage system demand fulfillment current business, by simply configuring the storage that just can manage mass data; On the other hand, in the process that business not very increases, storage system can expand, and storage resources of distributing according to need along with the growth of business.Thus, thin provisioning arises at the historic moment.
Thin provisioning is a kind of storage virtualization technology directly developed from traditional allocation of space mode, all storage spaces are pooled to simplify in pond by this technology carries out unified management, simplify on pond create simplify logical block, obtain amount of physical memory from simplifying pond as required.The use of thin provisioning can simplify the configuration management of storage resources, saves physical memory resources.
In thin provisioning, metadata is mainly used in the mapping relations of store logical addresses and physical address.The reliability of metadata is technology most crucial in thin provisioning.And in existing data processing method, storage system utilizes the metadata with B tree or B tree variant (as B+ tree) structure to carry out read-write process to data.Set at B in the node of metadata and can store logical address and physical address map relation, utilize B tree construction effectively can improve the access efficiency of metadata, there is good extendability.But, once the node that B sets metadata damages, logical address and the physical address map relation of the storage of this node can be lost, the lower level node that storage system searches this damage node can be affected simultaneously, the physical address that the logical address causing storage system cannot store in the affected part of metadata is corresponding with physical address map relation processes data.
Summary of the invention
In view of this, embodiments provide a kind of data processing method and storage system of apply metadata, to realize when metadata partial destruction, can not affect on logical address that storage system stores in other parts of the metadata physical address corresponding with physical address map relation and data are processed.
In first aspect, embodiments provide a kind of data processing method of apply metadata.Wherein, described metadata comprises: for store logical addresses and physical address map relation map section and close the index area of the position tied up in described map section for query logic address and physical address map; Described method comprises:
What storage system received data reads or writes request, described in read or write in request and carry logical address;
Inquired about in described map section the mapping relations of whether preserving between described logical address and physical address by the index information of described index area according to described logical address;
If preserve described mapping relations in described map section, the physical address that described mapping relations are corresponding reads or writes process to described data.
In the first possible implementation of first aspect, described map section comprises multiple region, and reserved one section of space, each region is used for store logical addresses and physical address map relation.
In conjunction with the first possible implementation of first aspect, in the implementation that the second is possible, described method also comprises: if do not preserve described mapping relations in described map section, and described in read or write request for write request, then the logical address allocated physical address of carrying for described write request; The physical address distributed carries out writing process to described data; According to the physical address of described distribution, described mapping relations corresponding for described write request are stored in map section; According to the logical address that described write request is carried, in described index area, record in map section the position of the mapping relations of preserving between described logical address and the physical address of distribution.
In conjunction with the first possible implementation of first aspect, in the implementation that the third is possible, described method also comprises: described to be inquired about the mapping relations of whether preserving described logical address and physical address in described map section by the index information of described index area according to described logical address while, metadata is detected; If preserve the area damage of described mapping relations in map section, and described in read or write request for write request, then the logical address of carrying for described write request redistributes physical address; The physical address redistributed carries out writing process to described data; According to the described physical address redistributed, described mapping relations corresponding for described write request are stored in map section; According to the logical address that described write request is carried, in described index area record, in map section, preserve the position of the mapping relations between described logical address and the physical address redistributed.
In conjunction with the third possible implementation of first aspect, in the 4th kind of possible implementation, described method also comprises: if local, described index area or all damage, recover described index area according to the logical address in the described mapping relations that each region in map section is preserved.
In second aspect, embodiments provide a kind of storage system of apply metadata.Wherein, described metadata comprises: for store logical addresses and physical address map relation map section and close the index area of the position tied up in described map section for query logic address and physical address map; Described storage system comprises:
Receiving element, reads or writes request for what receive data, described in read or write in request and carry logical address;
Query unit, for inquiring about in described map section the mapping relations of whether preserving between described logical address and physical address by the index information of described index area according to described logical address;
Processing unit, if for preserving described mapping relations in described map section, the physical address that described mapping relations are corresponding reads or writes process to described data.
In the first possible implementation of second aspect, described map section comprises multiple region, and reserved one section of space, each region is used for store logical addresses and physical address map relation.
In conjunction with the first possible implementation of first aspect, in the implementation that the second is possible, described storage system also comprises: allocation units, if for not preserving described mapping relations in described map section, and described in read or write request for write request, then the logical address allocated physical address of carrying for described write request; Described processing unit also for carrying out writing process to described data on the physical address distributed;
Record cell, for the physical address according to described distribution, described mapping relations corresponding for described write request are stored in map section, according to the logical address that described write request is carried, in described index area, record in map section the position of the mapping relations of preserving between described logical address and the physical address of distribution.
In conjunction with the first possible implementation of second aspect, in the implementation that the third is possible, described storage system also comprises: detecting unit, for to be inquired about by the index information of described index area according to described logical address in described query unit whether preserve described logical address and physical address in described map section mapping relations while, metadata is detected; Allocation units, if the area damage for preserving described mapping relations in map section, and described in read or write request for write request, then the logical address of carrying for described write request redistributes physical address; Described processing unit also for carrying out writing process to described data on the physical address redistributed; Record cell, for the physical address redistributed described in basis, described mapping relations corresponding for described write request are stored in map section, according to the logical address that described write request is carried, in described index area, record in map section the position of the mapping relations of preserving between described logical address and the physical address redistributed.
In conjunction with the third possible implementation of second aspect, in the 4th kind of possible implementation, described storage system also comprises: recovery unit, if for local, described index area or all damage, recover described index area according to the logical address in the described mapping relations that region each in map section is preserved.
Pass through such scheme, because the index area of metadata and map section are independently, storage system searches in map section the position of logical address corresponding to the request that reads or writes and the physical address map relation of preserving data during by index area, if the part of map section is damaged, only can lose logical address and the physical address map relation of the storage of broken parts, storage system searches the storage of map section other parts logical address and physical address map relation by index area can not be affected, thus can not affect on logical address that storage system preserves at the non-broken parts physical address corresponding with physical address map relation process is read or write to data.
Accompanying drawing explanation
Fig. 1 is the metadata structure schematic diagram of embodiment of the present invention application;
The process flow diagram of the data processing method of a kind of apply metadata that Fig. 2 provides for the embodiment of the present invention one;
The structural representation of the storage system of a kind of apply metadata that Fig. 3 provides for the embodiment of the present invention two;
The structural representation of the storage system of the another kind of apply metadata that Fig. 4 provides for the embodiment of the present invention two;
The structural representation of a kind of storage system that Fig. 5 provides for the embodiment of the present invention three.
Embodiment
In order to make the object, technical solutions and advantages of the present invention clearly, below in conjunction with accompanying drawing, the present invention is described in further detail, and obviously, described embodiment is only a part of embodiment of the present invention, instead of whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art, not making other embodiments all obtained under creative work prerequisite, belong to the scope of protection of the invention.
First, the metadata structure of embodiment of the present invention application is introduced for Fig. 1.Fig. 1 is the metadata structure schematic diagram of embodiment of the present invention application.This metadata comprises map section (MappingPartitn, MP, hereinafter referred to as MP district) and index area (IndexPartitn, IP, hereinafter referred to as IP district).Wherein, MP district is used for store logical addresses and physical address map relation (LogicaddressandPhysicaladdressMapping, LPM).MP district comprises multiple region.Concrete, LPM is preserved in the reserved one section of region of the every 1G physical space on physical disk, as shown in Figure 1, every 1G is a region, in each region, black part is divided into the LPM of storage, and each black square represents a LPM, and white portion is the data in this region stored on physical address corresponding to LPM.It should be noted that, the present invention does not limit the size in each region, MP district, and the size in each region, MP district is a kind of situation that 1G is only the embodiment of the present invention, and MP district can infinite expanding.IP district is used for inquiring about the position of LPM corresponding to this logical address in MP district according to logical address.IP district can be B tree construction, but is not limited only to B tree construction, and IP district can be the structure that other can realize straw line function.
As seen from Figure 1, the IP district of this metadata and MP district are two independently regions, when the subregion in MP district is damaged, can not affect the LPM that the non-failure area in MP district is searched in IP district.Further, when IP district is damaged, also can recover according to MP district.Therefore, apply this metadata within the storage system, when the part in Dang Yin MP district is damaged and causes part LPM to lose, the LPM that storage system utilizes IP district to search metadata MP district other parts to store can not be affected, thus can not affect on physical address corresponding to LPM that storage system stores in MP district other parts and carry out reading and writing data process.When IP district is damaged, the process of whole storage system to data can not be affected.
The data processing method of a kind of apply metadata that the embodiment of the present invention one provides is described below in detail, the process flow diagram of the data processing method of a kind of apply metadata that Fig. 2 provides for the embodiment of the present invention one for Fig. 2.The executive agent of the data processing method of this apply metadata is storage system, applies above-mentioned metadata in this storage system.As shown in Figure 2, the data processing method of this apply metadata comprises the following steps:
Step S201, storage system receive data read or write request.
The read request of each data and write request carry identical logical address, storage system receive data read or write request after, the logical address of carrying can be asked to search corresponding LPM according to reading or writing, thus find physical address corresponding to logical address, physical address carries out read or write to these data.
Step S202, asks the logical address of carrying to read or write by what whether preserve these data in the index information inquiry MP district in IP district the LPM asking logical address and the physical address carried according to reading or writing of data.
First storage system asks the logical address of carrying to inquire the region reading or writing MP district corresponding to request of these data by the index information in IP district according to reading or writing of data, then judge whether to preserve in this region to these data read or write the relevant LPM of the logical address of asking to carry.If inquire to these data read or write the relevant LPM of the logical address of asking to carry, then LPM corresponding to the request that reads or writes preserving these data is described in MP district, performs step S203.If do not inquire to these data read or write the relevant LPM of the logical address of asking to carry, then LPM corresponding to the request that reads or writes not preserving these data is described in MP district, performs step S204.
Such as, the data structure in IP district can be that Hash Hash shows, B sets and the various structures such as variant, sequence chained list.Describe the method for searching physical address according to logical address below for Hash table, during this Hash shows, Key value is logical address, and Value value is the address of LPM in MP district.This key value of first preserving in Hash table according to logical address is inquired about this with the corresponding relation of Value value and is read or write the address of asking corresponding LPM in MP district, then carry out reading the LPM that can obtain and read or write request correspondence in the address in MP district to LPM, thus obtain corresponding physical address according to LPM.
Step S203, if LPM corresponding to the request that reads or writes preserving these data in MP district, the physical address that this LPM is corresponding reads or writes process to these data.
If preserve the LPM corresponding to the request that reads or writes of these data in MP district, then know the physical address corresponding to the logical address that the request that reads or writes of these data is carried according to LPM, then on this physical address, process is read or write to these data.
Step S204, if LPM corresponding to the request that reads or writes not preserving these data in MP district, and the write request of the request that reads or writes for these data of these data, the logical address allocated physical address that the write request for these data is carried.
In addition, if do not preserve these data in MP district to read or write LPM corresponding to request, and the read request that the request that reads or writes of these data is these data, then return full 0 data, with inform read data this storage system of equipment in do not preserve corresponding data.
Step S205, the physical address distributed carries out writing process to these data.
After the logical address allocated physical address that storage system is carried in the write request for these data, just can write this data on this physical address.After this physical address completes write data, this physical address can be found by metadata after ensureing, process the data that this physical address writes, need LPM corresponding for the write request of these data to record in the metadata, therefore storage system also needs to perform following steps.
Step S206, according to the physical address distributed, is stored in LPM corresponding for the write request of these data in MP district.
The physical address of the logical address of carrying in write request and distribution is formed mapping relations LPM, LPM is kept at which position concrete in MP district, this patent does not limit, such as MP can be distinguished into the continuous physical space that 1G is unit, every 1G continuous space is designated as a region, the fixed area of foremost, each region is used for preserving LPM corresponding to all physical addresss in this region.
First storage system determines the region of preserving this LPM in MP district according to the physical address distributed, and then allocation space preserves this LPM in this region.
Step S207, according to the logical address that the write request of these data is carried, records the position of LPM corresponding to the write request of preserving these data in MP district in IP district.
For the data structure in IP district for Hash table, during this Hash shows, Key value is logical address, and Value value is the address of LPM in MP district.Namely this step is increase a record in Hash table.So that storage system receive again these data read or write request time, according to read or write ask the logical address of carrying by the Key value in the Hash table in IP district and the corresponding relation of Value value inquire about in MP preserve these data read or write the address of asking corresponding LPM, thus on the physical address reading or writing corresponding to the LPM asking correspondence of these data, process is read or write to these data.
In the disposal route of the apply metadata that the present embodiment provides, except above-mentioned storage system to read or write process method to data, also comprise when metadata is damaged, storage system is to the disposal route of data and the method recovering metadata.
Concrete, step S202, reads or writes the logical address of asking to carry by while LPM corresponding to the request that reads or writes that whether preserve these data in the index information inquiry MP district in IP district according to data, and storage system detects metadata.If metadata is not damaged, then perform above-mentioned steps S203-S207.
If preserve the area damage reading or writing LPM corresponding to request of these data in the MP district of metadata, and the write request of the request that reads or writes for data of these data, the logical address of then again carrying for the write request of these data redistributes physical address, the physical address redistributed carries out writing process to these data, to ensure that these data can normally write when subregion, MP district is damaged.
For storage system after making can find this physical address redistributed by metadata, the data that this physical address writes are processed, need LPM corresponding for the write request of these data to record in the metadata, recording method, with step S206-S207, does not repeat them here.
In addition, if preserve the area damage reading or writing LPM corresponding to request of these data in the MP district of metadata, and the read request of the request that reads or writes for data of these data, then return error message to the equipment reading these data.
If the IP district of metadata is damaged, then recover IP district according to logical address in the LPM of each region preservation in MP district.Concrete, storage system is accessed for storing the space of LPM in the every 1G space of physical disk successively, reads LPM.Ask the logical address of carrying because LPM comprises reading or writing of data, then storage system rebuilds IP district according to the LPM read, concrete, can adopt that Hash Hash shows, the various structures of B tree and variant, sequence chained list etc. rebuilds IP district according to logical address.After completing recovery, IP district normally can carry out index.
The disposal route of the apply metadata utilizing the embodiment of the present invention one to provide, storage system receive data read or write request after, the IP district of metadata is utilized to search the LPM stored in MP district, because IP district and MP district are independently, therefore, when Dang Yin MP district partial destruction causes partial data to be lost, the LPM that storage system utilizes the IP district of metadata to search the non-broken parts in MP district to preserve can not be affected, thus storage system can not be affected process is read or write to the data of not losing.Further, when the IP district of metadata is damaged, can recover according to the LPM preserved in MP district, storage system can not be affected and utilize IP district to inquire about LPM in MP district, then process be read or write to data.
Accordingly, the embodiment of the present invention two provides a kind of storage system in order to realize method described in embodiment one.As shown in Figure 3, the structural representation of the storage system of its a kind of apply metadata provided for the embodiment of the present invention two.The metadata applied in this storage system comprises the MP district for storing LPM and the IP district for inquiring about the position of LPM in described MP district.Wherein, MP district comprises multiple region, and reserved one section of space, each region is for storing LPM.This storage system comprises: receiving element 310, query unit 320 and processing unit 330.
Receiving element 310 reads or writes request for what receive data.Reading or writing in request of same data carries identical logical address.
Query unit 320 reads or writes by what whether preserve these data in the index information inquiry MP district in IP district the LPM asking logical address and the physical address carried for the logical address of asking to carry of reading or writing of the data that receive according to receiving element 310.
First query unit 320 asks the logical address of carrying to utilize IP district to inquire the region reading or writing MP district corresponding to request of these data according to reading or writing of data, then judge whether to preserve in this region to these data read or write the relevant LPM of the logical address of asking to carry.If query unit 320 inquire to these data read or write the relevant LPM of the logical address of asking to carry, then LPM corresponding to the request that reads or writes preserving these data is described in MP district.If query unit 320 do not inquire to these data read or write the relevant LPM of the logical address of asking to carry, then LPM corresponding to the request that reads or writes not preserving these data is described in MP district.
If processing unit 330, for preserving the LPM corresponding to the request that reads or writes of these data in MP district, the physical address corresponding to the LPM corresponding to the request that reads or writes of these data reads or writes process to these data.
In addition, this storage system also comprises: allocation units 340 and record cell 350.As shown in Figure 4, the structural representation of the storage system of its another kind of apply metadata provided for the embodiment of the present invention two.
If allocation units 340 are not for preserving the LPM corresponding to the request that reads or writes of these data in MP, and the write request of the request that reads or writes for these data of these data, the logical address allocated physical address that the write request for these data is carried.Then processing unit 330 also carries out writing process to these data on the physical address distributed at allocation units 340.
The physical address of record cell 350 for distributing according to allocation units 340, LPM corresponding for the write request of these data is stored in MP district, according to the logical address that the write request of these data is carried, in record MP district of described IP district, preserve the position of LPM corresponding to the write request of these data.After having recorded, the logical address that query unit 320 just can be carried according to the request that reads or writes of these data finds the physical address of this distribution by metadata, process the data that this physical address writes to make processing unit 330.
This storage system can also comprise: detecting unit 360 and recovery unit 370.
Detecting unit 360 for query unit 320 according to data read or write ask the logical address of carrying by the index information inquiry MP district in IP district in whether preserve the LPM corresponding to the request that reads or writes of these data while, metadata is detected, to make when metadata is damaged, data are done to the metadata of corresponding process and Recover from damaging.If then allocation units 340 are also for preserving the area damage reading or writing LPM corresponding to request of these data in MP district, and the write request of the request that reads or writes for these data of these data, the logical address that the write request for these data is carried redistributes physical address.Accordingly, processing unit 330 also for carrying out writing process to these data on the physical address redistributed.And record cell 350 is also for according to the physical address redistributed, LPM corresponding for the write request of these data is stored in MP district, then according to the logical address that the write request that changes data is carried, in record MP district of IP district, the position of LPM corresponding to the write request of these data is preserved.
If recovery unit 370 is for local, IP district or all damage, recover IP district according to the LPM that region each in MP district is preserved.Concrete, recovery unit 370 is accessed for storing the space of LPM in the every 1G space of physical disk successively, reads LPM.The logical address of carrying is asked, so recovery unit 370 rebuilds IP district according to the logical address in the LPM read because LPM comprises reading or writing of data.After completing recovery, IP district normally can carry out index.
It should be noted that, the unit included by storage system of a kind of apply metadata that the present embodiment provides is carry out dividing according to function logic, but is not limited to above-mentioned division, as long as can realize corresponding function; In addition, the concrete title of each functional unit, also just for the ease of mutual differentiation, is not limited to protection scope of the present invention.
The storage system of the apply metadata utilizing the embodiment of the present invention two to provide, storage system receive data read or write request after, the IP district of metadata is utilized to search the LPM stored in MP district, because IP district and MP district are independently, therefore, when Dang Yin MP district partial destruction causes partial data to be lost, the LPM that storage system utilizes the IP district of metadata to search the non-broken parts in MP district to preserve can not be affected, thus storage system can not be affected process is read or write to the data of not losing.Further, when the IP district of metadata is damaged, can recover according to the LPM preserved in MP district, storage system can not be affected and utilize IP district to inquire about LPM in MP district, then process be read or write to data.
On hardware implementing, above receiving element 310 can be receiver or transceiver.Other unit more than except receiving element 310 can be embedded in or in the form of hardware independent of in the processor of storage system, also can be stored in the storer of storage system in a software form, so that processor calls perform operation corresponding to above modules.This processor can be CPU (central processing unit) (CPU), microprocessor, single-chip microcomputer etc.
As shown in Figure 5, the structural representation of its a kind of storage system provided for the embodiment of the present invention three.The processor 530 that this storage system comprises transceiver 510, storer 520 and is connected with transceiver 510 and storer 520 respectively.Certainly, storage system can also comprise the universal components such as antenna, baseband process component, middle radio frequency processing parts, input-output unit, and the embodiment of the present invention is in this no longer any restriction.
Wherein, in storer 520, store batch processing code, and processor 530 is for calling the program code stored in storer 520, for performing following operation:
What receive data reads or writes request, described in read or write in request and carry logical address;
Inquired about in described map section the mapping relations of whether preserving between described logical address and physical address by the index information of described index area according to described logical address;
If preserve described mapping relations in described map section, the physical address that described mapping relations are corresponding reads or writes process to described data.
Further, in storer 520, store batch processing code, and processor 530 is for calling the program code stored in storer 520, can also be used for performing following operation:
If do not preserve described mapping relations in described map section, and described in read or write request for write request, then the logical address allocated physical address of carrying for described write request;
The physical address distributed carries out writing process to described data;
According to the physical address of described distribution, described mapping relations corresponding for described write request are stored in map section;
According to the logical address that described write request is carried, in described index area, record in map section the position of the mapping relations of preserving between described logical address and the physical address of distribution.
Further, in storer 520, store batch processing code, and processor 530 is for calling the program code stored in storer 520, can also be used for performing following operation:
Described to be inquired about the mapping relations of whether preserving described logical address and physical address in described map section by the index information of described index area according to described logical address while, metadata is detected;
If preserve the area damage of described mapping relations in map section, and described in read or write request for write request, then the logical address of carrying for described write request redistributes physical address;
The physical address redistributed carries out writing process to described data;
According to the described physical address redistributed, described mapping relations corresponding for described write request are stored in map section;
According to the logical address that described write request is carried, in described index area, record in map section the position of the mapping relations of preserving between described logical address and the physical address redistributed.
Further, in storer 520, store batch processing code, and processor 530 is for calling the program code stored in storer 520, can also be used for performing following operation:
If local, described index area or all damage, recover described index area according to the logical address in the described mapping relations that region each in map section is preserved.
It should be noted that, the storage system provided in embodiment two and embodiment three all may be used for realizing the method that embodiment one provides, and about the description of metadata etc. with the description in embodiment two, does not repeat them here.
Utilize the storage system that the embodiment of the present invention three provides, storage system receive data read or write request after, the IP district of metadata is utilized to search the LPM stored in MP district, because IP district and MP district are independently, therefore, when Dang Yin MP district partial destruction causes partial data to be lost, the LPM that storage system utilizes the IP district of metadata to search the non-broken parts in MP district to preserve can not be affected, thus storage system can not be affected process is read or write to the data of not losing.Further, when the IP district of metadata is damaged, can recover according to the LPM preserved in MP district, storage system can not be affected and utilize IP district to inquire about LPM in MP district, then process be read or write to data.
Professional should recognize further, in conjunction with unit and the algorithm steps of each example of embodiment disclosed herein description, can realize with electronic hardware, computer software or the combination of the two, in order to the interchangeability of hardware and software is clearly described, generally describe composition and the step of each example in the above description according to function.These functions perform with hardware or software mode actually, depend on application-specific and the design constraint of technical scheme.Professional and technical personnel can use distinct methods to realize described function to each specifically should being used for, but this realization should not thought and exceeds scope of the present invention.
The software module that the method described in conjunction with embodiment disclosed herein or the step of algorithm can use hardware, processor to perform, or the combination of the two is implemented.Software module can be placed in the storage medium of other form any known in random access memory (RAM), internal memory, ROM (read-only memory) (ROM), electrically programmable ROM, electrically erasable ROM, register, hard disk, moveable magnetic disc, CD-ROM or technical field.
Above-described embodiment; object of the present invention, technical scheme and beneficial effect are further described; be understood that; the foregoing is only the specific embodiment of the present invention; the protection domain be not intended to limit the present invention; within the spirit and principles in the present invention all, any amendment made, equivalent replacement, improvement etc., all should be included within protection scope of the present invention.
Claims (10)
1. the data processing method of an apply metadata, it is characterized in that, the data structure of described metadata comprises: for the map section of store logical addresses and physical address map relation and for store logical addresses and described mapping relations the index area of the corresponding relation of the position in described map section; Described method comprises:
What storage system received data reads or writes request, described in read or write in request and carry logical address;
Inquired about in described map section the mapping relations of whether preserving between described logical address and physical address by the index information of described index area according to described logical address;
If preserve described mapping relations in described map section, then in described index area, obtain the position of described mapping relations in described map section, find described mapping relations according to the position of obtained described mapping relations in described map section at described map section, the physical address that logical address described in described mapping relations is corresponding reads or writes process to described data.
2. method according to claim 1, is characterized in that, described map section comprises multiple region, and reserved one section of space, each region is for storing described mapping relations.
3. method according to claim 2, is characterized in that, described method also comprises:
If do not preserve described mapping relations in described map section, and described in read or write request for write request, then the logical address allocated physical address of carrying for described write request;
The physical address distributed carries out writing process to described data;
According to the physical address of described distribution, described mapping relations corresponding for described write request are stored in map section;
According to the logical address that described write request is carried, in described index area, record in map section the position of the mapping relations of preserving between described logical address and the physical address of distribution.
4. method according to claim 2, is characterized in that, described method also comprises:
Described to be inquired about the mapping relations of whether preserving described logical address and physical address in described map section by the index information of described index area according to described logical address while, metadata is detected;
If preserve the area damage of described mapping relations in map section, and described in read or write request for write request, then the logical address of carrying for described write request redistributes physical address;
The physical address redistributed carries out writing process to described data;
According to the described physical address redistributed, described mapping relations corresponding for described write request are stored in map section;
According to the logical address that described write request is carried, in described index area, record in map section the position of the mapping relations of preserving between described logical address and the physical address redistributed.
5. method according to claim 4, is characterized in that, described method also comprises:
If local, described index area or all damage, recover described index area according to the logical address in the described mapping relations that region each in map section is preserved.
6. the storage system of an apply metadata, it is characterized in that, the data structure of described metadata comprises: for the map section of store logical addresses and physical address map relation and for store logical addresses and described mapping relations the index area of the corresponding relation of the position in described map section; Described storage system comprises:
Receiving element, reads or writes request for what receive data, described in read or write in request and carry logical address;
Query unit, for inquiring about in described map section the mapping relations of whether preserving between described logical address and physical address by the index information of described index area according to described logical address;
Processing unit, if for preserving described mapping relations in described map section, then in described index area, obtain the position of described mapping relations in described map section, find described mapping relations according to the position of obtained described mapping relations in described map section at described map section, the physical address that logical address described in described mapping relations is corresponding reads or writes process to described data.
7. storage system according to claim 6, is characterized in that, described map section comprises multiple region, and reserved one section of space, each region is for storing described mapping relations.
8. storage system according to claim 7, is characterized in that, described storage system also comprises:
Allocation units, if for not preserving described mapping relations in described map section, and described in read or write request for write request, then the logical address allocated physical address of carrying for described write request;
Described processing unit also for carrying out writing process to described data on the physical address distributed;
Record cell, for the physical address according to described distribution, described mapping relations corresponding for described write request are stored in map section, according to the logical address that described write request is carried, in described index area, record in map section the position of preserving mapping relations between described logical address and the physical address of distribution.
9. storage system according to claim 7, is characterized in that, described storage system also comprises:
Detecting unit, for be inquired about by the index information of described index area according to described logical address in described query unit whether preserve described logical address and physical address in described map section mapping relations while, metadata is detected;
Allocation units, if the area damage for preserving described mapping relations in map section, and described in read or write request for write request, then the logical address of carrying for described write request redistributes physical address;
Described processing unit also for carrying out writing process to described data on the physical address redistributed;
Record cell, for the physical address redistributed described in basis, described mapping relations corresponding for described write request are stored in map section, according to the logical address that described write request is carried, in described index area, record in map section the position of the mapping relations of preserving between described logical address and the physical address redistributed.
10. storage system according to claim 9, is characterized in that, described storage system also comprises:
Recovery unit, if for local, described index area or all damage, recovers described index area according to the logical address in the described mapping relations that region each in map section is preserved.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210541231.4A CN103049390B (en) | 2012-12-14 | 2012-12-14 | The data processing method of apply metadata and storage system |
| PCT/CN2013/080973 WO2014089981A1 (en) | 2012-12-14 | 2013-08-07 | Data processing method and storing system for applied metadata |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210541231.4A CN103049390B (en) | 2012-12-14 | 2012-12-14 | The data processing method of apply metadata and storage system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103049390A CN103049390A (en) | 2013-04-17 |
| CN103049390B true CN103049390B (en) | 2016-03-09 |
Family
ID=48062037
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210541231.4A Active CN103049390B (en) | 2012-12-14 | 2012-12-14 | The data processing method of apply metadata and storage system |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN103049390B (en) |
| WO (1) | WO2014089981A1 (en) |
Families Citing this family (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103049390B (en) * | 2012-12-14 | 2016-03-09 | 华为技术有限公司 | The data processing method of apply metadata and storage system |
| CN104182508B (en) * | 2014-08-19 | 2018-10-30 | 华为技术有限公司 | A kind of data processing method and data processing equipment |
| CN105516001A (en) * | 2014-09-22 | 2016-04-20 | 中兴通讯股份有限公司 | Method and device for controlling data recovery after soft reset, and packet transport network equipment |
| CN106201923B (en) * | 2016-07-20 | 2019-05-07 | 杭州宏杉科技股份有限公司 | Method for reading and writing data and device |
| CN107885456B (en) * | 2016-09-29 | 2022-09-16 | 北京忆恒创源科技股份有限公司 | Reducing conflicts for IO command access to NVM |
| US11644992B2 (en) | 2016-11-23 | 2023-05-09 | Samsung Electronics Co., Ltd. | Storage system performing data deduplication, method of operating storage system, and method of operating data processing system |
| CN107329909B (en) * | 2017-06-27 | 2020-07-07 | 郑州云海信息技术有限公司 | Data management method and device |
| CN107480010A (en) * | 2017-08-21 | 2017-12-15 | 郑州云海信息技术有限公司 | A kind of method and device for recovering metadata |
| CN107632944B (en) * | 2017-09-22 | 2021-06-18 | 郑州云海信息技术有限公司 | Method and device for reading data |
| CN107704208B (en) * | 2017-10-13 | 2021-08-10 | 郑州云海信息技术有限公司 | Method, device and medium for repairing metadata |
| CN110019073A (en) * | 2017-12-29 | 2019-07-16 | 北京忆恒创源科技有限公司 | The method and its file system of file system are provided using KV storage equipment |
| CN109101194A (en) * | 2018-07-26 | 2018-12-28 | 郑州云海信息技术有限公司 | One kind writing with a brush dipped in Chinese ink performance optimization method and storage system |
| CN109508150B (en) * | 2018-11-13 | 2022-02-18 | 郑州云海信息技术有限公司 | Method and device for allocating storage space |
| CN111125447A (en) * | 2019-12-22 | 2020-05-08 | 北京浪潮数据技术有限公司 | Metadata access method, device and equipment and readable storage medium |
| CN111881064A (en) * | 2020-07-24 | 2020-11-03 | 北京浪潮数据技术有限公司 | Method, device and equipment for processing access request in full flash memory storage system |
| CN113282249B (en) * | 2021-07-19 | 2021-10-29 | 苏州浪潮智能科技有限公司 | Data processing method, system, device and medium |
| CN113568582B (en) * | 2021-07-30 | 2023-05-26 | 重庆紫光华山智安科技有限公司 | Data management method, device and storage equipment |
| CN115185723B (en) * | 2022-06-02 | 2023-06-02 | 广州市玄武无线科技股份有限公司 | Region meta-information restoration method, system, device and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101510144A (en) * | 2009-03-24 | 2009-08-19 | 中国科学院计算技术研究所 | Distributed cache system based on distributed virtual machine manager and working method thereof |
| CN102483714A (en) * | 2009-07-24 | 2012-05-30 | 苹果公司 | Restore Index Page |
| CN102651009A (en) * | 2011-02-28 | 2012-08-29 | 国际商业机器公司 | Method and equipment for retrieving data in storage system |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8005792B2 (en) * | 2004-05-14 | 2011-08-23 | Oracle International Corporation | System and method for managing versions of metadata |
| US7774575B2 (en) * | 2004-09-21 | 2010-08-10 | Intel Corporation | Integrated circuit capable of mapping logical block address data across multiple domains |
| CN101656094B (en) * | 2009-09-25 | 2012-04-18 | 杭州华三通信技术有限公司 | Data storage method and storage device |
| US9430155B2 (en) * | 2010-03-25 | 2016-08-30 | International Business Machines Corporation | File index, metadata storage, and file system management for magnetic tape |
| CN101814044B (en) * | 2010-04-19 | 2013-03-20 | 中兴通讯股份有限公司 | Method and device for processing metadata |
| CN102024044B (en) * | 2010-12-08 | 2012-11-21 | 华为技术有限公司 | Distributed file system |
| CN102629247B (en) * | 2011-12-31 | 2014-09-17 | 华为数字技术（成都）有限公司 | Method, device and system for data processing |
| CN102779080B (en) * | 2012-06-18 | 2015-05-27 | 上海爱数软件有限公司 | Method for generating snapshot, method and device for data recovery by using snapshot |
| CN103049390B (en) * | 2012-12-14 | 2016-03-09 | 华为技术有限公司 | The data processing method of apply metadata and storage system |
-
2012
- 2012-12-14 CN CN201210541231.4A patent/CN103049390B/en active Active
-
2013
- 2013-08-07 WO PCT/CN2013/080973 patent/WO2014089981A1/en active Application Filing
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101510144A (en) * | 2009-03-24 | 2009-08-19 | 中国科学院计算技术研究所 | Distributed cache system based on distributed virtual machine manager and working method thereof |
| CN102483714A (en) * | 2009-07-24 | 2012-05-30 | 苹果公司 | Restore Index Page |
| CN102651009A (en) * | 2011-02-28 | 2012-08-29 | 国际商业机器公司 | Method and equipment for retrieving data in storage system |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2014089981A1 (en) | 2014-06-19 |
| CN103049390A (en) | 2013-04-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103049390B (en) | The data processing method of apply metadata and storage system | |
| CN103019953B (en) | Construction system and construction method for metadata | |
| US11243706B2 (en) | Fragment management method and fragment management apparatus | |
| CN109542342B (en) | Metadata management and data reconstruction method, equipment and storage medium | |
| CN111125447A (en) | Metadata access method, device and equipment and readable storage medium | |
| US20160364407A1 (en) | Method and Device for Responding to Request, and Distributed File System | |
| CN105242881A (en) | Distributed storage system and data read-write method for same | |
| CN104461390A (en) | Method and device for writing data into imbricate magnetic recording SMR hard disk | |
| CN106569742A (en) | Storage management method and storage equipment | |
| CN102779180A (en) | Operation processing method of data storage system and data storage system | |
| CN100524237C (en) | Computer hard disk data continuous time point storage and recovery method | |
| CN104156380A (en) | Distributed memory Hash indexing method and system | |
| CN102694828B (en) | A kind of method of distributed cache system data access and device | |
| CN107273046B (en) | A kind of data processing method and system based on Solid-state disc array | |
| CN103049224B (en) | By the methods, devices and systems of data importing physical tape | |
| CN109976669B (en) | Edge storage method, device and storage medium | |
| CN107423233A (en) | A kind of writeable snapshot implementing method and device | |
| CN103399823A (en) | Method, equipment and system for storing service data | |
| CN105786410A (en) | Method for increasing processing speed of data storage system and data storage system | |
| CN109918316B (en) | Method and system for reducing FTL address mapping space | |
| CN102981979B (en) | A kind of method improving memory system data access speed | |
| US20240086332A1 (en) | Data processing method and system, device, and medium | |
| CN103440204B (en) | A kind of method of updating file system and memory device | |
| CN113094374A (en) | Distributed storage and retrieval method and device and computer equipment | |
| JP2007220107A (en) | Apparatus and method for managing mapping information of nonvolatile memory |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |