WO2020038186A1 - Appareil et procédé de migration de données, et dispositif de stockage - Google Patents

Appareil et procédé de migration de données, et dispositif de stockage Download PDF

Info

Publication number
WO2020038186A1
WO2020038186A1 PCT/CN2019/098061 CN2019098061W WO2020038186A1 WO 2020038186 A1 WO2020038186 A1 WO 2020038186A1 CN 2019098061 W CN2019098061 W CN 2019098061W WO 2020038186 A1 WO2020038186 A1 WO 2020038186A1
Authority
WO
WIPO (PCT)
Prior art keywords
storage
data
virtual
offset
address
Prior art date
Application number
PCT/CN2019/098061
Other languages
English (en)
Chinese (zh)
Inventor
贾胜迁
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2020038186A1 publication Critical patent/WO2020038186A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
    • G06F3/0647—Migration mechanisms
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0604—Improving or facilitating administration, e.g. storage management
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0662—Virtualisation aspects

Definitions

  • the embodiments of the present application relate to the field of communications technologies, and in particular, to a data migration method, device, and storage device.
  • Some electronic devices include storage media that need to be erased before writing, and these storage media include multiple storage areas. Data is written into the storage area in an additional write manner. After the storage addresses in the storage area are filled in order, a garbage collection (GC) process may need to be performed on the storage area.
  • the GC process refers to: after the storage address in the storage area is filled up in order, if a part of the data in the storage area is deleted, the storage space after the partial data is deleted because the remaining data occupies the storage space Not available, so you need to migrate the data in the storage area and reset the storage area.
  • the storage address information of data is recorded through metadata, and the metadata is organized in a tree structure. After the above GC process is performed, it is necessary to start from the leaf node metadata corresponding to the migrated data and modify the metadata layer by layer.
  • the embodiments of the present application provide a data migration method, device, and storage device, which are used to solve the problem of low data management efficiency in the prior art.
  • a first aspect of the embodiments of the present application provides a data migration method.
  • the method is applied to a storage device.
  • the storage device includes a controller and at least one hard disk.
  • the controller is configured to manage data stored in the hard disk.
  • the first data is stored in a first storage address, and a first virtual address is recorded in the metadata of the first data. There is a first mapping relationship between the first storage address and the first virtual address.
  • the method includes:
  • the storage device migrates the first data from the first storage address to the second storage address, and further, the storage device adjusts the first mapping relationship to a second mapping relationship, where the second mapping relationship is the second storage address and The corresponding relationship of the first virtual address.
  • a first virtual address is recorded in metadata of the first data
  • a first mapping relationship between the first virtual address and the first storage address storing the first data is recorded in the storage device before the first data is migrated
  • the first mapping relationship needs to be modified to the second mapping relationship between the first virtual address and the second storage address, so that there is no need to modify the first data
  • the metadata can also ensure that the first data can still be accessed correctly after the migration. Therefore, this embodiment greatly reduces the number of metadata modification times while ensuring that the data is correctly accessed, thereby greatly improving the metadata. Management efficiency reduces the load on the storage system.
  • the storage address is determined by a storage area ID and a storage offset in the storage area.
  • the first virtual address is jointly identified by the identifier of the first virtual area and the first virtual offset.
  • the first mapping relationship includes a mapping relationship between the identifier of the first virtual area and the first storage area, and a mapping relationship between the first virtual offset and the first storage offset.
  • the second mapping relationship includes a mapping relationship between the identifier of the first virtual area and the second storage area, and a mapping relationship between the first virtual offset and the second storage offset.
  • the following process may also be performed:
  • the storage device migrates all the data in the storage area corresponding to the first storage address to the storage area corresponding to the second storage address, the storage device updates a migration identifier, which is used to identify the migration of the first virtual area. frequency.
  • the above method further includes:
  • the third mapping relationship includes: a mapping relationship between an identifier of the second virtual area and a third storage area, and a mapping between the second virtual offset and a third storage offset relationship.
  • the determination is made in combination with the migration identifier, so that after the data is migrated, the virtual offset of the newly written data will not be different from the virtual offset of the already written data. Duplicates occur to avoid errors when reading and writing data.
  • the method further includes:
  • the determining the first storage address storing the first data according to the first mapping relationship and a first virtual address recorded in metadata of the first data include:
  • the mapping relationship between the first virtual offset and the first storage offset is identified by a mapping table of the first virtual offset and the first storage offset, or by a preset
  • the formula is OK.
  • a second aspect of the embodiments of the present application provides a data migration device, which is applied to a storage device, and the device may be a controller in the storage device or a part of the controller.
  • the first data is stored in a first storage address
  • a first virtual address is recorded in the metadata of the first data
  • This device can implement the functions in the first aspect described above. These functions can be realized by hardware, and can also be implemented by hardware executing corresponding software.
  • the hardware or software includes one or more modules corresponding to the functions described above.
  • the device may include a processing module, which may perform a corresponding function in the above method, for example, a processing module, configured to migrate the first data from the first storage address to the second storage. Address; and,
  • a third aspect of the embodiments of the present application provides a storage device.
  • the storage device includes a controller and at least one hard disk.
  • the controller is configured to manage data stored in the hard disk.
  • the first data is stored at a first storage address.
  • a first virtual address is recorded in the metadata of the data, and there is a first mapping relationship between the first storage address and the first virtual address.
  • the controller is further configured to execute the method according to the first aspect.
  • a fourth aspect of the embodiments of the present application provides a computer program product, where the computer program product includes computer program code, and when the computer program code is executed by a computer, causes the computer to execute the method described in the first aspect.
  • a fifth aspect of the embodiments of the present application provides a computer-readable storage medium.
  • the computer storage medium stores computer instructions.
  • the computer instructions When the computer instructions are executed by a computer, the computer is caused to execute the method described in the first aspect.
  • Figure 1 shows the storage structure of the file BB.doc in the tree structure of the file system
  • FIG. 2 is a schematic flowchart of an embodiment of a data migration method according to an embodiment of the present application
  • Figure 3 shows the storage structure of the file BB.doc in the tree structure of the file system
  • FIG. 4 is a schematic flowchart of writing second data in a storage device
  • FIG. 5 is a schematic flowchart of reading the first data from a storage device
  • FIG. 6 is a schematic block diagram of a data migration device 600 according to an embodiment of the present application.
  • FIG. 7 is a physical block diagram of a storage device 700 according to an embodiment of the present application.
  • the storage device can organize files through a file system, and the file system can be regarded as a method and a data structure of the storage device's operating system for specifying the files on the storage device.
  • data can be divided into data files and metadata, where the data file is a file stored on a storage device.
  • Metadata can be regarded as data describing data.
  • the metadata can hold information such as the owner, size, permissions, timestamp, link, and storage address of the data on the disk of the storage device.
  • the file system can be managed in a tree structure.
  • Figure 1 is the storage structure of the file BB.doc in the tree structure of the file system
  • the first layer of the tree structure, the root is the storage address of the root directory "/”.
  • the content stored in the storage address of the root directory includes metadata of "AA” and information of other files or directories included in the root directory. According to the metadata of "AA” recorded in the second layer, the storage address of "AA" can be obtained.
  • the content stored in the storage address of "AA” the content includes “BB.doc” metadata and information on other files or directories contained in the "AA” directory. According to the metadata of "AA” recorded in the third layer, the storage address of "BB.doc” can be obtained. Furthermore, in the fourth layer of the tree structure, the storage address of "BB.doc” is stored. Content, which is the data corresponding to the file "BB.doc”.
  • the storage address of a file in the file system is recorded by metadata, and is recorded layer by layer according to a tree structure.
  • SMR shingled magnetic recording
  • SSD solid-state drive
  • SMR is sometimes also called SMR disk or SMR hard disk.
  • the storage address of the data is represented by "zone + offset”.
  • the SMR hard disk is composed of multiple zones (areas), and each zone can be regarded as a storage area in the SMR hard disk.
  • offset describes a specific position within the zone, for example (zone3, 48KB) describes a position 48KB away from the starting position of zone3.
  • the offset can be any value greater than 0, as long as it does not exceed the length of the zone.
  • offset can use an integer multiple of 48K bytes as the granularity.
  • the storage address of the data is expressed in the form of "page group + offset".
  • the SSD can include multiple pages. These pages can be divided into multiple page groups. Each page group can be regarded as the SSD. Within a storage area. It can be divided into multiple offsets in each page group. Offset can be in bytes, or it can be in pages. Exemplarily, if the offset is in units of a page, an offset can be expressed as "beginning of the second page and ending at the third page" in a certain page group, or offset only describes "in a certain page group" From the second page as the starting page ", the end page is described with an additional length parameter.
  • the file “BB.doc” is stored on the SMR hard disk of a storage device, and the storage address of the file on the SMR hard disk is "zone1 + offset1", that is, the data of the file is stored in the SMR On the hard disk, zone1 is located in offset1 (in other words, offset1 in zone1 is used as the starting position for storing BB.doc).
  • the storage path of the file in the file system of the storage device is the path described in the corresponding description of FIG. 1.
  • the “zone1 + offset1” is recorded in the third layer.
  • Storage address, the storage address of "AA” is recorded in the second layer, and the storage address of "/" is recorded in the first layer.
  • the "storage address” described in the embodiments of the present application refers to the address used to identify the data storage location at the operating system level.
  • the "storage address” needs to pass through the bottom layer below the operating system. After the software performs further conversion, the actual physical address on the hard disk is obtained.
  • the specific process of the conversion may refer to the method in the prior art, which is not specifically limited in the embodiment of the present application.
  • the storage address can be zone + offset mentioned above.
  • FIG. 2 is a schematic flowchart of an embodiment of a data migration method provided by an embodiment of the present application.
  • the method is executed by the foregoing storage device, and the storage device may be a desktop computer, a notebook computer, a server, a storage controller + hard disk, and the like having data storage.
  • Functional device The storage device includes a controller and at least one hard disk, and the controller is configured to manage data stored in the hard disk.
  • a processor or a combination of a processor and other auxiliary devices, such as a combination of a processor and a memory
  • a storage controller from a functional perspective.
  • the first data may be any data in a hard disk of the storage device.
  • the first data may correspond to a file managed by the file system.
  • the first data is stored at a first storage address on a hard disk
  • the storage address recorded in the metadata of the first data is a first virtual address, and there is a one-to-one correspondence between the first storage address and the first virtual address.
  • a first mapping. relationship For convenience of description, we call it a first mapping. relationship.
  • the first virtual address is not a storage address, but a virtualized storage address, and an identifier recorded in the metadata of the first data.
  • the first virtual address and the first storage address have a first mapping relationship.
  • the first storage address of the first data can be obtained through the first virtual address recorded in the metadata and the first mapping relationship.
  • a virtual address virtualized storage address
  • the storage address recorded in the metadata is replaced with the virtual address recorded in the metadata. Therefore, when the storage address of the data is changed, it is only necessary to update the mapping relationship between the virtual address and the storage address, and the virtual address in the metadata can remain unchanged. Therefore, a large number of metadata updates caused by data migration are avoided, the metadata management efficiency is improved, and the load of the storage system is reduced.
  • the method includes:
  • the storage device migrates the first data from the first storage address to the second storage address.
  • the storage device has determined that the storage area where the first data is located needs to be GC. Further, in this step, the storage device migrates the first data from the first storage address to the second storage address.
  • the above-mentioned “migration” process may specifically be:
  • the first data is written into the second storage address.
  • the first data stored on the first storage address may be subsequently deleted.
  • the storage device adjusts the first mapping relationship to a second mapping relationship, where the second mapping relationship is a correspondence relationship between the second storage address and the first virtual address.
  • the storage device may allocate a first virtual address to the first data, and record the first virtual address in the metadata of the first data, and further After the first data is migrated from the first storage address to the second storage address, the storage device does not need to modify the first virtual address, but only the mapping relationship corresponding to the first data, that is, the first mapping relationship is modified to the second Mapping relations. Because the first storage address of the first data can be obtained through the mapping relationship between the first virtual address recorded in the metadata and the first data, after the first data is migrated, the first virtual address recorded in the metadata is obtained. As well as the modified second mapping relationship, the storage address of the first data can still be obtained. At the same time, in this process, the metadata of the first data is not modified.
  • mapping relationship (for example, the first mapping relationship and the second mapping relationship) may be recorded in the metadata, or may be recorded in another location by itself. As long as it can be read by the processor or memory controller.
  • the first mapping relationship and the second mapping relationship may be stored in the form of a mapping table.
  • a mapping table of data may be stored in the storage device.
  • the first mapping relationship is recorded in the mapping table before data migration
  • the second mapping is recorded in the mapping table after data migration. relationship.
  • Table 1 is an example of the mapping table before the first data migration
  • Table 2 is an example of the mapping table after the first data migration.
  • the above mapping table may be generated in the memory of the storage device.
  • the storage device updates and maintains the above mapping table.
  • the storage device can save the above mapping table to the hard disk of the storage device.
  • the mapping table stored in the hard disk is loaded into the memory, and then reading and writing are continued based on the mapping table, and the mapping table is updated and maintained.
  • a first virtual address is recorded in metadata of the first data, and before the first data is migrated, a first mapping relationship between the first virtual address and the first storage address storing the first data is recorded in the storage device.
  • the first data is migrated from the first storage address to the second storage address, it is only necessary to modify the first mapping relationship to the second mapping relationship between the first virtual address and the second storage address, thereby eliminating the need to modify the first data.
  • this embodiment greatly reduces the number of metadata modification times while ensuring that the data is correctly accessed, thereby greatly improving the metadata. Management efficiency, reducing the load on the storage system.
  • the storage address may be expressed differently.
  • the storage address representation method can be regarded as a "storage area + storage offset" method.
  • the hard disk is divided into multiple storage areas, and within each storage area, multiple storage offsets can be included.
  • a zone represents a storage area
  • an offset within each zone represents a storage offset within the zone.
  • a page group is used to represent a storage area, and an offset within each page group represents a storage offset within the page group.
  • the storage address of the data on the hard disk can be determined through the storage area ID and the storage offset in the storage area.
  • SMR hard disks and a type of hard disks represented by SSDs can write data in a sequential write manner.
  • new data cannot directly overwrite old data, and the space occupied by old data needs to be released through garbage collection for new data.
  • the storage address of the data D can be expressed as: zone1offset1, that is, the storage address of data D is identified by the storage area of zone1 and the storage offset of offset1.
  • zone1offset1 that is, the storage address of data D is identified by the storage area of zone1 and the storage offset of offset1.
  • the virtual address of the data may be jointly identified by the identifier of the virtual area and the virtual offset.
  • the first virtual address of the first data may be jointly identified by the identifier of the first virtual area and the first virtual offset.
  • the virtual address recorded in the metadata of the data D may be: vzone1voffset1.
  • vzone1 represents the identifier of the virtual zone of data D
  • voffset1 represents the virtual offset of data D.
  • vzone1 + voffset1 is highly similar to the way of representing the storage address "zone1 + offset1", and therefore has advantages such as being easily understood by device managers.
  • virtual addresses can also take other forms. For example, you can use hexadecimal numbers, Roman letters, English letters to describe virtual addresses, or a combination of multiple symbols to describe virtual addresses, as long as it can indicate that "zone + offset” has a corresponding relationship.
  • the first 48KB of zone1 is represented by the letter AAA1
  • the second 48KB of zone1 is represented by the letter AA2
  • the third 48KB of zone1 is represented by the letter AA3, and so on.
  • the storage address of the data in the storage device is jointly determined by the storage area ID and the storage offset in the storage area, and the virtual address of the data is collectively identified by the identification of the virtual area and the virtual offset
  • the corresponding mapping of the data The relationship can be expressed by the mapping relationship between the identification of the virtual area and the storage area, and the mapping relationship between the virtual offset and the storage offset.
  • the first mapping relationship corresponding to the first data may include:
  • mapping relationship between the identifier of the first virtual area and the first storage area and the mapping relationship between the first virtual offset and the first storage offset.
  • mapping relationship corresponding to the first data is modified to a second mapping relationship, and the second mapping relationship may include:
  • mapping relationship between the identifier of the first virtual area and the second storage area and the mapping relationship between the first virtual offset and the second storage offset.
  • mapping relationship corresponding to the data can be represented by the mapping relationship between the identification of the virtual area and the storage area, and the mapping relationship between the virtual offset and the storage offset.
  • mapping relationship between the identification of the virtual area and the storage area can be represented by the mapping relationship between the identification of the virtual area and the storage area, and the mapping relationship between the virtual offset and the storage offset.
  • mapping relationship between the identification of the virtual area and the storage area may be saved in the form of a mapping table.
  • the storage device For a piece of data in a storage device, when the data is written to the hard disk of the storage device for the first time, the storage device assigns a virtual area identifier to the data.
  • the virtual area identifier may be a virtual area identifier that already exists in the mapping table. ID for the new virtual area. If it is an existing virtual area identifier, the storage device may directly use the mapping relationship of the existing virtual area identifier. If it is the identifier of the new virtual area, the storage device adds a mapping relationship between the identifier of the new virtual area and the storage area to the mapping table according to the storage area allocated for the data. After the data is migrated, the storage device modifies the storage area corresponding to the identifier of the virtual area into the migrated storage area in the above mapping table. When the data needs to be read, query the above mapping table to get the data storage area.
  • mapping table may be as shown in Table 3 below.
  • mapping table After data D is migrated from zone1 to zone2, the above mapping table is modified into the form shown in Table 4 below.
  • mapping relationship between the virtual offset and the storage offset may be determined by a preset formula.
  • a virtual offset corresponding to the storage offset may be directly calculated by using a preset first formula.
  • a storage offset corresponding to the virtual offset may be calculated by a preset second formula, and further, data is read from a position corresponding to the storage offset.
  • the first formula may be the following formula (1)
  • the second formula may be the following formula (2).
  • Voffset offset + gctimes * sizeof (zone) (1)
  • offset is the storage offset allocated by the storage device for the data to be stored
  • zone is the ID of the storage area allocated by the storage device for the data to be stored
  • gctimes is the number of migrations of the virtual area allocated by the storage device for the data to be stored.
  • each virtual area allocated by the storage device may correspond to a migration identifier, and the migration identifier is used to identify the number of migration times of the virtual area.
  • the migration identifier corresponding to the first virtual area is used to identify Number of migrations of a virtual area.
  • the initial value of the migration identifier can be 0.
  • the storage device can update the migration identifier. For example, the storage device can add 1 to the value of the original migration identifier as the new migration identifier.
  • the virtual offset of the newly written data will not overlap with the virtual offset of the already written data. To avoid errors when reading and writing data.
  • voffset is the virtual offset of data
  • zone is the ID of the storage area allocated by the storage device for the data to be stored.
  • the storage area of the data may be preferably searched according to the identification of the virtual area of the data, and further, the above-mentioned formula (2) is used to determine the storage offset of the data.
  • mapping relationship between the virtual offset and the storage offset may be saved in the form of a mapping table.
  • the storage device For a piece of data in a storage device, when the data is written to the hard disk of the storage device for the first time, the storage device allocates a virtual area identifier, a storage area, and a storage offset for the data. Further, the storage device determines the virtual Offset, and saves the correspondence between the storage offset and the virtual offset in a mapping table. After the data is migrated, after the data is migrated, the storage device modifies the storage offset corresponding to the virtual offset into the migrated storage offset in the mapping table. When you need to read the data, you can query the mapping table to get the storage offset of the data.
  • the storage area allocated by the storage device for the data D is zone1
  • the virtual offset is voffset1
  • the storage offset allocated for the data D is offset1.
  • mapping table is modified into the form shown in Table 6 below.
  • the combined implementation process is:
  • the storage device can determine the virtual offset corresponding to the storage offset of the data according to the above formula (1). At this time, the correspondence between the storage offset and the virtual offset can be regarded as It is represented by the formula (1), and there is no need to record the corresponding relationship between the two in the storage device.
  • the storage offset corresponding to the virtual offset of the data changes. At this time, the correspondence between the virtual offset of the data and the migrated storage offset can be written into the mapping table.
  • the storage device may first determine whether a mapping table of the virtual offset and the storage offset currently exists, and if it exists, further determine whether there is a virtual offset of the data in the mapping table.
  • the storage offset corresponding to the virtual offset is the storage offset of the data. If there is no virtual offset of the data, or there is no mapping table of the virtual offset and the storage offset, you can follow
  • the above formula (1) determines the storage offset corresponding to the virtual offset of the data. At this time, the correspondence between the storage offset and the virtual offset can be regarded as represented by the formula (2).
  • the storage device may store data according to the foregoing optional representation manner, and establish a mapping relationship corresponding to the data.
  • FIG. 3 is the storage structure of the file BB.doc in the tree structure of the file system
  • the first layer of the tree structure that is, the root
  • the mapping relationship between the identifier of the virtual area and the storage area is stored in the first mapping table
  • the mapping relationship between the virtual offset and the storage offset is stored in the second mapping table or expressed by a formula.
  • the content stored in the storage area and the storage address determined by the storage offset is the tree.
  • the content of the second layer of the shape structure includes the identifier of the virtual area of "AA” and the virtual offset, and the mapping relationship between the identifier of the virtual area and the storage area is stored in the first mapping table, and the virtual offset
  • the mapping relationship with the storage offset is stored in the second mapping table or expressed by a formula. Through these two mapping relationships, the storage device can know the storage area and storage offset of "AA”.
  • the content stored in the storage area and the storage address determined by the storage offset is a tree structure.
  • the content of the third layer which includes the identification of the virtual area of "BB.doc" and the virtual offset, the storage device continues to follow the mapping relationship, and finally obtains the storage address of "BB.doc".
  • the metadata recorded in each layer of the tree structure includes the identification of the virtual area and the virtual offset, rather than the storage area ID and storage offset of the data. Therefore, after the data is migrated, The metadata in each layer of the tree structure does not need to be modified, but only the corresponding mapping relationship can be modified.
  • FIG. 4 is a schematic flowchart of writing second data in a storage device. As shown in FIG. 4, the writing process includes:
  • S401 Allocate an identifier of a second virtual area to the second data to be stored, and allocate a third storage area corresponding to a third storage address for storing the second data.
  • the storage device After determining the second data storage, the storage device first allocates a virtual area identifier and a storage area for the second data, that is, the identifier of the second virtual area and the third storage area.
  • the storage device may allocate a virtual area identifier and a storage area for the second data according to a preset policy.
  • the storage device After the identifier of the virtual area and the storage area are allocated for the second data, the storage device further allocates a storage offset for the second data, that is, a third storage offset.
  • the storage device may allocate a storage offset for the second data starting from the first free storage offset of the allocated storage area.
  • the size of the data is not the same. Therefore, the storage offset occupied by one data may be one or more. Therefore, the storage described in the embodiment of the present application
  • the offset can be regarded as the starting position of the data in the storage area.
  • the storage device needs to further combine the length of the data to obtain the complete storage offset of the data, which can also be stored in the metadata of the data at the same time.
  • the storage address allocated by the storage device for the second data is the storage area zone2 + storage offset offset2.
  • the starting position of the second data is a position 48KB from the starting position of zone2.
  • the second data has length information, for example, the length is 20 KB.
  • the storage device After determining the third storage area and the third storage offset for storing the second data, the storage device determines the storage address of the second data, that is, the third storage address. Further, in this step, the storage device The two data are written into a third storage address determined by the third storage area and the third storage offset.
  • the storage device starts from the starting position indicated by the third storage offset, and writes the second data into the starting position and the subsequent storage positions in a sequential writing manner.
  • the storage device records the identifier of the virtual area allocated for the second data and the determined second virtual offset in the metadata of the second data.
  • the storage device further writes length information of the second data into metadata of the second data.
  • the storage device may establish the mapping relationship between the identifier of the second virtual area and the third storage area according to the method described in the corresponding description in Table 3 above, and establish the first relationship according to the method described in the foregoing formula (1) or table (5).
  • the mapping relationship between the second virtual offset and the third storage offset is not repeated here.
  • FIG. 5 is a schematic flowchart of reading the first data from a storage device. As shown in FIG. 5, the process of reading the first data includes:
  • the first storage address when the foregoing first storage address is expressed in a "storage area + storage offset" manner, the first storage address includes a first storage area ID and a first storage offset, and the first virtual address includes a first The identification of the virtual area and the first virtual offset. Furthermore, according to the foregoing embodiment, it can be known that the first mapping relationship corresponding to the first data includes the mapping relationship between the identifier of the first virtual area and the first storage area, and the mapping between the first virtual offset and the first storage offset. relationship.
  • the identifier of the first virtual area corresponding to the first data and the first virtual offset may be obtained first through the metadata of the first data, and further, according to the identifier of the first virtual area and the first storage, Area mapping relationship to obtain a first storage area ID corresponding to the first storage address, and to obtain a first storage area corresponding to the first storage address according to the mapping relationship between the first virtual offset and the first storage offset Stores the offset.
  • the storage device may obtain the first storage area ID corresponding to the first storage address by reading the mapping table.
  • the storage device may obtain the first storage offset corresponding to the first storage address by using the foregoing formula (2).
  • the storage device After obtaining the first storage area of the first data and the first storage offset, the storage device obtains the starting storage position of the first data. The storage device can further combine the length information of the first data to read the complete first data. data.
  • the storage device first addresses a position within zone1 which is 48KB from the starting position of zone1. This position is the starting position of the first data. From this position, 20KB is sequentially read.
  • the read data is the first data. Full content.
  • FIG. 6 is a schematic block diagram of a data migration apparatus 600 according to an embodiment of the present application.
  • the apparatus is applied to a storage device.
  • the device may be a controller of the storage device, or be a part of the controller of the storage device.
  • the first data is stored in a first storage address
  • a first virtual address is recorded in the metadata of the first data
  • the apparatus 600 includes: a processing module 601.
  • a processing module 601 configured to migrate the first data from the first storage address to the second storage address
  • the first mapping relationship is adjusted to a second mapping relationship
  • the second mapping relationship is a correspondence relationship between the second storage address and the first virtual address.
  • the storage address is jointly determined by a storage area ID and a storage offset in the storage area.
  • the first virtual address is jointly identified by an identifier of the first virtual area and a first virtual offset.
  • the first mapping relationship includes a mapping relationship between the identifier of the first virtual area and the first storage area, and a relationship between the first virtual offset and the first storage offset. Mapping relations.
  • the second mapping relationship includes a mapping relationship between the identifier of the first virtual area and the second storage area, and a mapping relationship between the first virtual offset and the second storage offset.
  • processing module 601 is further configured to:
  • a migration identifier is updated, and the migration identifier is used to identify the number of migration times of the first virtual area.
  • processing module 601 is further configured to:
  • An identifier of a second virtual area is allocated to the second data to be stored, and a third storage area corresponding to a third storage address used to store the second data is allocated.
  • a third storage offset in a third storage area is allocated to the second data, and the third storage area and the third storage offset jointly determine the third storage address.
  • the third mapping relationship includes: a mapping relationship between an identifier of the second virtual area and a third storage area, and a mapping between the second virtual offset and a third storage offset relationship.
  • processing module 601 is further configured to:
  • processing module 601 is specifically configured to:
  • a first storage area ID corresponding to the first storage address is obtained.
  • a first storage offset corresponding to the first storage address is obtained.
  • the mapping relationship between the first virtual offset and the first storage offset is identified by a mapping table of the first virtual offset and the first storage offset, or by a preset
  • the formula is OK.
  • FIG. 7 is a physical block diagram of a storage device 700 according to an embodiment of the present application.
  • the storage device includes a controller 701 and at least one hard disk 702.
  • the controller 701 is configured to manage data stored in the hard disk 702.
  • the first data is stored in a first storage address, a first virtual address is recorded in the metadata of the first data, and there is a first mapping relationship between the first storage address and the first virtual address.
  • the controller 701 is also used for:
  • the first mapping relationship is adjusted to a second mapping relationship
  • the second mapping relationship is a correspondence relationship between the second storage address and the first virtual address.
  • the storage address is jointly determined by a storage area ID and a storage offset in the storage area.
  • the first virtual address is jointly identified by an identifier of the first virtual area and a first virtual offset.
  • the first mapping relationship includes a mapping relationship between the identifier of the first virtual area and the first storage area, and a relationship between the first virtual offset and the first storage offset. Mapping relations.
  • the second mapping relationship includes a mapping relationship between the identifier of the first virtual area and the second storage area, and a mapping relationship between the first virtual offset and the second storage offset.
  • controller 701 is further configured to:
  • a migration identifier is updated, and the migration identifier is used to identify the number of migration times of the first virtual area.
  • controller 701 is further configured to:
  • An identifier of a second virtual area is allocated to the second data to be stored, and a third storage area corresponding to a third storage address used to store the second data is allocated.
  • a third storage offset in a third storage area is allocated to the second data, and the third storage area and the third storage offset jointly determine the third storage address.
  • the third mapping relationship includes: a mapping relationship between an identifier of the second virtual area and a third storage area, and a mapping between the second virtual offset and a third storage offset relationship.
  • controller 701 is further configured to:
  • controller 701 is specifically configured to:
  • a first storage area ID corresponding to the first storage address is obtained.
  • a first storage offset corresponding to the first storage address is obtained.
  • the mapping relationship between the first virtual offset and the first storage offset is identified by a mapping table of the first virtual offset and the first storage offset, or by a preset
  • the formula is OK.
  • the computer program product includes one or more computer instructions.
  • the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.
  • the computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be from a website site, computer, server, or data center Transmission by wire (for example, coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (for example, infrared, wireless, microwave, etc.) to another website site, computer, server, or data center.
  • the computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, a data center, and the like that includes one or more available medium integration.
  • the available medium may be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (Solid State Disk (SSD)), and the like.
  • a magnetic medium for example, a floppy disk, a hard disk, a magnetic tape
  • an optical medium for example, a DVD
  • a semiconductor medium for example, a solid state disk (Solid State Disk (SSD)
  • this application may be provided as a method, a system, or a computer program product. Therefore, this application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, this application may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
  • computer-usable storage media including, but not limited to, disk storage, CD-ROM, optical storage, etc.
  • These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to work in a particular manner such that the instructions stored in the computer-readable memory produce a manufactured article including an instruction device, the instructions
  • the device implements the functions specified in one or more flowcharts and / or one or more blocks of the block diagram.
  • These computer program instructions can also be loaded on a computer or other programmable data processing device, so that a series of steps can be performed on the computer or other programmable device to produce a computer-implemented process, which can be executed on the computer or other programmable device.
  • the instructions provide steps for implementing the functions specified in one or more flowcharts and / or one or more blocks of the block diagrams.

Abstract

La présente invention concerne un procédé de migration de données et un appareil et un dispositif de stockage. Des premières données sont stockées dans une première adresse de stockage, une première adresse virtuelle est enregistrée dans des métadonnées des premières données, et il existe une première relation de mappage entre la première adresse de stockage et la première adresse virtuelle. Le procédé comprend les étapes suivantes : le dispositif de stockage fait migrer les premières données de la première adresse de stockage vers une seconde adresse de stockage ; et le dispositif de stockage règle la première relation de mappage à une seconde relation de mappage, la seconde relation de mappage étant une corrélation entre la seconde adresse de stockage et la première adresse virtuelle. Conformément au procédé, une fois que les premières données sont migrées, une adresse de stockage dans les métadonnées des premières données n'a pas besoin d'être modifiée, améliorant ainsi considérablement l'efficacité de gestion de métadonnées.
PCT/CN2019/098061 2018-08-24 2019-07-27 Appareil et procédé de migration de données, et dispositif de stockage WO2020038186A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810971371.2 2018-08-24
CN201810971371.2A CN109324758A (zh) 2018-08-24 2018-08-24 数据迁移方法、装置及存储设备

Publications (1)

Publication Number Publication Date
WO2020038186A1 true WO2020038186A1 (fr) 2020-02-27

Family

ID=65263397

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/098061 WO2020038186A1 (fr) 2018-08-24 2019-07-27 Appareil et procédé de migration de données, et dispositif de stockage

Country Status (2)

Country Link
CN (1) CN109324758A (fr)
WO (1) WO2020038186A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022268108A1 (fr) * 2021-06-22 2022-12-29 华为技术有限公司 Procédé de traitement de données et appareil associé

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109324758A (zh) * 2018-08-24 2019-02-12 华为技术有限公司 数据迁移方法、装置及存储设备
CN110007875B (zh) * 2019-04-15 2022-07-08 苏州浪潮智能科技有限公司 一种有效数据的迁移方法、装置及设备
CN110287169B (zh) * 2019-06-28 2022-02-01 四川长虹电器股份有限公司 一种云应用之间的主数据迁移方法
CN110928490B (zh) * 2019-10-28 2021-08-17 中国科学院上海微系统与信息技术研究所 一种数据的存储方法和装置
CN111078158A (zh) * 2019-12-29 2020-04-28 北京浪潮数据技术有限公司 一种数据迁移方法及相关装置
CN113467722B (zh) * 2021-07-26 2024-04-30 浙江大华技术股份有限公司 一种分布式存储系统的数据迁移方法及装置
CN113741812B (zh) * 2021-08-09 2024-04-12 浙江大华技术股份有限公司 存储器分区的更新方法、装置、电子装置和存储介质
CN117806526A (zh) * 2022-09-23 2024-04-02 华为技术有限公司 数据迁移方法、装置、芯片以及计算机可读存储介质
CN115442388B (zh) * 2022-10-21 2023-01-06 四川省华存智谷科技有限责任公司 用于分布式存储集群利用率100%的扩容方法、装置及系统
CN115718620A (zh) * 2022-11-22 2023-02-28 科东(广州)软件科技有限公司 一种代码程序迁移方法、装置、设备和存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9727485B1 (en) * 2014-11-24 2017-08-08 Pure Storage, Inc. Metadata rewrite and flatten optimization
CN107678981A (zh) * 2017-08-24 2018-02-09 北京盛和大地数据科技有限公司 数据处理方法及装置
CN108021336A (zh) * 2016-10-31 2018-05-11 甲骨文国际公司 用于无缝数据迁移的数据布局方案
CN109324758A (zh) * 2018-08-24 2019-02-12 华为技术有限公司 数据迁移方法、装置及存储设备

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9727485B1 (en) * 2014-11-24 2017-08-08 Pure Storage, Inc. Metadata rewrite and flatten optimization
CN108021336A (zh) * 2016-10-31 2018-05-11 甲骨文国际公司 用于无缝数据迁移的数据布局方案
CN107678981A (zh) * 2017-08-24 2018-02-09 北京盛和大地数据科技有限公司 数据处理方法及装置
CN109324758A (zh) * 2018-08-24 2019-02-12 华为技术有限公司 数据迁移方法、装置及存储设备

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022268108A1 (fr) * 2021-06-22 2022-12-29 华为技术有限公司 Procédé de traitement de données et appareil associé

Also Published As

Publication number Publication date
CN109324758A (zh) 2019-02-12

Similar Documents

Publication Publication Date Title
WO2020038186A1 (fr) Appareil et procédé de migration de données, et dispositif de stockage
WO2017000658A1 (fr) Système de stockage, dispositif de gestion de stockage, dispositif de stockage, dispositif de stockage hybride et procédé de gestion de stockage
US8521949B2 (en) Data deleting method and apparatus
US8924664B2 (en) Logical object deletion
US10282286B2 (en) Address mapping using a data unit type that is variable
EP3958107A1 (fr) Système de stockage, procédé de gestion de mémoire et noeud de gestion
TW201520889A (zh) 混合儲存的控制方法及混合儲存系統
CN110502455B (zh) 数据存储方法及系统
WO2022095346A1 (fr) Procédé de stockage de données de chaîne de blocs, système, dispositif et support de stockage lisible
WO2015018305A1 (fr) Procédé de stockage et système de stockage de mémoire
JPWO2009096180A1 (ja) メモリコントローラ、不揮発性記憶装置、及び、不揮発性記憶システム
US20180275889A1 (en) Append only streams for storing data on a solid state device
EP4012547B1 (fr) Procédé et appareil de mémoire pour une valeur de clé (kv) et dispositif de mémoire
WO2021159711A1 (fr) Procédé et appareil d'accès à une arborescence b+ et support de stockage lisible par ordinateur
EP3974974A1 (fr) Procédé et système de virtualisation pour une mémoire persistante
WO2020192710A1 (fr) Procédé de traitement des objets inutilisés au moyen d'une base de données lsm, d'un disque dur à semi-conducteurs et d'un appareil de stockage
WO2016082559A1 (fr) Procédé d'écriture de données et dispositif de stockage
WO2017000821A1 (fr) Système de mémoire, dispositif de gestion de mémoire, mémoire, mémoire hybride et procédé de gestion de mémoire
JP6531574B2 (ja) ストレージ装置、ストレージ装置制御プログラム及びストレージ装置制御方法
WO2016206070A1 (fr) Procédé de mise à jour de fichier et dispositif de mémorisation
US10768829B2 (en) Opportunistic use of streams for storing data on a solid state device
US20220350779A1 (en) File system cloning method and apparatus
US11132134B2 (en) Flexible over-provisioning of storage space within solid-state storage devices (SSDs)
JP6378364B2 (ja) ファイル管理方法およびファイルシステム
WO2022262381A1 (fr) Procédé et appareil de compression de données

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19851360

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19851360

Country of ref document: EP

Kind code of ref document: A1