WO2017092002A1 - 应用于计算机系统的数据迁移方法和装置、计算机系统 - Google Patents

应用于计算机系统的数据迁移方法和装置、计算机系统 Download PDF

Info

Publication number
WO2017092002A1
WO2017092002A1 PCT/CN2015/096285 CN2015096285W WO2017092002A1 WO 2017092002 A1 WO2017092002 A1 WO 2017092002A1 CN 2015096285 W CN2015096285 W CN 2015096285W WO 2017092002 A1 WO2017092002 A1 WO 2017092002A1
Authority
WO
WIPO (PCT)
Prior art keywords
address
memory device
target data
physical
file
Prior art date
Application number
PCT/CN2015/096285
Other languages
English (en)
French (fr)
Inventor
罗庆超
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to EP15909519.9A priority Critical patent/EP3361385A4/en
Priority to CN201580061176.0A priority patent/CN107111452B/zh
Priority to PCT/CN2015/096285 priority patent/WO2017092002A1/zh
Publication of WO2017092002A1 publication Critical patent/WO2017092002A1/zh
Priority to US15/980,630 priority patent/US10740018B2/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
    • G06F3/0647—Migration mechanisms
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061—Improving I/O performance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/14—Handling requests for interconnection or transfer
    • G06F13/20—Handling requests for interconnection or transfer for access to input/output bus
    • G06F13/28—Handling requests for interconnection or transfer for access to input/output bus using burst mode transfer, e.g. direct memory access DMA, cycle steal
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
    • G06F12/023—Free address space management
    • G06F12/0238—Memory management in non-volatile memory, e.g. resistive RAM or ferroelectric memory
    • G06F12/0246—Memory management in non-volatile memory, e.g. resistive RAM or ferroelectric memory in block erasable memory, e.g. flash memory
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/06—Addressing a physical block of locations, e.g. base addressing, module addressing, memory dedication
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
    • G06F12/10—Address translation
    • G06F12/1081—Address translation for peripheral access to main memory, e.g. direct memory access [DMA]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/13—File access structures, e.g. distributed indices
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0604—Improving or facilitating administration, e.g. storage management
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
    • G06F3/0683—Plurality of storage devices

Definitions

  • the present invention relates to the field of computer technologies, and in particular, to a data migration method and apparatus, and a computer system applied to a computer system.
  • NVM non-volatile memory
  • PM Persistent Memory
  • Persistent storage has a direct random access and non-volatile dual programming model that can be used to provide new storage behavior for file systems or databases using traditional virtual memory management interfaces.
  • the embodiment of the invention provides a data migration method and device and a computer system applied to a computer system, which can reduce the impact on the bandwidth of the memory controller during the data migration process of the memory device and the external storage device.
  • a first aspect of an embodiment of the present invention provides a data migration method applied to a computer system.
  • the method is used to migrate the target data stored in the memory device to the external storage device when the permanent memory is used as the memory device, and the operating system accessing the memory device in the computer system is the volume access mode or the file access mode. in.
  • the target data Before migrating the target data, obtaining a logical address of the target data in the memory device, and calculating a physical address of the target data in the memory device according to the logical address, and then constructing a scatter table, where the scatter table includes the target data in the memory device a physical address; a migration instruction containing information of the scatter table is sent to the DMA engine, and the information of the scatter table includes the target data
  • the physical address in the device is used to instruct the DMA engine to acquire the target data from the memory device according to the physical address, and write the target data to the external storage device.
  • the physical address of the target data in the memory device is calculated according to the logical address of the memory device according to the target data, so that the SGL can be directly generated according to the physical address of the target data in the memory device without using the target data from the memory.
  • the device After the device reads the RAM, it constructs the SGL according to the physical address of the target data in the RAM. Since the operation of the RAM is reduced in this process, the number of accesses to the memory controller is reduced, thereby reducing the impact on the bandwidth of the memory controller during data migration.
  • the logical address of the target data includes a volume identifier, an offset address, and a length
  • the volume identifier is an identifier of the logical volume to which the target data belongs.
  • the logical address of the target data includes a file identifier, an offset address, and a length
  • the file identifier is an identifier of the file to which the target data belongs in the file system.
  • the access mode of the memory device is a volume access mode
  • the physical address of the target data in the memory device is calculated according to the logical address of the target data in the memory device, including: according to the logical address
  • the included volume identifier determines a physical start address of the logical volume to which the target data belongs in the memory device; and calculates a physical start of the target data in the memory device according to the physical start address of the logical volume and the offset address included in the logical address Address; the physical address of the target data is determined according to the physical start address of the target data in the memory device and the length of the target data.
  • the physical address of the target data in the memory device can be calculated simply and quickly.
  • the computer system stores a mapping table, where the mapping table records the physical starting address of the logical volume corresponding to each volume identifier and The length is determined by querying the physical start address corresponding to the volume identifier in the mapping table to determine the physical start address of the logical volume in the memory device.
  • the mapping table By querying the mapping table, the physical starting address of the logical volume in the memory device can be quickly and accurately obtained.
  • the sum of the physical start address of the logical volume to which the target data belongs and the offset address included in the logical address of the target data is calculated.
  • the physical starting address of the target data in the memory device is the sum.
  • the access mode of the memory device is file access
  • the method of calculating the physical address of the target data in the memory device according to the logical address of the target data in the memory device includes: obtaining a physical start address of the file to which the target data belongs in the memory device according to the file identifier included in the logical address; The physical start address of the file and the offset address included in the logical address calculate the physical start address of the target data in the memory device; determine the physicality of the target data according to the physical start address of the target data in the memory device and the length of the target data address.
  • the physical address of the target data in the memory device can be calculated simply and quickly.
  • a mapping table is stored in the computer system, where the mapping table records each file. Identifying the full path information of the corresponding file, where the full path information includes the physical start address of the file; determining the physical location of the file to which the target data belongs in the memory device by querying the physical start address corresponding to the file identifier in the mapping table initial address. The physical start address of the logical volume in the memory device can be quickly and accurately obtained by querying the mapping table.
  • the operating system when a process in the processor accesses the file system according to the logical address of the target data, the operating system first opens the target to be accessed by the process. The target file where the data is located, and then read the open target file. During the process of opening the target file by the operating system, the operating system calls the kernel, and the kernel assigns a file identifier to the target file accessed by the process. During the process of reading the target file, the process can identify the target file to be accessed according to the file identifier. When a process accesses a file system, the process maintains a description of all open files accessed by the process. The file identifier is used as an index to find the description information of the file to which the target data belongs, and the physical start address of the file in the memory device is found in the description information of the file.
  • calculating a sum of a physical start address of a file to which the target data belongs and an offset address included in a logical address of the target data is the sum.
  • a second aspect of an embodiment of the present invention provides a data migration method applied to a computer system.
  • the method is used to migrate the target data stored in the external storage device to the memory device when the permanent memory is used as the memory device, and the operating system accessing the memory device in the computer system is the volume access mode or the file access mode. in.
  • the target data obtain the logical address of the target data to be migrated to the memory device, and calculate the target data to be migrated to the memory device according to the logical address.
  • the scatter table includes a physical address to be migrated to the memory device; the migration instruction including the information of the scatter table is sent to the DMA engine, and the information of the scatter table includes the target data to be The physical address is migrated to the memory device.
  • the migration instruction is used to instruct the DMA engine to obtain the target data from the external storage device, and write the target data to the memory device according to the physical address.
  • the corresponding physical address is calculated according to the logical address to be migrated to the memory device according to the target data, so that the scatter table can be directly constructed according to the physical address, so that the DMA engine can write the target data according to the scatter table.
  • the logical address to be migrated to the memory device includes the volume identifier, the offset address, and the length, and the volume identifier is the identifier of the logical volume to which the target data belongs.
  • the logical address of the target data to be migrated to the memory device includes a file identifier, an offset address, and a length, and the file identifier is a file to which the target data belongs in the file system. logo.
  • the access mode of the memory device is a volume access mode
  • the physical address to be migrated to the memory device according to the logical address of the target data to be migrated to the memory device includes: Determining, according to the volume identifier included in the logical address, a physical start address of the logical volume to which the target data is to be migrated in the memory device; calculating the target data to be migrated according to the physical start address of the logical volume and the offset address included in the logical address The physical starting address to the memory device; determining the physical address of the target data to be migrated to the memory device according to the physical starting address of the target data to be migrated to the memory device and the length of the target data.
  • the physical address of the target data in the memory device can be calculated simply and quickly.
  • the computer system stores a mapping table, where the mapping table records the physical starting address of the logical volume corresponding to each volume identifier and Length; determine the target by querying the physical start address corresponding to the volume identifier in the mapping table The physical start address of the logical volume to which the data is to be migrated in the memory device.
  • the physical start address of the logical volume in the memory device can be quickly and accurately obtained by querying the mapping table.
  • the physical start address of the logical volume to which the target data is to be migrated and the offset address included in the logical address of the target data are calculated.
  • the physical starting address of the target data in the memory device is the sum.
  • the access mode of the memory device is a file access mode
  • the physical address to be migrated to the memory device according to the logical address of the target data to be migrated to the memory device includes: And obtaining, according to the file identifier included in the logical address, a physical start address of the file to be migrated to the target data in the memory device; calculating the target data to be migrated according to the physical start address of the file and the offset address included in the logical address
  • the physical start address in the memory device; the physical address to which the target data is migrated to the memory device is determined according to the physical start address of the target data to be migrated to the memory device and the length of the target data.
  • the physical address of the target data in the memory device can be calculated simply and quickly.
  • a mapping table is stored in the computer system, where the mapping table records each file. Identifying the full path information of the corresponding file, where the full path information includes the physical start address of the file; determining the file to which the target data is migrated in the memory device by querying the physical start address corresponding to the file identifier in the mapping table Physical start address. By querying the mapping table, the physical starting address of the logical volume in the memory device can be quickly and accurately obtained.
  • the operating system when the process in the processor accesses the file system according to the logical address of the target data, the operating system first opens the target to be accessed by the process. The target file to which the data is to be migrated, and then the open target file is written. During the process of opening the target file by the operating system, the operating system calls the kernel, and the kernel assigns a file identifier to the target file accessed by the process. During the process of writing to the target file, the process can identify the target file to be accessed according to the file identifier. When a process accesses a file system, the process maintains a description of all open files accessed by the process. The file identifier is used as an index to find the description information of the file to which the target data is to be migrated, and the physical start address of the file in the memory device is found in the description information of the file.
  • the physical start address of the file to which the target data is to be migrated and the offset address included in the logical address of the target data are calculated. And, the physical start address of the target data to be migrated to the memory device is the sum.
  • an embodiment of the present invention further provides a data migration apparatus applied to a computer system for implementing the method of the above first aspect.
  • the data migration device applied to the computer system comprises: a determination module, a calculation module, a construction module and a transmission module, wherein each module performs a specific function, and each module cooperates with the first aspect to implement the first aspect and the first aspect of the first aspect
  • an embodiment of the present invention provides a computer system including a processor, a memory device, and an external storage device.
  • the memory device may be a permanent memory, and an access mode of the operating system in the computer system to access the memory device is a volume VOLUME access mode or a file FILE access mode.
  • the memory device stores target data, which is data that needs to be migrated from the memory device to the external storage device.
  • the processor can be a central processor.
  • the processor is specifically configured to perform the data migration method applied to the computer system provided in the above first aspect, the first to seventh possible implementation manners of the first aspect.
  • the embodiment of the present invention further provides a computer readable storage medium, where the instruction or software module for implementing the data migration method applied to the computer system described in the above first aspect is stored.
  • an embodiment of the present invention further provides a data migration apparatus applied to a computer system for implementing the method of the second aspect.
  • the data migration device applied to the computer system comprises: a determination module, a calculation module, a construction module and a transmission module, wherein each module performs a specific function, and each module cooperates to implement the first aspect of the second aspect and the second aspect to A data migration method for a computer system provided in a seventh possible implementation.
  • an embodiment of the present invention provides a computer system including a processor, a memory device, and an external storage device.
  • the memory device is a permanent memory
  • an access mode of the operating system in the computer system to access the memory device is a volume VOLUME access mode or a file FILE access mode.
  • the external storage device is configured to store target data, and the target data is data that needs to be migrated from the external storage device to the memory device.
  • the processor can be a central processor.
  • the processor is specifically configured to perform the foregoing second aspect, the first possible implementation manner of the second aspect, and the seventh possible implementation manner A data migration method applied to a computer system.
  • the embodiment of the present invention further provides a computer readable storage medium, where the first possible implementation to the seventh possible implementation manner for implementing the second aspect and the second aspect are stored.
  • the instructions or software modules described in the data migration method for computer systems are described.
  • FIG. 1 is a schematic structural view of an embodiment of a computer system of the present invention
  • FIG. 2 is a schematic structural diagram of an embodiment of a data migration method applied to a computer system according to an embodiment of the present invention
  • FIG. 3 is a schematic flowchart of an embodiment of an address calculation method when a memory device access mode is a volume access mode
  • FIG. 4 is a schematic flowchart of an embodiment of an address calculation method when a memory device access mode is a file access mode
  • FIG. 5 is a schematic structural diagram of another embodiment of a data migration method applied to a computer system according to an embodiment of the present invention.
  • FIG. 6 is a schematic structural diagram of an embodiment of a data migration apparatus applied to a computer system according to the present invention.
  • FIG. 7 is a schematic structural diagram of another embodiment of a data migration apparatus applied to a computer system according to the present invention.
  • DMA is a dual operation between external devices and system memory without the need for processor intervention.
  • the hardware mechanism for data transfer In the process of transferring data by DMA, the source physical address and the target physical address are required to be consecutive. However, in some computer systems, consecutive memory addresses are not necessarily physically contiguous, so DMA transfers are divided into multiple completions.
  • One solution to this problem is to use SGL to describe the physically non-contiguous storage space and then send the SGL information to the DAM engine. After the DMA engine transmits a piece of physical continuous data, it does not need to initiate an interrupt, but transmits the next piece of physically continuous data according to the SGL, and then initiates an interrupt after the transfer is completed.
  • FIG. 1 is a schematic structural view of an embodiment of a computer system of the present invention.
  • the computer system includes a processor 11, a Dynamic Random Access Memory (DRAM) 12, a Non-Volatile Memory (NVM) 13, a Host Bus Adapter (HBA) 14, and an external storage device.
  • DRAM Dynamic Random Access Memory
  • NVM Non-Volatile Memory
  • HBA Host Bus Adapter
  • the processor 11 may be a central processing unit (CPU).
  • the CPU 11 is the core of the computer system, and the CPU 11 can call different software programs in the computer system to implement different functions.
  • the CPU 11 can implement access to the memory device 12 and the external storage device 15. It can be understood that in the embodiment of the present invention, the CPU is only one example of the processor 11.
  • the processor 11 may be an Application Specific Integrated Circuit (ASIC) or one or more integrated circuits configured to implement embodiments of the present invention.
  • ASIC Application Specific Integrated Circuit
  • the DRAM 12 communicates with the processor 11 via a Double Data Rate (DDR) bus.
  • the processor 11 is capable of high speed access to the DRAM 12 for reading or writing to any of the DRAMs 12.
  • the DRAM 12 has the advantage of fast access speed, so DRAM is usually used as main memory.
  • DRAM 12 is used to store various running software, input and output data, and information exchanged with external memory in the operating system.
  • DRAM 12 is volatile, and the information in DRAM 12 will no longer be saved when the computer is powered off.
  • DRAM is a kind of volatile memory, and other random access memory (RAM) can be used as the memory of the computer system in practical applications.
  • a static random access memory (SRAM) can also be used as the memory of the computer system.
  • the NVM 13 communicates with the processor 11 via a DDR bus, and together with the DRAM 12 is the memory of the computer system 100. It should be noted that the memory device described below refers to the one shown in Figure 1. NVM13 in the architecture diagram. Because the new NVMs are fast and non-volatile, these new NVMs can be addressed in Bytes and write data in non-volatile memory in bits. Therefore, it can be used as a memory. Compared with DRAM 12, NVM13 is more non-volatile, so it can better save data. In the embodiment of the present invention, the NVM 13 is configured as a permanent memory, and in this manner, data stored in the NVM 13 is not lost even if the computer system is powered down.
  • the access mode of the operating system in the computer system to access the persistent storage may be a volume access mode or a file access mode.
  • the persistent storage in the volume access mode, is externally presented as multiple logical volumes.
  • the persistent storage In the file access mode, is externally presented as a file system including a plurality of files.
  • the storage medium of the external storage device 15 needs to have a non-volatile characteristic. When the computer is powered off, the data stored in the external storage is still saved. Moreover, the storage capacity of the external storage device is large.
  • the memory as an external storage device may be a disk, a solid state drive (SSD), a hard disk drive (HDD), an optical disk, a storage array, or the like, and other non-volatile storage devices capable of storing data.
  • the external storage device 15 and the HBA 13 are connected by a serial connection to a small computer system interface (English: Serial Attached SCSI, abbreviation: SAS), and the processor and the HBA pass through the peripheral device. Peripheral Component Interconnect Express (PCI-E) connection.
  • the HBA 13 includes a DMA engine for implementing data transfer between external storage devices and memory devices connected to the HBA 13.
  • the address of the target data stored in the memory device is logical as seen by the CPU.
  • the address (that is, the virtual address), not the actual physical address.
  • the CPU cannot generate the SGL according to the logical address of the target data, but first migrates the target data from the memory device to the RAM, and then generates the SGL according to the physical address of the target data in the RAM, thereby enabling the DMA engine to generate the SGL according to the generated data.
  • the computer system architecture diagram provided in FIG. 1 is only an application scenario in which NVM is used as a memory.
  • the memory of the computer system can also include only NVM without including DRAM.
  • the application of the data of the memory device as the permanent memory to the external storage device is mainly involved. Therefore, the structure of the specific computer system is not limited in the embodiment of the present invention, as long as it is included.
  • the memory device is a computer system that is a permanent memory.
  • FIG. 2 is a schematic flowchart diagram of an embodiment of a data migration method applied to a computer system according to the present invention.
  • the data migration method shown in FIG. 2 can be applied to the computer system shown in FIG. 1.
  • the data migration method applied to the computer system in this embodiment includes:
  • the target data is stored in the memory device of the computer system shown in FIG. 1 and needs to be migrated from the memory device to the external storage device.
  • the first address is a logical address of the target data in the memory device.
  • the memory device When the access mode of the memory device is a volume access mode, the memory device presents to the processor a plurality of logical volumes.
  • the first address includes a volume identifier, an offset address, and a length, and the volume identifier is an identifier of a logical volume to which the target data belongs.
  • a mapping table exists in the computer system, and the mapping table records the physical start address and length of the logical volume corresponding to each volume identifier.
  • a 128 GB memory device includes two logical volumes, the IDs of the two logical volumes are 1 and 2, and the physical address of the logical volume with ID 1 is 0 to 64 GB, and the logical volume of ID 2 is The physical address is 64GB to 128GB.
  • the first address of the target data includes the ID of the logical volume, the offset address of 2 GB, and the length of 6 GB.
  • the memory device When the access mode of the memory device is the file access mode, the memory device is presented to the processor is a file system for managing a plurality of files.
  • the first address includes a file identifier, an offset address, and a length
  • the file identifier is an identifier of the file to which the target data belongs in the file system, where the file identifier may be some number or other file descriptor (or file handle). .
  • FIG. 3 is a schematic flowchart of an embodiment of an address calculation method when a memory device access mode is a volume access mode.
  • the address calculation method includes:
  • S31 Determine, according to the volume identifier, a logical volume to which the target data belongs in the memory device. Physical start address.
  • the physical start address of the logical volume is queried by the ID of the logical volume where the target data is located.
  • paddr_start_vol volume_table[v_id], where volume_table[] is a function of the query mapping table, v_id is the ID of the logical volume to which the target data belongs, and paddr_start_vol is the physical starting address of the logical volume to which the target data belongs.
  • the offset address included in the first address is v_addr
  • m_addr is the physical start address of the target data in the memory device
  • m_addr paddr_start_vol+v_addr.
  • the first address of the target data includes the ID of the logical volume 2, the offset address 2GB, and the length 6GB.
  • the length of the target data is known, so the physical address of the target data in the memory device includes a physical start address of 66 GB and a data length of 2 GB.
  • the calculation method of one of the following is exemplified by taking the access mode of the memory device as a file access mode.
  • FIG. 4 is a schematic flowchart of an embodiment of an address calculation method when a memory device access mode is a file access mode.
  • the address calculation method includes:
  • mapping table in the process of constructing a file system in the memory device, a mapping table is stored in the computer system, and the mapping table records full path information of a file corresponding to each file identifier, where the full path information includes the file. Physical start address.
  • the mapping table stored in the piece system obtains the physical starting address of the file to which the target data belongs in the memory device.
  • the operating system when the process in the processor accesses the file system according to the logical address of the target data, the operating system first opens the target file where the target data to be accessed by the process, and then reads the opened target file.
  • the operating system calls the kernel, and the kernel assigns a file identifier to the target file accessed by the process.
  • the file identifier is the unique identifier of the current process identification target file.
  • the process can identify the target file to be accessed according to the file identifier.
  • the process maintains description information of all open files accessed by the process.
  • the file identifier is used as an index to find the description information of the file to which the target data belongs, and the file is found in the description information of the file.
  • the physical start address in the memory device when the process in the processor accesses the file system according to the logical address of the target data, the operating system first opens the target file where the target data to be accessed by the process, and then reads the opened target file.
  • the operating system calls the
  • a physical start address of the target data in the memory device is a sum of a physical start address of the file and the offset address.
  • SGL definition of SGL
  • Unsigned long page_link / / physical start address of the page where the target data in memory is located
  • the physical address of the target data in the memory device is filled into the scatter table. Specifically, the sg_list pointer is obtained, and the SGL information is filled according to the DMA specification, wherein the physical address of the target data in the memory device is filled in the sg_list pointer.
  • the physical address of the target data in the memory device includes 22 GB to 34 GB, 39 GB to 44 GB, and 56 GB to 78 GB, and then the physical addresses are sequentially filled into the sg_list pointer.
  • the processor needs to send a migration instruction to the DMA engine in the HBA to which the external device is connected, wherein the migration instruction includes the target number to be migrated to the external storage device.
  • the address in the address and the information of the SGL, the information of the SGL includes the physical address of the target data in the memory device, and the migration instruction is used to instruct the DMA engine to read the target data from the memory device according to the physical address in the SGL, and according to the target number
  • the target data to be migrated to the external storage device is written to the external storage device.
  • the processor calculates the physical address of the target data in the memory device according to the logical address of the memory device according to the target data, so that the SGL can be directly generated according to the physical address of the target data in the memory device without the target
  • the SGL is built according to the physical address of the target data in the RAM. Since the operation on the RAM is reduced, the number of accesses to the memory controller is reduced, thereby reducing the memory controller. The impact of bandwidth.
  • FIG. 5 is a schematic flowchart diagram of an embodiment of a data migration method applied to a computer system according to the present invention.
  • the data migration method shown in FIG. 5 can also be applied to the computer system shown in FIG. 1.
  • the data migration method applied to the computer system in this embodiment includes:
  • the target data is stored in the external storage device of the computer system shown in FIG. 1 and needs to be migrated from the external storage device to the memory device.
  • the first address is that the target data is to be migrated to the memory device
  • the destination address in the backup Since the memory device is a non-volatile memory, the access mode of the memory device is a volume VOLUME access mode or a file FILE access mode, so the first address is a logical address. For specific explanation, refer to the explanation in step 201, and details are not described herein again.
  • the method for calculating the physical address may refer to the explanation of step 202, and details are not described herein again.
  • SGL definition of SGL
  • Unsigned long page_link / / physical start address of the page in the memory target data to be written
  • Dma_addr_t dma_address //typedef u64 dma_addr_t; DMA address defined by 64bit
  • the physical address to which the target data is to be migrated to the memory device is filled in the scatter table. Specifically, the sg_list pointer is obtained, and the SGL information is filled according to the DMA specification, wherein the physical address to be migrated to the memory device of the target data is filled in the sg_list pointer.
  • the physical address of the target data to be migrated to the memory device includes 22 GB to 34 GB, 39 GB to 44 GB, and 56 GB to 78 GB, and then the physical addresses are sequentially filled in the sg_list pointer.
  • the processor needs to send an indication to the DMA engine in the HBA to which the external storage device is connected, the indication includes the information of the SGL, and the migration instruction is used for Instructs the DMA engine to read the target data from the external storage device and write the target data to the memory device according to the physical address in the SGL.
  • the external storage is set.
  • the data When the data is migrated to the memory device, the data needs to be first transferred to the RAM according to the SGL, and then written from the RAM to the memory device.
  • the logical address to be migrated to the memory device according to the target data Calculating the corresponding physical address, so that the scatter table can be constructed according to the physical address, so that the DMA engine can directly write the target data to the memory device according to the scatter table, thereby reducing the operation on the RAM, thereby reducing the memory controller The number of visits, which in turn reduces the impact on the bandwidth of the memory controller.
  • the access mode of the memory device is a volume access mode or a file access mode.
  • the address of the target data in the memory device is a logical address. Therefore, after the processor sends the migration instruction to the DMA engine, the processor also needs to calculate a corresponding logical address according to the target data in the physical address of the target memory device, and the processor calculates the corresponding physical address in the memory device according to the target data.
  • the method of logical address is described. Specifically, the access mode of the memory device is taken as an example of a volume access mode.
  • a volume identifier of the logical volume is determined according to a physical start address of the logical volume to which the target data is to be migrated. Specifically, in the mapping table described in step 201, the ID of the logical volume to which the target data is to be migrated is queried by the physical starting address of the target data to be migrated to the memory device.
  • an offset address is calculated according to the volume identifier of the logical volume and the physical start address of the target data in the memory device.
  • the physical start address of the logical volume can be queried in the mapping table. Since the physical start address of the target data in the memory device is the sum of the physical start address and the offset address of the logical volume, the offset address is the physical state of the target data in the memory device. The difference between the start address minus the physical start address of the logical volume.
  • the method for calculating the logical address of the target data in the memory device may refer to the above example, but only when querying the file identifier of the file to be migrated to the target data.
  • the query is made in a mapping table used to record the file system.
  • the data migration method applied to the computer system in the embodiment of the present invention is described above.
  • the data migration apparatus applied to the computer system in the embodiment of the present invention is described below, wherein the data migration apparatus applied to the computer system is used to execute the above-described data migration method applied to the computer system.
  • FIG. 6 is a schematic structural diagram of an embodiment of a data migration apparatus applied to a computer system according to the present invention.
  • the data migration apparatus 600 applied to the computer system includes:
  • a determining module 601 configured to determine a first address of the target data, where the target data is data that needs to be migrated from the memory device to the external storage device, where the first address is a logical address of the target data in the memory device
  • the memory device is a permanent memory PM, and an access mode of the operating system in the computer system accessing the memory device is a volume VOLUME access mode or a file FILE access mode;
  • the calculating module 602 is configured to calculate, according to the first address, a physical address of the target data in the memory device;
  • a building module 603 configured to construct a scatter table, where the scatter table includes a physical address of the target data in the memory device;
  • a sending module 604 configured to send a migration instruction to the direct memory access DMA engine, where the migration instruction includes information of the scatter table, where the migration instruction is used to instruct the DMA engine to use the memory according to the physical address
  • the target data is acquired in the device, and the target data is written to the external storage device.
  • the first address when the access mode of the memory device is a volume access mode, the first address includes a volume identifier, an offset address, and a length, where the volume identifier is an identifier of a logical volume to which the target data belongs.
  • the calculation module is specifically used to:
  • Determining a physical address of the target data according to a physical start address of the target data in the memory device and a length of the target data.
  • the access mode of the memory device is a file access mode
  • the first address includes a file identifier, an offset address, and a length
  • the file identifier is a file in which the target data belongs. Identification in the system; the calculation module is specifically used to:
  • Determining a physical address of the target data according to a physical start address of the target data in the memory device and a length of the target data.
  • FIG. 7 is a schematic structural diagram of an embodiment of a data transmission apparatus according to the present invention.
  • the data transmission device 700 includes:
  • the determining module 701 is configured to determine a first address of the target data to be migrated, where the target data is data that needs to be migrated from the external storage device to the memory device, where the first address is an address in the memory device,
  • the first address is a logical address
  • the memory device is a permanent memory
  • an access mode of the operating system in the computer system accessing the memory device is a volume VOLUME access mode or a file FILE access mode;
  • the calculating module 702 is configured to calculate, according to the first address, a physical address that the target data is to be migrated to the memory device;
  • the sending module 704 sends a migration instruction to the DMA engine, where the migration instruction includes information of the scatter table, the migration instruction is used to instruct the DMA engine to write the target data to the memory device according to the physical address in.
  • the first address when the access mode of the memory device is a volume access mode, the first address includes a volume identifier, an offset address, and a length, where the volume identifier is an identifier of a logical volume to which the target data is to be migrated.
  • the computing module is specifically configured to:
  • the access mode of the memory device is a file access mode
  • the first address includes a file identifier, an offset address, and a length, where the file identifier is in a file system to which the target data is to be migrated.
  • the calculation module is specifically used to:
  • the data migration device applied to the computer system in the embodiment of the present invention is described above from the perspective of the unitized functional entity.
  • the data migration device applied to the computer system in the embodiment of the present invention is described below from the perspective of hardware processing.
  • a specific embodiment of the data migration apparatus applied to the computer system in the present invention is a processor in the computer system shown in FIG. 2, which is used to construct a scatter table by using the method in the embodiment shown in FIG. 2 to FIG. And sending a migration instruction to the DMA engine to instruct the DMA engine to migrate data in the external storage device in the computer system to the memory device according to the migration instruction, or instruct the DMA engine to data the memory device in the computer system according to the migration instruction. Migrate to an external storage device.
  • each functional module in each embodiment of the present invention may be integrated into one processing unit, or each module may exist physically separately, or two or more modules may be integrated into one module.
  • the above integrated modules can be implemented in the form of hardware or in the form of software functional modules.
  • the integrated modules if implemented in the form of software functional modules and sold or used as separate products, may be stored in a computer readable storage medium. Based on this understanding, this issue
  • the technical solution of the present invention, or all or part of the technical solution, may be embodied in the form of a software product stored in a storage medium, including a plurality of instructions.
  • a computer device (which may be a personal computer, server, or network device, etc.) is caused to perform all or part of the steps of the methods described in various embodiments of the present invention.
  • the foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

一种应用于计算机系统的数据迁移方法和装置。包括：确定目标数据的第一地址，其中，目标数据为需要从内存设备迁移到外存设备的数据，第一地址为目标数据在内存设备中的逻辑地址，内存设备为永久性存储器PM，计算机系统中的操作系统访问内存设备的访问方式为卷VOLUME访问方式或文件FILE访问方式；根据第一地址计算目标数据在内存设备中的物理地址；构建散布表，散布表包括目标数据在内存设备中的物理地址；向直接内存存取DMA引擎发送迁移指令，迁移指令中包括散布表的信息，迁移指令用于指示DMA引擎根据物理地址从内存设备中获取目标数据，并将目标数据写到外存设备中。

Description

应用于计算机系统的数据迁移方法和装置、计算机系统 技术领域
本发明涉及计算机技术领域，尤其涉及一种应用于计算机系统的数据迁移方法和装置、计算机系统。
背景技术
存储器大致可分为两类：易失存储器和非易失存储器(NonVolatile Memory，NVM)。NVM包括多种模型，其中一种是永久性存储器(Persistent Memory，缩写：PM)。永久性存储器具有直接随机访问与非易失性双重特性的编程模型，可以采用传统虚拟内存管理接口，为文件系统或者数据库提供新的存储行为。
现有技术中，将作为内存设备的永久性存储器中的数据迁移到外存储设备中时，需要先将数据从永久性存储器中读取到随机存取存储器(Random-Access Memory，RAM)后，然后根据数据在RAM中的物理地址构建散布表(Scatter Gather List，SGL)。直接内存存取(Directional Memory Access，DMA)引擎根据该散布表来从RAM中读取数据并写入到硬盘中。在每一次对永久性存储器和随机存取存储器的操作时都需要访问内存控制器，因此上述数据传输流程中总共有3次对内存控制器的访问，大大影响了内存控制器的带宽。
发明内容
本发明实施例提供了一种应用于计算机系统的数据迁移方法和装置、计算机系统，能够在将内存设备与外存设备的数据迁移过程中，减少对内存控制器的带宽的影响。
本发明实施例第一方面提供了一种应用于计算机系统的数据迁移方法。该方法用于在永久性存储器作为内存设备，且计算机系统中的操作系统访问该内存设备的访问方式为卷访问方式或文件访问方式时，将存储在内存设备中的目标数据迁移到外存设备中。在迁移目标数据之前，获取该目标数据在内存设备中的逻辑地址，并根据该逻辑地址计算目标数据在内存设备中的物理地址，然后构建散布表，该散布表包括目标数据在内存设备中的物理地址；将包含有散布表的信息的迁移指令发送给DMA引擎，该散布表的信息包括目标数据在内 存设备中的物理地址，该迁移指令用于指示DMA引擎根据该物理地址从内存设备中获取该目标数据，并将目标数据写到外存设备中。
本方法中，通过根据目标数据在内存设备的逻辑地址计算目标数据在内存设备中的物理地址，这样，可直接根据该目标数据在内存设备中的物理地址生成SGL，而无需将目标数据从内存设备读取到RAM后，再根据目标数据在RAM中的物理地址构建SGL。由于在这个过程中减少了对RAM的操作，因此减少了对内存控制器的访问次数，进而在数据迁移过程中减少了对内存控制器的带宽的影响。
其中，当内存设备的访问方式为卷访问方式时，目标数据的逻辑地址包括卷标识、偏移地址及长度，该卷标识为该目标数据所属的逻辑卷的标识。
其中，当内存设备的访问方式为文件访问方式时，目标数据的逻辑地址包括文件标识、偏移地址及长度，该文件标识为该目标数据所属的文件在文件系统中的标识。
在第一方面的第一种可能的实现方式中，内存设备的访问方式为卷访问方式，根据目标数据在内存设备中的逻辑地址计算目标数据在内存设备中的物理地址包括：根据逻辑地址所包括的卷标识确定目标数据所属的逻辑卷在内存设备中的物理起始地址；根据该逻辑卷的物理起始地址和逻辑地址所包括的偏移地址计算目标数据在内存设备中的物理起始地址；根据目标数据在内存设备中的物理起始地址以及目标数据的长度确定目标数据的物理地址。本实现方式中能够简单快捷地计算出目标数据在内存设备中的物理地址。
根据第一方面的第一种可能的实现方式，在第二种可能的实现方式中，计算机系统中存储有映射表，该映射表记录了每个卷标识对应的逻辑卷的物理起始地址和长度；通过在映射表中查询卷标识所对应的物理起始地址来确定目标数据所述的逻辑卷在内存设备中的物理起始地址。通过查询映射表的的方式能够快速准确获取到逻辑卷在内存设备中的物理起始地址。
根据第一方面的第一种可能的实现方式，在第三种可能的实现方式中，计算目标数据所属的逻辑卷的物理起始地址和目标数据的逻辑地址中所包括的偏移地址的和，目标数据在内存设备中的物理起始地址为该和。
在第一方面的第四种可能的实现方式中，内存设备的访问方式为文件访问 方式，根据目标数据在内存设备中的逻辑地址计算目标数据在内存设备中的物理地址包括：根据逻辑地址所包括的文件标识获取目标数据所属的文件在内存设备中的物理起始地址；根据该文件的物理起始地址和逻辑地址所包括的偏移地址计算目标数据在内存设备中的物理起始地址；根据目标数据在内存设备中的物理起始地址以及目标数据的长度确定目标数据的物理地址。本实现方式中能够简单快捷地计算出目标数据在内存设备中的物理地址。
根据第一方面的第四种可能的实现方式，在第五种可能的实现方式中，在内存设备中构建文件系统的过程中，计算机系统中存储有映射表，该映射表记录了每个文件标识对应的文件的全路径信息，该全路径信息包括该文件的物理起始地址；通过在映射表中查询文件标识所对应的物理起始地址来确定目标数据所属的文件在内存设备中的物理起始地址。通过查询映射表的方式能够快速准确获取到逻辑卷在内存设备中的物理起始地址。
根据第一方面的第四种可能的实现方式，在第六种可能的实现方式中，当处理器中的进程根据目标数据的逻辑地址访问文件系统时，操作系统会先打开进程要访问的目标数据所在的目标文件，然后再对打开的目标文件进行读操作。在操作系统打开目标文件的过程中，操作系统会调用内核，内核会为该进程访问的目标文件分配一个文件标识。在对目标文件进行读操作的过程中，该进程可以根据该文件标识识别要访问的目标文件。当进程访问文件系统时，进程中维护有该进程访问的所有打开的文件的描述信息。以文件标识为索引可以找到目标数据所属的文件的描述信息，并在该文件的描述信息中找到该文件在所述内存设备中的物理起始地址。
根据第一方面的第四种可能的实现方式，在第七种可能的实现方式中，计算目标数据所属的文件的物理起始地址和目标数据的逻辑地址中所包括的偏移地址的和，目标数据在内存设备中的物理起始地址为该和。
本发明实施例第二方面提供了一种应用于计算机系统的数据迁移方法。该方法用于在永久性存储器作为内存设备，且计算机系统中的操作系统访问该内存设备的访问方式为卷访问方式或文件访问方式时，将存储在外存设备中的目标数据迁移到该内存设备中。在迁移目标数据之前，获取该目标数据待迁移到内存设备中的逻辑地址，并根据该逻辑地址计算目标数据待迁移到内存设备 中的物理地址，然后构建散布表，该散布表包括目标数据待迁移到内存设备中的物理地址；将包含有散布表的信息的迁移指令发送给DMA引擎，该散布表的信息包括目标数据待迁移到内存设备中的物理地址，该迁移指令用于指示DMA引擎从外存设备中获取该目标数据，并根据该物理地址将目标数据写到内存设备中。
本方法中，通过根据目标数据待迁移到所述内存设备中的逻辑地址计算对应的物理地址，这样可以直接根据该物理地址构建散布表，使得DMA引擎可以根据该散布表将目标数据写入到内存设备中，无需像背景技术中先将外存设备的数据迁移到RAM中再根据数据在RAM中的物理地址构建SGL，由于减少了对RAM的操作，因此减少了对内存控制器的访问次数，进而减少了对内存控制器的带宽的影响。
其中，当内存设备的访问方式为卷访问方式时，目标数据待迁移到内存设备中的逻辑地址包括卷标识、偏移地址及长度，该卷标识为该目标数据所属的逻辑卷的标识。
其中，当内存设备的访问方式为文件访问方式时，目标数据待迁移到内存设备中的逻辑地址包括文件标识、偏移地址及长度，该文件标识为该目标数据所属的文件在文件系统中的标识。
在第二方面的第一种可能的实现方式中，内存设备的访问方式为卷访问方式，根据目标数据待迁移到内存设备中的逻辑地址计算目标数据待迁移到内存设备中的物理地址包括：根据逻辑地址所包括的卷标识确定目标数据待迁移到的逻辑卷在内存设备中的物理起始地址；根据该逻辑卷的物理起始地址和逻辑地址所包括的偏移地址计算目标数据待迁移到内存设备中的物理起始地址；根据目标数据待迁移到内存设备中的物理起始地址以及目标数据的长度确定目标数据待迁移到内存设备中的物理地址。本实现方式中能够简单快捷地计算出目标数据在内存设备中的物理地址。
根据第二方面的第一种可能的实现方式，在第二种可能的实现方式中，计算机系统中存储有映射表，该映射表记录了每个卷标识对应的逻辑卷的物理起始地址和长度；通过在映射表中查询卷标识所对应的物理起始地址来确定目标 数据待迁移到的逻辑卷在内存设备中的物理起始地址。通过查询映射表的方式能够快速准确获取到逻辑卷在内存设备中的物理起始地址。
根据第二方面的第一种可能的实现方式，在第三种可能的实现方式中，计算目标数据待迁移到的逻辑卷的物理起始地址和目标数据的逻辑地址中所包括的偏移地址的和，目标数据在内存设备中的物理起始地址为该和。
在第二方面的第四种可能的实现方式中，内存设备的访问方式为文件访问方式，根据目标数据待迁移到内存设备中的逻辑地址计算目标数据待迁移到内存设备中的物理地址包括：根据该逻辑地址所包括的文件标识获取目标数据待迁移到的文件在内存设备中的物理起始地址；根据该文件的物理起始地址和逻辑地址所包括的偏移地址计算目标数据待迁移到内存设备中的物理起始地址；根据目标数据待迁移到内存设备中的物理起始地址以及目标数据的长度确定目标数据迁移到内存设备中的物理地址。本实现方式中能够简单快捷地计算出目标数据在内存设备中的物理地址。
根据第二方面的第四种可能的实现方式，在第五种可能的实现方式中，在内存设备中构建文件系统的过程中，计算机系统中存储有映射表，该映射表记录了每个文件标识对应的文件的全路径信息，该全路径信息包括该文件的物理起始地址；通过在映射表中查询文件标识所对应的物理起始地址来确定目标数据迁移到的文件在内存设备中的物理起始地址。通过查询映射表的的方式能够快速准确获取到逻辑卷在内存设备中的物理起始地址。
根据第二方面的第四种可能的实现方式，在第六种可能的实现方式中，当处理器中的进程根据目标数据的逻辑地址访问文件系统时，操作系统会先打开进程要访问的目标数据待迁移到的目标文件，然后再对打开的目标文件进行写操作。在操作系统打开目标文件的过程中，操作系统会调用内核，内核会为该进程访问的目标文件分配一个文件标识。在对目标文件进行写操作的过程中，该进程可以根据该文件标识识别要访问的目标文件。当进程访问文件系统时，进程中维护有该进程访问的所有打开的文件的描述信息。以文件标识为索引可以找到目标数据待迁移到的文件的描述信息，并在该文件的描述信息中找到该文件在内存设备中的物理起始地址。
根据第二方面的第四种可能的实现方式，在第七种可能的实现方式中，计算目标数据待迁移到的文件的物理起始地址和目标数据的逻辑地址中所包括的偏移地址的和，目标数据待迁移到内存设备中的物理起始地址为该和。
第三方面，本发明实施例还提供了用于实现上述第一方面的方法的应用于计算机系统的数据迁移装置。该应用于计算机系统的数据迁移装置包括：确定模块、计算模块、构建模块和发送模块，其中每个模块完成特定的功能，各个模块共同配合用于实现上述第一方面以及第一方面的第一种至第七种可能的实现方式中所提供的应用于计算机系统的数据迁移方法。
第四方面，本发明实施例提供了一种计算机系统，该计算机系统包括处理器、内存设备和外存设备。该内存设备可以是永久性存储器，所述计算机系统中的操作系统访问所述内存设备的访问方式为卷VOLUME访问方式或文件FILE访问方式。该内存设备中存储有目标数据，所述目标数据为需要从内存设备迁移到外存设备的数据。处理器可以为中央处理器。该处理器具体用于执行上述第一方面、第一方面的第一种至第七种可能的实现方式中所提供的应用于计算机系统的数据迁移方法。
第五方面，本发明实施例还提供了一种计算机可读存储介质，该存储介质中存储有用于实现上述第一方面中所描述的应用于计算机系统的数据迁移方法的指令或软件模块。
第六方面，本发明实施例还提供了用于实现上述第二方面的方法的应用于计算机系统的数据迁移装置。该应用于计算机系统的数据迁移装置包括：确定模块、计算模块、构建模块和发送模块，其中每个模块完成特定的功能，各个模块共同配合实现上述第二方面、第二方面的第一种至第七种可能的实现方式中所提供的应用于计算机系统的数据迁移方法。
第七方面，本发明实施例提供了一种计算机系统，该计算机系统包括处理器、内存设备和外存设备。该内存设备永久性存储器，所述计算机系统中的操作系统访问所述内存设备的访问方式为卷VOLUME访问方式或文件FILE访问方式。该外存设备用于存储目标数据，所述目标数据为需要从外存设备迁移到内存设备的数据。处理器可以为中央处理器。该处理器具体用于执行上述第二方面、第二方面的第一种可能的实现方式至第七种可能的实现方式中所提供 的应用于计算机系统的数据迁移方法。
第八方面，本发明实施例还提供一种计算机可读存储介质，该存储介质中存储有用于实现上述第二方面、第二方面的第一种可能的实现方式至第七种可能的实现方式中所描述的应用于计算机系统的数据迁移方法的指令或软件模块。
附图说明
为了更清楚地说明本发明实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍。
图1为本发明的计算机系统的一个实施例的结构示意图；
图2为本发明实施例的应用于计算机系统的数据迁移方法的一个实施例的结构示意图；
图3为内存设备的访问方式为卷访问方式时的地址计算方法的一个实施例的流程示意图；
图4为内存设备的访问方式为文件访问方式时的地址计算方法的一个实施例的流程示意图；
图5为本发明实施例的应用于计算机系统的数据迁移方法的另一个实施例的结构示意图；
图6为本发明的应用于计算机系统的数据迁移装置的一个实施例的结构示意图；
图7为本发明的应用于计算机系统的数据迁移装置的另一个实施例的结构示意图。
具体实施方式
为了使本技术领域的人员更好地理解本发明方案，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚地描述，显然，所描述的实施例仅仅是本发明一部分的实施例，而不是全部的实施例。
为便于理解，下面对本发明中的一些概念进行解释。
DMA是一种无需处理器的参与就可以让外在设备和系统内存之间进行双 向数据传输的硬件机制。在DMA传输数据的过程中，要求源物理地址和目标物理地址分别是连续的。但在某些计算机体系中，连续的存储器地址在物理上不一定是连续的，因此DMA传输要分成多次完成。针对这个问题，一种解决方法是使用SGL描述物理上不连续的存储空间，然后把该SGL的信息发送至DAM引擎。DMA引擎在传输完一块物理连续的数据后，不用发起中断，而是根据SGL来传输下一块物理上连续的数据，直到传输完毕后再发起一次中断。
如图1所示，图1为本发明的计算机系统的一个实施例的结构示意图。计算机系统包括处理器11、动态随机存储器(Dynamic Random Access Memory，DRAM)12、非易失性存储器(Non-Volatile Memory,NVM)13、主机总线适配器(Host Bus Adapter，HBA)14和外存设备15。
其中，处理器11可以是中央处理器(Central Processing Unit，CPU)。CPU11是计算机系统的核心，CPU11可以调用计算机系统中不同的软件程序实现不同的功能。例如，CPU11能够实现对内存设备12和外存设备15的访问。可以理解的是，在本发明实施例中，CPU仅仅是处理器11的一个示例。除了CPU外，处理器11还可以是其他特定集成电路(Application Specific Integrated Circuit，ASIC)，或者是被配置成实施本发明实施例的一个或多个集成电路。
DRAM 12通过双倍速率同步(Double Data Rate，DDR)总线和处理器11进行通信。处理器11能够高速访问DRAM 12，对DRAM 12中的任一存储单元进行读或写操作。DRAM 12具有访问速度快的优点，因此通常DRAM作为主内存使用。通常DRAM 12用来存放操作系统中各种正在运行的软件、输入和输出数据以及与外存交换的信息等。然而，DRAM 12是易失性的，当计算机关闭电源后，DRAM 12中的信息将不再保存。本领域技术人员可以知道，DRAM是易失性内存的一种，实际应用中还可以采用其他的随机存储器(Random Access Memory，RAM)作为计算机系统的内存。例如，还可以采用静态随机存储器(Static Random Access Memory，SRAM)作为计算机系统的内存。
NVM13通过DDR总线和处理器11进行通信，与DRAM12共同作为计算机系统100的内存。需注意的是，下文中所描述的内存设备指的是图1所示 架构图中的NVM13。由于新型NVM都具有访问速度快且具有非易失性的特点，并且，这些新型NVM能够按字节(Byte)寻址，将数据以位(bit)为单位写入非易失性存储器中，因而能够作为内存使用。与DRAM 12相比，由于NVM13具有非易失性的特点，从而能够更好的保存数据。在本发明实施例中，NVM 13被配置为永久性存储器，根据这种方式，即使在计算机系统掉电的情况下存储于NVM13中的数据也不会丢失。在本发明实施例中，计算机系统中的操作系统访问该永久性存储器的访问方式可以为卷(volume)访问方式或文件(file)访问方式。其中，在卷访问方式中，永久性存储器对外展现为多个逻辑卷。在文件访问方式中，永久性存储器对外展现为包括多个文件的文件系统。
外存设备15的存储介质需要具有非易失性的特点，当计算机关闭电源后，存储于外存的数据仍然会被保存。并且，外存设备的存储容量较大。作为外存设备的存储器可以是磁盘、固态硬盘(Solid State Drives，SSD)、机械硬盘(Hard Disk Drive，HDD)、光盘、存储阵列等其他能够存储数据的非易失性的存储设备。外存设备15和HBA 13通过串行连接小型计算机系统接口(英文：Small Computer System Interface；缩写：SCSI)接口(英文：Serial Attached SCSI，缩写：SAS)链路连接，处理器和HBA通过外设互联标准总线(Peripheral Component Interconnect Express，PCI-E)连接。HBA13包括DMA引擎，该DMA引擎用于实现数据在与该HBA13连接的外存设备和内存设备之间的传输。
实际应用中，当内存设备为永久性存储器时，在通过DMA的方式将内存设备中的数据迁移到外存设备的情况下，由于CPU所看到存储在内存设备中的目标数据的地址为逻辑地址(也即虚拟地址)，而不是实际的物理地址。那么CPU不能根据目标数据的逻辑地址生成SGL，而是先将目标数据从内存设备中迁移到RAM中，再根据该目标数据在RAM中的物理地址生成SGL，从而使DMA引擎能够根据生成的SGL将数据从非易失的内存设备迁移到外存设备中。
可以理解的是，图1提供的计算机系统架构图仅仅是NVM作为内存的一种应用场景。实际应用中，计算机系统的内存也可以只包括NVM而不包括 DRAM。由于在本发明实施例中，主要涉及将作为永久性存储器的内存设备的数据迁移到外存设备的应用场景，因此，在本发明实施例中不对具体的计算机系统的结构进行限定，只要是包括内存设备为永久性存储器的计算机系统即可。
请参阅图2，图2为本发明的应用于计算机系统的数据迁移方法的一个实施例的流程示意图。图2所示的数据迁移方法可以应用于图1所示的计算机系统中。如图2所示，本实施例中的应用于计算机系统的数据迁移方法包括：
201、确定目标数据的第一地址。
本实施例中，目标数据存储在图1所示计算机系统的内存设备中，并需要从内存设备迁移到外存设备中。第一地址为所述目标数据在内存设备中的逻辑地址。
当所述内存设备的访问方式为卷访问方式时，内存设备呈现给处理器的是多个逻辑卷。第一地址包括卷标识、偏移地址及长度，所述卷标识为所述目标数据所属的逻辑卷的标识。计算机系统中存有映射表，该映射表记录了每个卷标识对应的逻辑卷的物理起始地址和长度。
具体举例来说，一个128GB的内存设备包括两个逻辑卷，该两个逻辑卷的ID分别为1和2，ID为1的逻辑卷的物理地址为0～64GB，ID为2的逻辑卷的物理地址为64GB～128GB。当目标数据存储在该内存设备中物理地址为66GB～72GB处时，该目标数据的第一地址包括逻辑卷的ID2、偏移地址2GB和长度6GB。
当所述内存设备的访问方式为文件访问方式时，内存设备呈现给处理器的是用于管理多个文件的文件系统。第一地址包括文件标识、偏移地址及长度，所述文件标识为所述目标数据所属的文件在文件系统中的标识，其中，文件标识可以是一些数字或其他文件描述符(或者文件句柄)。
202、根据所述第一地址计算所述目标数据在所述内存设备中的物理地址。
下面以所述内存设备的访问方式为卷访问方式为例对其中一种的计算方法进行举例说明。如图3所示，图3为内存设备的访问方式为卷访问方式时的地址计算方法的一个实施例的流程示意图。该地址计算方法包括：
S31、根据所述卷标识确定所述目标数据所属的逻辑卷在所述内存设备中 的物理起始地址。
具体的，在步骤201中所描述的映射表中通过目标数据所在的逻辑卷的ID查询该逻辑卷的物理起始地址。例如，paddr_start_vol＝volume_table[v_id]，其中，volume_table[]为查询映射表的函数，v_id为目标数据所属的逻辑卷的ID，paddr_start_vol为目标数据所属的逻辑卷的物理起始地址。
S32、根据所述逻辑卷的物理起始地址和所述偏移地址计算所述目标数据在所述内存设备中的物理起始地址。
具体的，第一地址中包括的偏移地址为v_addr，m_addr为目标数据在所述内存设备中的物理起始地址，那么m_addr＝paddr_start_vol+v_addr。
S33、根据所述目标数据在所述内存设备中的物理起始地址以及所述目标数据的长度确定所述目标数据的物理地址。
计算出目标数据在所述内存设备中的物理起始地址m_addr后，确定所述目标数据在内存设备中的物理地址包括m_addr和所述目标数据的长度。
具体举例来说，目标数据的第一地址包括逻辑卷的ID 2、偏移地址2GB和长度6GB。那么可根据该逻辑卷的ID2在步骤201中所举例的内存设备对应的映射表中查询到该ID对应的逻辑卷的物理起始地址为64GB。由于第一地址中包括目标数据的偏移地址为2GB，那么可计算出目标数据在所述内存设备中的物理起始地址为64GB+2GB＝66GB。目标数据的长度是已知的，因此目标数据在所述内存设备中的的物理地址包括物理起始地址66GB和数据长度2GB。
下面以所述内存设备的访问方式为文件访问方式为例对其中一种的计算方法进行举例说明。
如图4所示，图4为内存设备的访问方式为文件访问方式时的地址计算方法的一个实施例的流程示意图。该地址计算方法包括：
S41、根据所述文件标识获取所述目标数据所属的文件在所述内存设备中的物理起始地址。
实际应用中，在所述内存设备中构建文件系统的过程中，计算机系统中存有映射表，该映射表记录了每个文件标识对应的文件的全路径信息，该全路径信息包括该文件的物理起始地址。在本步骤中，可以根据所述文件标识以及文 件系统中存储的映射表获得所述目标数据所属的文件在所述内存设备中的物理起始地址。
或者，当处理器中的进程根据目标数据的逻辑地址访问文件系统时，操作系统会先打开进程要访问的目标数据所在的目标文件，然后再对打开的目标文件进行读操作。在操作系统打开目标文件的过程中，操作系统会调用内核，内核会为该进程访问的目标文件分配一个文件标识。文件标识是当前进程识别目标文件的唯一标识。在对目标文件进行读操作的过程中，该进程可以根据该文件标识识别要访问的目标文件。当进程访问文件系统时，进程中维护有该进程访问的所有打开的文件的描述信息，以文件标识为索引可以找到目标数据所属的文件的描述信息，并在该文件的描述信息中找到该文件在所述内存设备中的物理起始地址。
S42、根据所述文件的物理起始地址和所述偏移地址计算所述目标数据在所述内存设备中的物理起始地址。
具体的，所述目标数据在所述内存设备中的物理起始地址为所述文件的物理起始地址和所述偏移地址的和。
S43、根据所述目标数据在所述内存设备中的物理起始地址以及所述目标数据的长度确定所述目标数据的物理地址。
计算出目标数据在所述内存设备中的物理起始地址m_addr后，确定所述目标数据在内存设备中的物理地址包括m_addr和所述目标数据的长度。
以上描述仅为对“根据所述第一地址计算所述目标数据在所述内存设备中的物理地址”的举例，并不作限制。
203、构建散布表，所述散布表包括所述物理地址。
具体的，SGL的定义如下：
struct scatterlist{
      unsigned long page_link；         //内存中目标数据所在页面的物理起始地址
      unsigned int offset；             //页面中的偏移
      unsigned int length；             //页面中的长度
      dma_addr_t dma_address；          //typedef u64 dma_addr_t；按照 64bit定义的DMA地址
}；
将目标数据在所述内存设备中的物理地址填写到散布表中。具体的，获取sg_list指针，根据DMA规范填写SGL信息，其中，将目标数据在内存设备中的物理地址填入sg_list指针中。
举例来说，目标数据在内存设备中的物理地址包括22GB～34GB、39GB～44GB、56GB～78GB，那么依次将各物理地址填入sg_list指针中。
204、向DMA引擎发送迁移指令，所述迁移指令中包括所述散布表的信息。
构建好SGL后，为将目标数据迁移到外存设备中，处理器需向外存设备所连接的HBA中的DMA引擎发送迁移指令，其中，该迁移指令中包括目标数要迁移到外存设备中的地址以及SGL的信息，该SGL的信息包括目标数据在内存设备中的物理地址，该迁移指令用于指示DMA引擎根据SGL中的物理地址从内存设备中读取目标数据，并根据目标数要迁移到外存设备中的地址将目标数据写入外存设备中。
本实施例中，处理器通过根据目标数据在内存设备的逻辑地址计算目标数据在内存设备中的物理地址，这样，可直接根据该目标数据在内存设备中的物理地址生成SGL，而无需将目标数据从内存设备读取到RAM后，再根据目标数据在RAM中的物理地址构建SGL，由于减少了对RAM的操作，因此减少了对内存控制器的访问次数，进而减少了对内存控制器的带宽的影响。
上面对将目标数据从操作系统中的内存设备迁移到外存设备的方法，下面对就将目标数据从操作系统中的外存设备传输到内存设备的方法进行描述。
如图5所示，图5为本发明的应用于计算机系统的数据迁移方法的一个实施例的流程示意图。图5所示的数据迁移方法也可以应用于图1所示的计算机系统中。如图5所示，本实施例中的应用于计算机系统的数据迁移方法包括：
501、确定目标数据的第一地址。
本实施例中，目标数据存储在图1所示计算机系统的外存设备中，并需要从外存设备迁移到内存设备中。第一地址为所述目标数据待迁移到所述内存设 备中的目的地址。由于内存设备为非易失性存储器，所述内存设备的访问方式为卷VOLUME访问方式或文件FILE访问方式，因此第一地址为逻辑地址。具体解释可参考步骤201中的解释说明，在此不再赘述。
502、根据所述第一地址计算所述目标数据待迁移到所述内存设备中的物理地址。
本实施例中，计算物理地址的方法可参考步骤202的解释说明，在此不再赘述。
503、构建散布表，所述散布表包括所述物理地址。
具体的，SGL的定义如下：
struct scatterlist{
      unsigned long page_link；       //内存中目标数据待写入的页面的物理起始地址
      unsigned int offset；           //页面中的偏移
      unsigned int length；           //页面中的长度
      dma_addr_t dma_address；        //typedef u64 dma_addr_t；按照64bit定义的DMA地址
}；
将目标数据待迁移到所述内存设备中的物理地址填写到散布表中。具体的，获取sg_list指针，根据DMA规范填写SGL信息，其中，将目标数据待迁移到所述内存设备中的物理地址填入sg_list指针中。
举例来说，目标数据待迁移到所述内存设备中的物理地址包括22GB～34GB、39GB～44GB、56GB～78GB，那么依次将各物理地址填入sg_list指针中。
504、向DMA引擎发送迁移指令，所述迁移指令包括所述散布表的信息。
构建好SGL后，为将目标数据从外存设备迁移到内存设备中，处理器需向该外存设备所连接的HBA中的DMA引擎发送指示，该指示包括SGL的信息，该迁移指令用于指示DMA引擎从外存设备中读取目标数据，并根据SGL中的物理地址将目标数据写入内存设备中。
现有技术中，由于构建SGL时需要根据物理地址来构建，因此将外存设 备的数据迁移到内存设备中时需要先根据SGL将数据先传输到RAM中，再从RAM写到内存设备中；本实施例中，通过根据目标数据待迁移到所述内存设备中的逻辑地址计算对应的物理地址，这样可以根据该物理地址构建散布表，使得DMA引擎可以根据该散布表直接将目标数据写入到内存设备中，由于减少了对RAM的操作，因此减少了对内存控制器的访问次数，进而减少了对内存控制器的带宽的影响。
进一步的，本实施例中，内存设备的访问模式为卷访问模式或者文件访问模式，在将目标数据迁移到内存设备后，目标数据在内存设备中的地址采用的是逻辑地址。因此，处理器在向DMA引擎发送迁移指令之后，还需根据目标数据在目标内存设备中物理地址计算对应的逻辑地址，下面对处理器根据所述目标数据在内存设备中的物理地址计算对应的逻辑地址的方法进行说明。具体的，以内存设备的访问模式为卷访问模式为例。
首先，根据所述目标数据待迁移到的逻辑卷在所述内存设备中的物理起始地址确定所述逻辑卷的卷标识。具体的，在步骤201中所描述的映射表中通过该目标数据待迁移到内存设备中的物理起始地址查询该目标数据待迁移到的逻辑卷的ID。
其次，根据所述逻辑卷的卷标识和所述目标数据在所述内存设备中的物理起始地址计算偏移地址。确定逻辑卷的卷标识后，可在映射表中查询到该逻辑卷的物理起始地址。由于所述目标数据在所述内存设备中的物理起始地址为该逻辑卷的物理起始地址和偏移地址的和，因此偏移地址为所述目标数据在所述内存设备中的物理起始地址减去该逻辑卷的物理起始地址得到的差。
根据所述逻辑卷的卷标识、偏移地址和所述目标数据的长度确定所述目标数据待迁移到所述内存设备中的逻辑地址。
当内存设备的访问方式为文件访问方式时，将所述目标数据在内存设备中的物理地址计算对应的逻辑地址的方法可以参考以上举例，只是在查询目标数据待迁移到的文件的文件标识时是在用于记录文件系统中的映射表中进行查询。
上面对本发明实施例中的应用于计算机系统的数据迁移方法进行了描述， 下面对本发明实施例中的应用于计算机系统的数据迁移装置进行描述，其中应用于计算机系统的数据迁移装置用于执行上述应用于计算机系统的数据迁移方法。
请参阅图6，图6为本发明的应用于计算机系统的数据迁移装置的一个实施例的结构示意图。本实施例中，应用于计算机系统的数据迁移装置600包括：
确定模块601，用于确定目标数据的第一地址，其中，所述目标数据为需要从内存设备迁移到外存设备的数据，所述第一地址为所述目标数据在内存设备中的逻辑地址，所述内存设备为永久性存储器PM，所述计算机系统中的操作系统访问所述内存设备的访问方式为卷VOLUME访问方式或文件FILE访问方式；
计算模块602，用于根据所述第一地址计算所述目标数据在所述内存设备中的物理地址；
构建模块603，用于构建散布表，所述散布表包括所述目标数据在所述内存设备中的物理地址；
发送模块604，用于向直接内存存取DMA引擎发送迁移指令，所述迁移指令中包括所述散布表的信息，所述迁移指令用于指示所述DMA引擎根据所述物理地址从所述内存设备中获取所述目标数据，并将所述目标数据写到外存设备中。
可选的，当所述内存设备的访问方式为卷访问方式时，所述第一地址包括卷标识、偏移地址及长度，所述卷标识为所述目标数据所属的逻辑卷的标识，所述计算模块具体用于：
根据所述卷标识确定所述目标数据所属的逻辑卷在所述内存设备中的物理起始地址；
根据所述逻辑卷的物理起始地址和所述偏移地址计算所述目标数据在所述内存设备中的物理起始地址；
根据所述目标数据在所述内存设备中的物理起始地址以及所述目标数据的长度确定所述目标数据的物理地址。
可选的，当所述内存设备的访问方式为文件访问方式时，所述第一地址包括文件标识、偏移地址及长度，所述文件标识为所述目标数据所属的文件在文 件系统中的标识；所述计算模块具体用于：
根据所述文件标识获取所述目标数据所属的文件在所述内存设备中的物理起始地址；
根据所述文件的物理起始地址和所述偏移地址计算所述目标数据在所述内存设备中的物理起始地址；
根据所述目标数据在所述内存设备中的物理起始地址以及所述目标数据的长度确定所述目标数据的物理地址。
对本实施例中的应用于计算机系统的数据迁移装置的具体说明可参考图2至图4所示实施例中的描述，在此不再赘述。
请参阅图7，图7为本发明的数据传输装置的一个实施例的结构示意图。本实施例中，数据传输装置700包括：
确定模块701，确定待迁移的目标数据的第一地址，其中，所述目标数据为需要从外存设备迁移到内存设备的数据，所述第一地址为所述内存设备中的地址，所述第一地址为逻辑地址，所述内存设备为为永久性存储器，所述计算机系统中的操作系统访问所述内存设备的访问方式为卷VOLUME访问方式或文件FILE访问方式；
计算模块702，根据所述第一地址计算所述目标数据待迁移到所述内存设备中的物理地址；
构建模块703，构建散布表，所述散布表包括所述物理地址；
发送模块704，向DMA引擎发送迁移指令，所述迁移指令包括所述散布表的信息，所述迁移指令用于指示所述DMA引擎根据所述物理地址将所述目标数据写到所述内存设备中。
可选的，当所述内存设备的访问方式为卷访问方式时，所述第一地址包括卷标识、偏移地址及长度，所述卷标识为所述目标数据待迁移到的逻辑卷的标识，所述计算模块具体用于：
根据所述卷标识确定所述目标数据待迁移到的逻辑卷在所述内存设备中的物理起始地址；
根据所述逻辑卷的物理起始地址和所述偏移地址计算所述目标数据待迁移到所述内存设备中的物理起始地址；
根据所述目标数据待迁移到所述内存设备中的物理起始地址以及所述目标数据的长度确定所述目标数据待迁移到所述内存设备中的物理地址。
可选的，当所述内存设备的访问方式为文件访问方式时，所述第一地址包括文件标识、偏移地址及长度，所述文件标识为所述目标数据待迁移到的文件系统中的标识，所述计算模块具体用于：
根据所述文件标识获取所述目标数据待迁移到的文件在所述内存设备中的物理起始地址；
根据所述文件的物理起始地址和所述偏移地址计算所述目标数据待迁移到所述内存设备中的物理起始地址；
根据所述目标数据待迁移到所述内存设备中的物理起始地址以及所述目标数据的长度确定所述目标数据待迁移到所述内存设备中的物理地址。
对本实施例中的应用于计算机系统的数据迁移装置的具体说明可参考图5所示实施例中的描述，在此不再赘述。
上面从单元化功能实体的角度对本发明实施例中的应用于计算机系统的数据迁移装置进行了描述，下面从硬件处理的角度对本发明实施例中的应用于计算机系统的数据迁移装置进行描述。
本发明中的应用于计算机系统的数据迁移装置的一个具体实施例为图2所示计算机系统中的处理器，该处理器用于采用图2至图5所示实施例中的方法来构建散布表并向DMA引擎发送迁移指令，以指示DMA引擎根据该迁移指令将计算机系统中的外存设备中的数据迁移到内存设备中，或者指示DMA引擎根据该迁移指令将计算机系统中的内存设备的数据迁移到外存设备中。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统，装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的。另外，在本发明各个实施例中的各功能模块可以集成在一个处理单元中，也可以是各个模块单独物理存在，也可以两个或两个以上模块集成在一个模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。
所述集成的模块如果以软件功能模块的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本发 明的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本发明各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。

Claims (18)

  1. 一种应用于计算机系统的数据迁移方法，其特征在于，包括：
    确定目标数据的第一地址，其中，所述目标数据为需要从内存设备迁移到外存设备的数据，所述第一地址为所述目标数据在内存设备中的逻辑地址，所述内存设备为永久性存储器PM，所述计算机系统中的操作系统访问所述内存设备的访问方式为卷VOLUME访问方式或文件FILE访问方式；
    根据所述第一地址计算所述目标数据在所述内存设备中的物理地址；
    构建散布表，所述散布表包括所述目标数据在所述内存设备中的物理地址；
    向直接内存存取DMA引擎发送迁移指令，所述迁移指令中包括所述散布表的信息，所述迁移指令用于指示所述DMA引擎根据所述物理地址从所述内存设备中获取所述目标数据，并将所述目标数据写到外存设备中。
  2. 根据权利要求1所述的方法，其特征在于，当所述内存设备的访问方式为卷访问方式时，所述第一地址包括卷标识、偏移地址及长度，所述卷标识为所述目标数据所属的逻辑卷的标识，所述根据所述第一地址计算所述目标数据在所述内存设备中的物理地址，包括：
    根据所述卷标识确定所述目标数据所属的逻辑卷在所述内存设备中的物理起始地址；
    根据所述逻辑卷的物理起始地址和所述偏移地址计算所述目标数据在所述内存设备中的物理起始地址；
    根据所述目标数据在所述内存设备中的物理起始地址以及所述目标数据的长度确定所述目标数据的物理地址。
  3. 根据权利要求1所述的方法，其特征在于，当所述内存设备的访问方式为文件访问方式时，所述第一地址包括文件标识、偏移地址及长度，所述文件标识为所述目标数据所属的文件在文件系统中的标识；所述根据所述第一地址计算所述目标数据在所述内存设备中的物理地址，包括：
    根据所述文件标识获取所述目标数据所属的文件在所述内存设备中的物理起始地址；
    根据所述文件的物理起始地址和所述偏移地址计算所述目标数据在所述 内存设备中的物理起始地址；
    根据所述目标数据在所述内存设备中的物理起始地址以及所述目标数据的长度确定所述目标数据的物理地址。
  4. 一种应用于计算机系统的数据迁移方法，其特征在于，包括：
    确定待迁移的目标数据的第一地址，其中，所述目标数据为需要从外存设备迁移到内存设备的数据，所述第一地址为所述内存设备中的地址，所述第一地址为逻辑地址，所述内存设备为永久性存储器，所述计算机系统中的操作系统访问所述内存设备的访问方式为卷VOLUME访问方式或文件FILE访问方式；
    根据所述第一地址计算所述目标数据待迁移到所述内存设备中的物理地址；
    构建散布表，所述散布表包括所述物理地址；
    向DMA引擎发送迁移指令，所述迁移指令包括所述散布表的信息，所述迁移指令用于指示所述DMA引擎根据所述物理地址将所述目标数据写到所述内存设备中。
  5. 根据权利要求4所述的方法，其特征在于，当所述内存设备的访问方式为卷访问方式时，所述第一地址包括卷标识、偏移地址及长度，所述卷标识为所述目标数据待迁移到的逻辑卷的标识，所述根据所述第一地址计算所述目标数据待迁移到所述内存设备中的物理地址，包括：
    根据所述卷标识确定所述目标数据待迁移到的逻辑卷在所述内存设备中的物理起始地址；
    根据所述逻辑卷的物理起始地址和所述偏移地址计算所述目标数据待迁移到所述内存设备中的物理起始地址；
    根据所述目标数据待迁移到所述内存设备中的物理起始地址以及所述目标数据的长度确定所述目标数据待迁移到所述内存设备中的物理地址。
  6. 根据权利要求5所述的方法，其特征在于，当所述内存设备的访问方式为文件访问方式时，所述第一地址包括文件标识、偏移地址及长度，所述文件标识为所述目标数据待迁移到的文件系统中的标识，所述根据所述第一地址计算所述目标数据待迁移到所述内存设备中的物理地址，包括：
    根据所述文件标识获取所述目标数据待迁移到的文件在所述内存设备中的物理起始地址；
    根据所述文件的物理起始地址和所述偏移地址计算所述目标数据待迁移到所述内存设备中的物理起始地址；
    根据所述目标数据待迁移到所述内存设备中的物理起始地址以及所述目标数据的长度确定所述目标数据待迁移到所述内存设备中的物理地址。
  7. 一种应用于计算机系统的数据迁移装置，其特征在于，包括：
    确定模块，用于确定目标数据的第一地址，其中，所述目标数据为需要从内存设备迁移到外存设备的数据，所述第一地址为所述目标数据在内存设备中的逻辑地址，所述内存设备为永久性存储器PM，所述计算机系统中的操作系统访问所述内存设备的访问方式为卷VOLUME访问方式或文件FILE访问方式；
    计算模块，用于根据所述第一地址计算所述目标数据在所述内存设备中的物理地址；
    构建模块，用于构建散布表，所述散布表包括所述目标数据在所述内存设备中的物理地址；
    发送模块，用于向直接内存存取DMA引擎发送迁移指令，所述迁移指令中包括所述散布表的信息，所述迁移指令用于指示所述DMA引擎根据所述物理地址从所述内存设备中获取所述目标数据，并将所述目标数据写到外存设备中。
  8. 根据权利要求7所述的装置，其特征在于，当所述内存设备的访问方式为卷访问方式时，所述第一地址包括卷标识、偏移地址及长度，所述卷标识为所述目标数据所属的逻辑卷的标识，所述计算模块具体用于：
    根据所述卷标识确定所述目标数据所属的逻辑卷在所述内存设备中的物理起始地址；
    根据所述逻辑卷的物理起始地址和所述偏移地址计算所述目标数据在所述内存设备中的物理起始地址；
    根据所述目标数据在所述内存设备中的物理起始地址以及所述目标数据的长度确定所述目标数据的物理地址。
  9. 根据权利要求7所述的装置，其特征在于，当所述内存设备的访问方式为文件访问方式时，所述第一地址包括文件标识、偏移地址及长度，所述文件标识为所述目标数据所属的文件在文件系统中的标识；所述计算模块具体用于：
    根据所述文件标识获取所述目标数据所属的文件在所述内存设备中的物理起始地址；
    根据所述文件的物理起始地址和所述偏移地址计算所述目标数据在所述内存设备中的物理起始地址；
    根据所述目标数据在所述内存设备中的物理起始地址以及所述目标数据的长度确定所述目标数据的物理地址。
  10. 一种应用于计算机系统的数据迁移装置，其特征在于，包括：
    确定模块，确定待迁移的目标数据的第一地址，其中，所述目标数据为需要从外存设备迁移到内存设备的数据，所述第一地址为所述内存设备中的地址，所述第一地址为逻辑地址，所述内存设备为为永久性存储器，所述计算机系统中的操作系统访问所述内存设备的访问方式为卷VOLUME访问方式或文件FILE访问方式；
    计算模块，根据所述第一地址计算所述目标数据待迁移到所述内存设备中的物理地址；
    构建模块，构建散布表，所述散布表包括所述物理地址；
    发送模块，向DMA引擎发送迁移指令，所述迁移指令包括所述散布表的信息，所述迁移指令用于指示所述DMA引擎根据所述物理地址将所述目标数据写到所述内存设备中。
  11. 根据权利要求10所述的装置，其特征在于，当所述内存设备的访问方式为卷访问方式时，所述第一地址包括卷标识、偏移地址及长度，所述卷标识为所述目标数据待迁移到的逻辑卷的标识，所述计算模块具体用于：
    根据所述卷标识确定所述目标数据待迁移到的逻辑卷在所述内存设备中的物理起始地址；
    根据所述逻辑卷的物理起始地址和所述偏移地址计算所述目标数据待迁移到所述内存设备中的物理起始地址；
    根据所述目标数据待迁移到所述内存设备中的物理起始地址以及所述目标数据的长度确定所述目标数据待迁移到所述内存设备中的物理地址。
  12. 根据权利要求10所述的装置，其特征在于，当所述内存设备的访问方式为文件访问方式时，所述第一地址包括文件标识、偏移地址及长度，所述文件标识为所述目标数据待迁移到的文件系统中的标识，所述计算模块具体用于：
    根据所述文件标识获取所述目标数据待迁移到的文件在所述内存设备中的物理起始地址；
    根据所述文件的物理起始地址和所述偏移地址计算所述目标数据待迁移到所述内存设备中的物理起始地址；
    根据所述目标数据待迁移到所述内存设备中的物理起始地址以及所述目标数据的长度确定所述目标数据待迁移到所述内存设备中的物理地址。
  13. 一种计算机系统，其特征在于，包括：
    内存设备，所述内存设备中存储有目标数据，所述目标数据为需要从内存设备迁移到外存设备的数据，其中，所述内存设备为永久性存储器PM，所述计算机系统中的操作系统访问所述内存设备的访问方式为卷VOLUME访问方式或文件FILE访问方式；
    处理器，用于：
    确定所述目标数据的第一地址，其中，所述第一地址为所述目标数据在内存设备中的逻辑地址；
    根据所述第一地址计算所述目标数据在所述内存设备中的物理地址；
    构建散布表，所述散布表包括所述目标数据在所述内存设备中的物理地址；
    向直接内存存取DMA引擎发送迁移指令，所述迁移指令中包括所述散布表的信息，所述迁移指令用于指示所述DMA引擎根据所述物理地址从所述内存设备中获取所述目标数据，并将所述目标数据写到外存设备中。
  14. 根据权利要求13的计算机系统，其特征在于，当所述内存设备的访问方式为卷访问方式时，所述第一地址包括卷标识、偏移地址及长度，所述卷标识为所述目标数据所属的逻辑卷的标识，所述处理器具体用于：
    根据所述卷标识确定所述目标数据所属的逻辑卷在所述内存设备中的物理起始地址；
    根据所述逻辑卷的物理起始地址和所述偏移地址计算所述目标数据在所述内存设备中的物理起始地址；
    根据所述目标数据在所述内存设备中的物理起始地址以及所述目标数据的长度确定所述目标数据的物理地址。
  15. 根据权利要求13所述的计算机系统，其特征在于，当所述内存设备的访问方式为文件访问方式时，所述第一地址包括文件标识、偏移地址及长度，所述文件标识为所述目标数据所属的文件在文件系统中的标识；所述处理器具体用于：
    根据所述文件标识获取所述目标数据所属的文件在所述内存设备中的物理起始地址；
    根据所述文件的物理起始地址和所述偏移地址计算所述目标数据在所述内存设备中的物理起始地址；
    根据所述目标数据在所述内存设备中的物理起始地址以及所述目标数据的长度确定所述目标数据的物理地址。
  16. 一种计算机系统，其特征在于，包括：
    内存设备，所述内存设备为永久性存储器，所述计算机系统中的操作系统访问所述内存设备的访问方式为卷VOLUME访问方式或文件FILE访问方式；
    外存设备，用于存储目标数据，所述目标数据为需要从外存设备迁移到内存设备的数据；
    处理器，用于：
    根据所述第一地址计算所述目标数据待迁移到所述内存设备中的物理地址，其中，所述第一地址为所述内存设备中的地址，所述第一地址为逻辑地址；
    构建散布表，所述散布表包括所述物理地址；
    向DMA引擎发送迁移指令，所述迁移指令包括所述散布表的信息，所述迁移指令用于指示所述DMA引擎根据所述物理地址将所述目标数据写到所述内存设备中。
  17. 根据权利要求16所述的计算机系统，其特征在于，当所述内存设备 的访问方式为卷访问方式时，所述第一地址包括卷标识、偏移地址及长度，所述卷标识为所述目标数据待迁移到的逻辑卷的标识，所述处理器具体用于：
    根据所述卷标识确定所述目标数据待迁移到的逻辑卷在所述内存设备中的物理起始地址；
    根据所述逻辑卷的物理起始地址和所述偏移地址计算所述目标数据待迁移到所述内存设备中的物理起始地址；
    根据所述目标数据待迁移到所述内存设备中的物理起始地址以及所述目标数据的长度确定所述目标数据待迁移到所述内存设备中的物理地址。
  18. 根据权利要求16所述的计算机系统，其特征在于，当所述内存设备的访问方式为文件访问方式时，所述第一地址包括文件标识、偏移地址及长度，所述文件标识为所述目标数据待迁移到的文件系统中的标识，所述处理器具体用于：
    根据所述文件标识获取所述目标数据待迁移到的文件在所述内存设备中的物理起始地址；
    根据所述文件的物理起始地址和所述偏移地址计算所述目标数据待迁移到所述内存设备中的物理起始地址；
    根据所述目标数据待迁移到所述内存设备中的物理起始地址以及所述目标数据的长度确定所述目标数据待迁移到所述内存设备中的物理地址。
PCT/CN2015/096285 2015-12-03 2015-12-03 应用于计算机系统的数据迁移方法和装置、计算机系统 WO2017092002A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
EP15909519.9A EP3361385A4 (en) 2015-12-03 2015-12-03 Data migration method applicable to computer system, and device and computer system utilizing same
CN201580061176.0A CN107111452B (zh) 2015-12-03 2015-12-03 应用于计算机系统的数据迁移方法和装置、计算机系统
PCT/CN2015/096285 WO2017092002A1 (zh) 2015-12-03 2015-12-03 应用于计算机系统的数据迁移方法和装置、计算机系统
US15/980,630 US10740018B2 (en) 2015-12-03 2018-05-15 Data migration method and apparatus applied to computer system, and computer system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2015/096285 WO2017092002A1 (zh) 2015-12-03 2015-12-03 应用于计算机系统的数据迁移方法和装置、计算机系统

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US15/980,630 Continuation US10740018B2 (en) 2015-12-03 2018-05-15 Data migration method and apparatus applied to computer system, and computer system

Publications (1)

Publication Number Publication Date
WO2017092002A1 true WO2017092002A1 (zh) 2017-06-08

Family

ID=58796043

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2015/096285 WO2017092002A1 (zh) 2015-12-03 2015-12-03 应用于计算机系统的数据迁移方法和装置、计算机系统

Country Status (4)

Country Link
US (1) US10740018B2 (zh)
EP (1) EP3361385A4 (zh)
CN (1) CN107111452B (zh)
WO (1) WO2017092002A1 (zh)

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11182090B2 (en) 2018-11-19 2021-11-23 Micron Technology, Inc. Systems, devices, and methods for data migration
US11256437B2 (en) 2018-11-19 2022-02-22 Micron Technology, Inc. Data migration for memory operation
US11163473B2 (en) * 2018-11-19 2021-11-02 Micron Technology, Inc. Systems, devices, techniques, and methods for data migration
CN112292660B (zh) * 2019-05-13 2022-05-31 华为技术有限公司 一种调度存储器中数据的方法、数据调度设备及系统
CN112540941B (zh) * 2019-09-21 2024-09-20 华为技术有限公司 一种数据转发芯片及服务器
CN115398403A (zh) * 2020-04-17 2022-11-25 华为技术有限公司 存储器的管理方法和控制装置
US11537479B2 (en) * 2020-04-29 2022-12-27 Memverge, Inc. Memory image capture
CN112486410B (zh) * 2020-11-23 2024-03-26 华南师范大学 一种持久性内存文件读写方法、系统、装置及存储介质
CN113282239A (zh) * 2021-05-21 2021-08-20 维沃移动通信有限公司 数据迁移方法、存储器及控制器
CN113687779B (zh) * 2021-07-29 2024-02-23 济南浪潮数据技术有限公司 数据迁移方法、装置、电子设备及可读存储介质
CN113821458B (zh) * 2021-09-18 2023-09-05 日立楼宇技术（广州）有限公司 一种数据操作方法、装置、计算机设备和存储介质
CN115659994B (zh) * 2022-12-09 2023-03-03 深圳市人马互动科技有限公司 人机交互系统中的数据处理方法及相关装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090164691A1 (en) * 2007-12-20 2009-06-25 Hsun-Wen Wang Io processor
CN101539894A (zh) * 2009-05-04 2009-09-23 成都市华为赛门铁克科技有限公司 一种将计算机外部设备接入计算机系统的方法和装置
CN102480516A (zh) * 2010-11-30 2012-05-30 英业达股份有限公司 网际网络小型电脑界面的数据单元的解析方法
CN104579695A (zh) * 2013-10-23 2015-04-29 杭州华三通信技术有限公司 一种数据转发装置和方法

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3217002B2 (ja) * 1996-11-19 2001-10-09 株式会社日立製作所 デジタルスタジオ装置およびその制御方法
US6021462A (en) * 1997-08-29 2000-02-01 Apple Computer, Inc. Methods and apparatus for system memory efficient disk access to a raid system using stripe control information
US20020144027A1 (en) * 2001-03-27 2002-10-03 Schmisseur Mark A. Multi-use data access descriptor
US8549252B2 (en) * 2005-12-13 2013-10-01 Emc Corporation File based volumes and file systems
US8775718B2 (en) * 2008-05-23 2014-07-08 Netapp, Inc. Use of RDMA to access non-volatile solid-state memory in a network storage system
CN102567244B (zh) * 2011-12-13 2014-06-04 清华大学 闪存和磁盘转换存取方法
US9223706B2 (en) * 2013-07-02 2015-12-29 Infinidat Ltd. System, method and a non-transitory computer readable medium for a pre-fetch operation
US9256373B1 (en) * 2014-05-19 2016-02-09 Emc Corporation Invulnerable data movement for file system upgrade
CN104965757B (zh) * 2015-01-21 2018-03-30 深圳市腾讯计算机系统有限公司 虚拟机热迁移的方法、虚拟机迁移管理装置及系统

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090164691A1 (en) * 2007-12-20 2009-06-25 Hsun-Wen Wang Io processor
CN101539894A (zh) * 2009-05-04 2009-09-23 成都市华为赛门铁克科技有限公司 一种将计算机外部设备接入计算机系统的方法和装置
CN102480516A (zh) * 2010-11-30 2012-05-30 英业达股份有限公司 网际网络小型电脑界面的数据单元的解析方法
CN104579695A (zh) * 2013-10-23 2015-04-29 杭州华三通信技术有限公司 一种数据转发装置和方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3361385A4 *

Also Published As

Publication number Publication date
CN107111452B (zh) 2020-01-10
EP3361385A1 (en) 2018-08-15
EP3361385A4 (en) 2018-11-21
US10740018B2 (en) 2020-08-11
US20180267739A1 (en) 2018-09-20
CN107111452A (zh) 2017-08-29

Similar Documents

Publication Publication Date Title
WO2017092002A1 (zh) 应用于计算机系统的数据迁移方法和装置、计算机系统
KR102395190B1 (ko) 호스트와 인터페이스를 수행하는 스토리지 장치, 호스트 및 스토리지 장치의 동작방법
US9304938B2 (en) Storage device and data transferring method thereof
US10359954B2 (en) Method and system for implementing byte-alterable write cache
US9092426B1 (en) Zero-copy direct memory access (DMA) network-attached storage (NAS) file system block writing
US20240264760A1 (en) Data Migration Method, Host, and Solid State Disk
US9558232B1 (en) Data movement bulk copy operation
JP6713934B2 (ja) 記憶装置及びその動作方法並びにシステム
WO2019090493A1 (zh) 内存块回收方法和装置
TW202101227A (zh) 閃存實體資源集合管理裝置及方法以及電腦程式產品
WO2019120226A1 (zh) 数据访问预测方法和装置
JP2020191055A (ja) 瞬断からの回復処理方法及び装置、並びにコンピュータ読み取り可能な記憶媒体
KR20140128819A (ko) 아토믹 라이트 방법
US20240192860A1 (en) Method and device for log structured merge-tree based key-value data storage
WO2017054714A1 (zh) 磁盘阵列的读方法及装置
US20230153006A1 (en) Data processing method and data processing device
US20230084539A1 (en) Computational storage device and storage system including the computational storage device
US11662949B2 (en) Storage server, a method of operating the same storage server and a data center including the same storage server
WO2021082877A1 (zh) 访问固态硬盘的方法及装置
KR101569049B1 (ko) 패스 스루 스토리지 디바이스들
US20230325110A1 (en) Operation method of host device and operation method of storage device
EP4258097A1 (en) Operation method of host device and operation method of storage device
KR20230144434A (ko) 호스트 장치의 동작 방법 및 스토리지 장치의 동작 방법
US11914879B2 (en) Storage controller and storage system comprising the same
KR102435910B1 (ko) 스토리지 장치 및 그것의 동작 방법

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15909519

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2015909519

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE