CN109791522A - 数据迁移的方法、系统及智能网卡 - Google Patents

数据迁移的方法、系统及智能网卡 Download PDF

Info

Publication number
CN109791522A
CN109791522A CN201780002932.1A CN201780002932A CN109791522A CN 109791522 A CN109791522 A CN 109791522A CN 201780002932 A CN201780002932 A CN 201780002932A CN 109791522 A CN109791522 A CN 109791522A
Authority
CN
China
Prior art keywords
target
source
migration
ssd
caching
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201780002932.1A
Other languages
English (en)
Other versions
CN109791522B (zh
Inventor
杜阁
胡瑜
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to CN202110043788.4A priority Critical patent/CN112835817A/zh
Publication of CN109791522A publication Critical patent/CN109791522A/zh
Application granted granted Critical
Publication of CN109791522B publication Critical patent/CN109791522B/zh
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
    • G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
    • G06F12/0866—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches for peripheral storage systems, e.g. disk cache
    • G06F12/0871—Allocation or management of cache space
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
    • G06F12/023—Free address space management
    • G06F12/0238—Memory management in non-volatile memory, e.g. resistive RAM or ferroelectric memory
    • G06F12/0246—Memory management in non-volatile memory, e.g. resistive RAM or ferroelectric memory in block erasable memory, e.g. flash memory
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
    • G06F3/0647—Migration mechanisms
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/004—Error avoidance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
    • G06F12/023—Free address space management
    • G06F12/0238—Memory management in non-volatile memory, e.g. resistive RAM or ferroelectric memory
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
    • G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
    • G06F12/0866—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches for peripheral storage systems, e.g. disk cache
    • G06F12/0868—Data transfer between cache memory and other subsystems, e.g. storage devices or host systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061—Improving I/O performance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0614—Improving the reliability of storage systems
    • G06F3/0619—Improving the reliability of storage systems in relation to data integrity, e.g. data losses, bit errors
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
    • G06F3/0656—Data buffering arrangements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
    • G06F3/0659—Command handling arrangements, e.g. command buffers, queues, command scheduling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
    • G06F3/0683—Plurality of storage devices
    • G06F3/0688—Non-volatile semiconductor memory arrays
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L61/00—Network arrangements, protocols or services for addressing or naming
    • H04L61/50—Address allocation
    • H04L61/5007—Internet protocol [IP] addresses
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/56—Provisioning of proxy services
    • H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/26—Using a specific storage system architecture
    • G06F2212/261—Storage comprising a plurality of storage devices
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/31—Providing disk cache in a specific location of a storage system
    • G06F2212/312—In storage controller
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/31—Providing disk cache in a specific location of a storage system
    • G06F2212/314—In storage network, e.g. network attached cache
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/72—Details relating to flash memory management
    • G06F2212/7203—Temporary buffering, e.g. using volatile buffer or dedicated buffer blocks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/72—Details relating to flash memory management
    • G06F2212/7208—Multiple device management, e.g. distributing data over multiple flash devices

Abstract

本发明实施例涉及一种应用于源智能网卡的数据迁移方法。所述方法包括：接收第一迁移指令，所述第一迁移指令携带源SSD中待迁移数据的地址，及迁入所述待迁移数据的目标SSD的信息，所述目标SSD属于目标存储阵列；发送读指令至所述源SSD，所述读指令包括所述第一迁移指令携带的所述源SSD的地址及所述源迁移缓存的访问信息，用于指示所述源SSD将所述待迁移数据读取至所述源迁移缓存；发送第二迁移指令至所述目标存储阵列的目标智能网卡，所述第二迁移指令包括所述源迁移缓存的访问信息及根据所述目标SSD信息确定的目标SSD地址，用于指示所述目标智能网卡所述源迁移缓存中的待迁移数据迁移至所述目标存储阵列。

Description

PCT国内申请，说明书已公开。

Claims (21)

  1. PCT国内申请，权利要求书已公开。
CN201780002932.1A 2017-09-05 2017-09-05 数据迁移的方法、系统及智能网卡 Active CN109791522B (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110043788.4A CN112835817A (zh) 2017-09-05 2017-09-05 数据迁移的方法及系统

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2017/100586 WO2019047026A1 (zh) 2017-09-05 2017-09-05 数据迁移的方法、系统及智能网卡

Related Child Applications (1)

Application Number Title Priority Date Filing Date
CN202110043788.4A Division CN112835817A (zh) 2017-09-05 2017-09-05 数据迁移的方法及系统

Publications (2)

Publication Number Publication Date
CN109791522A true CN109791522A (zh) 2019-05-21
CN109791522B CN109791522B (zh) 2021-01-15

Family

ID=65633432

Family Applications (2)

Application Number Title Priority Date Filing Date
CN202110043788.4A Pending CN112835817A (zh) 2017-09-05 2017-09-05 数据迁移的方法及系统
CN201780002932.1A Active CN109791522B (zh) 2017-09-05 2017-09-05 数据迁移的方法、系统及智能网卡

Family Applications Before (1)

Application Number Title Priority Date Filing Date
CN202110043788.4A Pending CN112835817A (zh) 2017-09-05 2017-09-05 数据迁移的方法及系统

Country Status (4)

Country Link
US (1) US11966612B2 (zh)
EP (2) EP3929756A1 (zh)
CN (2) CN112835817A (zh)
WO (1) WO2019047026A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112559245A (zh) * 2020-12-01 2021-03-26 深圳市欧瑞博科技股份有限公司 智能网关的数据迁移方法、装置、服务器以及存储介质
CN112578997A (zh) * 2019-09-30 2021-03-30 华为技术有限公司 一种数据迁移方法、系统及相关设备
CN117573043A (zh) * 2024-01-17 2024-02-20 济南浪潮数据技术有限公司 分布式存储数据的传输方法、装置、系统、设备和介质

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11068365B2 (en) * 2018-08-23 2021-07-20 Micron Technology, Inc. Data recovery within a memory sub-system without moving or processing the data through a host
KR20220055034A (ko) * 2020-10-26 2022-05-03 삼성전자주식회사 스토리지 장치의 구동 방법 및 이를 이용한 스토리지 시스템의 구동 방법
KR20220059266A (ko) * 2020-11-02 2022-05-10 에스케이하이닉스 주식회사 스토리지 시스템
CN113220346A (zh) * 2021-04-29 2021-08-06 上海阵量智能科技有限公司 一种硬件电路、数据搬移方法、芯片和电子设备
CN115374024A (zh) * 2021-05-21 2022-11-22 华为技术有限公司 一种内存数据排序方法及相关设备
CN114489517B (zh) * 2022-02-14 2023-09-08 苏州浪潮智能科技有限公司 一种离线加速迁移方法、系统、设备及存储介质

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1547348A (zh) * 2003-12-10 2004-11-17 中国科学院计算技术研究所 基于智能网卡的计算机间用户级并行通信的方法
CN101702688A (zh) * 2009-11-24 2010-05-05 武汉绿色网络信息服务有限责任公司 一种数据包收发方法
CN102244579A (zh) * 2010-05-12 2011-11-16 北京研华兴业电子科技有限公司 网卡及网络数据接收方法
CN102333037A (zh) * 2011-10-20 2012-01-25 中兴通讯股份有限公司 虚拟机迁移方法及装置
CN102651690A (zh) * 2011-02-28 2012-08-29 国际商业机器公司 对共享内存进行控制的方法和装置
WO2013024402A1 (en) * 2011-08-12 2013-02-21 International Business Machines Corporation Optimized virtual function translation entry memory caching
US20130046920A1 (en) * 2011-08-17 2013-02-21 Samsung Electronics Co., Ltd. Nonvolatile memory system with migration manager
CN103970481A (zh) * 2013-01-29 2014-08-06 国际商业机器公司 重建存储器阵列的方法和装置
US20160041915A1 (en) * 2014-07-15 2016-02-11 Oracle International Corporation Target and initiator mode configuration of tape drives for data transfer between source and destination tape drives
CN107077426A (zh) * 2016-12-05 2017-08-18 华为技术有限公司 NVMe over Fabric架构中数据读写命令的控制方法、设备和系统

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8782199B2 (en) * 1997-10-14 2014-07-15 A-Tech Llc Parsing a packet header
US20070041383A1 (en) * 2005-04-05 2007-02-22 Mohmmad Banikazemi Third party node initiated remote direct memory access
US8555019B2 (en) * 2010-09-08 2013-10-08 International Business Machines Corporation Using a migration cache to cache tracks during migration
EP2583162A1 (en) * 2010-12-22 2013-04-24 Hitachi, Ltd. Storage system comprising multiple storage apparatuses with both storage virtualization function and capacity virtualization function
US10382401B1 (en) * 2013-02-26 2019-08-13 Zentera Systems, Inc. Cloud over IP for enterprise hybrid cloud network and security
WO2014141411A1 (ja) * 2013-03-13 2014-09-18 株式会社日立製作所 ストレージシステムおよびストレージシステム制御方法
US20160364268A1 (en) * 2014-06-26 2016-12-15 Hitachi, Ltd. Computer system, management computer, and management method
US10162558B2 (en) * 2015-10-30 2018-12-25 Micron Technology, Inc. Data transfer techniques for multiple devices on a shared bus
US10447800B2 (en) * 2015-12-17 2019-10-15 Dell Products L.P. Network cache deduplication analytics based compute cluster load balancer
CN108228647B (zh) * 2016-12-21 2022-05-24 伊姆西Ip控股有限责任公司 用于数据拷贝的方法和设备

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1547348A (zh) * 2003-12-10 2004-11-17 中国科学院计算技术研究所 基于智能网卡的计算机间用户级并行通信的方法
CN101702688A (zh) * 2009-11-24 2010-05-05 武汉绿色网络信息服务有限责任公司 一种数据包收发方法
CN102244579A (zh) * 2010-05-12 2011-11-16 北京研华兴业电子科技有限公司 网卡及网络数据接收方法
CN102651690A (zh) * 2011-02-28 2012-08-29 国际商业机器公司 对共享内存进行控制的方法和装置
WO2013024402A1 (en) * 2011-08-12 2013-02-21 International Business Machines Corporation Optimized virtual function translation entry memory caching
US20130046920A1 (en) * 2011-08-17 2013-02-21 Samsung Electronics Co., Ltd. Nonvolatile memory system with migration manager
CN102333037A (zh) * 2011-10-20 2012-01-25 中兴通讯股份有限公司 虚拟机迁移方法及装置
CN103970481A (zh) * 2013-01-29 2014-08-06 国际商业机器公司 重建存储器阵列的方法和装置
US20160041915A1 (en) * 2014-07-15 2016-02-11 Oracle International Corporation Target and initiator mode configuration of tape drives for data transfer between source and destination tape drives
CN107077426A (zh) * 2016-12-05 2017-08-18 华为技术有限公司 NVMe over Fabric架构中数据读写命令的控制方法、设备和系统

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112578997A (zh) * 2019-09-30 2021-03-30 华为技术有限公司 一种数据迁移方法、系统及相关设备
CN112578997B (zh) * 2019-09-30 2022-07-22 华为云计算技术有限公司 一种数据迁移方法、系统及相关设备
CN112559245A (zh) * 2020-12-01 2021-03-26 深圳市欧瑞博科技股份有限公司 智能网关的数据迁移方法、装置、服务器以及存储介质
CN117573043A (zh) * 2024-01-17 2024-02-20 济南浪潮数据技术有限公司 分布式存储数据的传输方法、装置、系统、设备和介质

Also Published As

Publication number Publication date
EP3663928A4 (en) 2020-09-02
WO2019047026A1 (zh) 2019-03-14
EP3663928B1 (en) 2021-06-23
EP3929756A1 (en) 2021-12-29
EP3663928A1 (en) 2020-06-10
US11966612B2 (en) 2024-04-23
CN109791522B (zh) 2021-01-15
US20200233601A1 (en) 2020-07-23
CN112835817A (zh) 2021-05-25

Similar Documents

Publication Publication Date Title
CN109791522A (zh) 数据迁移的方法、系统及智能网卡
KR102457611B1 (ko) 터넌트-어웨어 스토리지 쉐어링 플랫폼을 위한 방법 및 장치
US10754575B2 (en) Storage system with replication process utilizing simulated target responses
US10372340B2 (en) Data distribution method in storage system, distribution apparatus, and storage system
US10296255B1 (en) Data migration techniques
US11036393B2 (en) Migrating data between volumes using virtual copy operation
US11144252B2 (en) Optimizing write IO bandwidth and latency in an active-active clustered system based on a single storage node having ownership of a storage object
US20220188249A1 (en) Memory appliance couplings and operations
CN108475229A (zh) 数据迁移的方法、主机、及固态存储设备
US20240045807A1 (en) Methods for managing input-output operations in zone translation layer architecture and devices thereof
CN109542814A (zh) 通过pci-express的p2p连接在存储设备之间传输数据的方法和系统
US20190114076A1 (en) Method and Apparatus for Storing Data in Distributed Block Storage System, and Computer Readable Storage Medium
CN109478171A (zh) 提高openfabrics环境中的吞吐量
CN104516687A (zh) 一种Linux块设备的Windows远程映射方法
CN108282516B (zh) 一种基于iSCSI的分布式存储集群负载均衡方法和装置
JP2004334481A (ja) 仮想化情報管理装置
CN112104729A (zh) 一种存储系统及其缓存方法
US20160283134A1 (en) Data replication across host systems via storage controller
US20210311654A1 (en) Distributed Storage System and Computer Program Product
WO2022073399A1 (zh) 存储节点、存储设备及网络芯片
CN117311593A (zh) 数据处理方法、装置及系统
CN105765542A (zh) 访问文件的方法、分布式存储系统和存储节点
WO2014077451A1 (ko) Iscsi 스토리지 시스템을 이용한 네트워크 분산 파일 시스템 및 방법
WO2020083106A1 (zh) 存储系统中的节点扩容方法和存储系统
US20220116451A1 (en) System and method for migrating volumes between storage appliances

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant