JP6778795B2 - データを記憶するための方法、装置及びシステム - Google Patents

データを記憶するための方法、装置及びシステム Download PDF

Info

Publication number
JP6778795B2
JP6778795B2 JP2019127387A JP2019127387A JP6778795B2 JP 6778795 B2 JP6778795 B2 JP 6778795B2 JP 2019127387 A JP2019127387 A JP 2019127387A JP 2019127387 A JP2019127387 A JP 2019127387A JP 6778795 B2 JP6778795 B2 JP 6778795B2
Authority
JP
Japan
Prior art keywords
data
storage
record
size
type
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2019127387A
Other languages
English (en)
Japanese (ja)
Other versions
JP2020038623A (ja
Inventor
チェン，ソンジャ
リュウ，ジャン
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Baidu Netcom Science and Technology Co Ltd
Original Assignee
Beijing Baidu Netcom Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Baidu Netcom Science and Technology Co Ltd filed Critical Beijing Baidu Netcom Science and Technology Co Ltd
Publication of JP2020038623A publication Critical patent/JP2020038623A/ja
Application granted granted Critical
Publication of JP6778795B2 publication Critical patent/JP6778795B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061—Improving I/O performance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0614—Improving the reliability of storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0629—Configuration or reconfiguration of storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0629—Configuration or reconfiguration of storage systems
    • G06F3/0631—Configuration or reconfiguration of storage systems by allocating resources to storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/064—Management of blocks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
    • G06F3/0652—Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0653—Monitoring storage devices or systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
JP2019127387A 2018-09-04 2019-07-09 データを記憶するための方法、装置及びシステム Active JP6778795B2 (ja)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811024928.8 2018-09-04
CN201811024928.8A CN109254733B (zh) 2018-09-04 2018-09-04 用于存储数据的方法、装置和系统

Publications (2)

Publication Number Publication Date
JP2020038623A JP2020038623A (ja) 2020-03-12
JP6778795B2 true JP6778795B2 (ja) 2020-11-04

Family

ID=65046720

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2019127387A Active JP6778795B2 (ja) 2018-09-04 2019-07-09 データを記憶するための方法、装置及びシステム

Country Status (3)

Country Link
JP (1) JP6778795B2 (zh)
KR (1) KR102240557B1 (zh)
CN (1) CN109254733B (zh)

Families Citing this family (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109918381B (zh) * 2019-03-13 2021-06-11 北京百度网讯科技有限公司 用于存储数据的方法和装置
CN110059087B (zh) * 2019-03-26 2023-02-28 创新先进技术有限公司 一种块链式账本中的数据属性标识方法、装置及设备
CN112085424A (zh) * 2019-06-14 2020-12-15 北京京东振世信息技术有限公司 存储物品的方法、装置、设备和计算机可读介质
CN110427377B (zh) * 2019-08-02 2023-12-26 北京博睿宏远数据科技股份有限公司 数据处理方法、装置、设备和存储介质
CN110727639B (zh) * 2019-10-08 2023-09-19 深圳市网心科技有限公司 碎片数据读取方法、电子设备、系统及介质
CN112765046A (zh) * 2019-10-21 2021-05-07 伊姆西Ip控股有限责任公司 用于回收存储空间的方法、设备和计算机程序产品
CN110781146A (zh) * 2019-10-28 2020-02-11 宁波三星智能电气有限公司 事件存储方法、电力终端及计算机可读存储介质
CN111399762B (zh) 2019-11-27 2021-06-18 杭州海康威视系统技术有限公司 数据存储方法、装置及存储系统
CN111198807B (zh) * 2019-12-18 2023-10-27 中移（杭州）信息技术有限公司 数据流分析方法、装置、计算机设备及存储介质
CN111522507B (zh) 2020-04-14 2021-10-01 中山大学 一种低延迟的文件系统地址空间管理方法、系统及介质
CN111694521B (zh) * 2020-06-17 2022-08-05 杭州海康威视系统技术有限公司 存储文件的方法、装置及系统
CN111797032B (zh) * 2020-06-19 2022-09-23 南京南瑞继保工程技术有限公司 存储结构、方法、装置、设备及可读存储介质
CN114430458B (zh) * 2020-10-29 2023-12-19 北京小米移动软件有限公司 防抖数据获取方法、防抖数据获取装置以及存储介质
CN112506918A (zh) * 2020-11-03 2021-03-16 深圳市宏电技术股份有限公司 数据存取方法、终端及计算机可读存储介质
CN112699092B (zh) * 2021-01-13 2023-02-03 浪潮云信息技术股份公司 一种RocksDB存储大值数据的方法
CN113010105A (zh) * 2021-02-25 2021-06-22 深圳市广和通无线股份有限公司 一种数据备份的方法、装置、设备和计算机可读存储介质
CN115190136B (zh) * 2021-04-21 2024-03-01 统信软件技术有限公司 一种数据存储方法、数据传输方法及计算设备
CN113495889B (zh) * 2021-07-07 2024-03-15 浙江大华技术股份有限公司 一种分布式对象存储方法、装置、电子设备及存储介质
CN113486026A (zh) * 2021-08-02 2021-10-08 北京字节跳动网络技术有限公司 数据处理方法、装置、设备及介质
CN114116795B (zh) * 2021-11-01 2022-12-20 北京力控元通科技有限公司 一种数据存储和查询方法、装置、存储介质及电子设备
CN114816277B (zh) * 2022-06-30 2022-11-11 广东睿江云计算股份有限公司 文件数据块顺序性保障的控制方法及控制系统
CN116010997B (zh) * 2023-03-20 2023-08-01 中国信息通信研究院 基于区块链的连续性校验的数据资源托管方法和装置
CN116521091B (zh) * 2023-06-28 2023-09-15 苏州浪潮智能科技有限公司 数据读取方法、装置、设备、数据传输系统及存储介质
CN117082083A (zh) * 2023-08-30 2023-11-17 杭州浩联智能科技有限公司 一种基于分布式物联网架构的数据存储方法、装置及介质
CN116910825B (zh) * 2023-09-12 2023-12-15 上海融和元储能源有限公司 一种基于kafka的数据同步方法及装置
CN117078213B (zh) * 2023-10-11 2024-03-29 潍坊威尔信息技术有限公司 基于大数据整合分析的建筑工程管理平台

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20030058687A (ko) * 2001-12-31 2003-07-07 주식회사 케이티 공간 데이터 특성에 따른 데이터 관리방법
KR100449419B1 (ko) * 2002-06-14 2004-09-18 학교법인 인하학원 데이터 크기에 따른 선별적 공간데이터 관리방법
KR100694069B1 (ko) * 2004-11-29 2007-03-12 삼성전자주식회사 상이한 크기를 가지는 복수 개의 데이터 블록들을포함하는 저장 장치 및 이를 이용한 파일 관리 방법 및이를 포함하는 인쇄 장치
EP1798934A1 (en) * 2005-12-13 2007-06-20 Deutsche Thomson-Brandt Gmbh Method and apparatus for organizing nodes in a network
KR100834574B1 (ko) * 2006-09-29 2008-06-02 한국전자통신연구원 파일 저장 시스템 및 그 시스템에서의 파일 저장 및 검색방법
CN100583832C (zh) * 2007-03-30 2010-01-20 华为技术有限公司 数据管理方法及系统
CN101446976A (zh) * 2008-12-26 2009-06-03 中兴通讯股份有限公司 一种分布式文件系统中的文件存储方法
JP5439236B2 (ja) * 2010-03-12 2014-03-12 株式会社日立製作所 計算機システムおよびアプリケーションプログラムの実行方法
JP2012073777A (ja) * 2010-09-28 2012-04-12 Kddi Corp 分散ファイルシステム制御装置
JP5677336B2 (ja) * 2011-08-01 2015-02-25 株式会社東芝 メモリ・デバイス
US9582204B2 (en) * 2014-01-07 2017-02-28 Apple Inc. Speculative prefetching of data stored in flash memory
KR101525453B1 (ko) * 2014-04-09 2015-06-04 인하대학교 산학협력단 데이터 접근빈도를 이용한 ｒａｉｄ 시스템의 데이터 복제 방법
WO2016106757A1 (zh) * 2014-12-31 2016-07-07 华为技术有限公司 一种存储数据的管理方法、存储管理器及存储系统
CN105205174B (zh) * 2015-10-14 2019-10-11 北京百度网讯科技有限公司 用于分布式系统的文件处理方法和装置
CN111488122A (zh) * 2017-10-25 2020-08-04 华为技术有限公司 数据写入方法和存储设备

Also Published As

Publication number Publication date
KR102240557B1 (ko) 2021-04-14
JP2020038623A (ja) 2020-03-12
CN109254733B (zh) 2021-10-01
CN109254733A (zh) 2019-01-22
KR20200027413A (ko) 2020-03-12

Similar Documents

Publication Publication Date Title
JP6778795B2 (ja) データを記憶するための方法、装置及びシステム
US9411685B2 (en) Parity chunk operating method and data server apparatus for supporting the same in distributed raid system
CN102169507B (zh) 一种分布式实时搜索引擎的实现方法
KR101694984B1 (ko) 비대칭 클러스터링 파일시스템에서의 패리티 산출 방법
US9507800B2 (en) Data management in distributed file systems
CN110262922B (zh) 基于副本数据日志的纠删码更新方法及系统
US8103621B2 (en) HSM two-way orphan reconciliation for extremely large file systems
US11093387B1 (en) Garbage collection based on transmission object models
JP5886447B2 (ja) ロケーション非依存のファイル
CN102938784A (zh) 应用于分布式存储系统中的数据存储方法和系统
CN104641365A (zh) 在文件存储系统中使用检查点管理去复制的系统和方法
CN104881466A (zh) 数据分片的处理以及垃圾文件的删除方法和装置
CN110347651A (zh) 基于云存储的数据同步方法、装置、设备及存储介质
US10346362B2 (en) Sparse file access
CN105183400A (zh) 一种基于内容寻址的对象存储方法和系统
US20190347165A1 (en) Apparatus and method for recovering distributed file system
CN105653209A (zh) 一种对象存储数据传输方法及装置
CN109597903A (zh) 图像文件处理装置和方法、文件存储系统及存储介质
CN112925859A (zh) 数据存储方法和装置
CN110245129B (zh) 一种分布式全局数据去重方法和装置
CN113190384B (zh) 基于纠删码的数据恢复控制方法、装置、设备及介质
CN113609090A (zh) 数据存储方法及装置、计算机可读存储介质、电子设备
CN112764662B (zh) 用于存储管理的方法、设备和计算机程序产品
CN112866406A (zh) 一种数据存储方法、系统、装置、设备及存储介质
CN115878625A (zh) 数据处理方法和装置及电子设备

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20190710

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20190710

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20200731

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20200915

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20201012

R150 Certificate of patent or registration of utility model

Ref document number: 6778795

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250