BR112014009477A2 - método e aparelho de processamento de dados em um sistema de agrupamento - Google Patents

método e aparelho de processamento de dados em um sistema de agrupamento

Info

Publication number
BR112014009477A2
BR112014009477A2 BR112014009477A BR112014009477A BR112014009477A2 BR 112014009477 A2 BR112014009477 A2 BR 112014009477A2 BR 112014009477 A BR112014009477 A BR 112014009477A BR 112014009477 A BR112014009477 A BR 112014009477A BR 112014009477 A2 BR112014009477 A2 BR 112014009477A2
Authority
BR
Brazil
Prior art keywords
processing method
increase
data processing
data stream
sketch value
Prior art date
Application number
BR112014009477A
Other languages
English (en)
Other versions
BR112014009477B1 (pt
Inventor
Zhou Dan
Yang Huadi
You Jun
Liu Qiang
Sun Quancheng
Liu Xiaobo
Huang Yan
Original Assignee
Huawei Tech Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Tech Co Ltd filed Critical Huawei Tech Co Ltd
Publication of BR112014009477A2 publication Critical patent/BR112014009477A2/pt
Publication of BR112014009477B1 publication Critical patent/BR112014009477B1/pt

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0608—Saving storage space on storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0626—Reducing size or complexity of storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/17—Details of further file system functions
    • G06F16/174—Redundancy elimination performed by the file system
    • G06F16/1748—De-duplication implemented within the file system, e.g. based on file segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/064—Management of blocks
    • G06F3/0641—De-duplication techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)

Abstract

resumo método e aparelho de processamento de dados em um sistema de agrupamento em modalidades da presente invenção, quando uma consulta de dados em duplicata é realizada em um fluxo de dados recebido, um primeiro nó físico o qual corresponde a cada primeiro valor de esboço e está em um sistema de agrupamento é identificado, de acordo com um primeiro valor de esboço representando o fluxo de dados, e, então, o primeiro valor de esboço representando o fluxo de dados é enviado para o nó físico identificado para a consulta de dados em duplicata, e um procedimento da consulta de dados em duplicata não muda com um aumento do número de nós no sistema de agrupamento; portanto, uma quantidade de cálculo de cada nó não aumenta com um aumento do número de nós no sistema de agrupamento.
BR112014009477A 2012-12-12 2012-12-12 método e aparelho de processamento de dados em um sistema de agrupamento BR112014009477B1 (pt)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2012/086413 WO2014089767A1 (zh) 2012-12-12 2012-12-12 集群系统中数据处理方法及装置

Publications (2)

Publication Number Publication Date
BR112014009477A2 true BR112014009477A2 (pt) 2017-05-09
BR112014009477B1 BR112014009477B1 (pt) 2018-10-16

Family

ID=49970062

Family Applications (1)

Application Number Title Priority Date Filing Date
BR112014009477A BR112014009477B1 (pt) 2012-12-12 2012-12-12 método e aparelho de processamento de dados em um sistema de agrupamento

Country Status (12)

Country Link
US (1) US8892529B2 (pt)
EP (3) EP2765524B1 (pt)
JP (1) JP5824167B2 (pt)
KR (2) KR101657561B1 (pt)
CN (2) CN106445413B (pt)
AU (1) AU2012389110B2 (pt)
BR (1) BR112014009477B1 (pt)
CA (1) CA2843922C (pt)
DK (2) DK3217298T3 (pt)
ES (2) ES2700431T3 (pt)
WO (1) WO2014089767A1 (pt)
ZA (1) ZA201401528B (pt)

Families Citing this family (27)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106663064A (zh) * 2014-09-04 2017-05-10 富士通株式会社 信息处理系统、信息处理装置、信息处理方法、以及信息处理程序
CN104239518B (zh) * 2014-09-17 2017-09-29 华为技术有限公司 重复数据删除方法和装置
CN105490903B (zh) * 2014-09-19 2019-07-19 中国银联股份有限公司 一种基于总线模式的集群架构
US10346075B2 (en) 2015-03-16 2019-07-09 Hitachi, Ltd. Distributed storage system and control method for distributed storage system
CN106294357B (zh) * 2015-05-14 2019-07-09 阿里巴巴集团控股有限公司 数据处理方法和流计算系统
US10657274B2 (en) * 2015-06-29 2020-05-19 Samsng Electronics Co., Ltd. Semiconductor device including memory protector
US10606499B2 (en) * 2015-09-11 2020-03-31 Hitachi, Ltd. Computer system, storage apparatus, and method of managing data
US10359942B2 (en) * 2016-10-31 2019-07-23 Pure Storage, Inc. Deduplication aware scalable content placement
US9946486B1 (en) 2016-11-29 2018-04-17 Veritas Technologies Llc Duplication between disparate deduplication systems
US10795860B1 (en) * 2017-04-13 2020-10-06 EMC IP Holding Company LLC WAN optimized micro-service based deduplication
CN107193503B (zh) * 2017-05-27 2020-05-29 杭州宏杉科技股份有限公司 一种数据重删方法及存储设备
JP6934068B2 (ja) 2017-06-08 2021-09-08 ヒタチ ヴァンタラ コーポレーションＨｉｔａｃｈｉ Ｖａｎｔａｒａ Ｃｏｒｐｏｒａｔｉｏｎ 分散されたイレイジャーコーディングされたオブジェクトの重複排除
US10055198B1 (en) * 2017-06-13 2018-08-21 Sap Se Systems and methods for probably approximate intent matching of procurement rules
CN107291396B (zh) * 2017-06-27 2018-04-13 北京华云网际科技有限公司 分布式块存储系统中的io数据的写入方法和装置
US10936543B1 (en) 2017-07-21 2021-03-02 EMC IP Holding Company LLC Metadata protected sparse block set for SSD cache space management
US11461269B2 (en) 2017-07-21 2022-10-04 EMC IP Holding Company Metadata separated container format
US11113153B2 (en) 2017-07-27 2021-09-07 EMC IP Holding Company LLC Method and system for sharing pre-calculated fingerprints and data chunks amongst storage systems on a cloud local area network
US11093453B1 (en) 2017-08-31 2021-08-17 EMC IP Holding Company LLC System and method for asynchronous cleaning of data objects on cloud partition in a file system with deduplication
CN108664217B (zh) * 2018-04-04 2021-07-13 安徽大学 一种降低固态盘存储系统写性能抖动的缓存方法及系统
CN110134331B (zh) * 2019-04-26 2020-06-05 重庆大学 路由路径规划方法、系统及可读存储介质
CN110837515A (zh) * 2019-11-06 2020-02-25 北京天融信网络安全技术有限公司 一种基于数据库的数据处理方法及电子设备
CN113220214A (zh) 2020-02-06 2021-08-06 伊姆西Ip控股有限责任公司 多节点存储系统及其数据去重方法
CN114185485B (zh) * 2021-11-04 2024-06-14 浙江华忆芯科技有限公司 静态电压表的节点处理方法、装置、计算机设备和存储介质
CN114265551B (zh) * 2021-12-02 2023-10-20 阿里巴巴（中国）有限公司 存储集群中的数据处理方法、存储集群、存储节点及设备
CN115604358A (zh) * 2022-09-21 2023-01-13 网易（杭州）网络有限公司（Cn） 业务处理方法、系统、装置、管理服务器及存储介质
US20240103976A1 (en) * 2022-09-23 2024-03-28 Dell Products L.P. Distributed and deduplicating file system for storing backup data to object storage
CN115904738B (zh) * 2023-01-05 2023-06-06 摩尔线程智能科技(北京)有限责任公司 数据处理装置集群的管理系统和控制方法

Family Cites Families (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1996025801A1 (en) * 1995-02-17 1996-08-22 Trustus Pty. Ltd. Method for partitioning a block of data into subblocks and for storing and communicating such subblocks
JP4068473B2 (ja) * 2003-02-19 2008-03-26 株式会社東芝 ストレージ装置、分担範囲決定方法及びプログラム
US8412682B2 (en) * 2006-06-29 2013-04-02 Netapp, Inc. System and method for retrieving and using block fingerprints for data deduplication
US8166012B2 (en) * 2007-04-11 2012-04-24 Emc Corporation Cluster storage using subsegmenting
US8195636B2 (en) * 2009-04-29 2012-06-05 Netapp, Inc. Predicting space reclamation in deduplicated datasets
US9058298B2 (en) * 2009-07-16 2015-06-16 International Business Machines Corporation Integrated approach for deduplicating data in a distributed environment that involves a source and a target
US8321648B2 (en) * 2009-10-26 2012-11-27 Netapp, Inc Use of similarity hash to route data for improved deduplication in a storage server cluster
US8244992B2 (en) * 2010-05-24 2012-08-14 Spackman Stephen P Policy based data retrieval performance for deduplicated data
US8370315B1 (en) * 2010-05-28 2013-02-05 Symantec Corporation System and method for high performance deduplication indexing
CN201804331U (zh) * 2010-09-21 2011-04-20 北京同有飞骥科技股份有限公司 一种基于协处理器的重复数据删除系统
CN102456059A (zh) * 2010-10-21 2012-05-16 英业达股份有限公司 重复数据删除的处理系统
CN102479245B (zh) * 2010-11-30 2013-07-17 英业达集团(天津)电子技术有限公司 数据区块的切分方法
US8898119B2 (en) * 2010-12-15 2014-11-25 Netapp, Inc. Fingerprints datastore and stale fingerprint removal in de-duplication environments
US8442941B2 (en) * 2011-01-19 2013-05-14 Microsoft Corporation Scalable database workload replay with mode selections
CN102323958A (zh) * 2011-10-27 2012-01-18 上海文广互动电视有限公司 重复数据删除方法
US9047304B2 (en) * 2011-11-28 2015-06-02 International Business Machines Corporation Optimization of fingerprint-based deduplication
CN102495894A (zh) * 2011-12-12 2012-06-13 成都市华为赛门铁克科技有限公司 重复数据查找方法、装置及系统
US8712978B1 (en) * 2012-06-13 2014-04-29 Emc Corporation Preferential selection of candidates for delta compression

Also Published As

Publication number Publication date
KR101657561B1 (ko) 2016-09-19
EP3217298B1 (en) 2018-08-29
CA2843922A1 (en) 2014-06-12
BR112014009477B1 (pt) 2018-10-16
CN103547329A (zh) 2014-01-29
AU2012389110B2 (en) 2016-03-17
ES2625690T3 (es) 2017-07-20
EP2765524A1 (en) 2014-08-13
EP3217298A1 (en) 2017-09-13
EP3460683B1 (en) 2021-08-11
AU2012389110A1 (en) 2014-06-26
EP3460683A1 (en) 2019-03-27
ZA201401528B (en) 2015-09-30
EP2765524A4 (en) 2015-06-03
CN106445413A (zh) 2017-02-22
CN103547329B (zh) 2016-11-02
CA2843922C (en) 2016-11-15
KR20140101720A (ko) 2014-08-20
CN106445413B (zh) 2019-10-25
US20140201169A1 (en) 2014-07-17
ES2700431T3 (es) 2019-02-15
DK2765524T3 (en) 2017-05-22
US8892529B2 (en) 2014-11-18
KR101575062B1 (ko) 2015-12-07
JP2015503174A (ja) 2015-01-29
EP2765524B1 (en) 2017-02-22
JP5824167B2 (ja) 2015-11-25
WO2014089767A1 (zh) 2014-06-19
KR20150140863A (ko) 2015-12-16
DK3217298T3 (da) 2019-01-02

Similar Documents

Publication Publication Date Title
BR112014009477A2 (pt) método e aparelho de processamento de dados em um sistema de agrupamento
BR112017011189A2 (pt) sistemas e métodos para fornecer redes sem fio virtuais customizadas com base em auto-criação de rede orientada a serviços
BR112019006260A2 (pt) método de acesso aleatório implementado por um dispositivo sem fio, método implementado por um nó de rede, dispositivo sem fio, nó de rede, programa de computador, portadora, e, mídia de armazenamento legível por computador não transitória.
BR112017026670A2 (pt) sistema e método para fornecer informações contextuais para uma localização
BR112015030590A2 (pt) sistema e método para mapear uma topologia de nível de serviço para uma topologia lógica de plano de dados específica quanto ao serviço
BR112018013550A2 (pt) identificação de entidades utilizando um modelo de aprendizado profundo
WO2014145076A3 (en) Crowdsourcing domain specific intelligence
BR112016029555A2 (pt) sistemas e métodos para a realização seletiva de uma verificação de conformidade de sequência de bits
MX369047B (es) Sistemas y metodos para la asignacion y enrutamiento basandose en agrupacion.
BR112017021925A2 (pt) gerenciamento de compromissos e de solicitações extraídos a partir de comunicações e conteúdo
IN2015DN02657A (pt)
BR112014018864A8 (pt) Sistema de categorização de resultado de pesquisa e método implementado por computador
BR112015027756A8 (pt) método, sistema e um ou mais meios de armazenamento não transitórios legíveis por computador para uso de operadores inversos para consultas em redes sociais online
BR112016002281A2 (pt) segurança de informação cognitiva usando um sistema de reconhecimento de comportamento
BR112015003406A8 (pt) Método implementado por computador e sistema de computação
MX2015008799A (es) Sistema y metodo para motores de consulta de base de datos distribuidos.
IN2013CH05115A (pt)
BR112017009726A2 (pt) métodos e aparelho para verificar informação de sistema
BR112017024449A2 (pt) definição de banda estreita para comunicação fortalecida do tipo máquina
BR112016004164A8 (pt) método e aparelho para seleção de nodo pai preferido em rede de sensor sem fio
WO2014190337A3 (en) Requesting proximate resources by learning devices
EP3005276A4 (en) Apparatus, system, and method of elastically processing message information from multiple sources
GB201310169D0 (en) A method for attesting a plurality of data processing systems
WO2015167427A3 (en) Data distribution based on network information
BR112016024885A2 (pt) identificação de intenção de pesquisa

Legal Events

Date Code Title Description
B09A Decision: intention to grant [chapter 9.1 patent gazette]
B16A Patent or certificate of addition of invention granted [chapter 16.1 patent gazette]

Free format text: PRAZO DE VALIDADE: 20 (VINTE) ANOS CONTADOS A PARTIR DE 12/12/2012, OBSERVADAS AS CONDICOES LEGAIS.