WO2013086867A1 - Procédé et équipement de traitement d'informations - Google Patents

Procédé et équipement de traitement d'informations Download PDF

Info

Publication number
WO2013086867A1
WO2013086867A1 PCT/CN2012/080274 CN2012080274W WO2013086867A1 WO 2013086867 A1 WO2013086867 A1 WO 2013086867A1 CN 2012080274 W CN2012080274 W CN 2012080274W WO 2013086867 A1 WO2013086867 A1 WO 2013086867A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
bitmap
storage device
network storage
hash values
Prior art date
Application number
PCT/CN2012/080274
Other languages
English (en)
Chinese (zh)
Inventor
陈跃泉
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2013086867A1 publication Critical patent/WO2013086867A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Definitions

  • the present invention relates to information processing technologies, and in particular, to an information processing method and device.
  • the terminal device or the network management system can periodically back up a large amount of information on the terminal device to the network backup device located on the network side, or the user/network administrator can manually manually use the terminal device as needed.
  • a large amount of information is backed up in a network backup device located on the network side.
  • the same information may be backed up multiple times in the network storage device, so that a large amount of the same information exists in the network storage device, thereby causing a decrease in the utilization rate of the network storage device, and Reduced efficiency of information maintenance and management.
  • the present invention provides an information processing method and device for improving the utilization of network storage devices and the efficiency of information maintenance and management.
  • an information processing method including:
  • the network storage device receives the information
  • the network storage device performs a hash operation on the information to obtain at least two hash values of the information
  • the network storage device stores the information or does not store the information according to the at least two hash values.
  • the storing, by the network storage device, the information according to the at least two hash values, or not storing the information includes:
  • the network storage device obtains a bitmap corresponding to the at least two hash values; the information does not store the information. Storing the information or not storing the information, comprising: comparing, by a bitmap of the at least one information received by the storage device, if the location corresponding to the bitmap corresponding to the at least two hash values is in the storage The network storage device does not store the information; if the location corresponding to the bitmap corresponding to the at least two hash values is not in the stored bitmap, the network storage device stores The information.
  • the method further includes: storing the bitmap corresponding to the at least two hash values. Before comparing the bitmap of at least one piece of information received by the network storage device, the method further includes:
  • the network storage device uses a bitmap corresponding to the hash value of the information as a reference bitmap, and stores the bitmap;
  • the network storage device If the network storage device stores at least two pieces of information, the network storage device performs or operates a bitmap corresponding to the hash value of the at least two pieces of information, and uses the obtained bitmap as a reference bitmap and stores the data; A bitmap of at least one piece of information received by the device is compared, including: Bitmaps for comparison.
  • the method further includes:
  • the network storage device performs an exclusive OR operation on the bitmap of the information within the selected range and the reference bitmap, and performs or operates with a bitmap of the information outside the selected range to obtain a new bitmap. As the reference bitmap after the update, and store it.
  • Another aspect provides an information processing apparatus, including:
  • a receiving unit configured to receive information
  • an obtaining unit configured to perform hash operation on the information, to obtain at least two hash values of the information
  • processing unit configured to store the information according to the at least two hash values or not to store the information.
  • processing unit is specifically configured to:
  • processing unit is specifically configured to:
  • processing unit is further configured to:
  • the network storage device uses a bitmap corresponding to the hash value of the information as a reference bitmap, and stores the bitmap;
  • the network storage device If the network storage device stores at least two pieces of information, the network storage device performs or operates a bitmap corresponding to the hash value of the at least two pieces of information, and uses the obtained bitmap as a reference bitmap and stores the bitmap;
  • the processing unit is specifically configured to
  • processing unit is further configured to:
  • the network storage device after receiving the information, obtains at least two hash values of the information by performing a hash operation on the foregoing information, so that the network storage device can store the foregoing according to the at least two hash values.
  • the information does not store the above information, which can avoid the problem that a large amount of the same information exists in the network storage device due to the fact that the same information may be backed up multiple times in the network storage device in the prior art, thereby improving the utilization of the network storage device. Reduced rates, as well as the efficiency of information maintenance and management.
  • FIG. 1 is a schematic flowchart of an information processing method according to an embodiment of the present invention.
  • FIG. 2 is a schematic flowchart of an information processing method according to another embodiment of the present invention.
  • FIG. 3 is a schematic structural diagram of an information processing device according to another embodiment of the present invention.
  • FIG. 1 is a schematic flowchart of an information processing method according to an embodiment of the present invention. As shown in FIG. 1, the information processing method in this embodiment may include:
  • the network storage device receives the information.
  • the foregoing information may include, but is not limited to, a short message, a multimedia message, a mail, or an instant message.
  • the network storage device performs a hash operation on the foregoing information to obtain at least two hash values of the foregoing information.
  • the network storage device performs a hash operation such as MD5 or CityHash on the information (that is, the content of the information), and uses at least two hash functions to map the binary value of the information into at least two fixed length smaller binary values. Hash value.
  • non-encrypted hash operations can be performed, such as: CityHash and other hash operations.
  • the network storage device stores the foregoing information or does not store the information according to the at least two hash values.
  • FIG. 2 is a schematic flowchart of an information processing method according to another embodiment of the present invention. This embodiment is a specific embodiment of the embodiment corresponding to FIG. 1. As shown in FIG. 2, the information processing method in this embodiment may include:
  • the network storage device receives the information.
  • the foregoing information may include, but is not limited to, a short message, a multimedia message, a mail, or an instant message.
  • the foregoing network storage device performs a hash operation on the foregoing information to obtain at least two hash values of the foregoing information.
  • the network storage device performs a hash operation such as MD5 or CityHash on the information (that is, the content of the information), and uses at least two hash functions to map the binary value of the information into at least two fixed length smaller binary values. Hash value.
  • non-encrypted hash operations can be performed, such as: CityHash and other hash operations.
  • the network storage device obtains a bitmap corresponding to the at least two hash values. For example, if the network storage device performs a hash operation on the information, the three hash values of the foregoing information are respectively obtained as 2 6, and 14 can be assumed that the bit corresponding to the hash value is 1, and the bitmap corresponding to the hash value can be 010001000000010000.
  • the foregoing network storage device stores the foregoing information or does not store the foregoing information according to the bitmap.
  • the network storage device Comparing the bitmaps of the at least one information received by the network storage device, if the location corresponding to the bitmap corresponding to the at least two hash values is in the stored bitmap, the network storage device does not store the information; The location corresponding to the bitmap corresponding to the at least two hash values is not in the stored bitmap, and the network storage device stores the information.
  • the bitmap corresponding to the at least two hash values may be further stored.
  • the network storage device may store a bitmap of the information after storing one piece of information each time.
  • the network storage device may further store the bitmap of the information into a bitmap of the previously stored information after storing one information each time, for example: a bitmap of the information. Perform or operate with a bitmap of previously stored information, use the resulting bitmap as a reference bitmap, and store the reference bitmap.
  • the network storage device Before the foregoing network storage device receives the bitmap and the network storage device, Before the bitmap of the at least one information is compared, if the network storage device stores a message, the network storage device may further use the bitmap corresponding to the hash value of the above information as the reference bitmap and store the data; if the network storage The device stores at least two pieces of information, and the network storage device may further perform or operate on the bitmap corresponding to the hash value of the at least two pieces of information, and use the obtained bitmap as a reference bitmap and store the bitmap.
  • the network storage device can compare the bitmap corresponding to the at least two hash values with the reference bitmap, if the location corresponding to the bitmap corresponding to the at least two hash values is stored in the foregoing In the reference bitmap, the network storage device does not store the information; if the location corresponding to the bitmap corresponding to the at least two hash values is not in the stored reference bitmap, the network storage device stores the information.
  • the network storage device in this embodiment may further clear the bits in the expired bitmap or bitmap in an instant.
  • the network storage device may perform an exclusive OR operation on the bitmap of the information within the selected range and the above reference bitmap, and perform or operate with a bitmap of the information outside the selected range to obtain a new bitmap.
  • the reference bitmap after the update, and stored it can meet a huge amount of information backup, and some information is time-effective (for example: a short message about a product promotion issued by a certain brand group, the effective time limit of the short message When the promotion is stopped, the short message has a time validity requirement, which saves the memory overhead occupied by the bitmap.
  • the network storage device after receiving the information, obtains at least two hash values of the foregoing information by performing a hash operation on the foregoing information, and obtains a bitmap corresponding to the at least two hash values, so that the network is configured.
  • the storage device can store the foregoing information or not store the foregoing information according to the foregoing bitmap, and can avoid a large amount of the same information in the network storage device caused by the same information being backed up in the network storage device multiple times in the prior art.
  • the problem is to improve the utilization of network storage devices and the efficiency of information maintenance and management.
  • an 18-bit bitmap will be exemplified below.
  • the network storage device does not currently store any information.
  • the first cycle is within T1
  • the network storage device stores information A, and obtains the bitmap corresponding to at least two hash values corresponding to A.
  • the T1 bitmap is used as a bitmap S1 bitmap of the network storage device in T1 and stored in the disk.
  • the second cycle is within T2
  • the network storage device performs or operates on the C bitmap and the T1 bitmap to obtain the reference bitmap T2 bitmap.
  • the C bitmap is used as a bitmap S2 bitmap of the network storage device in T2, and stored in the disk;
  • the third cycle is within T3
  • the network storage device receives the information E, and the bitmap corresponding to the at least two hash values corresponding to the information E is the E bitmap 00001 1000001000000. After comparison, the E bitmap is in the T2 bitmap, indicating that the network storage device has been stored. After the same information as the information E, the information E is a duplicate information, and the network storage device no longer stores the information E;
  • FIG. 3 is a schematic structural diagram of an information processing device according to another embodiment of the present invention.
  • the information processing device of this embodiment may include a receiving unit 31, an obtaining unit 32, and a processing unit 33.
  • the receiving unit 31 is configured to receive information
  • the obtaining unit 32 is configured to perform hash operation on the information received by the receiving unit 31 to obtain at least two hash values of the foregoing information.
  • the processing unit 33 is configured to obtain the information according to the obtaining unit 32.
  • the at least two hash values described above store the above information or do not store the above information.
  • the functions of the network storage device in the embodiments corresponding to the foregoing FIG. 1 and FIG. 2 can be implemented by the information processing device provided in this embodiment.
  • the bitmap should be stored, and the above information is stored or not stored according to the above bitmap.
  • the processing unit 33 compares the bitmap with the bitmap of the at least one information previously received by the network storage device, and if the location corresponding to the bitmap is in the stored bitmap, the information is not stored; The corresponding location is not in the bitmap stored above, and the above information is stored.
  • the processing unit 33 in this embodiment may further use a bitmap corresponding to the hash value of the one information as a reference bitmap, and store the data storage device; For at least two pieces of information, the processing unit 33 in this embodiment may further perform or operate on the bitmap corresponding to the hash value of the at least two pieces of information, and use the obtained bitmap as a reference bitmap and store the bitmap. The corresponding bitmap is compared to the reference bitmap.
  • the information processing device in this embodiment may further clear the bits in the expired bitmap or the bitmap in an instant.
  • the processing unit 33 may be further configured to perform an exclusive OR operation on the bitmap of the information within the selected range and the reference, and perform or operate with a bitmap of the information outside the selected range to obtain a new bit.
  • the map is used as a reference bitmap after the update, and is stored, so as to satisfy a huge amount of information backup, and some information is time-effective (for example: a short message about a product promotion issued by a certain brand group, the effective time of the short message) When the promotion is stopped, it indicates that the short message has time validity), which saves the memory overhead occupied by the bitmap.
  • the obtaining unit performs a hash operation on the information received by the receiving unit to obtain at least two hash values of the information, so that the processing unit can obtain the information according to the obtaining unit.
  • the foregoing at least two hash values store the foregoing information or do not store the foregoing information, which can avoid the existence of a large amount of the same information in the network storage device caused by the same information being backed up in the network storage device multiple times in the prior art.
  • the problem is to improve the utilization of network storage devices and the efficiency of information maintenance and management.
  • the disclosed systems, devices, and methods may be implemented in other ways.
  • the device embodiments described above are merely illustrative.
  • the division of the unit is only a logical function division.
  • there may be another division manner for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not executed.
  • the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be electrical, mechanical or otherwise.
  • the units described as separate components may or may not be physically separate, and the components displayed as the units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solution of the embodiment.
  • each functional unit in various embodiments of the present invention may be integrated in one processing unit. It is also possible that each unit physically exists alone, or two or more units may be integrated in one unit.
  • the above integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional units.
  • the above-described integrated unit implemented in the form of a software functional unit can be stored in a computer readable storage medium.
  • the above software functional units are stored in a storage medium and include a number of instructions for causing a computer device (which may be a personal computer, server, or network device, etc.) to perform part of the steps of the method of the various embodiments of the present invention.
  • the foregoing storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, and the like.
  • the medium of the program code includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, and the like.

Abstract

La présente invention concerne un procédé de traitement d'informations consistant en ce qui suit : l'équipement de stockage de réseau reçoit des informations ; l'équipement de stockage de réseau effectue une opération de hachage sur les informations, et obtient au moins deux valeurs de hachage des informations ; selon lesdites au moins deux valeurs de hachage, l'équipement de stockage de réseau enregistre ou n'enregistre pas les informations. Et un équipement de traitement d'informations comprend : une unité de réception, qui sert à recevoir des informations ; une unité d'obtention, qui sert à réaliser une opération de hachage sur les informations et obtenir au moins deux valeurs de hachage des informations ; une unité de traitement, qui sert à enregistrer ou à ne pas enregistrer les informations selon lesdites au moins deux valeurs de hachage. Grâce à l'invention, la réduction du taux d'utilisation de l'équipement de stockage de réseau et l'efficacité de la maintenance et de la gestion des informations sont améliorées.
PCT/CN2012/080274 2011-12-14 2012-08-17 Procédé et équipement de traitement d'informations WO2013086867A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201110416964.0 2011-12-14
CN201110416964.0A CN102523112B (zh) 2011-12-14 2011-12-14 信息处理方法及设备

Publications (1)

Publication Number Publication Date
WO2013086867A1 true WO2013086867A1 (fr) 2013-06-20

Family

ID=46293911

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2012/080274 WO2013086867A1 (fr) 2011-12-14 2012-08-17 Procédé et équipement de traitement d'informations

Country Status (2)

Country Link
CN (1) CN102523112B (fr)
WO (1) WO2013086867A1 (fr)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102523112B (zh) * 2011-12-14 2015-09-09 华为技术有限公司 信息处理方法及设备
CN105760404A (zh) * 2014-12-19 2016-07-13 富士通株式会社 数据集成方法和装置
CN110335125A (zh) * 2019-07-12 2019-10-15 政采云有限公司 一种信息报备方法及装置
CN112330398B (zh) * 2020-10-30 2024-04-09 京东科技控股股份有限公司 一种对象处理方法、装置、电子设备及存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100312752A1 (en) * 2009-06-08 2010-12-09 Symantec Corporation Source Classification For Performing Deduplication In A Backup Operation
US20110093439A1 (en) * 2009-10-16 2011-04-21 Fanglu Guo De-duplication Storage System with Multiple Indices for Efficient File Storage
CN102194002A (zh) * 2011-05-25 2011-09-21 中兴通讯股份有限公司 哈希表的表项添加、删除、查找方法及哈希表存储装置
CN102523112A (zh) * 2011-12-14 2012-06-27 华为技术有限公司 信息处理方法及设备

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CA2706007C (fr) * 2006-12-22 2016-08-30 Commvault Systems, Inc. Systeme et procede pour stocker des informations redondantes
CN101882141A (zh) * 2009-05-08 2010-11-10 北京众志和达信息技术有限公司 一种实现重复数据数据删除的方法和系统
CN102184128B (zh) * 2011-05-26 2013-04-10 成都易我科技开发有限责任公司 快速磁盘增量备份方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100312752A1 (en) * 2009-06-08 2010-12-09 Symantec Corporation Source Classification For Performing Deduplication In A Backup Operation
US20110093439A1 (en) * 2009-10-16 2011-04-21 Fanglu Guo De-duplication Storage System with Multiple Indices for Efficient File Storage
CN102194002A (zh) * 2011-05-25 2011-09-21 中兴通讯股份有限公司 哈希表的表项添加、删除、查找方法及哈希表存储装置
CN102523112A (zh) * 2011-12-14 2012-06-27 华为技术有限公司 信息处理方法及设备

Also Published As

Publication number Publication date
CN102523112A (zh) 2012-06-27
CN102523112B (zh) 2015-09-09

Similar Documents

Publication Publication Date Title
WO2021164178A1 (fr) Procédé et appareil de téléversement de fragment de fichier basé sur la technologie en nuage, et dispositif et support de stockage
EP3208714B1 (fr) Procédé, appareil et système de reconstruction de données dans un système de stockage distribué
CN109949111B (zh) 电子票据标识分配方法、电子票据生成方法、装置及系统
CN110569309B (zh) 用于实现区块链的设备、方法、系统以及介质
EP3128719B1 (fr) Procédé de traitement de quorum après déconnexion de deux parties d'une grappe, et dispositif et système de stockage de quorum
WO2018076759A1 (fr) Procédé et système de gestion multi-chaînes fondés sur une chaîne de blocs, dispositif électronique et support d'informations
TWI669620B (zh) Database switching method, server, storage medium, electronic device and product
US8375200B2 (en) Embedded device and file change notification method of the embedded device
CN110377570B (zh) 节点切换方法、装置、计算机设备及存储介质
KR20160139493A (ko) 클라우드 서비스를 위한 암호화 키 관리 방법 및 그 장치
US10943010B2 (en) Generation of a random value for a child process
WO2021121407A1 (fr) Procédé et appareil de changement de capacité pour machine virtuelle
CN111880967A (zh) 云场景下的文件备份方法、装置、介质和电子设备
WO2013086867A1 (fr) Procédé et équipement de traitement d'informations
CN111338834B (zh) 数据存储方法和装置
CN115934414A (zh) 数据备份方法、数据恢复方法、装置、设备及存储介质
CN110442601B (zh) 一种Openstack镜像数据并行加速的方法和装置
CN111857609A (zh) 一种上传云打印日志的方法、装置和计算机存储介质
WO2014107836A1 (fr) Dispositif et procédé d'alarme de réseau
US8412844B2 (en) Validating download success
EP4142258A1 (fr) Réseau informatique périphérique, procédé et appareil de transmission de données, dispositif et support de stockage
WO2012146091A1 (fr) Traitement d'informations d'authentification
CN107395772B (zh) 一种重复数据的管理方法及管理系统
CN111432357B (zh) 信息处理方法、系统及计算设备
US10250515B2 (en) Method and device for forwarding data messages

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 12857167

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 12857167

Country of ref document: EP

Kind code of ref document: A1