CN109150537B - File ownership proving method based on dynamic Bloom Filter - Google Patents

File ownership proving method based on dynamic Bloom Filter Download PDF

Info

Publication number
CN109150537B
CN109150537B CN201810660485.5A CN201810660485A CN109150537B CN 109150537 B CN109150537 B CN 109150537B CN 201810660485 A CN201810660485 A CN 201810660485A CN 109150537 B CN109150537 B CN 109150537B
Authority
CN
China
Prior art keywords
file
bloom filter
user
server
data block
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201810660485.5A
Other languages
Chinese (zh)
Other versions
CN109150537A (en
Inventor
柳毅
王平雁
凌捷
欧毓毅
罗玉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong University of Technology
Original Assignee
Guangdong University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong University of Technology filed Critical Guangdong University of Technology
Priority to CN201810660485.5A priority Critical patent/CN109150537B/en
Publication of CN109150537A publication Critical patent/CN109150537A/en
Application granted granted Critical
Publication of CN109150537B publication Critical patent/CN109150537B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/32—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
    • H04L9/3271—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials using challenge-response
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data
    • G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
    • G06F21/6209—Protecting access to data via a platform, e.g. using keys or access control rules to a single file or object, e.g. in a secure envelope, encrypted and accessed using a key, or with access control rules appended to the object itself
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/32—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
    • H04L9/321—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials involving a third party or a trusted authority
    • H04L9/3213—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials involving a third party or a trusted authority using tickets or tokens, e.g. Kerberos
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/32—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
    • H04L9/3236—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials using cryptographic hash functions

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Security & Cryptography (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Theoretical Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Bioethics (AREA)
  • Computer Hardware Design (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a file ownership proving method based on dynamic Bloom Filter, which comprises the following steps: s1, the user calculates the abstract h of the file F to be uploadedfUploading the data to a server; s2, the server checks whether the duplication exists, if so, the server initiates a verification challenge to the user, and goes to the S3 challenge stage; otherwise, the user is required to upload the file, and the process goes to the uploading stage of S4; the file ownership proving method based on the dynamic Bloom Filter ensures that an attacker cannot obtain a complete file only by virtue of the file abstract, and meanwhile, on the premise of ensuring the high verification efficiency of the Bloom Filter, the problem of increase of the misjudgment rate is solved, and the practicability of the scheme is greatly improved; the method adopts the dynamic Bloom Filter to dynamically manage the size of the Bloom Filter, and when the number of files in the cloud storage system is increased, the dynamic Bloom Filter can be enlarged, so that the false positive misjudgment rate can be effectively controlled within an acceptable range.

Description

File ownership proving method based on dynamic Bloom Filter
Technical Field
The invention relates to the technical field of file ownership certification, in particular to a file ownership certification method based on dynamic Bloom Filter.
Background
With the continuous development of Cloud computing technology, more and more users choose to outsource data to the Cloud for storage and management, and the problem that how to economically, efficiently and safely store data due to unprecedented large-scale data volume is a problem that Cloud Service Providers (CSP) need to solve urgently is faced. One of the technologies used for reducing the cloud storage overhead is a data deduplication (deduplication) technology, also called a deduplication technology, which only retains one copy of data, eliminates redundant data inside and between files in cloud storage, and greatly reduces the consumption of storage space and network bandwidth.
Typical storage systems often adopt the abstract of a file as a file Ownership certificate of a user, so that an attacker can obtain a complete file only by the abstract of the file, and in order to prevent the complete file from being obtained by the abstract of the file or part of the file content, researchers have proposed Proof of Ownership (PoW), and the user must pass authentication under the condition of owning the file, so as to obtain the file authority.
The Bloom Filter is an efficient probabilistic data structure for determining whether an element belongs to a specific set, and generally consists of 1 binary vector and k mutually independent hash functions. Setting a binary vector with m bits in a Bloom Filter, and initializing all bits to be 0; there are n elements in the set, each element passing through k hash functions { H }1，H2，…，HkMap into the range of {1, 2, …, m }; when inserting element x, the location H to which the kth hash function is mappedi(x) Setting the value to 1, as shown in fig. 1, where n is 2, k is 3, and the position pointed by the arrow is the bit mapped by the hash function, and setting the bit to 1; when querying a certain data object s, { H } is computed1(s)，H2(s)，…，Hk(s), checking whether all mapping positions are 1, and if not all mapping positions are 1, judging that the set does not contain s; if all 1 s are likely to belong to the set, but there is a certain probability of misjudgment. Assuming that the k function mapping positions of an element not belonging to a set are exactly all 1, the element is misjudged to belong to the set, which is called false positive (false positives), and the misjudgment rate is higher when the number of elements in the set is more.
The Dynamic Bloom Filter (DBF, Dynamic Bloom Filter) consists of several Standard Bloom filters (SBF, Standard Bloom Filter); the number of SBFs in the DBF is 1 in the initial stage, the state is active, namely the misjudgment rate is smaller than the upper limit value, with the continuous insertion of new elements, the final state becomes full, namely the misjudgment rate reaches the upper limit value, a new SBF is added at the moment, the state is ensured to be active all the time, and therefore the DBF can control the misjudgment rate. Besides the insertion operation, the DBF can also execute operations such as query, deletion, combination and the like; the DBF needs to initialize several parameters: the maximum misjudgment rate of the DBF, the upper limit value of the SBF number s, the maximum misjudgment rate of the SBF, the size m of a single SBF, the capacity c of the single SBF and the hash function number k of the single SBF.
In the existing cloud storage ownership proof deduplication scheme, the deduplication scheme based on Bloom filters is high in efficiency, but improvements still remain in aspects of scalability, extensibility, performance and the like, for example, the Bloom filters adopted in the existing scheme are fixed in size, the misjudgment rate is continuously increased along with the increase of cloud storage files, and elements cannot be deleted.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provide a file ownership proving method based on a dynamic Bloom Filter, which enables an attacker not to obtain a complete file only by virtue of a file abstract, solves the problem of increase of misjudgment rate on the premise of ensuring the high verification efficiency of the Bloom Filter, and greatly improves the practicability of the scheme.
The purpose of the invention is realized by the following technical scheme:
a file ownership certification method based on dynamic Bloom Filter comprises the following steps:
s1, the user calculates the abstract h of the file F to be uploadedfUploading the data to a server;
s2, the server checks whether the duplication exists, if so, the server initiates a verification challenge to the user, and goes to the S3 challenge stage; otherwise, the user is required to upload the file, and the process goes to the uploading stage of S4;
s3, challenge phase:
s3.1, the server sends the data block division parameter p and the randomly selected J data block indexes to the user;
s3.2, dividing the file into n data blocks { B ] by the user according to the data block dividing parameter piCalculating labels { token n of corresponding J data blocks (i is more than or equal to 1 and less than or equal to n)jAnd return it to the server;
s3.3, the server takes the token value as seed initialization PRF and generates corresponding data block index, and the Bloom Filter calculates k hash functions { H }1，H2，…，HkChecking whether the index value is in the set; if all the bits mapped in the Bloom Filter are 1, indicating that the user passes the verification to obtain the file authority; otherwise, the user challenge fails;
s4, uploading:
s4.1, the user sends the size of the file to be uploaded to a server, and the server returns a data block division parameter p according to the size of the file;
s4.2, calculating the file abstract h by the userfDividing the file into n data blocks { B ] according to the data block dividing parameter piCalculating the label of the data block to obtain { token } (i is more than or equal to 1 and less than or equal to n)iFinal user will { token }i}、{BiH andfuploading to a server;
s4.3, the server creates a dynamic Bloom Filter, takes the token value as seed initialization PRF and generates a corresponding data block index, inserts the data block index into the Bloom Filter, and sets the mapping bit position to be 1; and detecting whether the state of the Bloom Filter is active or full, and if the state of the Bloom Filter is full, creating a new SBF in the DBF so that the misjudgment rate is in a controllable range.
Compared with the prior art, the invention has the following beneficial effects:
the file ownership proving method based on the dynamic Bloom Filter ensures that an attacker cannot obtain a complete file only by virtue of the file abstract, and meanwhile, on the premise of ensuring the high verification efficiency of the Bloom Filter, the problem of increase of the misjudgment rate is solved, and the practicability of the scheme is greatly improved; the method adopts the dynamic Bloom Filter to dynamically manage the size of the Bloom Filter, and when the number of files in the cloud storage system is increased, the dynamic Bloom Filter can be enlarged, so that the false positive misjudgment rate can be effectively controlled within an acceptable range.
Drawings
FIG. 1 is a diagram of a standard Bloom Filter mapping;
FIG. 2 is a schematic diagram of the dynamic Bloom Filter structure of the present invention;
FIG. 3 is a flow chart of the present invention.
Symbol description table
Figure BDA0001706301530000041
Figure BDA0001706301530000051
Detailed Description
The present invention will be described in further detail with reference to examples and drawings, but the present invention is not limited thereto.
As shown in fig. 2 to 3, a file ownership proving method based on dynamic Bloom Filter includes the following steps:
s1, the user calculates the abstract h of the file F to be uploadedfUploading the data to a server;
s2, the server checks whether the duplication exists, if so, the server initiates a verification challenge to the user, and goes to the S3 challenge stage; otherwise, the user is required to upload the file, and the process goes to the uploading stage of S4;
s3, challenge phase:
s3.1, the server sends the data block division parameter p and the randomly selected J data block indexes to the user;
s3.2, dividing the file into n data blocks { B ] by the user according to the data block dividing parameter piCalculating labels { token n of corresponding J data blocks (i is more than or equal to 1 and less than or equal to n)jAnd return it to the server;
s3.3, the server takes the token value as seed initialization PRF and generates corresponding data block index, and the Bloom Filter calculates k hash functions { H }1，H2，…，HkChecking whether the index value is in the set; if all the bits mapped in the Bloom Filter are 1, indicating that the user passes the verification to obtain the file authority; otherwise, the user challenge fails;
s4, uploading:
s4.1, the user sends the size of the file to be uploaded to a server, and the server returns a data block division parameter p according to the size of the file;
s4.2, calculating the file abstract h by the userfDividing the file into n data blocks { B ] according to the data block dividing parameter piCalculating the label of the data block to obtain { token } (i is more than or equal to 1 and less than or equal to n)iFinal user will { token }i}、{BiH andfuploading to a server;
s4.3, the server creates a dynamic Bloom Filter, takes the token value as seed initialization PRF and generates a corresponding data block index, inserts the data block index into the Bloom Filter, and sets the mapping bit position to be 1; and detecting whether the state of the Bloom Filter is active or full, and if the state of the Bloom Filter is full, creating a new SBF in the DBF so that the misjudgment rate is in a controllable range.
The user can obtain the file authority only through the verification of the ownership certificate, the method can be used for cloud storage cross-client duplicate removal, and the potential safety hazard that the user can obtain a complete file only by means of the file abstract can be avoided; the invention is also suitable for removing the duplication of the ciphertext data, and the ciphertext data duplication removing scheme is similar, for example, the data can be encrypted by adopting convergent encryption on the basis of the method, so that the same encryption result of different users on the same data is ensured, and the cross-user ciphertext data duplication removing can be realized.
The file ownership proving method based on the dynamic Bloom Filter ensures that an attacker cannot obtain a complete file only by virtue of the file abstract, and meanwhile, on the premise of ensuring the high verification efficiency of the Bloom Filter, the problem of increase of the misjudgment rate is solved, and the practicability of the scheme is greatly improved; when the server carries out repeated detection and finds that a file to be uploaded by a user exists in a cloud storage system, the server initiates a challenge to the user, as challenge data blocks are random, the probability of correct response of an illegal user to the challenge is negligible under the condition that no complete file exists, however, the illegal user still can pass verification when a Bloom Filter generates false positive misjudgment, as the existing scheme adopts a standard Bloom Filter, the size is fixed, and the false positive misjudgment rate is continuously increased along with the increase of the file in the cloud storage system, the practicability of the scheme is greatly limited.
The present invention is not limited to the above embodiments, and any other changes, modifications, substitutions, combinations, and simplifications which do not depart from the spirit and principle of the present invention should be construed as equivalents and are included in the scope of the present invention.

Claims (1)

1. A file ownership certification method based on dynamic Bloom Filter is characterized by comprising the following steps:
s1, the user calculates the abstract h of the file F to be uploadedfUploading the data to a server;
s2, the server checks whether the duplication exists, if so, the server initiates a verification challenge to the user, and goes to the S3 challenge stage; otherwise, the user is required to upload the file, and the process goes to the uploading stage of S4;
s3, challenge phase:
s3.1, the server sends the data block division parameter p and the randomly selected J data block indexes to the user;
s3.2, dividing the file into n data blocks { B ] by the user according to the data block dividing parameter piI is more than or equal to 1 and less than or equal to n, and labels { token n of corresponding J data blocks in the n data blocks are calculatedjAnd return it to the server;
s3.3, the server takes the token value as a seed initialization pseudo random function and generates a corresponding data block index, and the Bloom Filter calculates k hash functionsNumber { H1，H2，…，HkChecking whether the index value is in the set; if all the bits mapped in the Bloom Filter are 1, indicating that the user passes the verification to obtain the file authority; otherwise, the user challenge fails;
s4, uploading:
s4.1, the user sends the size of the file to be uploaded to a server, and the server returns a data block division parameter p according to the size of the file;
s4.2, calculating the file abstract h by the userfDividing the file into n data blocks { B ] according to the data block dividing parameter piI is more than or equal to 1 and less than or equal to n, and calculating the label of the data block to obtain { token }iFinal user will { token }i}、{BiH andfuploading to a server;
s4.3, the server creates a dynamic Bloom Filter, takes the token value as a seed initialization pseudo-random function and generates a corresponding data block index, the corresponding data block index is inserted into the Bloom Filter, and the mapped bit positions are all set to be 1; and detecting whether the state of the Bloom Filter is active or full, and if the state of the Bloom Filter is full, creating a new SBF in the DBF so that the misjudgment rate is in a controllable range.
CN201810660485.5A 2018-06-25 2018-06-25 File ownership proving method based on dynamic Bloom Filter Active CN109150537B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810660485.5A CN109150537B (en) 2018-06-25 2018-06-25 File ownership proving method based on dynamic Bloom Filter

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810660485.5A CN109150537B (en) 2018-06-25 2018-06-25 File ownership proving method based on dynamic Bloom Filter

Publications (2)

Publication Number Publication Date
CN109150537A CN109150537A (en) 2019-01-04
CN109150537B true CN109150537B (en) 2021-08-17

Family

ID=64802284

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810660485.5A Active CN109150537B (en) 2018-06-25 2018-06-25 File ownership proving method based on dynamic Bloom Filter

Country Status (1)

Country Link
CN (1) CN109150537B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114760061B (en) * 2020-12-29 2023-09-05 深信服科技股份有限公司 Method, device, equipment and storage medium for uploading data
US11741258B2 (en) 2021-04-16 2023-08-29 International Business Machines Corporation Dynamic data dissemination under declarative data subject constraints

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102810107A (en) * 2011-06-01 2012-12-05 英业达股份有限公司 Processing method for repeating data
CN103838850A (en) * 2014-03-11 2014-06-04 湖州师范学院 Hashing data representing and querying method based on dynamic counting type Bloom filter
CN105320654A (en) * 2014-05-28 2016-02-10 中国科学院深圳先进技术研究院 Dynamic bloom filter and element operating method based on same
CN105897921A (en) * 2016-05-27 2016-08-24 重庆大学 Data block routing method combining fingerprint sampling and reducing data fragments
CN105938480A (en) * 2016-04-07 2016-09-14 重庆大学 RFID redundant data cleansing method and system based on DTBF
CN106612320A (en) * 2016-06-14 2017-05-03 四川用联信息技术有限公司 Encrypted data dereplication method for cloud storage

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102810107A (en) * 2011-06-01 2012-12-05 英业达股份有限公司 Processing method for repeating data
CN103838850A (en) * 2014-03-11 2014-06-04 湖州师范学院 Hashing data representing and querying method based on dynamic counting type Bloom filter
CN105320654A (en) * 2014-05-28 2016-02-10 中国科学院深圳先进技术研究院 Dynamic bloom filter and element operating method based on same
CN105938480A (en) * 2016-04-07 2016-09-14 重庆大学 RFID redundant data cleansing method and system based on DTBF
CN105897921A (en) * 2016-05-27 2016-08-24 重庆大学 Data block routing method combining fingerprint sampling and reducing data fragments
CN106612320A (en) * 2016-06-14 2017-05-03 四川用联信息技术有限公司 Encrypted data dereplication method for cloud storage

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
Bloom Filter研究进展;严华云，关佶红;《电信科学》;20100215;第26卷(第02期);全文 *
The Dynamic Bloom Filter;D.Guo，J.Wu;《IEEE》;20090227;第22卷(第01期);全文 *
基于Bloom Filter的去重方法研究;赵艳红，李洪奇;《计算技术与自动化》;20160315;第35卷(第01期);全文 *

Also Published As

Publication number Publication date
CN109150537A (en) 2019-01-04

Similar Documents

Publication Publication Date Title
Xia et al. A comprehensive study of the past, present, and future of data deduplication
US11153094B2 (en) Secure data deduplication with smaller hash values
US9201800B2 (en) Restoring temporal locality in global and local deduplication storage systems
Storer et al. Secure data deduplication
US8165221B2 (en) System and method for sampling based elimination of duplicate data
US10380073B2 (en) Use of solid state storage devices and the like in data deduplication
US9727573B1 (en) Out-of core similarity matching
US10222987B2 (en) Data deduplication with augmented cuckoo filters
US9753648B2 (en) Fixed size extents for variable size deduplication segments
CN105027069B (en) Deduplication of volume regions
CN106101257B (en) A kind of cloud storage data managing method and device based on Bloom filter
US9569357B1 (en) Managing compressed data in a storage system
US7373520B1 (en) Method for computing data signatures
US20160196320A1 (en) Replication to the cloud
CN106776967A (en) Mass small documents real-time storage method and device based on sequential aggregating algorithm
CN103944988A (en) Repeating data deleting system and method applicable to cloud storage
US8386717B1 (en) Method and apparatus to free up cache memory space with a pseudo least recently used scheme
CN104836862B (en) A kind of Intelligent terminal data storage method
WO2016075562A1 (en) Exploiting node-local deduplication in distributed storage system
WO2018032375A1 (en) Survivable storage system and method for block chain
CN109150537B (en) File ownership proving method based on dynamic Bloom Filter
CN106407224A (en) Method and device for file compaction in KV (Key-Value)-Store system
CN106603561A (en) Block level encryption method in cloud storage and multi-granularity deduplication method
US20200065306A1 (en) Bloom filter partitioning
CN110968452A (en) Data integrity verification method capable of safely removing duplicate in cloud storage of smart power grid

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant