CN116894013A - File processing method and device, storage medium and electronic equipment - Google Patents
File processing method and device, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN116894013A CN116894013A CN202310892266.0A CN202310892266A CN116894013A CN 116894013 A CN116894013 A CN 116894013A CN 202310892266 A CN202310892266 A CN 202310892266A CN 116894013 A CN116894013 A CN 116894013A
- Authority
- CN
- China
- Prior art keywords
- file
- subfiles
- target
- storage space
- sub
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 12
- 238000012545 processing Methods 0.000 claims abstract description 64
- 238000001914 filtration Methods 0.000 claims abstract description 52
- 238000000034 method Methods 0.000 claims abstract description 42
- 230000006870 function Effects 0.000 claims abstract description 40
- 230000011218 segmentation Effects 0.000 claims abstract description 7
- 238000006243 chemical reaction Methods 0.000 claims description 16
- 238000001514 detection method Methods 0.000 claims description 7
- 230000002159 abnormal effect Effects 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 6
- 230000000694 effects Effects 0.000 description 11
- 238000005516 engineering process Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 4
- 238000012217 deletion Methods 0.000 description 4
- 230000037430 deletion Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 230000008878 coupling Effects 0.000 description 2
- 238000010168 coupling process Methods 0.000 description 2
- 238000005859 coupling reaction Methods 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 239000002699 waste material Substances 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013475 authorization Methods 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/172—Caching, prefetching or hoarding of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
- G06F16/162—Delete operations
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a file processing method, a file processing device, a storage medium and electronic equipment. The method relates to the technical field of cloud computing, wherein the method comprises the following steps: responding to a file storage request, and carrying out segmentation processing on a target file to obtain a sub-file set, wherein the types of the target storage space comprise: cloud storage space, including in the sub-file collection: n subfiles; carrying out hash processing on each sub-file in the sub-file set through a hash function set to obtain N first sets, wherein the hash function set comprises: m hash functions, each first set including M hash values; filtering the subfiles in the subfiles set based on the N first sets to obtain a filtered subfiles set; and storing the subfiles in the filtered subfiles set to a target storage space. The method and the device solve the technical problems that in the related art, the whole file is stored in the cloud storage space in a serialization mode, and the occupancy rate of the cloud storage space is high.
Description
Technical Field
The invention relates to the technical field of cloud computing, in particular to a file processing method, a device, a storage medium and electronic equipment.
Background
With the continuous improvement of the informatization degree of the domestic internet, the development of information technology is continuous, and the internet technology is penetrated into various industries. Currently, cloud storage is widely popularized, and almost every user has more than one cloud storage space, so that the users can access and share files on different devices at any time and any place. Based on this, many companies want to build their own private cloud storage systems. If cloud storage is to be built by oneself, a large amount of hardware devices are required besides the support on software. The required hardware cost is increased, and the number of servers as a system and hard disks as storage space is increased with the use requirement and the use time, besides, the maintenance cost of hardware failure is also required to be considered, and in some cases, the risk of data loss is also faced.
In the related art, in the process of storing a file to be stored in a disk of a cloud storage space, the file to be stored is stored in an uploading file unit, and the whole file is serialized to the disk, because different files possibly contain the same parts, in the process of storing the disk, along with the increase of the data volume stored in the cloud storage space, more repeated files exist in a cloud file system, so that a large space is occupied, and serious resource waste is caused. With the increase of cloud files, redundant files are increased, and the maintenance cost and hardware cost of the cloud storage space are also increased continuously.
In view of the above problems, no effective solution has been proposed at present.
Disclosure of Invention
The embodiment of the invention provides a file processing method, a device, a storage medium and electronic equipment, which at least solve the technical problem that the whole file is stored in a cloud storage space in a serialization manner in the related technology, and the occupancy rate of the cloud storage space is high.
According to an aspect of an embodiment of the present invention, there is provided a method for processing a file, including: responding to a file storage request, and carrying out segmentation processing on a target file to obtain a sub-file set, wherein the file storage request is used for requesting to store the target file into a target storage space, and the types of the target storage space comprise: cloud storage space, the sub-file set includes: n subfiles, N is an integer greater than 1; carrying out hash processing on each sub-file in the sub-file set through a hash function set to obtain N first sets, wherein the hash function set comprises: m hash functions, wherein each first set comprises M hash values, and M is a positive integer greater than 1; filtering the subfiles in the subfiles set based on the N first sets to obtain a filtered subfiles set; and storing the subfiles in the filtered subfile set to the target storage space.
Further, based on the N first sets, filtering the subfiles in the subfiles set to obtain a filtered subfile set, including: filtering repeated subfiles in the subfiles set based on the hash values in the N first sets to obtain a target file set, wherein the target file set comprises: s sub-files, wherein S is a positive integer smaller than N; acquiring file information of the files stored in the target storage space; and filtering the subfiles which are repeated between the target file set and the stored files based on the hash values in the N first sets and the file information, so as to obtain the filtered subfile set.
Further, the step of filtering the subfiles that overlap with the stored files in the target file set based on the hash values in the N first sets and the file information to obtain the filtered subfile set includes: binary conversion is carried out on the hash values in the N first sets to obtain conversion results, wherein the conversion results comprise: n second sets, including: m binary data; and filtering the subfiles which are repeated between the target file set and the stored files by adopting a target filter based on the N second sets and the file information to obtain the filtered subfile set.
Further, based on the N second sets and the file information, filtering the subfiles that are repeated between the files in the target file set and the stored files by using a target filter, so as to obtain a filtered subfile set, including: determining a target storage file in the target storage space and a target sub-file in the sub-file set based on the binary data in the N second sets and the file information, and taking a storage address of the target storage file in the target storage space as a storage address of the target sub-file, wherein the contents of the target storage file and the target sub-file are the same; and deleting the target subfile in the subfile set to obtain the filtered subfile set.
Further, the step of storing the subfiles in the filtered subfile set to the target storage space includes: creating a file object of the target file, wherein the file object comprises: a storage address of each subfile in the set of subfiles in the target storage space; creating a sub-file object of each sub-file, wherein the sub-file object comprises: the storage addresses of two subfiles adjacent to the front and the back of the subfiles, the storage address of the subfiles, and the hash value of the subfiles; and storing the subfiles in the filtered subfile set to the target storage space based on the file object and the subfile object.
Further, after storing the subfiles in the filtered set of subfiles to the target storage space based on the file object and the subfiles object, the method further comprises: responding to a file inquiry request, and acquiring storage addresses of all subfiles associated with the target file and storage addresses of two subfiles adjacent to each subfile before and after the target file through a file object of the target file and each subfile object associated with the target file, wherein the file inquiry request is used for requesting to inquire file contents of the target file; and acquiring all subfiles associated with the target file based on the storage addresses of all subfiles of the target file and the storage addresses of two subfiles adjacent to each subfile before and after the subfiles, and merging all subfiles associated with the target file into the target file.
Further, after storing the subfiles in the filtered set of subfiles to the target storage space based on the file object and the subfiles object, the method further comprises: based on a preset backup rule, backing up all sub files of the target file in the target storage space to obtain a backup file of the target file; and detecting whether the backup file is abnormal or not based on a first preset period, and recording a detection result.
Further, after storing the subfiles in the filtered set of subfiles to the target storage space based on the file object and the subfiles object, the method further comprises: based on a second preset period, deleting the subfiles with the cited times smaller than a preset threshold value in the target storage space by adopting a soft deleting mode, wherein the soft deleting mode indicates that the subfiles to be deleted are logically deleted; and deleting the target data in all the subfiles in the target storage space by adopting a hard deleting mode, wherein the hard deleting mode represents deleting the subfiles to be deleted physically, and the target data comprises the following steps: privacy data.
According to another aspect of the embodiment of the present invention, there is also provided a processing apparatus for a file, including: the splitting unit is used for responding to a file storage request and splitting a target file to obtain a sub-file set, wherein the file storage request is used for requesting to store the target file into a target storage space, and the types of the target storage space comprise: cloud storage space, the sub-file set includes: n subfiles, N is an integer greater than 1; the processing unit is configured to perform hash processing on each of the subfiles in the subfiles set through a hash function set to obtain N first sets, where the hash function set includes: m hash functions, wherein each first set comprises M hash values, and M is a positive integer greater than 1; the filtering unit is used for filtering the subfiles in the subfiles set based on the N first sets to obtain a filtered subfile set; and the storage unit is used for storing the subfiles in the filtered subfile set into the target storage space.
Further, the filter unit includes: the first filtering subunit is configured to filter, based on hash values in N first sets, repeated subfiles in the subfiles set to obtain a target file set, where the target file set includes: s sub-files, wherein S is a positive integer smaller than N; an obtaining subunit, configured to obtain file information of a file stored in the target storage space; and the second filtering subunit is used for filtering the subfiles which are repeated between the target file set and the stored files based on the hash values in the N first sets and the file information, so as to obtain the filtered subfile set.
Further, the second filtering subunit includes: the conversion module is configured to binary convert hash values in the N first sets to obtain a conversion result, where the conversion result includes: n second sets, including: m binary data; and the filtering module is used for filtering the subfiles which are repeated between the target file set and the stored files by adopting a target filter based on the N second sets and the file information to obtain the filtered subfile set.
Further, the filter module includes: a processing sub-module, configured to determine, based on the binary data in the N second sets and the file information, a target storage file in the target storage space, and a target sub-file in the sub-file set, and use a storage address of the target storage file in the target storage space as a storage address of the target sub-file, where contents of the target storage file and the target sub-file are the same; and the deleting sub-module is used for deleting the target sub-file in the sub-file set to obtain the filtered sub-file set.
Further, the storage unit includes: a first creating subunit, configured to create a file object of the target file, where the file object includes: a storage address of each subfile in the set of subfiles in the target storage space; a second creating subunit, configured to create a subfile object of each subfile, where the subfile object includes: the storage addresses of two subfiles adjacent to the front and the back of the subfiles, the storage address of the subfiles, and the hash value of the subfiles; and the storage subunit is used for storing the subfiles in the filtered subfile set to the target storage space based on the file object and the subfile object.
Further, the file processing device further includes: the query unit is used for responding to a file query request after storing the subfiles in the filtered subfiles set to the target storage space based on the file object and the subfiles object, and acquiring storage addresses of all subfiles associated with the target file and storage addresses of two subfiles adjacent to each subfile before and after the target file through the file object of the target file and each subfile object associated with the target file, wherein the file query request is used for requesting to query file contents of the target file; and the acquisition unit is used for acquiring all the subfiles associated with the target file based on the storage addresses of all the subfiles of the target file and the storage addresses of two subfiles adjacent to each subfile before and after the subfiles, and merging all the subfiles associated with the target file into the target file.
Further, the file processing device includes: the backup unit is used for backing up all the subfiles of the target file in the target storage space based on a preset backup rule after storing the subfiles in the filtered subfiles set to the target storage space based on the file object and the subfiles object to obtain a backup file of the target file; and the detection unit is used for detecting whether the backup file is abnormal or not based on the first preset period and recording a detection result.
Further, the file processing device further includes: a first deleting unit, configured to, after storing the subfiles in the filtered subfile set to the target storage space based on the file object and the subfile object, delete the subfiles in the target storage space that are referenced less than a preset threshold value by adopting a soft deleting method based on a second preset period, where the soft deleting method indicates that the subfiles to be deleted logically are deleted; the second deleting unit is configured to delete target data in all subfiles in the target storage space by adopting a hard deleting manner, where the hard deleting manner indicates that subfiles to be deleted are physically deleted, and the target data includes: privacy data.
According to another aspect of the embodiment of the present invention, there is also provided an electronic device, including: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the method of processing a file of any of the above via execution of the executable instructions.
According to another aspect of the embodiment of the present invention, there is also provided a computer readable storage medium, in which a computer program is stored, where the computer readable storage medium is controlled to execute the method for processing a file according to any one of the above steps when the computer program is executed.
In the invention, a target file is subjected to segmentation processing in response to a file storage request to obtain a sub-file set, wherein the file storage request is used for requesting to store the target file into a target storage space, and the types of the target storage space comprise: cloud storage space, including in the sub-file collection: n subfiles, N is an integer greater than 1; carrying out hash processing on each sub-file in the sub-file set through a hash function set to obtain N first sets, wherein the hash function set comprises: m hash functions, wherein each first set comprises M hash values, and M is a positive integer greater than 1; filtering the subfiles in the subfiles set based on the N first sets to obtain a filtered subfiles set; and storing the subfiles in the filtered subfiles set to a target storage space. The method further solves the technical problems that in the related art, the whole file is stored in the cloud storage space in a serialization mode, and the occupancy rate of the cloud storage space is high. According to the method and the device, the hash values of the segmented subfiles are calculated through various hash functions, repeated subfiles are filtered according to the hash values, the filtered subfiles are stored in the target storage space, the situation that the whole file is directly stored in the cloud storage space in a sequential mode in the related technology, the number of redundant files in the storage space is high, and the storage space is wasted is avoided, and therefore the technical effect of improving the storage rate of the storage file in the storage space is achieved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute a limitation on the application. In the drawings:
FIG. 1 is a flow chart of an alternative method of processing a file according to an embodiment of the present application;
FIG. 2 is a schematic illustration of the storage of an alternative file according to an embodiment of the present application;
FIG. 3 is a schematic diagram of an alternative document processing apparatus according to an embodiment of the present application;
fig. 4 is a schematic diagram of an electronic device according to an embodiment of the application.
Detailed Description
In order that those skilled in the art will better understand the present application, a technical solution in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present application without making any inventive effort, shall fall within the scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It should be noted that, the method and the device for processing a file in the present disclosure may be used in the cloud computing technical field when the file is stored, and may also be used in any field other than the cloud computing technical field when the file is stored, and the application field of the method and the device for processing the file in the present disclosure is not limited.
It should be noted that, the user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, displayed data, file data, etc.) related to the present application are information and data authorized by the user or fully authorized by each party, and the collection, use and processing of the related data need to comply with the related laws and regulations and standards of the related country and region, and provide corresponding operation entries for the user to select authorization or rejection.
For convenience of description, some terms or nouns involved in the embodiments of the present application are explained below:
MD5: message-Digest Algorithm, a widely used cryptographic hash function, can generate a 128-bit (16-byte) hash value (hash value) to ensure that the information transfer is complete and consistent;
MurmurHash: a non-cryptographic hash function adapted for hash-based lookups;
CityHash: the character string hash algorithm is a very fast and efficient hash algorithm and is mainly used for generating hash values;
hash: "hashing" can be directly transliterated into "hashing", i.e., transforming an input of arbitrary length (also called pre-map) into an output of fixed length, which is the hash value, by a hashing algorithm. This transformation is a compressed mapping, i.e. the hash value is typically much smaller in space than the input, different inputs may be hashed to the same output, so it is not possible to uniquely determine the input value from the hash value. Is a message pick-up function that compresses messages of arbitrary length to a certain fixed length.
Example 1
According to an embodiment of the present invention, there is provided an alternative method embodiment of a method of processing a file, it being noted that the steps shown in the flowchart of the figures may be performed in a computer system such as a set of computer executable instructions, and that although a logical order is shown in the flowchart, in some cases the steps shown or described may be performed in an order different from that shown or described herein.
FIG. 1 is a flowchart of an alternative method of processing a file according to an embodiment of the present invention, as shown in FIG. 1, the method comprising the steps of:
step S101, in response to a file storage request, performing segmentation processing on a target file to obtain a sub-file set, wherein the file storage request is used for requesting to store the target file into a target storage space, and the types of the target storage space comprise: cloud storage space, including in the sub-file collection: n subfiles, N is an integer greater than 1.
The file storage request is used for requesting to store the target file in the target storage space, where the target storage space may include, but is not limited to, a cloud storage space, in this embodiment, the target file may be subjected to splitting processing based on a preset splitting rule to obtain N subfiles, where the N subfiles form the above subfiles set, for example: the target file is divided into pieces of a predetermined length (granularity is as small as possible). Cut into different subfiles.
Step S102, carrying out hash processing on each sub-file in the sub-file set through a hash function set to obtain N first sets, wherein the hash function set comprises: m hash functions, each first set includes M hash values, M is a positive integer greater than 1.
In this embodiment, each subfile may be calculated by using M hash functions in the hash function set, and M hash values may be obtained after each subfile is calculated, where M hash values form a first set, that is, each subfile correspondingly obtains a first set, where the first set includes M hash values, and N subfiles obtain N first sets after being processed by the hash functions in the hash function set.
Step S103, filtering the subfiles in the subfiles set based on the N first sets to obtain a filtered subfile set.
In this embodiment, the subfiles of the set of subfiles that have a duplicate with the files stored in the target storage space may be filtered based on the M hash values in each first set, for a sub-file in which a repeated condition exists between the sub-file set and a stored file, the storage address of the stored repeated file can be directly used as the storage address of the sub-file. In this embodiment, the subfiles with the repetition condition with the stored files in the subfile set may be filtered by the target filter, and it should be noted that the repeated subfiles in the subfile set may be filtered based on M hash values in each of the N first sets. And (3) processing each sub-file through M hash functions to obtain M hash values in each first set, and filtering repeated sub-files, so that the technical effect of improving the accuracy of filtering repeated files is achieved.
Step S104, storing the subfiles in the filtered subfile set into a target storage space.
In this embodiment, the subfiles in the filtered file set are stored in the target storage space, so as to avoid the situations that a large number of redundant files exist in the storage space and storage resources are occupied, and it should be noted that, in order to facilitate subsequent query on file contents of the target file, when the subfiles in the subfiles set are stored in the target storage space, storage addresses of adjacent subfiles before and after each subfile can be recorded.
Through the steps, in the embodiment, the hash value of the segmented sub-file is calculated through multiple hash functions, repeated sub-files are filtered according to the hash value, and the filtered sub-files are stored in the target storage space, so that the situation that the whole file is directly stored in the cloud storage space in a sequential mode in the related technology, the number of redundant files in the storage space is high, and the storage space is wasted is avoided, and the technical effect of improving the storage rate of the storage file in the storage space is achieved. The method further solves the technical problems that in the related art, the whole file is stored in the cloud storage space in a serialization mode, and the occupancy rate of the cloud storage space is high.
Optionally, filtering the subfiles in the subfile set based on the N first sets to obtain a filtered subfile set, including: filtering repeated subfiles in the subfiles set based on the hash values in the N first sets to obtain a target file set, wherein the target file set comprises: s subfiles, S is a positive integer less than N; acquiring file information of a file stored in a target storage space; and filtering repeated subfiles between the target file set and the stored files based on the hash values and the file information in the N first sets to obtain a filtered subfile set.
In this embodiment, the repeated subfiles in the subfiles set may be filtered based on the hash values in the N first sets to obtain the target file set, for example: if the hash values in the two first sets are completely repeated, two subfiles corresponding to the two sets can be determined to be subfiles with repeated contents, one of the subfiles can be filtered, and repeated files in the target storage space after the subfiles are stored in the target storage space are avoided, so that storage resources of the target storage space are wasted.
In this embodiment, file information of a file stored in the target storage space may also be obtained, where the file information of the stored file may include, but is not limited to: the file id, the hash value of the file and the like can load the file information of the stored file into a target filter, and based on the hash values in the N first sets, the target filter is used for judging the sub-file which is repeated with the stored file in the target file set, if the sub-file is repeated with the stored file, the stored address of the file which is repeated with the sub-file can be directly used as the storage address of the sub-file, the sub-file is filtered out from the target file set, the filtered sub-file set is obtained, and the technical effect of improving the filtering accuracy of repeated file filtering is achieved.
Optionally, based on the hash values and the file information in the N first sets, filtering sub files that are repeated between the target file set and the stored files to obtain a filtered sub file set, including: binary conversion is carried out on the hash values in the N first sets to obtain conversion results, wherein the conversion results comprise: and N second sets, wherein the second sets comprise: m binary data; and filtering the subfiles which are repeated between the target file set and the stored files by adopting a target filter based on the N second sets and the file information to obtain a filtered subfile set.
The target filter may be a bloom filter, in this embodiment, a binary conversion may be performed on hash values in each first set to obtain N second sets, and then, whether a sub-file already has a duplicate file in the target storage space is determined by the target filter. If the file exists, directly acquiring the file address of the file in the target storage space as the storage address of the sub-file, so as to filter the sub-file which is repeated between the target file set and the stored file, and obtain a filtered sub-file set. If not present. The sub-file may be stored to the target storage space and the binary file corresponding to the sub-file with the bloom filter set to 1, indicating that the sub-file is stored.
Whether repeated files exist in the target storage space or not is judged through the target filter, and the technical effect of improving the filtering efficiency of file filtering is achieved.
Optionally, filtering, based on the N second sets and the file information, sub files that are repeated between the target file set and the stored files by using a target filter, to obtain a filtered sub file set, including: determining a target storage file in a target storage space and a target sub-file in a sub-file set based on binary data and file information in the N second sets, and taking a storage address of the target storage file in the target storage space as a storage address of the target sub-file, wherein the contents of the target storage file and the target sub-file are the same; and deleting the target subfiles in the subfile set to obtain a filtered subfile set.
In this embodiment, file information (e.g., hash value of the stored file) of the stored file may be read into the target filter, and then whether binary data of the hash value of the sub-file is identical to binary data of the hash value in the file information or not is judged by the target filter, if so, it is determined that the sub-file is identical to file content corresponding to the file information of the target storage space, i.e., the sub-file already exists in the target storage space, otherwise, it does not exist. If so, the file address of the file which is repeated with the sub-file in the target storage space can be directly acquired, and the file address is used as the storage address of the sub-file. And deleting the subfiles in the target set to filter the subfiles which are repeated between the target file set and the stored files, thereby obtaining a filtered subfile set. If not present. The sub-file can be stored in the target storage space, and the binary file corresponding to the sub-file is set to 1 by the bloom filter, so that the sub-file is stored, and the technical effect of improving the file filtering precision of the repeated sub-file is realized.
Optionally, the step of storing the subfiles in the filtered set of subfiles to the target storage space includes: creating a file object of the target file, wherein the file object comprises: a storage address of each subfile in the set of subfiles in the target storage space; creating a subfile object for each subfile, wherein the subfile object comprises: the storage addresses of two subfiles adjacent to the front and back subfiles, the storage address of the subfiles, and the hash value of the subfiles; and storing the subfiles in the filtered subfile set to a target storage space based on the file object and the subfile object.
In this embodiment, fig. 2 is a schematic diagram of storing an optional file according to an embodiment of the present invention, as shown in fig. 2, a file Object (such as a file Object1, object2, and Object3 on the right side of the file 1 in fig. 2, and an Object1, object3, and Object4 on the right side of the file 2) may be created for the target file, where the Object includes a storage address of each sub-file. Sub-file objects (e.g., document Object1, document Object2, document Object3, and document Object4 in fig. 2) may also be created, each of which contains a last sub-file Object address (storage address, e.g., upper file address in fig. 2), a next sub-file address (e.g., lower file address in fig. 2), and an address of the file's storage, a Hash value, where the last sub-file Object address and the next sub-file address, i.e., the storage addresses of two sub-files that are adjacent to the sub-file in front of and behind the sub-file. By storing the file objects and the subfiles in the filtered subfile set to the target storage space (such as the cloud space in fig. 2), the subfiles can be conveniently and sequentially acquired according to the file objects, so as to achieve the purpose of quickly acquiring the target file.
Before storing the subfiles in the target storage space, the subfiles can be encrypted, and then stored in the target storage space after being encrypted, so that the safety of file data is improved, repeated subfiles are filtered, the redundant space occupied by the repeated files in a storage disk is reduced, and the storage rate of the disk is improved.
Optionally, after storing the subfiles in the filtered set of subfiles to the target storage space based on the file object and the subfile object, the method further comprises: responding to a file inquiry request, and acquiring storage addresses of all subfiles associated with the target file and storage addresses of two subfiles adjacent to each subfile in front of and behind through a file object of the target file and each subfile object associated with the target file, wherein the file inquiry request is used for requesting to inquire file contents of the target file; and acquiring all the subfiles associated with the target file based on the storage addresses of all the subfiles of the target file and the storage addresses of two subfiles adjacent to each subfile before and after, and merging all the subfiles associated with the target file into the target file.
In this embodiment, when the target file is queried, information (including, but not limited to, storage addresses of subfiles adjacent to the subfiles before and after the subfiles, hash values, etc.) of all subfiles split by the target file can be obtained according to the target file id or file object and subfile object of the target file, then all subfiles of the target file are read in the target storage space based on the storage addresses of the subfiles, the storage addresses of the subfiles adjacent to the subfiles before and after the subfiles, and the target file is obtained by merging.
In this embodiment, the file content of the subfile may also be obtained according to the subfile address recorded in the subfile object. If the sub-file is encrypted before being stored in the target storage space, the sub-file may be decrypted to obtain the file content of the sub-file, and in this embodiment, the original file (i.e., the target file) content is synthesized according to the file information (including, but not limited to, the storage address of the sub-file adjacent to the sub-file before and after the sub-file, the hash value, etc.), thereby realizing the technical effect of improving the file searching efficiency.
Optionally, after storing the subfiles in the filtered set of subfiles to the target storage space based on the file object and the subfile object, the method further comprises: based on a preset backup rule, backing up all sub files of the target file in the target storage space to obtain a backup file of the target file; based on the first preset period, detecting whether the backup file is abnormal or not, and recording a detection result.
In this embodiment, in order to improve the security and reliability of the file data, the sub-files of the target file may be backed up, and an appropriate backup mode (corresponding to the preset backup rule described above) may be selected, for example, full-scale backup, consolidated backup, and the like. Dividing the fragmented files (sub files obtained by splitting the target files) into different backup directories according to a certain rule (for example, according to time, size, data volume and the like), and after backup, detecting whether the backup files are abnormal or not at regular intervals. For example: the integrity of the backup file (namely whether the backup file has a deletion) and the availability of the backup file (namely whether the backup file can be read and used normally) are detected, so that the integrity and the availability of the backup are ensured. In this embodiment, a redundant backup mechanism (such as multiple backup and remote backup) may be further used to improve the reliability of the backup file, thereby implementing the technical effect of improving the reliability and security of storing the target file.
Optionally, after storing the subfiles in the filtered set of subfiles to the target storage space based on the file object and the subfile object, the method further comprises: based on a second preset period, deleting the subfiles with the cited times smaller than a preset threshold value in the target storage space in a soft deleting mode, wherein the soft deleting mode means that the subfiles to be deleted are logically deleted; and deleting the target data in all the subfiles in the target storage space by adopting a hard deleting mode, wherein the hard deleting mode is used for deleting the subfiles to be deleted physically, and the target data comprises the following steps: privacy data.
In this embodiment, in order to further perform effective utilization rate on the storage resource of the target storage space, according to a certain rule (for example, according to time, size, data volume, etc.), whether a document (sub-file) is referenced or not may be periodically determined, and if the number of times of referencing is smaller than a preset threshold, the sub-file is cleaned, and if the preset threshold is 1, that is, if the sub-file that is not referenced is cleaned. The use of soft deletion means, i.e. the removal of the fragmented files (subfiles) to be deleted into a specific directory, is considered when cleaning the files, so that the files can be restored in the target storage space when necessary. For sensitive data, for example: the method can thoroughly remove the file content by adopting a safe deletion mode (for example, a hard deletion mode) to prevent the data from being divulged, thereby realizing the technical effect of improving the safety of the data in the file.
In this embodiment, the storage mode of the file is changed, so that the redundant space occupied by the storage disk is reduced if the file is split more carefully, the storage rate of the disk is improved, the cost of an enterprise is reduced, and the file transmission efficiency can be improved: for example: when data transmission is carried out, a large file often needs longer transmission time, so that the large file can be divided into a plurality of small files by adopting fragmented storage, and each small file only needs to be independently transmitted and received, thereby improving the transmission efficiency; the file storage efficiency can also be improved: for example: for the situation that the capacity of a storage medium is smaller, the storage space can be more effectively utilized by using file splitting storage, and the problem of storage waste caused by overlarge single large file is avoided; reliability can be improved: for example: the adoption of file sharded storage can reduce the risk of data loss. When a small file is damaged or lost, only the small file needs to be downloaded again, and the whole large file does not need to be downloaded again. Files in the target storage space can be conveniently managed: for example: the file can be managed more flexibly by adopting the file slicing storage, and operations such as periodically backing up certain small files, deleting unnecessary small files and the like can be performed; data security can also be improved: for example: by performing operations such as encryption, signature and the like on the small file, the security of the file can be improved, and illegal tampering, theft and the like of the file can be prevented.
Example two
An embodiment II of the present application provides an optional file processing device, where each implementation unit in the processing device corresponds to each implementation step in the embodiment I.
FIG. 3 is a schematic diagram of an alternative document processing apparatus according to an embodiment of the present application, as shown in FIG. 3, the processing apparatus includes: a segmentation unit 31, a processing unit 32, a filtering unit 33 and a storage unit 34.
Specifically, the splitting unit 31 is configured to perform splitting processing on a target file in response to a file storage request to obtain a sub-file set, where the file storage request is used to request that the target file be stored in a target storage space, and a type of the target storage space includes: cloud storage space, including in the sub-file collection: n subfiles, N is an integer greater than 1;
the processing unit 32 is configured to perform hash processing on each sub-file in the sub-file set through a hash function set to obtain N first sets, where the hash function set includes: m hash functions, wherein each first set comprises M hash values, and M is a positive integer greater than 1;
a filtering unit 33, configured to filter subfiles in the subfiles set based on the N first sets, to obtain a filtered subfile set;
And the storage unit 34 is used for storing the subfiles in the filtered subfile set to the target storage space.
In the processing device for a file provided in the second embodiment of the present application, a target file may be subjected to splitting processing by a splitting unit 31 in response to a file storage request to obtain a sub-file set, where the file storage request is used to request that the target file be stored in a target storage space, and the type of the target storage space includes: cloud storage space, including in the sub-file collection: n subfiles, N is an integer greater than 1, and each subfile in the subfile set is hashed by the processing unit 32 through a hash function set to obtain N first sets, wherein the hash function set includes: m hash functions, wherein each first set comprises M hash values, M is a positive integer greater than 1, subfiles in the subfiles set are filtered by the filtering unit 33 based on the N first sets, a filtered subfile set is obtained, and the subfiles in the filtered subfiles set are stored into a target storage space by the storage unit 34. The method further solves the technical problems that in the related art, the whole file is stored in the cloud storage space in a serialization mode, and the occupancy rate of the cloud storage space is high. In this embodiment, hash values of the segmented subfiles are calculated through multiple hash functions, repeated subfiles are filtered according to the hash values, and the filtered subfiles are stored in a target storage space, so that the situation that the whole file is directly stored in a cloud storage space in a sequential manner in the related art is avoided, the number of redundant files in the storage space is high, and the storage space is wasted, and therefore the technical effect of improving the storage rate of the storage file in the storage space is achieved.
Optionally, in the processing device for a file provided in the second embodiment of the present application, the filtering unit includes: the first filtering subunit is configured to filter repeated subfiles in the subfiles set based on hash values in the N first sets to obtain a target file set, where the target file set includes: s subfiles, S is a positive integer less than N; an acquisition subunit, configured to acquire file information of a file stored in the target storage space; and the second filtering subunit is used for filtering repeated subfiles between the target file set and the stored files based on the hash values and the file information in the N first sets to obtain a filtered subfile set.
Optionally, in the processing device for a file provided in the second embodiment of the present application, the second filtering subunit includes: the conversion module is used for binary converting the hash values in the N first sets to obtain conversion results, wherein the conversion results comprise: and N second sets, wherein the second sets comprise: m binary data; and the filtering module is used for filtering repeated subfiles between the target file set and the stored files by adopting a target filter based on the N second sets and the file information to obtain a filtered subfile set.
Optionally, in the processing device for a file provided in the second embodiment of the present application, the filtering module includes: the processing sub-module is used for determining a target storage file in a target storage space and a target sub-file in a sub-file set based on binary data and file information in the N second sets, and taking a storage address of the target storage file in the target storage space as a storage address of the target sub-file, wherein the contents of the target storage file and the target sub-file are the same; and the deleting sub-module is used for deleting the target sub-file in the sub-file set to obtain a filtered sub-file set.
Optionally, in the processing device for a file provided in the second embodiment of the present application, the storage unit includes: a first creation subunit, configured to create a file object of the target file, where the file object includes: a storage address of each subfile in the set of subfiles in the target storage space; a second creation subunit, configured to create a subfile object of each subfile, where the subfile object includes: the storage addresses of two subfiles adjacent to the subfiles, the storage address of the subfiles, and the hash value of the subfiles; and the storage sub-unit is used for storing the subfiles in the filtered subfile set to the target storage space based on the file object and the subfile object.
Optionally, in the file processing device provided in the second embodiment of the present application, the file processing device further includes: the query unit is used for responding to a file query request after storing the subfiles in the filtered subfile set into the target storage space based on the file object and the subfile object, and acquiring the storage addresses of all subfiles associated with the target file and the storage addresses of two subfiles adjacent to each subfile before and after the target file through the file object of the target file and each subfile object associated with the target file, wherein the file query request is used for requesting to query the file content of the target file; the acquisition unit is used for acquiring all the subfiles associated with the target file based on the storage addresses of all the subfiles of the target file and the storage addresses of two subfiles adjacent to each subfile before and after the subfiles, and merging all the subfiles associated with the target file into the target file.
Optionally, in the file processing device provided in the second embodiment of the present application, the file processing device includes: the backup unit is used for backing up all the subfiles of the target file in the target storage space based on a preset backup rule after storing the subfiles in the filtered subfiles set in the target storage space based on the file object and the subfile object to obtain a backup file of the target file; the detection unit is used for detecting whether the backup file is abnormal or not based on the first preset period and recording a detection result.
Optionally, in the file processing device provided in the second embodiment of the present application, the file processing device further includes: the first deleting unit is used for deleting the subfiles with the times of being cited in the target storage space being smaller than a preset threshold value based on a second preset period after storing the subfiles in the filtered subfiles set to the target storage space based on the file objects and the subfiles objects, wherein the soft deleting mode indicates that the subfiles to be deleted logically; the second deleting unit is configured to delete target data in all subfiles in the target storage space by adopting a hard deleting mode, where the hard deleting mode indicates that the subfiles to be deleted are physically deleted, and the target data includes: privacy data.
The processing device of the above-mentioned document may further include a processor and a memory, where the above-mentioned segmentation unit 31, processing unit 32, filtering unit 33, storage unit 34, and the like are stored as program units, and the processor executes the above-mentioned program units stored in the memory to implement the corresponding functions.
The processor includes a kernel, and the kernel fetches a corresponding program unit from the memory. The kernel can set one or more than one kernel, the hash value of the segmented subfiles is calculated through adjusting kernel parameters, repeated subfiles are filtered according to the hash value, and the filtered subfiles are stored in the target storage space, so that the situation that the whole file is directly stored in the cloud storage space in a serialization mode in the related technology, the number of redundant files in the storage space is high, and the storage space is wasted is avoided, and the technical effect of improving the storage rate of the storage file in the storage space is achieved.
The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM), which includes at least one memory chip.
According to another aspect of the embodiment of the present invention, there is also provided an electronic device, including: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the method of processing a file of any of the above via execution of the executable instructions.
According to another aspect of the embodiment of the present invention, there is also provided a computer readable storage medium, in which a computer program is stored, where the computer readable storage medium is controlled to execute the method for processing a file according to any one of the above steps when the computer program is executed.
Fig. 4 is a schematic diagram of an electronic device according to an embodiment of the present invention, and as shown in fig. 4, an embodiment of the present invention provides an electronic device 40, where the electronic device includes a processor, a memory, and a program stored on the memory and executable on the processor, and the processor implements a method for processing a file according to any one of the above when executing the program.
The foregoing embodiment numbers of the present application are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
In the foregoing embodiments of the present application, the descriptions of the embodiments are emphasized, and for a portion of this disclosure that is not described in detail in this embodiment, reference is made to the related descriptions of other embodiments.
In the several embodiments provided in the present application, it should be understood that the disclosed technology may be implemented in other manners. The above-described embodiments of the apparatus are merely exemplary, and the division of the units, for example, may be a logic function division, and may be implemented in another manner, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some interfaces, units or modules, or may be in electrical or other forms.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied essentially or in part or all of the technical solution or in part in the form of a software product stored in a storage medium, including instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a removable hard disk, a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The foregoing is merely a preferred embodiment of the present invention and it should be noted that modifications and adaptations to those skilled in the art may be made without departing from the principles of the present invention, which are intended to be comprehended within the scope of the present invention.
Claims (11)
1. A method for processing a document, comprising:
responding to a file storage request, and carrying out segmentation processing on a target file to obtain a sub-file set, wherein the file storage request is used for requesting to store the target file into a target storage space, and the types of the target storage space comprise: cloud storage space, the sub-file set includes: n subfiles, N is an integer greater than 1;
carrying out hash processing on each sub-file in the sub-file set through a hash function set to obtain N first sets, wherein the hash function set comprises: m hash functions, wherein each first set comprises M hash values, and M is a positive integer greater than 1;
filtering the subfiles in the subfiles set based on the N first sets to obtain a filtered subfiles set;
and storing the subfiles in the filtered subfile set to the target storage space.
2. The processing method according to claim 1, wherein the step of filtering the subfiles in the subfiles set based on the N first sets to obtain a filtered subfile set includes:
filtering repeated subfiles in the subfiles set based on the hash values in the N first sets to obtain a target file set, wherein the target file set comprises: s sub-files, wherein S is a positive integer smaller than N;
acquiring file information of the files stored in the target storage space;
and filtering the subfiles which are repeated between the target file set and the stored files based on the hash values in the N first sets and the file information, so as to obtain the filtered subfile set.
3. The processing method according to claim 2, wherein the step of filtering the sub-files in the target file set, which are repeated with the stored files, based on the hash values in the N first sets and the file information, to obtain the filtered sub-file set, includes:
binary conversion is carried out on the hash values in the N first sets to obtain conversion results, wherein the conversion results comprise: n second sets, including: m binary data;
And filtering the subfiles which are repeated between the target file set and the stored files by adopting a target filter based on the N second sets and the file information to obtain the filtered subfile set.
4. A processing method according to claim 3, wherein the step of filtering, based on the N second sets and the file information, the subfiles in the target file set that are repeated with the stored files with a target filter to obtain the filtered subfiles set includes:
determining a target storage file in the target storage space and a target sub-file in the sub-file set based on the binary data in the N second sets and the file information, and taking a storage address of the target storage file in the target storage space as a storage address of the target sub-file, wherein the contents of the target storage file and the target sub-file are the same;
and deleting the target subfile in the subfile set to obtain the filtered subfile set.
5. A processing method according to claim 3, wherein the step of storing subfiles in the filtered set of subfiles to the target storage space comprises:
Creating a file object of the target file, wherein the file object comprises: a storage address of each subfile in the set of subfiles in the target storage space;
creating a sub-file object of each sub-file, wherein the sub-file object comprises: the storage addresses of two subfiles adjacent to the front and the back of the subfiles, the storage address of the subfiles, and the hash value of the subfiles;
and storing the subfiles in the filtered subfile set to the target storage space based on the file object and the subfile object.
6. The processing method of claim 5, wherein after storing subfiles in the filtered set of subfiles to the target storage space based on the file object and the subfiles object, the method further comprises:
responding to a file inquiry request, and acquiring storage addresses of all subfiles associated with the target file and storage addresses of two subfiles adjacent to each subfile before and after the target file through a file object of the target file and each subfile object associated with the target file, wherein the file inquiry request is used for requesting to inquire file contents of the target file;
And acquiring all subfiles associated with the target file based on the storage addresses of all subfiles of the target file and the storage addresses of two subfiles adjacent to each subfile before and after the subfiles, and merging all subfiles associated with the target file into the target file.
7. The processing method of claim 5, wherein after storing subfiles in the filtered set of subfiles to the target storage space based on the file object and the subfiles object, the method further comprises:
based on a preset backup rule, backing up all sub files of the target file in the target storage space to obtain a backup file of the target file;
and detecting whether the backup file is abnormal or not based on a first preset period, and recording a detection result.
8. The processing method of claim 5, wherein after storing subfiles in the filtered set of subfiles to the target storage space based on the file object and the subfiles object, the method further comprises:
based on a second preset period, deleting the subfiles with the cited times smaller than a preset threshold value in the target storage space by adopting a soft deleting mode, wherein the soft deleting mode indicates that the subfiles to be deleted are logically deleted;
And deleting the target data in all the subfiles in the target storage space by adopting a hard deleting mode, wherein the hard deleting mode represents deleting the subfiles to be deleted physically, and the target data comprises the following steps: privacy data.
9. A document processing apparatus, comprising:
the splitting unit is used for responding to a file storage request and splitting a target file to obtain a sub-file set, wherein the file storage request is used for requesting to store the target file into a target storage space, and the types of the target storage space comprise: cloud storage space, the sub-file set includes: n subfiles, N is an integer greater than 1;
the processing unit is configured to perform hash processing on each of the subfiles in the subfiles set through a hash function set to obtain N first sets, where the hash function set includes: m hash functions, wherein each first set comprises M hash values, and M is a positive integer greater than 1;
the filtering unit is used for filtering the subfiles in the subfiles set based on the N first sets to obtain a filtered subfile set;
And the storage unit is used for storing the subfiles in the filtered subfile set into the target storage space.
10. A computer readable storage medium, wherein a computer program is stored in the computer readable storage medium, and wherein the computer program, when executed, controls a device in which the computer readable storage medium is located to perform the method for processing a file according to any one of claims 1 to 8.
11. An electronic device comprising one or more processors and a memory for storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of processing a file of any of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310892266.0A CN116894013A (en) | 2023-07-19 | 2023-07-19 | File processing method and device, storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310892266.0A CN116894013A (en) | 2023-07-19 | 2023-07-19 | File processing method and device, storage medium and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116894013A true CN116894013A (en) | 2023-10-17 |
Family
ID=88310523
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310892266.0A Pending CN116894013A (en) | 2023-07-19 | 2023-07-19 | File processing method and device, storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116894013A (en) |
-
2023
- 2023-07-19 CN CN202310892266.0A patent/CN116894013A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11558450B2 (en) | Systems and methods for aggregation of cloud storage | |
| US11960486B2 (en) | Systems and methods for secure file management via an aggregation of cloud storage services | |
| US11818211B2 (en) | Aggregation and management among a plurality of storage providers | |
| US10158483B1 (en) | Systems and methods for efficiently and securely storing data in a distributed data storage system | |
| US10264072B2 (en) | Systems and methods for processing-based file distribution in an aggregation of cloud storage services | |
| US9792306B1 (en) | Data transfer between dissimilar deduplication systems | |
| CN109597717B (en) | Data backup and recovery method and device, electronic equipment and storage medium | |
| US20190306240A1 (en) | Systems and methods for third-party policy-based file distribution in an aggregation of cloud storage services | |
| US9830231B2 (en) | Processes and methods for client-side fingerprint caching to improve deduplication system backup performance | |
| US8838968B2 (en) | System and method for virtual machine data protection in a public cloud | |
| US9922041B2 (en) | Storing data files in a file system | |
| US9390101B1 (en) | Social deduplication using trust networks | |
| CN106649676A (en) | Duplication eliminating method and device based on HDFS storage file | |
| US8474038B1 (en) | Software inventory derivation | |
| US8655844B1 (en) | File version tracking via signature indices | |
| CN112925750B (en) | Method, electronic device and computer program product for accessing data | |
| TWI420333B (en) | A distributed de-duplication system and the method therefore | |
| Su et al. | Design and implementation of various file deduplication schemes on storage devices | |
| CN116894013A (en) | File processing method and device, storage medium and electronic equipment | |
| Neuner et al. | PeekaTorrent: Leveraging P2P hash values for digital forensics | |
| US8701193B1 (en) | Malware detection via signature indices | |
| Kumar et al. | Cloud storage performance improvement using deduplication and compression techniques | |
| Periasamy et al. | The Enhancement of Storage and Bandwidth Optimization Using Data De-Duplication | |
| CN116049891A (en) | Operation request processing method, device, electronic equipment and storage medium | |
| JP6459707B2 (en) | Directory information management apparatus, information processing system, directory information management method, and directory information management program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |