CN110995788B - Method for realizing breakpoint continuous uploading and file storage of HTTP (hyper text transport protocol) server - Google Patents
Method for realizing breakpoint continuous uploading and file storage of HTTP (hyper text transport protocol) server Download PDFInfo
- Publication number
- CN110995788B CN110995788B CN201911107977.2A CN201911107977A CN110995788B CN 110995788 B CN110995788 B CN 110995788B CN 201911107977 A CN201911107977 A CN 201911107977A CN 110995788 B CN110995788 B CN 110995788B
- Authority
- CN
- China
- Prior art keywords
- data block
- file
- stored
- data
- offset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/172—Caching, prefetching or hoarding of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/104—Peer-to-peer [P2P] networks
- H04L67/1074—Peer-to-peer [P2P] networks for supporting data block transmission mechanisms
- H04L67/1078—Resource delivery mechanisms
- H04L67/108—Resource delivery mechanisms characterised by resources being split in blocks or fragments
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method for realizing breakpoint continuous uploading and file storage of an HTTP server, which specifically comprises the following steps: s1, after receiving the fragments of the uploaded files, the server divides the fragments into smaller data blocks according to the multiple of the memory pages, and marks the data blocks to be completely received and stored through bitmap mapping files; s2, when the received data blocks are continuous and exceed the set number, calculating the MD5 value of the continuous data block set; and S3, after all the fragments of the file are received and stored, the MD5 values are verified by the client, and the stored fragments are combined into a complete file. The invention fragments the uploaded file into smaller data blocks according to rules, maps the file through a bitmap, marks, receives and stores the file, and realizes the problem of repeated coverage of the file through a specific algorithm, thereby realizing the breakpoint continuous uploading of an HTTP protocol.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a method for realizing breakpoint continuous uploading file storage of an HTTP server.
Background
The HTTP breakpoint resuming refers to that based on the HTTP protocol, when uploading/downloading, a file is divided into several parts, each part is uploaded/downloaded by using a thread, and if a fault interrupt is encountered, the uncompleted part can be uploaded/downloaded continuously from the part that has been uploaded/downloaded, without uploading/downloading from the beginning.
HTTP/1.1 starts to support breakpoint continuous transmission downloading, and through a GET method of an HTTP protocol, a Range and a Content-Range entity header are used for realizing the exchange between a client and a server to acquire a specified file segment.
The URI corresponding to the PUT method of the HTTP protocol is the resource itself to be created or updated. Such as: PUT http: the semantics of// www.example.com/file are to create or update a file. The side effects of multiple PUTs on the same URI are the same as those of one PUT; thus, the PUT method has idempotency. The more correct definition of a PUT is replace (create or update), i.e., if present, a replacement is created, and if not present, it is created.
However, in many current HTTP servers, it is considered that overlapping coverage of files may cause a problem of resource consistency, and a Content-Range entity header under the PUT method is not implemented, so that the PUT method cannot implement breakpoint continuous uploading.
Disclosure of Invention
Aiming at the defects of the prior art, the invention aims to provide a method for realizing the storage of the breakpoint continuous uploading file of the HTTP server, which divides the uploaded file into smaller data blocks according to rules, maps the file through a bitmap, marks the file for receiving and storing, and realizes the problem of repeated coverage of the file through a specific algorithm, thereby realizing the breakpoint continuous uploading of the HTTP protocol.
In order to achieve the purpose, the invention adopts the following technical scheme:
a method for realizing breakpoint continuous uploading file storage of an HTTP server specifically comprises the following steps:
s1, after receiving the fragments of the uploaded files, the server divides the fragments into smaller data blocks according to the multiple of the memory pages, and marks the data blocks to be completely received and stored through bitmap mapping files;
s2, when the received data blocks are continuous and exceed the set number, calculating the MD5 value of the continuous data block set;
s3, after all the fragments of the file are received and stored, the MD5 values are verified by the client, and the stored fragments are combined into a complete file;
the specific process of step S1 is as follows:
s1.1, establishing a bitmap mapping file FMAP by a server according to the total length L of an uploaded file, a preset multiple factor F and a memory page size PS; mapping every 2 bits of the bitmap mapping file into 1 data block, wherein binary 00 represents blank and no data block is stored, binary 11 represents a complete data block, binary 01 or 10 represents that a data block is incomplete and the data length is less than the length of the mapping data block, 01 represents that the tail offset of the stored data block is aligned with the initial position of the data block with the complete size, and 10 represents that the stored data block starts from the end position of the data block with the complete size; the size of a complete data block is BS (F) PS, and the size of the bitmap mapping file is equal to ((L + BS-1)/F) BS + 3)/4;
s1.2, a server receives a fragment SP, an offset SO of the fragment SP relative to the starting position of a complete file and the length SL of the fragment SP; if the length SL of the fragment is less than BS, go to step S1.3, otherwise go to step S1.4;
s1.3, data block merging processing:
s1.3.1, calculating an offset interval according to the offset SO of the slice SP:
the offset interval is [ S0, rounded ((SO + PS-1)/PS) × PS ];
s1.3.2, checking whether there is a data block within the offset interval range calculated in step S1.3.1: if the fragment SP does not exist or the fragment SP does not overlap with the existing fragment, the fragment SP and the offset SO thereof are saved; if the fragment SP is overlapped with the existing fragment, comparing the overlapped part, and if the overlapped part is different, performing difference data processing;
s1.3.3, combining the fragment SP with the existing fragment, if the offset after combination is an integral multiple of BS and the length of the combined data block is equal to BS, setting the bit position corresponding to the bitmap mapping file FMAP as 11, which indicates that the corresponding data block is completely stored; finishing the processing;
s1.4, converting the offset SO of the fragment SP into an offset set SO1，SO2，SO3...SOn－1，SOnThen calculates the corresponding data size L1，L2，L3，...，Ln－1，LnAnd according to the data size L1，L2，L3，...，Ln－1，LnData block DB is cut out1，DB2，DB3，...，DBn－1，DBnStoring; wherein:
SO1equal to SO;
if S0 is not an integer multiple of BS, then adjust L1To make SO2Is an integer multiple of BS;
SO2＝S01+L1，SO3＝SO2+L2，…，SOn＝SOn－1+Ln－1；
L2，L3，...，Ln－1are the same in size and equal to the BS;
Lnis less than or equal to BS;
SO3＝SO2+BS，SO4＝SO3+BS，…，SOn＝SOn－1+BS；
s1.5, according to the offset set obtained in the step S1.4, sequentially checking bits corresponding to a bitmap mapping file FMAP, if the bits are 11, comparing the stored data block with the corresponding data block to be stored, and if the data are different, performing difference data processing; if the bit is 10, comparing the data block to be stored with the stored data block from the beginning, wherein the comparison data length is the data block with the smaller length, and if the data is different, performing difference data processing; if the bit is 01, comparing the reverse direction from the tail of the data block to be stored with the stored data block, wherein the comparison data length is the smaller of the length of the two data blocks, and if the data are different, performing difference data processing;
the specific process of the difference data processing is as follows:
(1) if the rule is set to overwrite:
if the size of the data block to be stored is the same as that of the stored data block, the stored data block is stored in an overlaying mode;
if the data block to be stored is different from the stored data block, deleting the stored data block, storing the data block to be stored, and updating the offset of the data block relative to the start position of the complete file;
(2) if the rule is set to prohibit overwriting:
ending the file fragmentation processing operation;
s1.6, data block DB obtained by dividing fragment SP1，DB2，DB3，...，DBn－1，DBnStored as a plurality of small files F1、F2....FnRecording and storing the offset of each small file relative to the start position of the complete file; setting a bitmap mapping file FMAP according to the offset set in the step S1.4; wherein:
if the bit of the bitmap mapping file corresponding to the data block to be stored is 00, storing the data block;
if SOn+LnEqual to the full file length, indicates DBnAnd if the data block is the last data block, the bit position of the corresponding bitmap mapping file is 11.
Further, the specific process of step S2 is: and checking the bitmap mapping file, and if the bits in a certain fixed range are all set, namely a set number of continuous data blocks are stored completely, calculating and recording the MD5 value of the continuous data block set.
Further, the set number is 1/N of the total number of the data blocks.
Still further, as a preferable mode, N ═ 10.
Further, the specific process of step S3 is: checking the bitmap mapping file, if all the bit positions are set, checking the MD5 value through the client, sequentially combining all the corresponding data blocks into a large file, calculating the MD5 value of the large file, and finishing uploading after final checking through the client.
And further, after the uploading is finished, clearing the MD5 values of the deleted data blocks, the bitmap mapping file and the continuous data block set.
The invention has the beneficial effects that:
1. the method of the invention decomposes the file fragments into data blocks with fixed size through a specific algorithm and solves the possible overlapping problem of a plurality of file fragments, thereby realizing the breakpoint continuous uploading of HTTP protocol. The invention can ensure that the single file is uploaded and merged and kept consistent after being randomly fragmented.
Detailed Description
The present invention will be further described below, and it should be noted that the present embodiment is based on the technical solution, and a detailed implementation manner and a specific operation process are provided, but the protection scope of the present invention is not limited to the present embodiment.
Example 1
The embodiment provides a method for implementing breakpoint continuous uploading file storage of an HTTP server, which specifically includes:
s1, after receiving the fragments of the uploaded files, the server divides the fragments into smaller data blocks according to the multiple of the memory pages, and marks the data blocks to be completely received and stored through bitmap mapping files;
s2, when the number of the continuous received data blocks exceeds the set number, calculating the MD5 value of the continuous data block set;
and S3, after all the fragments of the uploaded file are received and stored, the client checks all MD5 values, and after the check is passed, the stored fragments are combined into a complete file.
Further, the specific process of step S1 is as follows:
s1.1, establishing a bitmap mapping file FMAP by a server according to the total length L of an uploaded file, a preset multiple factor F and a memory page size PS; mapping every 2 bits of the bitmap mapping file into 1 data block, wherein binary 00 represents blank and no data block is stored, binary 11 represents a complete data block, binary 01 or 10 represents that a data block is incomplete and the data length is less than the length of the mapping data block, 01 represents that the tail offset of the stored data block is aligned with the initial position of the data block with the complete size, and 10 represents that the stored data block starts from the end position of the data block with the complete size; the size of a complete data block is BS (F) PS, and the size of the bitmap mapping file is equal to ((L + BS-1)/F) BS + 3)/4;
s1.2, when the server receives one fragment SP, receiving the offset SO of the fragment SP relative to the starting position of the complete file and the length SL of the fragment SP at the same time; if the length SL of the fragment SP is smaller than BS, turning to step S1.3, otherwise, turning to step S1.4;
s1.3, data block merging processing:
s1.3.1, calculating an offset interval according to the offset SO of the slice SP:
the offset interval is [ S0, rounded ((SO + PS-1)/PS) × PS ];
s1.3.2, checking whether the slice in the offset interval range calculated in step S1.3.1 exists, if the slice does not exist or the slice SP does not overlap with the existing slice, saving the slice SP and the offset SO thereof; if the fragment SP is overlapped with the existing fragment, comparing the overlapped part, and if the overlapped part is different, performing difference data processing;
the specific process of the difference data processing is as follows:
(1) if the rule is set to overwrite:
if the size of the data block to be stored is the same as that of the stored data block, the stored data block is stored in an overlaying mode;
if the size of the data block to be stored is different from that of the stored data block, deleting the stored data block, storing the data block to be stored, and updating the offset of the data block relative to the start position of the complete file;
(2) if the rule is set to prohibit overwriting:
ending the file fragmentation processing operation;
s1.3.3, combining the fragment SP with the existing fragment, if the offset after combination is integral multiple of BS and the length of the combined data block is equal to BS, setting the bit position corresponding to the bitmap mapping file FMAP as 11, which indicates that the corresponding data block is completely stored;
s1.4, slicingConversion of offset SO of SP into offset set SO1，SO2，SO3...SOn－1，SOnThen calculates the corresponding data size L1，L2，L3，...，Ln－1，LnAnd according to the data size L1，L2，L3，...，Ln－1，LnData block DB to be stored is cut out1，DB2，DB3，...，DBn－1，DBnStoring; wherein:
SO1equal to SO;
if SO is not an integer multiple of BS, adjusting L1To make SO2Is an integer multiple of BS;
SO2＝S01+L1，SO3＝SO2+L2，…，SOn＝SOn－1+Ln－1；
L2，L3，...，Ln－1are the same in size and equal to the BS;
Lnis less than or equal to BS; therefore, there are:
SO3＝SO2+BS，SO4＝SO3+BS，…，SOn＝SOn－1+BS；
s1.5, according to the offset set obtained in the step S1.4 and the cut data blocks to be stored, sequentially checking bit positions corresponding to a bitmap mapping file FMAP, if the bit positions are 11, comparing the stored data blocks with the corresponding data blocks to be stored, and if the data are different, performing difference data processing; if the bit is 10, comparing the data block to be stored with the stored data block from the beginning, wherein the comparison data length is the smaller of the two data block lengths, and if the data are different, performing difference data processing; if the bit is 01, comparing the reverse direction from the tail of the data block to be stored with the stored data block, and if the data length is different, performing difference data processing;
the specific process of the difference data processing is as follows:
(1) if the rule is set to overwrite:
if the size of the data block to be stored is the same as that of the stored data block, the stored data block is stored in an overlaying mode;
if the size of the data block to be stored is different from that of the stored data block, deleting the stored data block, storing the data block to be stored, and updating the offset of the data block relative to the start position of the complete file;
(2) if the rule is set to prohibit overwriting:
ending the file fragmentation processing operation;
s1.6, data block DB obtained by dividing fragment SP1，DB2，DB3，...，DBn－1，DBnStored as a plurality of small files F1、F2....FnRecording and storing the offset of each small file relative to the start position of the complete file; setting a bitmap mapping file FMAP according to the offset set in the step S1.4; wherein:
if the bit of the bitmap mapping file corresponding to the data block to be stored is 00, storing the data block;
if SOn+LnEqual to the full file length, indicates DBnAnd if the data block is the last data block, the bit position of the corresponding bitmap mapping file is 11.
Further, the specific process of step S2 is: and checking the bitmap mapping file, and if the bits in a certain fixed range are all set, namely a set number of continuous data blocks are stored completely, calculating and recording the MD5 value of the continuous data block set.
In this embodiment, the set number is 1/N of the total number of data blocks. More specifically, in the present embodiment, N is 10. The reason for taking N to 10 is to assume that the file is divided into 10 shares, because many download tools divide the file into 10 downloads. This value can be modified.
Still further, the specific process of step S3 is: checking the bitmap mapping file, if all the bit positions are set, checking the MD5 value through the client, sequentially combining all the corresponding data blocks into a large file, calculating the MD5 value of the large file, and finishing uploading after final checking through the client.
And further, after the uploading is finished, clearing the MD5 values of the deleted data blocks, the bitmap mapping file and the continuous data block set.
Examples
Assuming that the memory page size is 4096 bytes, the multiple factor is 1, and the size of the complete data block is 4096 bytes; the original file a.zip is 23345, and is divided into 3 fragments; slicing SP0The relative complete file offset is 1 and the length is 5000; SP1The relative full file offset is 5001, length 18000, SP2The relative complete file offset is 23001, and 345 length; the stored tile used to calculate MD5 is 2, the first set of data blocks is 3, and the second set of data blocks is 3.
1. Based on the file length 23345, a bitmap file a.zip.bmap is created, with a size of 2 bytes and a length calculation formula of ((23345+4095)/4096+ 3)/4. Initialize all bits of a.zip.bmap to 0.
2. Processing SP0The offset is 1, length 5000, length greater than 4096 bytes, and the slice is divided into 2 shares, the first share being 4096 bytes and the second share being 904 bytes. Respectively stored as a.zip.p. 1, a.zip.p4097, which corresponds to the 1 st, 2 nd, 3 rd, 4 th bit of the first byte of a.zip.bmap being 0, the 1 st, 2 nd bit of the first byte of a.zip.bmap being set 11, the 3 rd, 4 th bit being set 10.
3. Processing SP1The offset is 5001, the length is 18000, the length is larger than 4096 bytes, the slice is divided into 5 parts, the 1 st part is 3192 bytes, the 2 nd part is 4096 bytes, the 3 rd part is 4096 bytes, the 4 th part is 4096 bytes, and the 5 th part is 2520 bytes; the offsets are 5001, 8193, 12289, 16385, 20481 respectively.
The 2 nd, 3 rd and 4 th offsets are all 4096 times, the length is also 4096 times, the 5 th, 6 th, 7 th and 8 th bit bits of the first byte and the 1 st and 2 nd bit bits of the second byte of a.zip.
The 1 st byte is 3192 bytes and the offset is 5001. checking that the 3 rd and 4 th bits of the corresponding a.zip. bmap first byte are 10 indicates that there is a slice smaller than one data block size from 4097 times. The size of the corresponding fragment is read to be 904 bytes, and it is determined that the 1 st part and the stored fragment can be merged into a complete data block. Setting the 3 rd and 4 th bits of the first byte of the corresponding a.zip.bmap to 11, and merging the storage data blocks.
2520 byte for the 5 th byte, 20481 for the offset, check the bit 3, 4 of the corresponding second byte of a.zip.bmap to 0, set it to 10, and store the slice a.zip.p20481
4. Processing SP2The offset is 23001, the length is 345, the length is smaller than 4096 bytes, the sum of the length and the offset is the size of the complete file, the last fragment is represented, the bit 3 and the bit 4 of the second byte corresponding to the a.zip.bmap are 10, the stored data block is represented, the length is 2520, and the display and SP are respectively shown2Joining with SP2Merge and overwrite storage and set the corresponding bit position 11.
5. Checking a.zip.bmap, completely storing the 1 st data block to the 3 rd data block, and calculating a corresponding MD5 value; the 4 th to 6 th data blocks are completely stored, and the corresponding MD5 is calculated; total 2 MD5 values
6. Checking a.zip.bmap, setting all valid bits, calculating the MD5 value a.zip.md5 of all data blocks, and totaling 1 MD5 value
7. Sending 3 MD5 values of 5 and 6 to a file uploader, after the uploader checks that no errors exist, the server side combines the data blocks into a complete file in sequence, and reserving the MD5 value a.zip.md5
8. The data block, a.zip.bmap, 1 st-3 rd and 4 th-6 th data block MD5 values of the delete slice are cleared.
Various corresponding changes and modifications can be made by those skilled in the art based on the above technical solutions and concepts, and all such changes and modifications should be included in the protection scope of the present invention.
Claims (6)
1. A method for realizing the breakpoint continuous uploading file storage of an HTTP server is characterized by comprising the following steps:
s1, after receiving the fragments of the uploaded files, the server divides the fragments into smaller data blocks according to the multiple of the memory pages, and marks the data blocks to be completely received and stored through bitmap mapping files;
s2, when the received data blocks are continuous and exceed the set number, calculating the MD5 value of the continuous data block set;
s3, after all the fragments of the file are received and stored, the MD5 values are verified by the client, and the stored fragments are combined into a complete file;
the specific process of step S1 is as follows:
s1.1, establishing a bitmap mapping file FMAP by a server according to the total length L of an uploaded file, a preset multiple factor F and a memory page size PS; mapping every 2 bits of the bitmap mapping file into 1 data block, wherein binary 00 represents blank and no data block is stored, binary 11 represents a complete data block, binary 01 or 10 represents that a data block is incomplete and the data length is less than the length of the mapping data block, 01 represents that the tail offset of the stored data block is aligned with the initial position of the data block with the complete size, and 10 represents that the stored data block starts from the end position of the data block with the complete size; the size of a complete data block is BS (F) PS, and the size of the bitmap mapping file is equal to ((L + BS-1)/F) BS + 3)/4;
s1.2, a server receives a fragment SP, an offset SO of the fragment SP relative to the starting position of a complete file and the length SL of the fragment SP; if the length SL of the fragment is less than BS, go to step S1.3, otherwise go to step S1.4;
s1.3, data block merging processing:
s1.3.1, calculating an offset interval according to the offset SO of the slice SP:
the offset interval is [ S0, rounded ((SO + PS-1)/PS) × PS ];
s1.3.2, checking whether there is a data block within the offset interval range calculated in step S1.3.1: if the fragment SP does not exist or the fragment SP does not overlap with the existing fragment, the fragment SP and the offset SO thereof are saved; if the fragment SP is overlapped with the existing fragment, comparing the overlapped part, and if the overlapped part is different, performing difference data processing;
s1.3.3, combining the fragment SP with the existing fragment, if the offset after combination is an integral multiple of BS and the length of the combined data block is equal to BS, setting the bit position corresponding to the bitmap mapping file FMAP as 11, which indicates that the corresponding data block is completely stored; finishing the processing;
s1.4, converting the offset SO of the fragment SP into an offset set SO1，SO2，SO3...SOn－1，SOnThen calculates the corresponding data size L1，L2，L3，...，Ln－1，LnAnd according to the data size L1，L2，L3，...，Ln－1，LnData block DB is cut out1，DB2，DB3，...，DBn－1，DBnStoring; wherein:
SO1equal to SO;
if S0 is not an integer multiple of BS, then adjust L1To make SO2Is an integer multiple of BS;
SO2＝S01+L1，SO3＝SO2+L2，…，SOn＝SOn－1+Ln－1；
L2，L3，...，Ln－1are the same in size and equal to the BS;
Lnis less than or equal to BS;
SO3＝SO2+BS，SO4＝SO3+BS，…，SOn＝SOn－1+BS；
s1.5, according to the offset set obtained in the step S1.4, sequentially checking bits corresponding to a bitmap mapping file FMAP, if the bits are 11, comparing the stored data block with the corresponding data block to be stored, and if the data are different, performing difference data processing; if the bit is 10, comparing the data block to be stored with the stored data block from the beginning, wherein the comparison data length is the data block with the smaller length, and if the data is different, performing difference data processing; if the bit is 01, comparing the reverse direction from the tail of the data block to be stored with the stored data block, wherein the comparison data length is the smaller of the length of the two data blocks, and if the data are different, performing difference data processing;
the specific process of the difference data processing is as follows:
(1) if the rule is set to overwrite:
if the size of the data block to be stored is the same as that of the stored data block, the stored data block is stored in an overlaying mode;
if the data block to be stored is different from the stored data block, deleting the stored data block, storing the data block to be stored, and updating the offset of the data block relative to the start position of the complete file;
(2) if the rule is set to prohibit overwriting:
ending the file fragmentation processing operation;
s1.6, data block DB obtained by dividing fragment SP1，DB2，DB3，...，DBn－1，DBnStored as a plurality of small files F1、F2....FnRecording and storing the offset of each small file relative to the start position of the complete file; setting a bitmap mapping file FMAP according to the offset set in the step S1.4; wherein:
if the bit of the bitmap mapping file corresponding to the data block to be stored is 00, storing the data block;
if SOn+LnEqual to the full file length, indicates DBnAnd if the data block is the last data block, the bit position of the corresponding bitmap mapping file is 11.
2. The method for implementing HTTP server breakpoint resuming uploading of file storage according to claim 1, wherein the specific process of step S2 is as follows: and checking the bitmap mapping file, and if the bits in a certain fixed range are all set, namely a set number of continuous data blocks are stored completely, calculating and recording the MD5 value of the continuous data block set.
3. The method as claimed in claim 2, wherein the predetermined number is 1/N of the total number of the data blocks.
4. The method of claim 3, wherein N is 10.
5. The method for implementing HTTP server breakpoint resuming uploading of file storage according to claim 2, wherein the specific process of step S3 is as follows: checking the bitmap mapping file, if all the bit positions are set, checking the MD5 value through the client, sequentially combining all the corresponding data blocks into a large file, calculating the MD5 value of the large file, and finishing uploading after final checking through the client.
6. The method for implementing storage of the HTTP server breakpoint continuous upload file according to claim 5, wherein after uploading is finished, MD5 values of deleted data blocks, bitmap mapping files and continuous data block sets are cleared.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911107977.2A CN110995788B (en) | 2019-11-13 | 2019-11-13 | Method for realizing breakpoint continuous uploading and file storage of HTTP (hyper text transport protocol) server |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911107977.2A CN110995788B (en) | 2019-11-13 | 2019-11-13 | Method for realizing breakpoint continuous uploading and file storage of HTTP (hyper text transport protocol) server |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110995788A CN110995788A (en) | 2020-04-10 |
| CN110995788B true CN110995788B (en) | 2022-02-22 |
Family
ID=70084167
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911107977.2A Active CN110995788B (en) | 2019-11-13 | 2019-11-13 | Method for realizing breakpoint continuous uploading and file storage of HTTP (hyper text transport protocol) server |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110995788B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112073535A (en) * | 2020-09-21 | 2020-12-11 | 烟台云朵软件有限公司 | Bitmap-based data packet fragment transmission method |
| CN114051152A (en) * | 2022-01-17 | 2022-02-15 | 飞狐信息技术（天津）有限公司 | Video playing method and device, storage medium and electronic equipment |
| CN115277723B (en) * | 2022-07-19 | 2024-06-18 | 国能信控互联技术有限公司 | Breakpoint continuous transmission method and system of edge collection history module based on buffer event |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102710758A (en) * | 2012-05-22 | 2012-10-03 | 苏州云博信息技术有限公司 | Method for continuously transferring breakpoint of data stream based on single chip system |
| CN103858440A (en) * | 2011-08-31 | 2014-06-11 | 高通股份有限公司 | Switch signaling methods providing improved switching between representations for adaptive http streaming |
| CN104253816A (en) * | 2014-09-25 | 2014-12-31 | 武汉传神信息技术有限公司 | HTTP (hyper text transport protocol)-based large file page breakpoint resuming method |
| CN104679830A (en) * | 2015-01-30 | 2015-06-03 | 乐视网信息技术（北京）股份有限公司 | File processing method and device |
| EP3065341A1 (en) * | 2015-03-05 | 2016-09-07 | Mitsubishi Electric R&D Centre Europe B.V. | Content classification medthod and device |
| CN107566540A (en) * | 2017-10-31 | 2018-01-09 | 云宏信息科技股份有限公司 | Big document breakpoint transmission method, apparatus and system |
| CN108540566A (en) * | 2018-04-18 | 2018-09-14 | 暴风集团股份有限公司 | file uploading method, device, system and client and server |
| CN109639807A (en) * | 2018-12-19 | 2019-04-16 | 中国四维测绘技术有限公司 | A kind of massive remote sensing image file network transmission method based on slice slice |
| US10303797B1 (en) * | 2015-12-18 | 2019-05-28 | EMC IP Holding Company LLC | Clustering files in deduplication systems |
| US10445292B1 (en) * | 2016-03-30 | 2019-10-15 | Veritas Technologies Llc | Sharing services between deduplication systems |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190236102A1 (en) * | 2018-01-29 | 2019-08-01 | Planet Data Solutions | System and method for differential document analysis and storage |
-
2019
- 2019-11-13 CN CN201911107977.2A patent/CN110995788B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103858440A (en) * | 2011-08-31 | 2014-06-11 | 高通股份有限公司 | Switch signaling methods providing improved switching between representations for adaptive http streaming |
| CN102710758A (en) * | 2012-05-22 | 2012-10-03 | 苏州云博信息技术有限公司 | Method for continuously transferring breakpoint of data stream based on single chip system |
| CN104253816A (en) * | 2014-09-25 | 2014-12-31 | 武汉传神信息技术有限公司 | HTTP (hyper text transport protocol)-based large file page breakpoint resuming method |
| CN104679830A (en) * | 2015-01-30 | 2015-06-03 | 乐视网信息技术（北京）股份有限公司 | File processing method and device |
| EP3065341A1 (en) * | 2015-03-05 | 2016-09-07 | Mitsubishi Electric R&D Centre Europe B.V. | Content classification medthod and device |
| US10303797B1 (en) * | 2015-12-18 | 2019-05-28 | EMC IP Holding Company LLC | Clustering files in deduplication systems |
| US10445292B1 (en) * | 2016-03-30 | 2019-10-15 | Veritas Technologies Llc | Sharing services between deduplication systems |
| CN107566540A (en) * | 2017-10-31 | 2018-01-09 | 云宏信息科技股份有限公司 | Big document breakpoint transmission method, apparatus and system |
| CN108540566A (en) * | 2018-04-18 | 2018-09-14 | 暴风集团股份有限公司 | file uploading method, device, system and client and server |
| CN109639807A (en) * | 2018-12-19 | 2019-04-16 | 中国四维测绘技术有限公司 | A kind of massive remote sensing image file network transmission method based on slice slice |
Non-Patent Citations (4)
| Title |
|---|
| "FMAP_a_technology_mapping_algorithm_for_FPGA_with_MUX-LUT_mixed_architecture";Wen Yujie，Tong Jiarong，Charles Chiang;《ASIC, 2003. Proceedings. 5th International Conference on》;20031024;全文 * |
| "一种改进的P2P文件传输系统的研究与实现";李杰绪;《中国优秀硕士学位论文全文数据库-信息科技辑》;20180131;全文 * |
| "基于应用层动态特征识别技术的数据流分析软件工具包";王恒亮;《中国优秀硕士学位论文全文数据库-信息科技辑》;20190531;全文 * |
| "轻量级BitTorrent网络平台设计与实现";陈威;《中国优秀硕士学位论文全文数据库-信息科技辑》;20110430;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110995788A (en) | 2020-04-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110995788B (en) | Method for realizing breakpoint continuous uploading and file storage of HTTP (hyper text transport protocol) server | |
| US10725772B2 (en) | Electric control device for vehicles, program updating method, and server apparatus | |
| EP3847602B1 (en) | Secure and transparent pruning for blockchains | |
| DE69429482T2 (en) | Method and device for data compression | |
| US7546492B2 (en) | Remotely repairing files by hierarchical and segmented cyclic redundancy checks | |
| US20120185612A1 (en) | Apparatus and method of delta compression | |
| US20050010916A1 (en) | System for providing software application updates to multiple clients on a network | |
| US20080034346A1 (en) | Preparation for Software on Demand System | |
| US10732881B1 (en) | Region cloning for deduplication | |
| US7222137B2 (en) | Difference updating method, program and apparatus | |
| US20080275923A1 (en) | Method for the expungement of backup versions of files on server targets that are configured to be updated sequentially | |
| US20110246763A1 (en) | Parallel method, machine, and computer program product for data transmission and reception over a network | |
| US11169796B2 (en) | Methods and systems for remote software update | |
| US20170060957A1 (en) | Optimized and consistent replication of file overwrites | |
| CN111385354A (en) | File transmission method and device, electronic equipment and nonvolatile storage medium | |
| US11797488B2 (en) | Methods for managing storage in a distributed de-duplication system and devices thereof | |
| US11860739B2 (en) | Methods for managing snapshots in a distributed de-duplication system and devices thereof | |
| CN112748877A (en) | File integration uploading method and device and file downloading method and device | |
| EP3510483B1 (en) | Binary image differential patching | |
| CN114036004A (en) | Data backup method, data recovery method and data recovery device | |
| JP4285679B2 (en) | Differential transfer method, program and system | |
| US20160037119A1 (en) | Video recording apparatus, and video recording method when temporary network failure occurs | |
| US10901952B2 (en) | Method for transferring a difference file | |
| WO2017022034A1 (en) | Information processing device, information processing method, and information processing program | |
| JP2005100007A (en) | Data file saving system and allocation method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |