CN112583889A - Large file transmission method and device - Google Patents
Large file transmission method and device Download PDFInfo
- Publication number
- CN112583889A CN112583889A CN202011293127.9A CN202011293127A CN112583889A CN 112583889 A CN112583889 A CN 112583889A CN 202011293127 A CN202011293127 A CN 202011293127A CN 112583889 A CN112583889 A CN 112583889A
- Authority
- CN
- China
- Prior art keywords
- file
- transmission
- data
- information
- database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/14—Session management
- H04L67/141—Setup of application sessions
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/16—Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
- H04L69/161—Implementation details of TCP/IP or UDP/IP stack architecture; Specification of modified or new header fields
- H04L69/162—Implementation details of TCP/IP or UDP/IP stack architecture; Specification of modified or new header fields involving adaptations of sockets based mechanisms
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention relates to a method and a device for transmitting a large file. The transmission method is used for transmitting files in a Netty framework. The transmission method comprises the following steps: acquiring transmission request information and data information of a file to be transmitted; carrying out http coding and http compression on file data; carrying out block transmission coding on the file data based on an http1.1 protocol; judging the transmission request, judging whether the transmission request is legal or not, and judging whether the transmission request is breakpoint continuous transmission or not; carrying out block transmission on the file, decompressing and decoding the received block file data, and then writing in a temporary file for file assembly; and when the data is transmitted to the last block of data, renaming the temporary file, feeding back the transmission success information and updating the file transmission state. According to the invention, the block transmission coding based on the http1.1 protocol in the Netty framework realizes the block transmission aiming at the large file, realizes the safe, stable and efficient transmission of the large file, and meets the requirement of the large file transmission.
Description
Technical Field
The present invention relates to file transmission technologies, and in particular, to a method and an apparatus for transmitting a large file.
Background
Data is growing explosively, almost doubling every two years. The increase is not only the amount of data, but also the capacity of the monomer file: a picture 2-3G, a manuscript 4-5G, a video clip 3-4G, a dozen or so design drawings G … … and even hundreds of G of large documents. The file is used as one of common data forms of storage and exchange, a client needs to upload local file data to a storage platform or exchange the local file data for a third party to use, and the whole file transmission process needs to be completed through a file uploading and downloading function mode provided by an application system.
In the field of file uploading or file exchange application, an http interface or an FTP protocol is mainly adopted for file data transmission at present. When the http interface is used for file transmission, the maximum capacity is limited, and meanwhile, due to network or connection problems, the terminal cannot perform efficient transmission in the transmission process of large files (more than 2-3G). When the FTP protocol is adopted for file transmission, the transmission process is interrupted due to network factors, and connection transmission needs to be carried out again, so that the requirement for efficient and stable transmission of large files cannot be met.
Disclosure of Invention
The present invention is directed to overcome the above drawbacks of the prior art, and provides a method and an apparatus for transmitting a large file, so as to meet the requirement of efficient and stable transmission of the large file.
In order to achieve the purpose, the invention adopts the following technical scheme: a large file transfer method for transferring a file in a Netty framework, comprising the steps of:
acquiring transmission request information and data information of a file to be transmitted;
carrying out http coding and http compression on file data;
carrying out block transmission coding on the file data based on an http1.1 protocol;
judging the transmission request, judging whether the transmission request is legal or not, and judging whether the transmission request is breakpoint continuous transmission or not;
carrying out block transmission on the file, decompressing and decoding the received block file data, and then writing in a temporary file for file assembly;
and when the data is transmitted to the last block of data, renaming the temporary file, feeding back the transmission success information and updating the file transmission state.
After the transmission request information and the data information of the file to be transmitted are obtained, the WebSocket long connection is established and used for transmitting the transmission progress information of the file.
When the transmission request is judged, inquiring in the database through the transmission ID of the file, if the same transmission ID exists in the database and the corresponding file transmission is the breakpoint continuous transmission, modifying the corresponding file transmission state in the database and subsequently finding the corresponding temporary file according to the original transmission progress and the file information for writing; if the same transmission ID does not exist in the database, the corresponding file transmission is indicated as the first writing, a new temporary file needs to be created subsequently for writing, and the file transmission is recorded in the database.
When the file is transmitted in blocks, if the transmission process is interrupted, the transmission state of the file is modified to be suspended, and the transmission progress and the file information are recorded in the database.
And when the file is subjected to breakpoint continuous transmission, acquiring the transmission progress and file information, then segmenting the file to obtain file data to be continuously transmitted, and continuing the transmission progress.
When the file is transmitted in a blocking mode, the transmission progress and the transmission speed are obtained according to the size of the transmitted file data and the transmission time consumption, and feedback is carried out through WebSocket long connection.
And when the last block of data is transmitted, renaming the temporary file into the original name of the file, feeding back the transmission success information through the WebSocket long connection, and updating the file transmission state to the database.
When the file is transmitted in blocks and each file block data is received, whether a temporary file corresponding to the file block data exists or not is judged firstly: if yes, directly writing the file block data into the temporary file; if the file transmission information does not exist, a temporary file is created by using the transmission ID, the file name and the file size of the file as parameters, and the file transmission information is updated to the database and the memory.
The invention also discloses an electronic device, which is characterized by comprising: a processor; and a memory having computer readable instructions stored thereon which, when executed by the processor, implement the large file transfer method of the present invention.
The invention also discloses a computer readable storage medium on which a computer program is stored, which when executed by a processor implements the large file transfer method of the invention.
Compared with the prior art, the invention has the beneficial effects that: the blocking transmission coding based on the http1.1 protocol in the Netty framework realizes the blocking transmission aiming at the large file, realizes the safe, stable and efficient transmission of the large file, realizes the intermittent continuous transmission in the Netty framework through secondary development, meets the user requirement, and greatly improves the working efficiency of the user in the file transmission and exchange process.
Drawings
FIG. 1 is a flow chart of a large file transmission method according to the present invention.
It should be noted that, the products shown in the above views are all appropriately reduced/enlarged according to the size of the drawing and the clear view, and the size of the products shown in the views is not limited.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The same reference numerals denote the same or similar parts in the drawings, and thus, a repetitive description thereof will be omitted.
Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the disclosure. One skilled in the relevant art will recognize, however, that the embodiments of the disclosure can be practiced without one or more of the specific details, or with other methods, components, materials, devices, steps, and so forth. In other instances, well-known structures, methods, devices, implementations, materials, or operations are not shown or described in detail to avoid obscuring aspects of the disclosure.
The block diagrams shown in the figures are functional entities only and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in the form of software, or in one or more software-hardened modules, or in different networks and/or processor devices and/or microcontroller devices.
The invention relates to a large file transmission method which is used for transmitting files in a Netty framework. As shown in fig. 1, the method for transmitting a large file includes the following steps: step S1, acquiring transmission request information and data information of a file to be transmitted; step S2, http encoding and http compressing are carried out on the file data; step S3, block transmission coding based on http1.1 protocol is carried out on the file data; step S4, the transmission request is judged, whether the transmission request is legal is judged, and whether the transmission request is breakpoint transmission is judged; step S5, the file is transmitted in blocks, the received data of the block file is decompressed and decoded, and then the data is written into a temporary file to be assembled; and step S6, renaming the temporary file when the last block of data is transmitted, feeding back the transmission success information and updating the file transmission state.
In step S1, after the transmission request information and the data information of the file to be transmitted are obtained, a WebSocket long connection is also established for transmitting the transmission progress information of the file. In step S4, when the transmission request is judged, the transmission ID of the file is searched for in the database. And if the same transmission ID exists in the database and indicates that the corresponding file transmission is the breakpoint continuous transmission, modifying the corresponding file transmission state in the database and subsequently finding the corresponding temporary file according to the original transmission progress and the file information for writing. If the same transmission ID does not exist in the database, the corresponding file transmission is indicated as the first writing, a new temporary file needs to be created subsequently for writing, and the file transmission is recorded in the database.
In step S5, when the file is transferred in blocks and each file block data is received, it is first determined whether a temporary file corresponding to the file block data exists: if yes, directly writing the file block data into the temporary file; if the file transmission information does not exist, a temporary file is created by using the transmission ID, the file name and the file size of the file as parameters, and the file transmission information is updated to the database and the memory. In step S5, when the file is transferred in blocks, if the transfer process is interrupted, the transfer status of the file is modified to pause and the transfer progress and the file information are recorded in the database (step S51 in fig. 1). When the file is subjected to breakpoint resuming, the transmission progress and the file information are acquired from the database (step S52 in fig. 1), and then the file is divided into file data to be resumed (step S53 in fig. 1), and the transmission process is resumed. In step S5, when the file is transmitted in blocks, the transmission progress and the transmission speed are obtained according to the size of the transmitted file data and the transmission time, and feedback is performed through the WebSocket long connection. In step S6, when the last piece of data is transmitted, the temporary file is renamed to the original name of the file, and the transmission success information is fed back through the WebSocket long connection and the file transmission state is updated to the database.
The large file transmission method realizes the block transmission aiming at the large file in the Netty framework based on the block transmission coding of the http1.1 protocol, and realizes the safe, stable and efficient transmission of the large file. In addition, intermittent continuous transmission is realized through secondary development in the Netty framework, the user requirements are met, and the working efficiency of the user in the file transmission and exchange process is greatly improved.
The large file transfer method is further described in detail below with reference to specific embodiments.
First embodiment
The first embodiment is that the large file transmission method is applied to file uploading operation in a Netty framework.
In step S1, a local file to be uploaded is selected on the browser page, and the Netty client obtains the transmission request information to be uploaded and the data information of the file to be transmitted, where the transmission request information at least includes a request type of the request, a request link, a parameter of the get request, and the like. And then establishing a WebSocket long connection between the browser page end and the Netty server end for transmitting the transmission progress information of the file, so that the browser page end can instantly display the uploading progress and the uploading speed of the current file.
In step S2, http encoding and http compressing are performed on the file data of the upload request. And the Netty client adds an http codec, an http message data compressor and a service processor to the uploading request. The http codec is used for encoding and transmitting the file data of the uploading request. The http message data compressor is used for compressing the file data of the uploading request, so that the network transmission data volume can be greatly reduced, and the transmission speed is improved. The service processor is used for processing result data returned by the Netty server. Meanwhile, the Netty server also needs to add an http codec, an http message data decompressor and a service processor. The http codec is used for decoding the file data of the uploading request. The http message data decompressor is used for decompressing the transmitted file data. The service processor is used for service logic processing of the request data.
In step S3, block transmission coding based on the http1.1 protocol is performed on the file data of the upload request of this time. The transmission message data can be automatically blocked based on the http1.1 protocol blocking-transmission coding (chunked), the file data does not need to be manually blocked in advance, the file block data does not need to be sequenced and transmitted one by one, and finally safe transmission and complete transmission are achieved.
In step S4, it is necessary to make necessary judgment on the current upload request, such as judging whether the transmission request is legal, judging whether the transmission request is a breakpoint resume, and the like. When the transmission request is judged, whether the request link of the uploading request is a link needing to be processed is judged, and if the request link does not belong to a link processed by a Netty server or a Netty client, the uploading request is rejected and a request failure prompt message is fed back. Then finding the transmission ID (uploading ID) of the uploading request file at the parameter of the get request and inquiring in a database (MongoDB). If the same transmission ID and the same file transmission information exist in the database, the uploading request is a breakpoint continuous transmission request, the corresponding file transmission state in the database is modified to be 'uploading in progress', and the corresponding temporary file needs to be found and written in subsequently according to the original transmission progress and the file information. If the same transmission ID and file transmission information do not exist in the database, the uploading request is the first time of writing, a new temporary file needs to be created subsequently for writing, and the file transmission is recorded in the database.
In step S5, the Netty client file starts block upload, the Netty server decompresses and decodes the received block file data, then writes to the temporary file for file assembly, and writes the current upload file information data in the database (MongoDB). In step S5, when the file is uploaded in blocks, the Netty server determines whether a temporary file corresponding to the block data exists after receiving each block data. And if the corresponding temporary file exists, directly writing the file block data into the temporary file. If the corresponding temporary file does not exist, which indicates that the file is uploaded for the first time and is the data of the first file block, a temporary file is created by using the transmission ID, the file name and the file size of the file as parameters for writing, and file transmission information is updated in a database and a memory. In step S5, when the file is uploaded in blocks, the transmission progress and the transmission speed are obtained according to the size of the file data uploaded to the Netty server and the transmission time consumption, and then feedback is performed through the WebSocket long connection, so that the browser page can see the real-time transmission progress and speed.
In step S5, when the file is uploaded in blocks, if the browser page selects to suspend uploading or the uploading is interrupted due to a network reason, the Netty server monitors the uploading interruption and suspends the uploading, and meanwhile, modifies the transmission status of the file to suspend uploading and records the transmission progress and the file information in the database for the subsequent continuous transmission to find the corresponding file transmission information. If the browser page terminal selects to continue uploading or automatically continues the uploading process after the network is recovered, the Netty client acquires the corresponding transmission progress and the file information from the database according to the transmission ID of the file, then divides the file, discards the uploaded part to obtain the file data to be continuously transmitted, and then restarts the uploading process from step S1.
The file block data is continuously uploaded in step S5, and when the last block data is transferred, the process proceeds to step S6. In step S6, the temporary file is renamed to the original name of the uploaded file, the uploaded file is dropped to the Netty server, the uploading success information is fed back through the long WebSocket connection, and the file transmission state is updated to the database (MongoDB) to be the uploading success.
Second embodiment
The second embodiment is that the large file transmission method is applied to the Netty framework for file downloading operation.
In step S1, the browser page requests the Netty server to obtain a file list available for downloading, a file to be downloaded is selected from the file list, and the Netty client obtains the transmission request information of the download and the data information of the file to be transmitted. Wherein the transmission request information at least comprises the request type of the request, the request link, the parameter of the get request and the like. And then establishing a WebSocket long connection between the browser page end and the Netty server end for transmitting the transmission progress information of the file, so that the browser page end can instantly display the downloading progress and the downloading speed of the current file.
In step S2, http encoding and http compression are required to be performed on the file data of the download request. The Netty server side and the Netty client side are both added with an http codec, an http message data compressor, an http message data decompressor and a service processor in the downloading request. The http codec is used for encoding and decoding the file data of the download request. The http message data compressor is used for compressing the file data of the downloading request, so that the network transmission data volume can be greatly reduced, and the transmission speed is improved. The http message data decompressor is used for decompressing the transmitted file data. The service processor is used for processing the service logic of the request data or processing the received result data.
In step S3, the file data of the download request of this time is subjected to http1.1 protocol-based block transfer coding. The transmission message data can be automatically blocked based on the http1.1 protocol blocking-transmission coding (chunked), the file data does not need to be manually blocked in advance, the file block data does not need to be sequenced and transmitted one by one, and finally safe transmission and complete transmission are achieved.
In step S4, the download request is determined to be valid, and the transmission request is determined to be a breakpoint transmission. When the download request is judged, whether the request link of the download request is a link needing to be processed is judged, and if the request link does not belong to a link processed by a Netty server or a Netty client, the download request is rejected and a request failure prompt message is fed back. Then find the transmission ID (download ID) of the download request file at the parameter of the get request and inquire in the database (MongoDB). If the same transmission ID and file transmission information exist in the database, which indicates that the downloading request is the breakpoint continuous transmission, the corresponding file transmission state in the database is modified to be 'downloading', and a corresponding temporary file needs to be found and written in subsequently according to the original transmission progress and the file information. If the same transmission ID and file transmission information do not exist in the database, which indicates that the downloading request is written for the first time, a new temporary file needs to be created subsequently for writing and the file transmission is recorded in the database.
In step S5, the Netty server starts the file to be downloaded in blocks, the Netty client decompresses and decodes the received blocked file data, writes the decompressed file data to a temporary file to perform file assembly, and writes the currently downloaded file information data in the database (MongoDB). In step S5, when the file is downloaded in blocks, the Netty client determines whether a temporary file corresponding to the block data exists after receiving each block data. And if the corresponding temporary file exists, directly writing the file block data into the temporary file. If the corresponding temporary file does not exist, which indicates that the file is downloaded for the first time and is the first file block data, a temporary file is created by using the transmission ID, the file name and the file size of the file as parameters for writing, and file transmission information is updated in a database and a memory. In step S5, when the file is downloaded in blocks, the transmission progress and the transmission speed are obtained according to the size of the downloaded file data and the transmission time, and then feedback is performed through the WebSocket long connection, so that the browser page can see the real-time transmission progress and speed.
In step S5, when the file is downloaded in blocks, if the browser page selects to suspend downloading or the downloading is interrupted due to a network, the Netty server monitors the downloading interruption and suspends the downloading, and meanwhile, modifies the transmission status of the file to suspend the downloading and records the transmission progress and the file information in the database. If the browser page end selects to continue downloading or automatically continues downloading the process after the network is recovered, the Netty server locally obtains the corresponding transmission progress and the file information from the Netty server according to the transmission ID of the file, then divides the file, discards the downloaded part to obtain the file data to be continuously transmitted, and then restarts the downloading process from step S1.
The file block data is continuously downloaded in step S5, and when it is transferred to the last block data, the flow proceeds to step S6. In step S6, the temporary file is renamed to the original name of the downloaded file, the file is landed, and the downloading success information is fed back through the WebSocket long connection and the file transmission status is updated to the database (MongoDB) as downloading success.
In addition, in the embodiment of the invention, the electronic equipment capable of realizing the large file transmission method is also provided.
As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or program product. Thus, various aspects of the invention may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" system.
The electronic device is in the form of a general purpose computing device. Components of the electronic device may include, but are not limited to: the system comprises at least one processing unit, at least one storage unit, a bus for connecting different system components (comprising the storage unit and the processing unit), and a display unit.
Wherein the storage unit stores program code which is executable by the processing unit to cause the processing unit to perform steps according to various exemplary embodiments of the present invention as described in the above section "exemplary methods" of the present description. For example, the processing unit may perform all the steps of the large file transfer method of the present invention.
The memory unit may include a readable medium in the form of a volatile memory unit, such as a random access memory unit (RAM) and/or a cache memory unit, and may further include a read only memory unit (ROM).
The storage unit may also include a program/utility having a set (at least one) of program modules including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.
The bus may be any of several types of bus structures including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.
The electronic device may also communicate with one or more external devices (e.g., keyboard, pointing device, bluetooth device, etc.), with one or more devices that enable a user to interact with the electronic device, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device to communicate with one or more other computing devices. Such communication may be through an input/output (I/O) interface. Also, the electronic device may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the internet) via a network adapter. As shown, the network adapter communicates with other modules of the electronic device over a bus. It should be appreciated that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which may be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.
In an exemplary embodiment of the present disclosure, there is also provided a computer-readable storage medium having stored thereon a program product capable of implementing the above-described large file transfer method of the present specification. In some possible embodiments, aspects of the invention may also be implemented in the form of a program product comprising program code means for causing a terminal device to carry out the steps according to various exemplary embodiments of the invention described in the above-mentioned "exemplary methods" section of the description, when the program product is run on the terminal device.
According to the program product for realizing the method, the portable compact disc read only memory (CD-ROM) can be adopted, the program code is included, and the program product can be operated on terminal equipment, such as a personal computer. However, the program product of the present invention is not limited in this regard and, in the present document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
A computer readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).
Furthermore, the above-described figures are merely schematic illustrations of processes involved in methods according to exemplary embodiments of the invention, and are not intended to be limiting. It will be readily understood that the processes shown in the above figures are not intended to indicate or limit the chronological order of the processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, e.g., in multiple modules.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is to be limited only by the following claims.
Claims (10)
1. A large file transfer method for transferring a file in a Netty framework, comprising the steps of:
acquiring transmission request information and data information of a file to be transmitted;
carrying out http coding and http compression on file data;
carrying out block transmission coding on the file data based on an http1.1 protocol;
judging the transmission request, judging whether the transmission request is legal or not, and judging whether the transmission request is breakpoint continuous transmission or not;
carrying out block transmission on the file, decompressing and decoding the received block file data, and then writing in a temporary file for file assembly;
and when the data is transmitted to the last block of data, renaming the temporary file, feeding back the transmission success information and updating the file transmission state.
2. The large file transmission method according to claim 1, wherein after the transmission request information and the data information of the file to be transmitted are acquired, a WebSocket long connection is also established for transmitting the transmission progress information of the file.
3. The large file transmission method according to claim 1, wherein when the transmission request is judged, the transmission ID of the file is queried in the database, and if the same transmission ID exists in the database, indicating that the corresponding file transmission is a breakpoint transmission, the corresponding file transmission state in the database is modified and the corresponding temporary file needs to be found for writing subsequently according to the original transmission schedule and the file information; if the same transmission ID does not exist in the database, the corresponding file transmission is indicated as the first writing, a new temporary file needs to be created subsequently for writing, and the file transmission is recorded in the database.
4. The large file transmission method according to claim 1, wherein when the file is transmitted in blocks, if the transmission process is interrupted, the transmission status of the file is modified to be suspended and the transmission progress and the file information are recorded in the database.
5. The large file transmission method according to claim 4, wherein when the file is subjected to breakpoint resume, the transmission progress and the file information are acquired, and then the file is divided to obtain the file data to be resumed, and the transmission progress is resumed.
6. The large file transmission method according to claim 2, wherein when the file is transmitted in blocks, a transmission progress and a transmission speed are obtained according to the size of the transmitted file data and the transmission time consumption, and feedback is performed through the WebSocket long connection.
7. The large file transmission method according to claim 2, wherein when the last block of data is transmitted, the temporary file is renamed to the original name of the file, the transmission success information is fed back through the WebSocket long connection, and the file transmission state is updated to the database.
8. The large file transmission method according to claim 3, wherein when the file is transferred in blocks and each file block data is received, it is first determined whether a temporary file corresponding to the file block data exists: if yes, directly writing the file block data into the temporary file; if the file transmission information does not exist, a temporary file is created by using the transmission ID, the file name and the file size of the file as parameters, and the file transmission information is updated to the database and the memory.
9. An electronic device, comprising:
a processor; and
a memory having computer readable instructions stored thereon which, when executed by the processor, implement the method of any of claims 1 to 8.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011293127.9A CN112583889A (en) | 2020-11-18 | 2020-11-18 | Large file transmission method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011293127.9A CN112583889A (en) | 2020-11-18 | 2020-11-18 | Large file transmission method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112583889A true CN112583889A (en) | 2021-03-30 |
Family
ID=75122735
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011293127.9A Pending CN112583889A (en) | 2020-11-18 | 2020-11-18 | Large file transmission method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112583889A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113612821A (en) * | 2021-07-14 | 2021-11-05 | 支付宝(杭州)信息技术有限公司 | Data interaction method and device in multi-party security computing |
| CN113934553A (en) * | 2021-10-14 | 2022-01-14 | 西安万像电子科技有限公司 | Shear plate redirection transmission method and device, storage medium and electronic equipment |
| CN115277594A (en) * | 2022-07-18 | 2022-11-01 | 浪潮软件股份有限公司 | Method and system for accelerating file uploading based on HTTP |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101174954A (en) * | 2006-10-31 | 2008-05-07 | 上海高勤通信科技有限公司 | Document breaking point uploading method based on internet hypertext transfer protocol |
| CN105635186A (en) * | 2016-03-28 | 2016-06-01 | 浪潮通用软件有限公司 | HTTP transmission protocol-based file reading and writing service downloading method |
| CN107566540A (en) * | 2017-10-31 | 2018-01-09 | 云宏信息科技股份有限公司 | Big document breakpoint transmission method, apparatus and system |
-
2020
- 2020-11-18 CN CN202011293127.9A patent/CN112583889A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101174954A (en) * | 2006-10-31 | 2008-05-07 | 上海高勤通信科技有限公司 | Document breaking point uploading method based on internet hypertext transfer protocol |
| CN105635186A (en) * | 2016-03-28 | 2016-06-01 | 浪潮通用软件有限公司 | HTTP transmission protocol-based file reading and writing service downloading method |
| CN107566540A (en) * | 2017-10-31 | 2018-01-09 | 云宏信息科技股份有限公司 | Big document breakpoint transmission method, apparatus and system |
Non-Patent Citations (5)
| Title |
|---|
| GITEE: "【源码学习】基于Netty的大文件断点续传开源解决方案", 《检索自互联网：<URL:HTTPS://M.TQWBA.COM/X_D/JISHU/242882.HTML>》 * |
| GITEE项目: "【源码学习】基于Netty的大文件断点续传开源解决方案", 《检索自互联网：<URL:HTTPS://ZHUANLAN.ZHIHU.COM/P/242319005>》 * |
| GJJ: "frontend-service代码", 《检索自互联网：<URL:HTTPS://GITEE.COM/GAOJUNJIE03/FILEEX/COMMIT/0D52F0FB82474B46FB63D8CEAED531AB3C05CEC3>》 * |
| GJJ: "netty-server代码", 《检索自互联网：<URL:HTTPS://GITEE.COM/GAOJUNJIE03/FILEEX/COMMIT/E037FABA8ECB313521666B8FCFF8B1A5F5153C6D>》 * |
| 鹿哥说: "socket分块接收_fileex: 一款基于netty、http1.1 transfer-encoding:chunked、webs", 《检索自互联网：<URL:HTTPS://BLOG.CSDN.NET/WEIXIN_36403165/ARTICLE/DETAILS/111965408>》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113612821A (en) * | 2021-07-14 | 2021-11-05 | 支付宝(杭州)信息技术有限公司 | Data interaction method and device in multi-party security computing |
| CN113934553A (en) * | 2021-10-14 | 2022-01-14 | 西安万像电子科技有限公司 | Shear plate redirection transmission method and device, storage medium and electronic equipment |
| CN115277594A (en) * | 2022-07-18 | 2022-11-01 | 浪潮软件股份有限公司 | Method and system for accelerating file uploading based on HTTP |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112583889A (en) | Large file transmission method and device | |
| CN109510754B (en) | Online document generation method, device and system and electronic equipment | |
| US20110167173A1 (en) | Optimal Compression Process Selection Methods | |
| US10965732B2 (en) | Streaming zip | |
| WO2021052058A1 (en) | Video processing method and apparatus, electronic device and storage medium | |
| US20140250158A1 (en) | Method and device for obtaining file | |
| CN102098330A (en) | Method, device and system for asynchronous transmission based on json data format | |
| CN103532984A (en) | Data transmission method, device and system of websocket protocol | |
| CN108874825B (en) | Abnormal data verification method and device | |
| US10701150B2 (en) | Method and apparatus for storing network data | |
| CN112596920A (en) | Message processing method and device, electronic equipment and storage medium | |
| CN110753020A (en) | Network request processing method and device | |
| CN105491132A (en) | File server, terminal and file subpackage transmission method | |
| CN104123244A (en) | USB redirection system and method | |
| CN112583743B (en) | Distributed file exchange method and device | |
| CN110677443A (en) | Data transmitting and receiving method, transmitting end, receiving end, system and storage medium | |
| CN110912948A (en) | Method and device for reporting problems | |
| CN111818500A (en) | Method and device for establishing Bluetooth connection between mobile terminal and motor vehicle | |
| CN116684703A (en) | Streaming media data transmission method and related equipment based on proximity service communication protocol | |
| AU2018403361B2 (en) | Data transmission | |
| CN112149392A (en) | Rich text editing method and device | |
| CN105159638A (en) | Data transmission method, data transmission system and portable display apparatus | |
| CN102710673A (en) | Method and device for downloading data by terminal | |
| CN114201234A (en) | File processing method, device, system, equipment and storage medium | |
| CN116437377A (en) | Communication method, system, electronic device and medium for device control communication protocol |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20210330 |
|
| WD01 | Invention patent application deemed withdrawn after publication |