CN113766006B - Real-time file transmission method and system based on FTP protocol - Google Patents
Real-time file transmission method and system based on FTP protocol Download PDFInfo
- Publication number
- CN113766006B CN113766006B CN202110864371.4A CN202110864371A CN113766006B CN 113766006 B CN113766006 B CN 113766006B CN 202110864371 A CN202110864371 A CN 202110864371A CN 113766006 B CN113766006 B CN 113766006B
- Authority
- CN
- China
- Prior art keywords
- file
- directory
- attribute information
- shared queue
- real
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000005540 biological transmission Effects 0.000 title claims abstract description 48
- 238000000034 method Methods 0.000 title claims abstract description 31
- 238000004806 packaging method and process Methods 0.000 claims abstract description 13
- 238000012544 monitoring process Methods 0.000 claims abstract description 11
- 238000012546 transfer Methods 0.000 claims description 5
- 230000002159 abnormal effect Effects 0.000 claims description 4
- 230000005856 abnormality Effects 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention provides a real-time file transmission method and a real-time file transmission system based on an FTP protocol, which relate to the technical field of file transmission and comprise the following steps: starting a multithreading connection FTP server, which comprises a main thread and a plurality of sub threads; the main thread monitors the file directory in real time and inquires the files under the file directory according to the need; judging whether the inquired file is transmitted or not; if the judgment result is negative, acquiring attribute information of the file, packaging the attribute information into an object, and putting the object into a shared queue; and the thread-dividing obtains the object from the shared queue and executes the transmission task. According to the invention, the FTP server is connected through multiple threads, one thread is used as a main thread, and the real-time monitoring of the directory files and the local directory files of the FTP server is realized; and further, the file is acquired to the shared queue through the main thread, and the file is rapidly uploaded and downloaded between the directory file of the FTP server and the local directory file in real time in a mode of being matched with a multithreaded file transmission mode.
Description
Technical Field
The invention relates to the technical field of file transmission, in particular to a real-time file transmission method and system based on an FTP protocol.
Background
In the business handling process of the telecommunication industry, files are required to be transmitted between a server and a local area in real time; therefore, it is necessary to search the files in the file directory in real time, and transmit the files to the server or local after searching the files meeting the conditions.
The file transmission efficiency of the current service handling system is low, and the lower file transmission efficiency cannot meet the service handling requirement along with the increase of the service, so that the problems of backlog, time delay and the like of the service are easily caused.
Disclosure of Invention
In order to solve the problems, the invention provides a real-time file transmission method and a real-time file transmission system based on an FTP protocol, which realize real-time file transmission of a specified directory by connecting an FTP server in a multithreading manner.
In order to achieve the above object, the present invention provides an emergency multitasking short message group sending system, comprising:
starting a multithreading connection FTP server, which comprises a main thread and a plurality of sub threads;
the main thread monitors a file directory in real time, and queries files under the file directory according to the need;
judging whether the inquired file is transmitted or not;
if the judgment result is negative, acquiring attribute information of the file, packaging the attribute information into an object, and putting the object into a shared queue;
and the branching process acquires the object from the shared queue and executes a transmission task.
As a further improvement of the invention, the main thread monitors the file directory in real time and inquires the files under the file directory according to the need; comprising the following steps:
the main thread monitors the file catalogue and the local file catalogue of the FTP server in real time;
inquiring a file directory of the FTP server according to the downloading requirement;
and inquiring the local file catalogue according to the uploading requirement.
As a further improvement of the present invention, the attribute information of the file includes: source file directory, source file name, destination server, and destination directory.
As a further improvement of the invention, the thread obtains the object from the shared queue and executes a transmission task; comprising the following steps:
each branching process respectively acquires different objects from the shared queue;
analyzing the attribute information encapsulated in the object;
and transmitting the file from the source end to the target end according to the attribute information.
As a further improvement of the invention, if an abnormality occurs in the process of executing the transmission task by the branching program, the object is replaced in the shared queue.
The invention also provides a real-time file transmission system based on the FTP protocol, which comprises: the system comprises a thread connection module, a monitoring module, a transmission condition judging module, a packaging module and a file transmission module;
the thread connection module is used for:
starting a multithreading connection FTP server, which comprises a main thread and a plurality of sub threads;
the monitoring module is used for:
the main thread monitors a file directory in real time, and queries files under the file directory according to the need;
the transmission condition judging module is used for:
judging whether the inquired file is transmitted or not;
the packaging module is used for:
if the judgment result is negative, acquiring attribute information of the file, packaging the attribute information into an object, and putting the object into a shared queue;
the file transmission module is used for:
and the branching process acquires the object from the shared queue and executes a transmission task.
As a further improvement of the invention, the main thread monitors the file directory in real time and inquires the files under the file directory according to the need; comprising the following steps:
the main thread monitors the file catalogue and the local file catalogue of the FTP server in real time;
inquiring a file directory of the FTP server according to the downloading requirement;
and inquiring the local file catalogue according to the uploading requirement.
As a further improvement of the present invention, the attribute information of the file includes: source file directory, source file name, destination server, and destination directory.
As a further improvement of the invention, the thread obtains the object from the shared queue and executes a transmission task; comprising the following steps:
each branching process respectively acquires different objects from the shared queue;
analyzing the attribute information encapsulated in the object;
and transmitting the file from the source end to the target end according to the attribute information.
As a further improvement of the invention, if an abnormality occurs in the process of executing the transmission task by the branching program, the object is replaced in the shared queue.
Compared with the prior art, the invention has the beneficial effects that:
according to the invention, the FTP server is connected through multiple threads, one thread is used as a main thread, and the real-time monitoring of the directory files and the local directory files of the FTP server is realized; and further, the file is acquired to the shared queue through the main thread, and the file is rapidly uploaded and downloaded between the directory file of the FTP server and the local directory file in real time in a mode of being matched with a multithreaded file transmission mode.
According to the scheme, an operation and maintenance personnel can automatically set specific uploading and downloading tasks, specific FTP servers and the number of threads connected with the FTP, so that a program is started, and the uploading and downloading tasks set by the main thread branch office scan local directory files and directory files of the FTP servers respectively, so that uploading and downloading file transmission work is completed.
Drawings
FIG. 1 is a flow chart of a method for transmitting real-time files based on an FTP protocol according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a real-time file transfer method system based on an FTP protocol according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The invention is described in further detail below with reference to the attached drawing figures:
examples:
as shown in fig. 1, the method for transmitting real-time files based on FTP protocol provided by the present invention includes:
s1, starting a multithreading connection FTP server, wherein the FTP server comprises a main thread and a plurality of sub threads;
s2, the main thread monitors the file directory in real time, and queries the files under the file directory according to the requirements;
wherein, include:
the main thread monitors the file catalogue and the local file catalogue of the FTP server in real time;
inquiring a file directory of the FTP server according to the downloading requirement;
and inquiring the local file directory according to the uploading requirement.
S3, judging whether the inquired file is transmitted or not;
s4, if the judgment result is negative, acquiring attribute information of the file, packaging the attribute information as an object, and putting the object into a shared queue;
wherein,
the attribute information of the file includes: source file directory, source file name, destination server, and destination directory.
S5, the object is obtained from the shared queue by the threads, and the transmission task is executed.
Wherein, include:
each branch thread respectively acquires different objects from the shared queue;
analyzing the attribute information encapsulated in the object;
and transmitting the file from the source end to the target end according to the attribute information.
Further, the method comprises the steps of,
and transmitting the file from the source file directory to the target directory according to the source file directory and the target directory in the attribute information.
In the invention, if the thread is abnormal in the process of executing the transmission task, the object is replaced in the shared queue.
As shown in fig. 2, the present invention further provides a real-time file transfer system based on FTP protocol, including: the system comprises a thread connection module, a monitoring module, a transmission condition judging module, a packaging module and a file transmission module;
a thread connection module for:
starting a multithreading connection FTP server, which comprises a main thread and a plurality of sub threads;
the monitoring module is used for:
the main thread monitors the file directory in real time and inquires the files under the file directory according to the need;
wherein, include:
the main thread monitors the file catalogue and the local file catalogue of the FTP server in real time;
inquiring a file directory of the FTP server according to the downloading requirement;
and inquiring the local file directory according to the uploading requirement.
The transmission condition judging module is used for:
judging whether the inquired file is transmitted or not;
and a packaging module for:
if the judgment result is negative, acquiring attribute information of the file, packaging the attribute information into an object, and putting the object into a shared queue;
wherein,
the attribute information of the file includes: source file directory, source file name, destination server, and destination directory.
A file transmission module, configured to:
and the thread-dividing obtains the object from the shared queue and executes the transmission task.
Wherein, include:
each branch thread respectively acquires different objects from the shared queue;
analyzing the attribute information encapsulated in the object;
and transmitting the file from the source end to the target end according to the attribute information.
Further, according to the source file directory and the target directory in the attribute information, the file is transferred from the source file directory to the target directory.
In the invention, if the thread is abnormal in the process of executing the transmission task, the object is replaced in the shared queue.
The invention has the advantages that:
(1) The method comprises the steps of connecting an FTP server through multiple threads, taking one thread as a main thread, and realizing real-time monitoring of directory files and local directory files of the FTP server; and further, the file is acquired to the shared queue through the main thread, and the file is rapidly uploaded and downloaded between the directory file of the FTP server and the local directory file in real time in a mode of being matched with a multithreaded file transmission mode.
(2) The operation and maintenance personnel can automatically set specific uploading and downloading tasks, specific FTP servers and the number of threads connected with the FTP, the program automatically initializes the connection of the FTP servers according to the setting information, then the program is started, and the uploading and downloading tasks set by the main thread branch office respectively scan local directory files and directory files of the FTP servers to finish uploading and downloading file transmission work.
(3) The thread real-time checks whether the shared queue has an object, if so, the object is taken out from the shared queue, and the file transmission task is executed according to the information in the object until the shared queue is empty
(4) The main thread regularly checks whether the task needs to be stopped, restarted and the like, so that the front end is convenient for controlling and managing the task.
(5) Monitoring the directory file of the FTP server in real time, and downloading the file from the FTP server to a local directory when the file meeting the downloading requirement exists under the directory file; and monitoring the local directory file in real time, and uploading the file to a specified directory of a specified FTP server, which can be a plurality of FTP servers, when the file meeting the uploading condition exists under the directory file.
The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (8)
1. A method for transmitting real-time files based on FTP protocol, comprising:
starting a multithreading connection FTP server, which comprises a main thread and a plurality of sub threads;
the main thread monitors a file directory in real time, and queries files under the file directory according to the need; the main thread monitors the file catalogue and the local file catalogue of the FTP server in real time; inquiring a file directory of the FTP server according to the downloading requirement; inquiring the local file catalogue according to uploading requirements;
judging whether the inquired file is transmitted or not;
if the judgment result is negative, acquiring attribute information of the file, packaging the attribute information into an object, and putting the object into a shared queue;
and the branching process acquires the object from the shared queue and executes a transmission task.
2. The real-time file transfer method according to claim 1, wherein: the attribute information of the file includes: source file directory, source file name, destination server, and destination directory.
3. The method according to claim 1, wherein the thread obtains the object from the shared queue and performs a transmission task; comprising the following steps:
each branching process respectively acquires different objects from the shared queue;
analyzing the attribute information encapsulated in the object;
and transmitting the file from the source end to the target end according to the attribute information.
4. The real-time file transfer method according to claim 1, wherein: and if the branching program is abnormal in the process of executing the transmission task, the object is replaced in the shared queue.
5. A system for implementing the real-time file transfer method of any one of claims 1 to 4, comprising: the system comprises a thread connection module, a monitoring module, a transmission condition judging module, a packaging module and a file transmission module;
the thread connection module is used for:
starting a multithreading connection FTP server, which comprises a main thread and a plurality of sub threads;
the monitoring module is used for:
the main thread monitors a file directory in real time, and queries files under the file directory according to the need; the main thread monitors the file catalogue and the local file catalogue of the FTP server in real time;
inquiring a file directory of the FTP server according to the downloading requirement;
inquiring the local file catalogue according to uploading requirements;
the transmission condition judging module is used for:
judging whether the inquired file is transmitted or not;
the packaging module is used for:
if the judgment result is negative, acquiring attribute information of the file, packaging the attribute information into an object, and putting the object into a shared queue;
the file transmission module is used for:
and the branching process acquires the object from the shared queue and executes a transmission task.
6. The system according to claim 5, wherein: the attribute information of the file includes: source file directory, source file name, destination server, and destination directory.
7. The system according to claim 5, wherein: the branching process acquires the object from the shared queue and executes a transmission task; comprising the following steps:
each branching process respectively acquires different objects from the shared queue;
analyzing the attribute information encapsulated in the object;
and transmitting the file from the source end to the target end according to the attribute information.
8. The system according to claim 5, wherein: and if the branching program is abnormal in the process of executing the transmission task, the object is replaced in the shared queue.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110864371.4A CN113766006B (en) | 2021-07-29 | 2021-07-29 | Real-time file transmission method and system based on FTP protocol |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110864371.4A CN113766006B (en) | 2021-07-29 | 2021-07-29 | Real-time file transmission method and system based on FTP protocol |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113766006A CN113766006A (en) | 2021-12-07 |
| CN113766006B true CN113766006B (en) | 2024-02-23 |
Family
ID=78788250
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110864371.4A Active CN113766006B (en) | 2021-07-29 | 2021-07-29 | Real-time file transmission method and system based on FTP protocol |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113766006B (en) |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20020019170A (en) * | 2000-09-05 | 2002-03-12 | 신동호 | Simultaneous multiple file downloads uploads in Internet's FTP |
| CN1863050A (en) * | 2005-09-15 | 2006-11-15 | 上海华为技术有限公司 | Method of document synchronization between server and system thereof |
| CN103605568A (en) * | 2013-10-29 | 2014-02-26 | 北京奇虎科技有限公司 | Multithread management method and device |
| CN104750811A (en) * | 2015-03-30 | 2015-07-01 | 浪潮通信信息系统有限公司 | Multithreading real-time acquisition method for mobile communication data file |
| CN106372526A (en) * | 2016-08-19 | 2017-02-01 | 上海爱数信息技术股份有限公司 | File change monitoring and synchronization method and system |
| CN106911811A (en) * | 2017-05-04 | 2017-06-30 | 郑州云海信息技术有限公司 | A kind of method based on ftp file high efficiency of transmission |
| CN108156199A (en) * | 2016-12-05 | 2018-06-12 | 上海宝信软件股份有限公司 | The system and method for user-defined file transport protocol is realized based on plug-in type module |
| CN108198367A (en) * | 2018-01-16 | 2018-06-22 | 西门子工厂自动化工程有限公司 | Data acquire and monitoring control method, system, device and computer storage media |
| CN109766131A (en) * | 2017-11-06 | 2019-05-17 | 上海宝信软件股份有限公司 | The system and method for the intelligent automatic upgrading of software is realized based on multithreading |
| CN110198334A (en) * | 2018-04-19 | 2019-09-03 | 腾讯科技（深圳）有限公司 | Access method, device and storage medium based on object storage service |
| CN110401724A (en) * | 2019-08-22 | 2019-11-01 | 北京旷视科技有限公司 | File management method, ftp server and storage medium |
| CN110784528A (en) * | 2019-10-22 | 2020-02-11 | 北京天融信网络安全技术有限公司 | File downloading method and device and storage medium |
| CN111970355A (en) * | 2020-08-14 | 2020-11-20 | 苏州浪潮智能科技有限公司 | Method for automatically downloading numbered files |
| CN112684982A (en) * | 2020-12-25 | 2021-04-20 | 北京浪潮数据技术有限公司 | Data migration method, system, equipment and computer readable storage medium |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9203892B2 (en) * | 2011-04-19 | 2015-12-01 | Accenture Global Services Limited | Content transfer accelerator |
| US10686867B2 (en) * | 2017-05-12 | 2020-06-16 | Guavus, Inc. | Scaled in-order record input ingestion for file-based streams in multi-threaded environments |
| US10846180B2 (en) * | 2017-09-14 | 2020-11-24 | Commvault Systems, Inc. | Distributed framework for task splitting and task assignments in a content indexing system |
-
2021
- 2021-07-29 CN CN202110864371.4A patent/CN113766006B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20020019170A (en) * | 2000-09-05 | 2002-03-12 | 신동호 | Simultaneous multiple file downloads uploads in Internet's FTP |
| CN1863050A (en) * | 2005-09-15 | 2006-11-15 | 上海华为技术有限公司 | Method of document synchronization between server and system thereof |
| CN103605568A (en) * | 2013-10-29 | 2014-02-26 | 北京奇虎科技有限公司 | Multithread management method and device |
| CN104750811A (en) * | 2015-03-30 | 2015-07-01 | 浪潮通信信息系统有限公司 | Multithreading real-time acquisition method for mobile communication data file |
| CN106372526A (en) * | 2016-08-19 | 2017-02-01 | 上海爱数信息技术股份有限公司 | File change monitoring and synchronization method and system |
| CN108156199A (en) * | 2016-12-05 | 2018-06-12 | 上海宝信软件股份有限公司 | The system and method for user-defined file transport protocol is realized based on plug-in type module |
| CN106911811A (en) * | 2017-05-04 | 2017-06-30 | 郑州云海信息技术有限公司 | A kind of method based on ftp file high efficiency of transmission |
| CN109766131A (en) * | 2017-11-06 | 2019-05-17 | 上海宝信软件股份有限公司 | The system and method for the intelligent automatic upgrading of software is realized based on multithreading |
| CN108198367A (en) * | 2018-01-16 | 2018-06-22 | 西门子工厂自动化工程有限公司 | Data acquire and monitoring control method, system, device and computer storage media |
| CN110198334A (en) * | 2018-04-19 | 2019-09-03 | 腾讯科技（深圳）有限公司 | Access method, device and storage medium based on object storage service |
| CN110401724A (en) * | 2019-08-22 | 2019-11-01 | 北京旷视科技有限公司 | File management method, ftp server and storage medium |
| CN110784528A (en) * | 2019-10-22 | 2020-02-11 | 北京天融信网络安全技术有限公司 | File downloading method and device and storage medium |
| CN111970355A (en) * | 2020-08-14 | 2020-11-20 | 苏州浪潮智能科技有限公司 | Method for automatically downloading numbered files |
| CN112684982A (en) * | 2020-12-25 | 2021-04-20 | 北京浪潮数据技术有限公司 | Data migration method, system, equipment and computer readable storage medium |
Non-Patent Citations (3)
| Title |
|---|
| FTP upload based on multithreaded and broken transfer resume technology;Yimin Wu et al.;International conference on computer, communications and information technology;20141231;全文 * |
| FTP服务器枢纽中的文件处理引擎设计;聂岩峰;彭向军;徐先超;;计算机工程;20101120(第22期);全文 * |
| 支持共享的Ftp客户端的设计;林耿仕;赵建华;郑国梁;;计算机科学;20071125(第11期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113766006A (en) | 2021-12-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11774952B2 (en) | Management apparatus and management method | |
| CN106850830B (en) | Service request processing method, device and system and related server | |
| CN107968797B (en) | Video transmission method, device and system | |
| CN103150324A (en) | Chained processing-based data collecting system and method | |
| CN113766006B (en) | Real-time file transmission method and system based on FTP protocol | |
| CN110875887A (en) | MQTT protocol-based communication interaction method and communication interaction system | |
| CN112241401A (en) | Knowledge graph-based digital quality management system and method | |
| CN106354870B (en) | A kind of method and apparatus of data load | |
| CN111222350B (en) | System and method for associating optical fibers with equipment ports | |
| CN104618879A (en) | Distributed quick short message sending system | |
| CN109765833A (en) | A method of monitoring Internet of Things equipment state of a control | |
| US9860292B2 (en) | Method and system for delivering data to a batch consumer and a streaming consumer | |
| CN105447000A (en) | Transactional message transmission method and device | |
| CN111488997A (en) | Automatic operation and maintenance method, device, equipment and storage medium | |
| CN107844954A (en) | A kind of artificial intelligence system method of automatic factory | |
| CN113419745A (en) | Application instance number management method and device, electronic equipment and computer readable medium | |
| CN104915246A (en) | High-configurable distributed real-time calculation engine based on workflow and control method | |
| CN113934187A (en) | Remote control system, method and device for annealing tinning machine and storage medium | |
| CN111866118A (en) | Working platform file storage and transmission method and system | |
| CN115705259A (en) | Fault processing method, related device and storage medium | |
| CN107370612B (en) | Network quality management system detection task scheduling method and device | |
| CN110262882A (en) | A kind of distributed communication command scheduling system and method | |
| CN104833882A (en) | Distributed remote automatic frequency converter fault diagnosis method | |
| CN111665749A (en) | Remote supervision and management system | |
| CN112965844A (en) | CPU surge accident processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |