CN113766006A - Real-time file transmission method and system based on FTP (file transfer protocol) - Google Patents
Real-time file transmission method and system based on FTP (file transfer protocol) Download PDFInfo
- Publication number
- CN113766006A CN113766006A CN202110864371.4A CN202110864371A CN113766006A CN 113766006 A CN113766006 A CN 113766006A CN 202110864371 A CN202110864371 A CN 202110864371A CN 113766006 A CN113766006 A CN 113766006A
- Authority
- CN
- China
- Prior art keywords
- file
- directory
- attribute information
- thread
- real
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000005540 biological transmission Effects 0.000 title claims abstract description 50
- 238000000034 method Methods 0.000 title claims abstract description 32
- 238000012546 transfer Methods 0.000 title claims description 10
- 238000004806 packaging method and process Methods 0.000 claims abstract description 10
- 238000012544 monitoring process Methods 0.000 claims description 9
- 238000005538 encapsulation Methods 0.000 claims description 3
- 230000002159 abnormal effect Effects 0.000 claims 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 239000000126 substance Substances 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
Abstract
The invention provides a real-time file transmission method and a system based on an FTP protocol, which relate to the technical field of file transmission and comprise the following steps: starting a multithreading connection FTP server, which comprises a main thread and a plurality of sub threads; the main thread monitors a file directory in real time and inquires files under the file directory as required; judging whether the inquired file is transmitted or not; if the judgment result is negative, acquiring the attribute information of the file, packaging the attribute information as an object, and putting the object into a sharing queue; and the sub-threads acquire the object from the shared queue and execute the transmission task. The FTP server is connected through multiple threads, one thread is used as a main thread, and directory files of the FTP server and local directory files are monitored in real time; and then, files are acquired to the sharing queue through the main thread, and the files are uploaded and downloaded between the directory files of the FTP server and the local directory files in real time and rapidly in a multithread file transmission mode.
Description
Technical Field
The invention relates to the technical field of file transmission, in particular to a real-time file transmission method and a real-time file transmission system based on an FTP protocol.
Background
In the process of business handling in the telecommunication industry, files need to be transmitted between a server and the local in real time; therefore, files in the file directory need to be searched in real time, and after files meeting the conditions are searched, the files are transmitted to the server or the local.
The file transmission efficiency of the current service handling system is low, and the low file transmission efficiency cannot meet the service handling requirements along with the increase of services, so that the problems of service backlog, delay and the like are easily caused.
Disclosure of Invention
Aiming at the problems, the invention provides a real-time file transmission method and a real-time file transmission system based on an FTP protocol, which realize the real-time file transmission of a designated directory by connecting an FTP server in a multithreading way.
In order to achieve the above object, the present invention provides an emergency multitasking short message group sending system, which comprises:
starting a multithreading connection FTP server, which comprises a main thread and a plurality of sub threads;
the main thread monitors a file directory in real time and inquires files under the file directory as required;
judging whether the inquired file is transmitted or not;
if the judgment result is negative, acquiring the attribute information of the file, packaging the attribute information as an object, and putting the object into a sharing queue;
and the sub-thread acquires the object from the shared queue and executes a transmission task.
As a further improvement of the present invention, the main thread monitors the file directory in real time, and queries the files in the file directory as required; the method comprises the following steps:
the main thread monitors a file directory and a local file directory of the FTP server in real time;
inquiring a file directory of the FTP server according to downloading requirements;
and inquiring the local file directory according to uploading requirements.
As a further improvement of the present invention, the attribute information of the file includes: a source file directory, a source file name, a target server, and a target directory.
As a further improvement of the present invention, the sub-thread acquires the object from the shared queue and executes a transmission task; the method comprises the following steps:
each branch thread acquires different objects from the shared queue respectively;
analyzing the attribute information encapsulated in the object;
and transmitting the file from the source end to the target end according to the attribute information.
As a further improvement of the invention, if an exception occurs in the sub-thread executing transmission task process, the object is placed back into the shared queue again.
The invention also provides a system for real-time file transmission based on the FTP protocol, which comprises: the device comprises a thread connection module, a monitoring module, a transmission condition judgment module, a packaging module and a file transmission module;
the thread connection module is configured to:
starting a multithreading connection FTP server, which comprises a main thread and a plurality of sub threads;
the monitoring module is used for:
the main thread monitors a file directory in real time and inquires files under the file directory as required;
the transmission condition judgment module is used for:
judging whether the inquired file is transmitted or not;
the encapsulation module is used for:
if the judgment result is negative, acquiring the attribute information of the file, packaging the attribute information as an object, and putting the object into a sharing queue;
the file transmission module is used for:
and the sub-thread acquires the object from the shared queue and executes a transmission task.
As a further improvement of the present invention, the main thread monitors the file directory in real time, and queries the files in the file directory as required; the method comprises the following steps:
the main thread monitors a file directory and a local file directory of the FTP server in real time;
inquiring a file directory of the FTP server according to downloading requirements;
and inquiring the local file directory according to uploading requirements.
As a further improvement of the present invention, the attribute information of the file includes: a source file directory, a source file name, a target server, and a target directory.
As a further improvement of the present invention, the sub-thread acquires the object from the shared queue and executes a transmission task; the method comprises the following steps:
each branch thread acquires different objects from the shared queue respectively;
analyzing the attribute information encapsulated in the object;
and transmitting the file from the source end to the target end according to the attribute information.
As a further improvement of the invention, if an exception occurs in the sub-thread executing transmission task process, the object is placed back into the shared queue again.
Compared with the prior art, the invention has the beneficial effects that:
the FTP server is connected through multiple threads, one thread is used as a main thread, and directory files of the FTP server and local directory files are monitored in real time; and then, files are acquired to the sharing queue through the main thread, and the files are uploaded and downloaded between the directory files of the FTP server and the local directory files in real time and rapidly in a multithread file transmission mode.
In the scheme of the invention, operation and maintenance personnel can automatically set the specific uploading and downloading tasks, the specific FTP server and the thread number for connecting the FTP, further start a program, and respectively scan the local directory files and the directory files of the FTP server by the uploading and downloading tasks set by the main thread branch office to finish the file transmission work of uploading and downloading.
Drawings
FIG. 1 is a flowchart illustrating a real-time file transfer method based on FTP protocol according to an embodiment of the present invention;
fig. 2 is a system diagram of a real-time file transfer method based on the FTP protocol according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.
The invention is described in further detail below with reference to the attached drawing figures:
example (b):
as shown in fig. 1, a real-time file transfer method based on the FTP protocol provided by the present invention includes:
s1, starting a multithreading connection FTP server, which comprises a main thread and a plurality of sub threads;
s2, the main thread monitors the file directory in real time and inquires files under the file directory as required;
wherein, include:
the main thread monitors a file directory and a local file directory of the FTP server in real time;
inquiring a file directory of the FTP server according to downloading requirements;
and inquiring the local file directory according to uploading requirements.
S3, judging whether the inquired file is transmitted or not;
s4, if the judgment result is negative, acquiring the attribute information of the file, packaging the attribute information as an object, and putting the object into a sharing queue;
wherein the content of the first and second substances,
the attribute information of the file includes: a source file directory, a source file name, a target server, and a target directory.
And S5, the sub-thread acquires the object from the shared queue and executes the transmission task.
Wherein, include:
each branch process respectively acquires different objects from the shared queue;
analyzing the encapsulated attribute information in the object;
and transmitting the file from the source end to the target end according to the attribute information.
Further, in the above-mentioned case,
and transmitting the file from the source file directory to the target directory according to the source file directory and the target directory in the attribute information.
In the invention, if an exception occurs in the process of executing the transmission task by the branch thread, the object is placed back to the shared queue again.
As shown in fig. 2, the present invention further provides a real-time file transfer system based on the FTP protocol, including: the device comprises a thread connection module, a monitoring module, a transmission condition judgment module, a packaging module and a file transmission module;
a thread connection module to:
starting a multithreading connection FTP server, which comprises a main thread and a plurality of sub threads;
a monitoring module to:
the main thread monitors a file directory in real time and inquires files under the file directory as required;
wherein, include:
the main thread monitors a file directory and a local file directory of the FTP server in real time;
inquiring a file directory of the FTP server according to downloading requirements;
and inquiring the local file directory according to uploading requirements.
A transmission condition judgment module for:
judging whether the inquired file is transmitted or not;
an encapsulation module to:
if the judgment result is negative, acquiring the attribute information of the file, packaging the attribute information as an object, and putting the object into a sharing queue;
wherein the content of the first and second substances,
the attribute information of the file includes: a source file directory, a source file name, a target server, and a target directory.
A file transfer module to:
and the sub-threads acquire the object from the shared queue and execute the transmission task.
Wherein, include:
each branch process respectively acquires different objects from the shared queue;
analyzing the encapsulated attribute information in the object;
and transmitting the file from the source end to the target end according to the attribute information.
Further, the file is transmitted from the source file directory to the target directory according to the source file directory and the target directory in the attribute information.
In the invention, if an exception occurs in the process of executing the transmission task by the branch thread, the object is placed back to the shared queue again.
The invention has the advantages that:
(1) connecting the FTP servers through multiple threads, and using one thread as a main thread to realize real-time monitoring of directory files of the FTP servers and local directory files; and then, files are acquired to the sharing queue through the main thread, and the files are uploaded and downloaded between the directory files of the FTP server and the local directory files in real time and rapidly in a multithread file transmission mode.
(2) The operation and maintenance personnel can automatically set specific uploading and downloading tasks, specific FTP servers and the number of threads for connecting the FTP, the program automatically initializes the connection of the FTP servers according to the set information, and then starts the program, and the uploading and downloading tasks set by the main thread branch office respectively scan local directory files and directory files of the FTP servers to finish the file transmission work of uploading and downloading.
(3) The branch process checks whether the shared queue has an object in real time, if so, the object is taken out from the shared queue, and the file transmission task is executed according to the information in the object until the shared queue is empty
(4) The main thread regularly checks whether the task needs to be stopped, restarted and the like, so that the front end can conveniently control and manage the task.
(5) Monitoring a directory file of the FTP server in real time, and downloading the file to a local directory from the FTP server when the file meeting the downloading requirement exists under the directory file; and monitoring local directory files in real time, and uploading the files to a designated directory of a designated FTP server when the files meet uploading conditions, wherein the files can be a plurality of FTP servers.
The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes will occur to those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A real-time file transmission method based on FTP protocol is characterized by comprising the following steps:
starting a multithreading connection FTP server, which comprises a main thread and a plurality of sub threads;
the main thread monitors a file directory in real time and inquires files under the file directory as required;
judging whether the inquired file is transmitted or not;
if the judgment result is negative, acquiring the attribute information of the file, packaging the attribute information as an object, and putting the object into a sharing queue;
and the sub-thread acquires the object from the shared queue and executes a transmission task.
2. The real-time file transfer method according to claim 1, wherein: the main thread monitors a file directory in real time and inquires files under the file directory as required; the method comprises the following steps:
the main thread monitors a file directory and a local file directory of the FTP server in real time;
inquiring a file directory of the FTP server according to downloading requirements;
and inquiring the local file directory according to uploading requirements.
3. The real-time file transfer method according to claim 1, wherein: the attribute information of the file includes: a source file directory, a source file name, a target server, and a target directory.
4. The real-time file transmission method according to claim 1, wherein the sub-thread acquires the object from the shared queue and executes a transmission task; the method comprises the following steps:
each branch thread acquires different objects from the shared queue respectively;
analyzing the attribute information encapsulated in the object;
and transmitting the file from the source end to the target end according to the attribute information.
5. The real-time file transfer method according to claim 1, wherein: and if the sub-thread is abnormal in the process of executing the transmission task, the object is placed back into the shared queue again.
6. A system for implementing the real-time file transmission method according to any one of claims 1 to 5, comprising: the device comprises a thread connection module, a monitoring module, a transmission condition judgment module, a packaging module and a file transmission module;
the thread connection module is configured to:
starting a multithreading connection FTP server, which comprises a main thread and a plurality of sub threads;
the monitoring module is used for:
the main thread monitors a file directory in real time and inquires files under the file directory as required;
the transmission condition judgment module is used for:
judging whether the inquired file is transmitted or not;
the encapsulation module is used for:
if the judgment result is negative, acquiring the attribute information of the file, packaging the attribute information as an object, and putting the object into a sharing queue;
the file transmission module is used for:
and the sub-thread acquires the object from the shared queue and executes a transmission task.
7. The system of claim 6, wherein: the main thread monitors a file directory in real time and inquires files under the file directory as required; the method comprises the following steps:
the main thread monitors a file directory and a local file directory of the FTP server in real time;
inquiring a file directory of the FTP server according to downloading requirements;
and inquiring the local file directory according to uploading requirements.
8. The system of claim 6, wherein: the attribute information of the file includes: a source file directory, a source file name, a target server, and a target directory.
9. The system of claim 6, wherein: the sub-thread acquires the object from the shared queue and executes a transmission task; the method comprises the following steps:
each branch thread acquires different objects from the shared queue respectively;
analyzing the attribute information encapsulated in the object;
and transmitting the file from the source end to the target end according to the attribute information.
10. The system of claim 6, wherein: and if the sub-thread is abnormal in the process of executing the transmission task, the object is placed back into the shared queue again.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110864371.4A CN113766006B (en) | 2021-07-29 | 2021-07-29 | Real-time file transmission method and system based on FTP protocol |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110864371.4A CN113766006B (en) | 2021-07-29 | 2021-07-29 | Real-time file transmission method and system based on FTP protocol |
Publications (2)
Publication Number | Publication Date |
---|---|
CN113766006A true CN113766006A (en) | 2021-12-07 |
CN113766006B CN113766006B (en) | 2024-02-23 |
Family
ID=78788250
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202110864371.4A Active CN113766006B (en) | 2021-07-29 | 2021-07-29 | Real-time file transmission method and system based on FTP protocol |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN113766006B (en) |
Citations (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR20020019170A (en) * | 2000-09-05 | 2002-03-12 | 신동호 | Simultaneous multiple file downloads uploads in Internet's FTP |
CN1863050A (en) * | 2005-09-15 | 2006-11-15 | 上海华为技术有限公司 | Method of document synchronization between server and system thereof |
US20120271880A1 (en) * | 2011-04-19 | 2012-10-25 | Accenture Global Services Limited | Content transfer accelerator |
CN103605568A (en) * | 2013-10-29 | 2014-02-26 | 北京奇虎科技有限公司 | Multithread management method and device |
CN104750811A (en) * | 2015-03-30 | 2015-07-01 | 浪潮通信信息系统有限公司 | Mobile communication data file multithread real-time collection method |
CN106372526A (en) * | 2016-08-19 | 2017-02-01 | 上海爱数信息技术股份有限公司 | File change monitoring and synchronization method and system |
CN106911811A (en) * | 2017-05-04 | 2017-06-30 | 郑州云海信息技术有限公司 | A kind of method based on ftp file high efficiency of transmission |
CN108156199A (en) * | 2016-12-05 | 2018-06-12 | 上海宝信软件股份有限公司 | The system and method for user-defined file transport protocol is realized based on plug-in type module |
CN108198367A (en) * | 2018-01-16 | 2018-06-22 | 西门子工厂自动化工程有限公司 | Data acquire and monitoring control method, system, device and computer storage media |
US20180332100A1 (en) * | 2017-05-12 | 2018-11-15 | Priyanka Bhaskar | Scaled in-order record input ingestion for file-based streams in multi-threaded environments |
US20190087280A1 (en) * | 2017-09-14 | 2019-03-21 | Commvault Systems, Inc. | Distributed framework for task splitting and task assignments in a content indexing system |
CN109766131A (en) * | 2017-11-06 | 2019-05-17 | 上海宝信软件股份有限公司 | The system and method for the intelligent automatic upgrading of software is realized based on multithreading |
CN110198334A (en) * | 2018-04-19 | 2019-09-03 | 腾讯科技(深圳)有限公司 | Access method, device and storage medium based on object storage service |
CN110401724A (en) * | 2019-08-22 | 2019-11-01 | 北京旷视科技有限公司 | File management method, ftp server and storage medium |
CN110784528A (en) * | 2019-10-22 | 2020-02-11 | 北京天融信网络安全技术有限公司 | File downloading method and device and storage medium |
CN111970355A (en) * | 2020-08-14 | 2020-11-20 | 苏州浪潮智能科技有限公司 | Method for automatically downloading numbered files |
CN112684982A (en) * | 2020-12-25 | 2021-04-20 | 北京浪潮数据技术有限公司 | Data migration method, system, equipment and computer readable storage medium |
-
2021
- 2021-07-29 CN CN202110864371.4A patent/CN113766006B/en active Active
Patent Citations (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR20020019170A (en) * | 2000-09-05 | 2002-03-12 | 신동호 | Simultaneous multiple file downloads uploads in Internet's FTP |
CN1863050A (en) * | 2005-09-15 | 2006-11-15 | 上海华为技术有限公司 | Method of document synchronization between server and system thereof |
US20120271880A1 (en) * | 2011-04-19 | 2012-10-25 | Accenture Global Services Limited | Content transfer accelerator |
CN103605568A (en) * | 2013-10-29 | 2014-02-26 | 北京奇虎科技有限公司 | Multithread management method and device |
CN104750811A (en) * | 2015-03-30 | 2015-07-01 | 浪潮通信信息系统有限公司 | Mobile communication data file multithread real-time collection method |
CN106372526A (en) * | 2016-08-19 | 2017-02-01 | 上海爱数信息技术股份有限公司 | File change monitoring and synchronization method and system |
CN108156199A (en) * | 2016-12-05 | 2018-06-12 | 上海宝信软件股份有限公司 | The system and method for user-defined file transport protocol is realized based on plug-in type module |
CN106911811A (en) * | 2017-05-04 | 2017-06-30 | 郑州云海信息技术有限公司 | A kind of method based on ftp file high efficiency of transmission |
US20180332100A1 (en) * | 2017-05-12 | 2018-11-15 | Priyanka Bhaskar | Scaled in-order record input ingestion for file-based streams in multi-threaded environments |
US20190087280A1 (en) * | 2017-09-14 | 2019-03-21 | Commvault Systems, Inc. | Distributed framework for task splitting and task assignments in a content indexing system |
CN109766131A (en) * | 2017-11-06 | 2019-05-17 | 上海宝信软件股份有限公司 | The system and method for the intelligent automatic upgrading of software is realized based on multithreading |
CN108198367A (en) * | 2018-01-16 | 2018-06-22 | 西门子工厂自动化工程有限公司 | Data acquire and monitoring control method, system, device and computer storage media |
CN110198334A (en) * | 2018-04-19 | 2019-09-03 | 腾讯科技(深圳)有限公司 | Access method, device and storage medium based on object storage service |
CN110401724A (en) * | 2019-08-22 | 2019-11-01 | 北京旷视科技有限公司 | File management method, ftp server and storage medium |
CN110784528A (en) * | 2019-10-22 | 2020-02-11 | 北京天融信网络安全技术有限公司 | File downloading method and device and storage medium |
CN111970355A (en) * | 2020-08-14 | 2020-11-20 | 苏州浪潮智能科技有限公司 | Method for automatically downloading numbered files |
CN112684982A (en) * | 2020-12-25 | 2021-04-20 | 北京浪潮数据技术有限公司 | Data migration method, system, equipment and computer readable storage medium |
Non-Patent Citations (3)
Title |
---|
YIMIN WU ET AL.: "FTP upload based on multithreaded and broken transfer resume technology", INTERNATIONAL CONFERENCE ON COMPUTER, COMMUNICATIONS AND INFORMATION TECHNOLOGY, 31 December 2014 (2014-12-31) * |
林耿仕;赵建华;郑国梁;: "支持共享的Ftp客户端的设计", 计算机科学, no. 11, 25 November 2007 (2007-11-25) * |
聂岩峰;彭向军;徐先超;: "FTP服务器枢纽中的文件处理引擎设计", 计算机工程, no. 22, 20 November 2010 (2010-11-20) * |
Also Published As
Publication number | Publication date |
---|---|
CN113766006B (en) | 2024-02-23 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN112134802A (en) | Edge computing power resource scheduling method and system based on terminal triggering | |
CN110231998B (en) | Detection method and device for distributed timing task and storage medium | |
CN113312165B (en) | Task processing method and device | |
CN108833565A (en) | A kind of method, apparatus of monitoring server, server and storage medium | |
CN112948212A (en) | RPA task state monitoring method, device and computer storage medium | |
CN107844339B (en) | Task scheduling method, plug-in and server | |
CN111526049B (en) | Operation and maintenance system, operation and maintenance method, electronic device and storage medium | |
CN113672934A (en) | Security vulnerability scanning system and method, terminal and storage medium | |
CN107423122B (en) | Complex operation and maintenance operation arranging and scheduling system and method thereof | |
CN115622906A (en) | Application log capturing system and method | |
JP2007199956A (en) | Logging information management method and logging information management system | |
CN111464824A (en) | Message sending method and system, server and storage medium | |
CN113766006A (en) | Real-time file transmission method and system based on FTP (file transfer protocol) | |
CN104618879A (en) | Distributed quick short message sending system | |
CN110072199B (en) | Method and system for monitoring short message sending abnormity | |
CN116737395A (en) | Asynchronous information processing system and method | |
CN113760634A (en) | Data processing method and device | |
CN115509714A (en) | Task processing method and device, electronic equipment and storage medium | |
CN113055493B (en) | Data packet processing method, device, system, scheduling device and storage medium | |
CN115499435B (en) | Task scheduling method, system, electronic device and computer readable storage medium | |
CN113467902A (en) | Task scheduling method and device combining RPA and AI, client and server | |
CN111866118A (en) | Working platform file storage and transmission method and system | |
CN112965844A (en) | CPU surge accident processing method and device | |
CN114327838B (en) | Video scheduling engine device and method based on distributed management | |
KR102134552B1 (en) | System and method for transmitting message capable of seperate accounting |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |