CN113766006A - Real-time file transmission method and system based on FTP (file transfer protocol) - Google Patents

Real-time file transmission method and system based on FTP (file transfer protocol) Download PDF

Info

Publication number
CN113766006A
CN113766006A CN202110864371.4A CN202110864371A CN113766006A CN 113766006 A CN113766006 A CN 113766006A CN 202110864371 A CN202110864371 A CN 202110864371A CN 113766006 A CN113766006 A CN 113766006A
Authority
CN
China
Prior art keywords
file
directory
attribute information
thread
real
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110864371.4A
Other languages
Chinese (zh)
Other versions
CN113766006B (en
Inventor
黄志平
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Si Tech Information Technology Co Ltd
Original Assignee
Beijing Si Tech Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Si Tech Information Technology Co Ltd filed Critical Beijing Si Tech Information Technology Co Ltd
Priority to CN202110864371.4A priority Critical patent/CN113766006B/en
Publication of CN113766006A publication Critical patent/CN113766006A/en
Application granted granted Critical
Publication of CN113766006B publication Critical patent/CN113766006B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/06Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/18File system types
    • G06F16/182Distributed file systems

Abstract

The invention provides a real-time file transmission method and a system based on an FTP protocol, which relate to the technical field of file transmission and comprise the following steps: starting a multithreading connection FTP server, which comprises a main thread and a plurality of sub threads; the main thread monitors a file directory in real time and inquires files under the file directory as required; judging whether the inquired file is transmitted or not; if the judgment result is negative, acquiring the attribute information of the file, packaging the attribute information as an object, and putting the object into a sharing queue; and the sub-threads acquire the object from the shared queue and execute the transmission task. The FTP server is connected through multiple threads, one thread is used as a main thread, and directory files of the FTP server and local directory files are monitored in real time; and then, files are acquired to the sharing queue through the main thread, and the files are uploaded and downloaded between the directory files of the FTP server and the local directory files in real time and rapidly in a multithread file transmission mode.

Description

Real-time file transmission method and system based on FTP (file transfer protocol)
Technical Field
The invention relates to the technical field of file transmission, in particular to a real-time file transmission method and a real-time file transmission system based on an FTP protocol.
Background
In the process of business handling in the telecommunication industry, files need to be transmitted between a server and the local in real time; therefore, files in the file directory need to be searched in real time, and after files meeting the conditions are searched, the files are transmitted to the server or the local.
The file transmission efficiency of the current service handling system is low, and the low file transmission efficiency cannot meet the service handling requirements along with the increase of services, so that the problems of service backlog, delay and the like are easily caused.
Disclosure of Invention
Aiming at the problems, the invention provides a real-time file transmission method and a real-time file transmission system based on an FTP protocol, which realize the real-time file transmission of a designated directory by connecting an FTP server in a multithreading way.
In order to achieve the above object, the present invention provides an emergency multitasking short message group sending system, which comprises:
starting a multithreading connection FTP server, which comprises a main thread and a plurality of sub threads;
the main thread monitors a file directory in real time and inquires files under the file directory as required;
judging whether the inquired file is transmitted or not;
if the judgment result is negative, acquiring the attribute information of the file, packaging the attribute information as an object, and putting the object into a sharing queue;
and the sub-thread acquires the object from the shared queue and executes a transmission task.
As a further improvement of the present invention, the main thread monitors the file directory in real time, and queries the files in the file directory as required; the method comprises the following steps:
the main thread monitors a file directory and a local file directory of the FTP server in real time;
inquiring a file directory of the FTP server according to downloading requirements;
and inquiring the local file directory according to uploading requirements.
As a further improvement of the present invention, the attribute information of the file includes: a source file directory, a source file name, a target server, and a target directory.
As a further improvement of the present invention, the sub-thread acquires the object from the shared queue and executes a transmission task; the method comprises the following steps:
each branch thread acquires different objects from the shared queue respectively;
analyzing the attribute information encapsulated in the object;
and transmitting the file from the source end to the target end according to the attribute information.
As a further improvement of the invention, if an exception occurs in the sub-thread executing transmission task process, the object is placed back into the shared queue again.
The invention also provides a system for real-time file transmission based on the FTP protocol, which comprises: the device comprises a thread connection module, a monitoring module, a transmission condition judgment module, a packaging module and a file transmission module;
the thread connection module is configured to:
starting a multithreading connection FTP server, which comprises a main thread and a plurality of sub threads;
the monitoring module is used for:
the main thread monitors a file directory in real time and inquires files under the file directory as required;
the transmission condition judgment module is used for:
judging whether the inquired file is transmitted or not;
the encapsulation module is used for:
if the judgment result is negative, acquiring the attribute information of the file, packaging the attribute information as an object, and putting the object into a sharing queue;
the file transmission module is used for:
and the sub-thread acquires the object from the shared queue and executes a transmission task.
As a further improvement of the present invention, the main thread monitors the file directory in real time, and queries the files in the file directory as required; the method comprises the following steps:
the main thread monitors a file directory and a local file directory of the FTP server in real time;
inquiring a file directory of the FTP server according to downloading requirements;
and inquiring the local file directory according to uploading requirements.
As a further improvement of the present invention, the attribute information of the file includes: a source file directory, a source file name, a target server, and a target directory.
As a further improvement of the present invention, the sub-thread acquires the object from the shared queue and executes a transmission task; the method comprises the following steps:
each branch thread acquires different objects from the shared queue respectively;
analyzing the attribute information encapsulated in the object;
and transmitting the file from the source end to the target end according to the attribute information.
As a further improvement of the invention, if an exception occurs in the sub-thread executing transmission task process, the object is placed back into the shared queue again.
Compared with the prior art, the invention has the beneficial effects that:
the FTP server is connected through multiple threads, one thread is used as a main thread, and directory files of the FTP server and local directory files are monitored in real time; and then, files are acquired to the sharing queue through the main thread, and the files are uploaded and downloaded between the directory files of the FTP server and the local directory files in real time and rapidly in a multithread file transmission mode.
In the scheme of the invention, operation and maintenance personnel can automatically set the specific uploading and downloading tasks, the specific FTP server and the thread number for connecting the FTP, further start a program, and respectively scan the local directory files and the directory files of the FTP server by the uploading and downloading tasks set by the main thread branch office to finish the file transmission work of uploading and downloading.
Drawings
FIG. 1 is a flowchart illustrating a real-time file transfer method based on FTP protocol according to an embodiment of the present invention;
fig. 2 is a system diagram of a real-time file transfer method based on the FTP protocol according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.
The invention is described in further detail below with reference to the attached drawing figures:
example (b):
as shown in fig. 1, a real-time file transfer method based on the FTP protocol provided by the present invention includes:
s1, starting a multithreading connection FTP server, which comprises a main thread and a plurality of sub threads;
s2, the main thread monitors the file directory in real time and inquires files under the file directory as required;
wherein, include:
the main thread monitors a file directory and a local file directory of the FTP server in real time;
inquiring a file directory of the FTP server according to downloading requirements;
and inquiring the local file directory according to uploading requirements.
S3, judging whether the inquired file is transmitted or not;
s4, if the judgment result is negative, acquiring the attribute information of the file, packaging the attribute information as an object, and putting the object into a sharing queue;
wherein the content of the first and second substances,
the attribute information of the file includes: a source file directory, a source file name, a target server, and a target directory.
And S5, the sub-thread acquires the object from the shared queue and executes the transmission task.
Wherein, include:
each branch process respectively acquires different objects from the shared queue;
analyzing the encapsulated attribute information in the object;
and transmitting the file from the source end to the target end according to the attribute information.
Further, in the above-mentioned case,
and transmitting the file from the source file directory to the target directory according to the source file directory and the target directory in the attribute information.
In the invention, if an exception occurs in the process of executing the transmission task by the branch thread, the object is placed back to the shared queue again.
As shown in fig. 2, the present invention further provides a real-time file transfer system based on the FTP protocol, including: the device comprises a thread connection module, a monitoring module, a transmission condition judgment module, a packaging module and a file transmission module;
a thread connection module to:
starting a multithreading connection FTP server, which comprises a main thread and a plurality of sub threads;
a monitoring module to:
the main thread monitors a file directory in real time and inquires files under the file directory as required;
wherein, include:
the main thread monitors a file directory and a local file directory of the FTP server in real time;
inquiring a file directory of the FTP server according to downloading requirements;
and inquiring the local file directory according to uploading requirements.
A transmission condition judgment module for:
judging whether the inquired file is transmitted or not;
an encapsulation module to:
if the judgment result is negative, acquiring the attribute information of the file, packaging the attribute information as an object, and putting the object into a sharing queue;
wherein the content of the first and second substances,
the attribute information of the file includes: a source file directory, a source file name, a target server, and a target directory.
A file transfer module to:
and the sub-threads acquire the object from the shared queue and execute the transmission task.
Wherein, include:
each branch process respectively acquires different objects from the shared queue;
analyzing the encapsulated attribute information in the object;
and transmitting the file from the source end to the target end according to the attribute information.
Further, the file is transmitted from the source file directory to the target directory according to the source file directory and the target directory in the attribute information.
In the invention, if an exception occurs in the process of executing the transmission task by the branch thread, the object is placed back to the shared queue again.
The invention has the advantages that:
(1) connecting the FTP servers through multiple threads, and using one thread as a main thread to realize real-time monitoring of directory files of the FTP servers and local directory files; and then, files are acquired to the sharing queue through the main thread, and the files are uploaded and downloaded between the directory files of the FTP server and the local directory files in real time and rapidly in a multithread file transmission mode.
(2) The operation and maintenance personnel can automatically set specific uploading and downloading tasks, specific FTP servers and the number of threads for connecting the FTP, the program automatically initializes the connection of the FTP servers according to the set information, and then starts the program, and the uploading and downloading tasks set by the main thread branch office respectively scan local directory files and directory files of the FTP servers to finish the file transmission work of uploading and downloading.
(3) The branch process checks whether the shared queue has an object in real time, if so, the object is taken out from the shared queue, and the file transmission task is executed according to the information in the object until the shared queue is empty
(4) The main thread regularly checks whether the task needs to be stopped, restarted and the like, so that the front end can conveniently control and manage the task.
(5) Monitoring a directory file of the FTP server in real time, and downloading the file to a local directory from the FTP server when the file meeting the downloading requirement exists under the directory file; and monitoring local directory files in real time, and uploading the files to a designated directory of a designated FTP server when the files meet uploading conditions, wherein the files can be a plurality of FTP servers.
The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes will occur to those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (10)

1. A real-time file transmission method based on FTP protocol is characterized by comprising the following steps:
starting a multithreading connection FTP server, which comprises a main thread and a plurality of sub threads;
the main thread monitors a file directory in real time and inquires files under the file directory as required;
judging whether the inquired file is transmitted or not;
if the judgment result is negative, acquiring the attribute information of the file, packaging the attribute information as an object, and putting the object into a sharing queue;
and the sub-thread acquires the object from the shared queue and executes a transmission task.
2. The real-time file transfer method according to claim 1, wherein: the main thread monitors a file directory in real time and inquires files under the file directory as required; the method comprises the following steps:
the main thread monitors a file directory and a local file directory of the FTP server in real time;
inquiring a file directory of the FTP server according to downloading requirements;
and inquiring the local file directory according to uploading requirements.
3. The real-time file transfer method according to claim 1, wherein: the attribute information of the file includes: a source file directory, a source file name, a target server, and a target directory.
4. The real-time file transmission method according to claim 1, wherein the sub-thread acquires the object from the shared queue and executes a transmission task; the method comprises the following steps:
each branch thread acquires different objects from the shared queue respectively;
analyzing the attribute information encapsulated in the object;
and transmitting the file from the source end to the target end according to the attribute information.
5. The real-time file transfer method according to claim 1, wherein: and if the sub-thread is abnormal in the process of executing the transmission task, the object is placed back into the shared queue again.
6. A system for implementing the real-time file transmission method according to any one of claims 1 to 5, comprising: the device comprises a thread connection module, a monitoring module, a transmission condition judgment module, a packaging module and a file transmission module;
the thread connection module is configured to:
starting a multithreading connection FTP server, which comprises a main thread and a plurality of sub threads;
the monitoring module is used for:
the main thread monitors a file directory in real time and inquires files under the file directory as required;
the transmission condition judgment module is used for:
judging whether the inquired file is transmitted or not;
the encapsulation module is used for:
if the judgment result is negative, acquiring the attribute information of the file, packaging the attribute information as an object, and putting the object into a sharing queue;
the file transmission module is used for:
and the sub-thread acquires the object from the shared queue and executes a transmission task.
7. The system of claim 6, wherein: the main thread monitors a file directory in real time and inquires files under the file directory as required; the method comprises the following steps:
the main thread monitors a file directory and a local file directory of the FTP server in real time;
inquiring a file directory of the FTP server according to downloading requirements;
and inquiring the local file directory according to uploading requirements.
8. The system of claim 6, wherein: the attribute information of the file includes: a source file directory, a source file name, a target server, and a target directory.
9. The system of claim 6, wherein: the sub-thread acquires the object from the shared queue and executes a transmission task; the method comprises the following steps:
each branch thread acquires different objects from the shared queue respectively;
analyzing the attribute information encapsulated in the object;
and transmitting the file from the source end to the target end according to the attribute information.
10. The system of claim 6, wherein: and if the sub-thread is abnormal in the process of executing the transmission task, the object is placed back into the shared queue again.
CN202110864371.4A 2021-07-29 2021-07-29 Real-time file transmission method and system based on FTP protocol Active CN113766006B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110864371.4A CN113766006B (en) 2021-07-29 2021-07-29 Real-time file transmission method and system based on FTP protocol

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110864371.4A CN113766006B (en) 2021-07-29 2021-07-29 Real-time file transmission method and system based on FTP protocol

Publications (2)

Publication Number Publication Date
CN113766006A true CN113766006A (en) 2021-12-07
CN113766006B CN113766006B (en) 2024-02-23

Family

ID=78788250

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110864371.4A Active CN113766006B (en) 2021-07-29 2021-07-29 Real-time file transmission method and system based on FTP protocol

Country Status (1)

Country Link
CN (1) CN113766006B (en)

Citations (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20020019170A (en) * 2000-09-05 2002-03-12 신동호 Simultaneous multiple file downloads uploads in Internet's FTP
CN1863050A (en) * 2005-09-15 2006-11-15 上海华为技术有限公司 Method of document synchronization between server and system thereof
US20120271880A1 (en) * 2011-04-19 2012-10-25 Accenture Global Services Limited Content transfer accelerator
CN103605568A (en) * 2013-10-29 2014-02-26 北京奇虎科技有限公司 Multithread management method and device
CN104750811A (en) * 2015-03-30 2015-07-01 浪潮通信信息系统有限公司 Mobile communication data file multithread real-time collection method
CN106372526A (en) * 2016-08-19 2017-02-01 上海爱数信息技术股份有限公司 File change monitoring and synchronization method and system
CN106911811A (en) * 2017-05-04 2017-06-30 郑州云海信息技术有限公司 A kind of method based on ftp file high efficiency of transmission
CN108156199A (en) * 2016-12-05 2018-06-12 上海宝信软件股份有限公司 The system and method for user-defined file transport protocol is realized based on plug-in type module
CN108198367A (en) * 2018-01-16 2018-06-22 西门子工厂自动化工程有限公司 Data acquire and monitoring control method, system, device and computer storage media
US20180332100A1 (en) * 2017-05-12 2018-11-15 Priyanka Bhaskar Scaled in-order record input ingestion for file-based streams in multi-threaded environments
US20190087280A1 (en) * 2017-09-14 2019-03-21 Commvault Systems, Inc. Distributed framework for task splitting and task assignments in a content indexing system
CN109766131A (en) * 2017-11-06 2019-05-17 上海宝信软件股份有限公司 The system and method for the intelligent automatic upgrading of software is realized based on multithreading
CN110198334A (en) * 2018-04-19 2019-09-03 腾讯科技(深圳)有限公司 Access method, device and storage medium based on object storage service
CN110401724A (en) * 2019-08-22 2019-11-01 北京旷视科技有限公司 File management method, ftp server and storage medium
CN110784528A (en) * 2019-10-22 2020-02-11 北京天融信网络安全技术有限公司 File downloading method and device and storage medium
CN111970355A (en) * 2020-08-14 2020-11-20 苏州浪潮智能科技有限公司 Method for automatically downloading numbered files
CN112684982A (en) * 2020-12-25 2021-04-20 北京浪潮数据技术有限公司 Data migration method, system, equipment and computer readable storage medium

Patent Citations (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20020019170A (en) * 2000-09-05 2002-03-12 신동호 Simultaneous multiple file downloads uploads in Internet's FTP
CN1863050A (en) * 2005-09-15 2006-11-15 上海华为技术有限公司 Method of document synchronization between server and system thereof
US20120271880A1 (en) * 2011-04-19 2012-10-25 Accenture Global Services Limited Content transfer accelerator
CN103605568A (en) * 2013-10-29 2014-02-26 北京奇虎科技有限公司 Multithread management method and device
CN104750811A (en) * 2015-03-30 2015-07-01 浪潮通信信息系统有限公司 Mobile communication data file multithread real-time collection method
CN106372526A (en) * 2016-08-19 2017-02-01 上海爱数信息技术股份有限公司 File change monitoring and synchronization method and system
CN108156199A (en) * 2016-12-05 2018-06-12 上海宝信软件股份有限公司 The system and method for user-defined file transport protocol is realized based on plug-in type module
CN106911811A (en) * 2017-05-04 2017-06-30 郑州云海信息技术有限公司 A kind of method based on ftp file high efficiency of transmission
US20180332100A1 (en) * 2017-05-12 2018-11-15 Priyanka Bhaskar Scaled in-order record input ingestion for file-based streams in multi-threaded environments
US20190087280A1 (en) * 2017-09-14 2019-03-21 Commvault Systems, Inc. Distributed framework for task splitting and task assignments in a content indexing system
CN109766131A (en) * 2017-11-06 2019-05-17 上海宝信软件股份有限公司 The system and method for the intelligent automatic upgrading of software is realized based on multithreading
CN108198367A (en) * 2018-01-16 2018-06-22 西门子工厂自动化工程有限公司 Data acquire and monitoring control method, system, device and computer storage media
CN110198334A (en) * 2018-04-19 2019-09-03 腾讯科技(深圳)有限公司 Access method, device and storage medium based on object storage service
CN110401724A (en) * 2019-08-22 2019-11-01 北京旷视科技有限公司 File management method, ftp server and storage medium
CN110784528A (en) * 2019-10-22 2020-02-11 北京天融信网络安全技术有限公司 File downloading method and device and storage medium
CN111970355A (en) * 2020-08-14 2020-11-20 苏州浪潮智能科技有限公司 Method for automatically downloading numbered files
CN112684982A (en) * 2020-12-25 2021-04-20 北京浪潮数据技术有限公司 Data migration method, system, equipment and computer readable storage medium

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
YIMIN WU ET AL.: "FTP upload based on multithreaded and broken transfer resume technology", INTERNATIONAL CONFERENCE ON COMPUTER, COMMUNICATIONS AND INFORMATION TECHNOLOGY, 31 December 2014 (2014-12-31) *
林耿仕;赵建华;郑国梁;: "支持共享的Ftp客户端的设计", 计算机科学, no. 11, 25 November 2007 (2007-11-25) *
聂岩峰;彭向军;徐先超;: "FTP服务器枢纽中的文件处理引擎设计", 计算机工程, no. 22, 20 November 2010 (2010-11-20) *

Also Published As

Publication number Publication date
CN113766006B (en) 2024-02-23

Similar Documents

Publication Publication Date Title
CN112134802A (en) Edge computing power resource scheduling method and system based on terminal triggering
CN110231998B (en) Detection method and device for distributed timing task and storage medium
CN113312165B (en) Task processing method and device
CN108833565A (en) A kind of method, apparatus of monitoring server, server and storage medium
CN112948212A (en) RPA task state monitoring method, device and computer storage medium
CN107844339B (en) Task scheduling method, plug-in and server
CN111526049B (en) Operation and maintenance system, operation and maintenance method, electronic device and storage medium
CN113672934A (en) Security vulnerability scanning system and method, terminal and storage medium
CN107423122B (en) Complex operation and maintenance operation arranging and scheduling system and method thereof
CN115622906A (en) Application log capturing system and method
JP2007199956A (en) Logging information management method and logging information management system
CN111464824A (en) Message sending method and system, server and storage medium
CN113766006A (en) Real-time file transmission method and system based on FTP (file transfer protocol)
CN104618879A (en) Distributed quick short message sending system
CN110072199B (en) Method and system for monitoring short message sending abnormity
CN116737395A (en) Asynchronous information processing system and method
CN113760634A (en) Data processing method and device
CN115509714A (en) Task processing method and device, electronic equipment and storage medium
CN113055493B (en) Data packet processing method, device, system, scheduling device and storage medium
CN115499435B (en) Task scheduling method, system, electronic device and computer readable storage medium
CN113467902A (en) Task scheduling method and device combining RPA and AI, client and server
CN111866118A (en) Working platform file storage and transmission method and system
CN112965844A (en) CPU surge accident processing method and device
CN114327838B (en) Video scheduling engine device and method based on distributed management
KR102134552B1 (en) System and method for transmitting message capable of seperate accounting

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant