CN108769222A - A kind of high-performance treatments method that files in batch uploads - Google Patents
A kind of high-performance treatments method that files in batch uploads Download PDFInfo
- Publication number
- CN108769222A CN108769222A CN201810567184.8A CN201810567184A CN108769222A CN 108769222 A CN108769222 A CN 108769222A CN 201810567184 A CN201810567184 A CN 201810567184A CN 108769222 A CN108769222 A CN 108769222A
- Authority
- CN
- China
- Prior art keywords
- server
- servers
- transmitting file
- wheel
- files
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 17
- 238000011282 treatment Methods 0.000 title claims abstract description 9
- 230000005540 biological transmission Effects 0.000 claims abstract description 9
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/06—Network architectures or network communication protocols for network security for supporting key management in a packet data network
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Hardware Design (AREA)
- Computer Security & Cryptography (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention discloses a kind of high-performance treatments methods that files in batch uploads, the present invention provides a kind of high performance solution, method by authorizing child servers temporary authority, so that every all has permission identical with host, it, can be with the transmission speed of the promotion file of geometry multiple and then when completing file upload.
Description
Technical field
The present invention relates to technical field of data processing, the high-performance treatments method of specifically a kind of files in batch upload.
Background technology
In the prior art, in one big file copy to 2000 Linux servers (Gigabit Ethernet), one one is needed
The sequentially batch of platform uploads, and transmission bottleneck is limited by physical network card rate (i.e. Gigabit Ethernet, most fast 128MB/s) big files of 4GB
It is transferred to 2000 server needs and takes and be：32 seconds/platform × 2000 of 128MB/s=32 seconds/platforms of 4096MB (4GB) ÷=
64000 seconds (about 18 hours) are if similarly the method needs similarly the method completion 8000 in 36 hours if completing 4000 servers
Platform server then needs 72 hours.
Invention content
The purpose of the present invention is to provide a kind of high-performance treatments methods that files in batch uploads, to solve above-mentioned background skill
The problem of being proposed in art.
To achieve the above object, the present invention provides the following technical solutions：
A kind of high-performance treatments method that files in batch uploads, comprises the steps of：
A, the SSH keys in home server are lent into each server temporarily, make every all have it is identical with host
Permission；
B, start the transmission of SCP files；The specific method is as follows：1st wheel) home server transmitting file give the 1st server；2nd
Wheel) the 1st server of control, enable its transmitting file to the 2nd server；Home server continues transmitting file to the 3rd server control simultaneously
1-3 servers are made, it is enabled while transmitting file is to 4-6 servers；Home server continues transmitting file to the 7th service simultaneously
Device；4th wheel) control 1-7 servers, it enables it while transmitting file is to 8-14 servers；Home server is after resuming text simultaneously
Part gives the 15th server；5th wheel) control 1-15 servers, it enables it while transmitting file is to 16-30 servers；Host simultaneously
Server continues transmitting file to the 31st server；6th wheel) control 1-31 servers, enabling it, transmitting file is serviced to 32-62 simultaneously
Device；Home server continues transmitting file to the 63rd server controls 1-63 servers simultaneously, enables it while transmitting file is to 64-
126 servers；Home server continues transmitting file to the 127th server simultaneously；8th wheel) control 1-127 servers, enable it
Transmitting file gives 128-254 servers simultaneously；Home server continues transmitting file to the 255th server simultaneously；
C, restore original key of each server.
Further technical solution as the present invention：The step C's is specifically the temporary authority for deleting each server.
Compared with prior art, the beneficial effects of the invention are as follows：The present invention provides a kind of high performance solution, passes through
The method for authorizing child servers temporary authority so that every all has permission identical with host, and then uploaded completing file
When, can with geometry multiple promotion file transmission speed,
Description of the drawings
Fig. 1 is the method flow schematic diagram of the present invention.
Specific implementation mode
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation describes, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment shall fall within the protection scope of the present invention.
Referring to Fig. 1, a kind of high-performance treatments method that files in batch uploads, comprises the steps of：
A, the SSH keys in home server are lent into each server temporarily, make every all have it is identical with host
Permission；
B, start the transmission of SCP files；The specific method is as follows：1st wheel) home server transmitting file gives the 1st server (consumption
When 32 seconds)；2nd wheel) the 1st server of control, enable its transmitting file to the 2nd server；Home server is after resuming text simultaneously
Part gives the 3rd server (taking 32 seconds)；3rd wheel) control 1-3 servers, enabling it, transmitting file gives 4-6 servers simultaneously；Simultaneously
Home server continues transmitting file and gives the 7th server (taking 32 seconds)；4th wheel) control 1-7 servers, enable its transmitting file simultaneously
Give 8-14 servers；Home server continuation transmitting file gives the 15th server (taking 32 seconds) simultaneously；5th wheel) control 1-15 clothes
Business device, enabling it, transmitting file gives 16-30 servers simultaneously；Home server continues transmitting file to the 31st server (time-consuming 32 simultaneously
Second)；6th wheel) control 1-31 servers, enabling it, transmitting file gives 32-62 servers simultaneously；Home server continues transmitting file simultaneously
Give the 63rd server (taking 32 seconds)；7th wheel) control 1-63 servers, enabling it, transmitting file gives 64-126 servers simultaneously；Simultaneously
Home server continues transmitting file and gives the 127th server (taking 32 seconds)；8th wheel) control 1-127 servers, it enables it while passing
File gives 128-254 servers；Home server continuation transmitting file gives the 255th server (taking 32 seconds) simultaneously；
C, the original key for restoring each server, specifically deletes the temporary authority of each server.
In such a mode：
The file transmission of 2000 servers can be completed in 11th round；
The file transmission of 4000 servers can be completed in 12nd wheel；
The file transmission of 8000 servers can be completed in 13rd wheel；
× 11 wheel=352 seconds (about 6 minutes) of 32 seconds/wheel；
× 12 wheel=384 seconds (about 6.5 minutes) of 32 seconds/wheel；
× 13 wheel=416 seconds (about 7 minutes) of 32 seconds/wheel.
It is obvious to a person skilled in the art that invention is not limited to the details of the above exemplary embodiments, Er Qie
In the case of without departing substantially from spirit or essential attributes of the invention, the present invention can be realized in other specific forms.Therefore, no matter
From the point of view of which point, the present embodiments are to be considered as illustrative and not restrictive, and the scope of the present invention is by appended power
Profit requires rather than above description limits, it is intended that all by what is fallen within the meaning and scope of the equivalent requirements of the claims
Variation is included within the present invention.Any reference signs in the claims should not be construed as limiting the involved claims.
In addition, it should be understood that although this specification is described in terms of embodiments, but not each embodiment is only wrapped
Containing an independent technical solution, this description of the specification is merely for the sake of clarity, and those skilled in the art should
It considers the specification as a whole, the technical solutions in the various embodiments may also be suitably combined, forms those skilled in the art
The other embodiment being appreciated that.
Claims (2)
1. a kind of high-performance treatments method that files in batch uploads, which is characterized in that comprise the steps of：
A, the SSH keys in home server are lent into each server temporarily, makes every to all have permission identical with host；
B, start the transmission of SCP files；The specific method is as follows：1st wheel) home server transmitting file give the 1st server；2nd wheel) control
The 1st server is made, enables its transmitting file to the 2nd server；Home server continues transmitting file to the 3rd server controls 1- simultaneously
3 servers enable it while transmitting file are to 4-6 servers；Home server continues transmitting file to the 7th server simultaneously；4th
Wheel) control 1-7 servers, it enables it while transmitting file is to 8-14 servers；Home server continues transmitting file to the simultaneously
15 servers；5th wheel) control 1-15 servers, it enables it while transmitting file is to 16-30 servers；Home server simultaneously
Continue transmitting file to the 31st server；6th wheel) control 1-31 servers, it enables it while transmitting file is to 32-62 servers；
Home server continues transmitting file to the 63rd server controls 1-63 servers simultaneously, enables it while transmitting file is to 64-126
Server；Home server continues transmitting file to the 127th server simultaneously；8th wheel) control 1-127 servers, enable it simultaneously
Transmitting file gives 128-254 servers；Home server continues transmitting file to the 255th server simultaneously；
C, restore original key of each server.
2. the high-performance treatments method that a kind of files in batch according to claim 1 uploads, which is characterized in that the step
C's is specifically the temporary authority for deleting each server.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810567184.8A CN108769222A (en) | 2018-06-05 | 2018-06-05 | A kind of high-performance treatments method that files in batch uploads |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810567184.8A CN108769222A (en) | 2018-06-05 | 2018-06-05 | A kind of high-performance treatments method that files in batch uploads |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108769222A true CN108769222A (en) | 2018-11-06 |
Family
ID=63998944
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810567184.8A Pending CN108769222A (en) | 2018-06-05 | 2018-06-05 | A kind of high-performance treatments method that files in batch uploads |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108769222A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2010244432A (en) * | 2009-04-08 | 2010-10-28 | Nippon Telegr & Teleph Corp <Ntt> | File sharing system, shared file server device, file sharing method, access control method of shared file server device, and programs thereof |
| US20110184998A1 (en) * | 2010-01-22 | 2011-07-28 | Palahnuk Samuel L | Universally accessible encrypted internet file system for wired and wireless computing devices supplanting synchronization, backup and email file attachment |
| US20130239014A1 (en) * | 2012-03-07 | 2013-09-12 | Salesforce.Com, Inc. | File transfer methodology for a desktop sharing system |
| CN103546546A (en) * | 2013-09-30 | 2014-01-29 | 蓝盾信息安全技术股份有限公司 | Large-scale cluster file distribution method |
| CN104732151A (en) * | 2015-03-17 | 2015-06-24 | 浪潮集团有限公司 | Remote safety batch BIOS updating method under Linux system |
| US9213718B1 (en) * | 2011-06-22 | 2015-12-15 | Emc Corporation | Synchronized file management across multiple disparate endpoints |
-
2018
- 2018-06-05 CN CN201810567184.8A patent/CN108769222A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2010244432A (en) * | 2009-04-08 | 2010-10-28 | Nippon Telegr & Teleph Corp <Ntt> | File sharing system, shared file server device, file sharing method, access control method of shared file server device, and programs thereof |
| US20110184998A1 (en) * | 2010-01-22 | 2011-07-28 | Palahnuk Samuel L | Universally accessible encrypted internet file system for wired and wireless computing devices supplanting synchronization, backup and email file attachment |
| US9213718B1 (en) * | 2011-06-22 | 2015-12-15 | Emc Corporation | Synchronized file management across multiple disparate endpoints |
| US20130239014A1 (en) * | 2012-03-07 | 2013-09-12 | Salesforce.Com, Inc. | File transfer methodology for a desktop sharing system |
| CN103546546A (en) * | 2013-09-30 | 2014-01-29 | 蓝盾信息安全技术股份有限公司 | Large-scale cluster file distribution method |
| CN104732151A (en) * | 2015-03-17 | 2015-06-24 | 浪潮集团有限公司 | Remote safety batch BIOS updating method under Linux system |
Non-Patent Citations (1)
| Title |
|---|
| 刘佳旭;刘万军;: "基于云计算平台的学生作业提交系统设计与实现" * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Crozier et al. | Distance upper bounds and true minimum distance results for turbo-codes designed with DRP interleavers | |
| CN104717133A (en) | Distributed message push system and method | |
| CN106227780A (en) | Automatization's sectional drawing evidence collecting method of a kind of magnanimity webpage and system | |
| CN102917012A (en) | Storage and communication de-duplication | |
| CN104967660B (en) | A kind of network performance method for improving towards more cloud frameworks | |
| CN104771903A (en) | Login queuing method for online game | |
| CN104301438A (en) | File uploading method | |
| CN105607933A (en) | Heat updating method for mobile games | |
| CN106846189A (en) | A kind of intellectual property interactive management system | |
| CN108769222A (en) | A kind of high-performance treatments method that files in batch uploads | |
| CN104636397B (en) | Resource allocation methods, calculating accelerated method and device for Distributed Calculation | |
| US20160156610A1 (en) | Message Pushing System And Method | |
| CN104079472A (en) | Method for mobile game products to achieve real-time social contact functions | |
| CN107423446A (en) | New media based on cloud storage automates credible deployment system and method | |
| CN106846190A (en) | A kind of intellectual property case management system | |
| CN106791855A (en) | A kind of method and system for reducing management system cost of attending a banquet | |
| CN109491807A (en) | Data exchange method, device and system | |
| WO2015038273A1 (en) | Apparatus, system, and method for event-identified content exchange and management | |
| WO2023173551A1 (en) | Short address generation method and apparatus, and electronic device and computer-readable storage medium | |
| CN103977560A (en) | Real-time social function implementing method for mobile game products | |
| CN106411720A (en) | WEB page instant messaging system and method based on user permission | |
| CN104866463A (en) | Information arrangement method and device | |
| CN104901869A (en) | Interaction method for online group members | |
| CN104166820A (en) | Data management method and system | |
| CN105100176B (en) | A kind of resource sharing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181106 |
|
| RJ01 | Rejection of invention patent application after publication |