CN107861686A - File memory method, service end and computer-readable recording medium - Google Patents
File memory method, service end and computer-readable recording medium Download PDFInfo
- Publication number
- CN107861686A CN107861686A CN201710885384.3A CN201710885384A CN107861686A CN 107861686 A CN107861686 A CN 107861686A CN 201710885384 A CN201710885384 A CN 201710885384A CN 107861686 A CN107861686 A CN 107861686A
- Authority
- CN
- China
- Prior art keywords
- file
- service end
- distributed
- storage
- merging
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0608—Saving storage space on storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/0643—Management of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of file memory method, and applied to Internet data center, the Internet data center includes document handling system, distributed file system, distributed memory system, and document handling system includes service end and client, and this method includes：Service end receives the file of sender's upload by client；By the file cache received into temporary folder, and record in distributed memory system the storage location information of each file；Processing is merged to each file in temporary folder, the file after being merged, and the file after merging is stored into distributed file system；Based on the file after merging, corresponding storage location information in distributed memory system is updated.The invention also discloses a kind of service end and computer-readable recording medium.The present invention can improve the quantity of data storage by the merging of file, and pass through distributed file system storage file so that storable quantity of documents is more.
Description
Technical field
The present invention relates to applied technical field, more particularly to a kind of file memory method, service end and computer-readable deposit
Storage media.
Background technology
Traditional mass file storage is used in special file server such as NAS (Network Attached
Storage, network attached storage) stored, NAS is defined as a kind of special exclusive data storage server, including deposits
Memory device (such as disk array, CD/DVD drivers, tape drive or moveable storage medium) and embedded systems software,
Cross-platform file-sharing function can be provided.NAS generally occupies certainly on a LAN (Local Area Network, LAN)
Oneself node, the intervention without application server, it is allowed to which user accesses data on network.
Existing file storage architecture, more front-end servers pass through specific store network share rear end NAS device, rear end
Memory space on NAS device passes through CIFS (Commom Internet File System, universal network file system), NFS
(Network File System, NFS) agreement shares to preceding end main frame, and same catalogue or file can be entered simultaneously
Row concurrent reading and writing.File system is located at back end storage system, and connection uses standard ethernet link and TCP (Transmission
Control Protocol, transmission control protocol)/IP (Internet Protocol, Internet Protocol) agreement, can be real
File storage between existing multisystem is shared.However, over time and business development, data scale is increasingly
Greatly, the memory capacity of NAS device is limited, and traditional file memory module has been difficult to tackle the eruptive growth of data, that is,
Say, in the case where data volume is increasing, existing document storage mode, the capacity of data storage is difficult meet demand.
The content of the invention
It is a primary object of the present invention to provide a kind of file memory method, service end and computer-readable recording medium,
Existing document storage mode is aimed to solve the problem that, in the case where data capacity increases, it is difficult to meet the technical problem of storage demand.
To achieve the above object, the present invention provides a kind of file memory method, described mutual applied to Internet data center
Networking data center includes document handling system, distributed file system, distributed memory system, the document handling system bag
Service end and client are included, the file memory method includes：
The service end of document handling system receives the file of sender's upload by client；
By the file cache received into temporary folder, and depositing for each file is recorded in distributed memory system
Store up positional information；
Processing is merged to each file in temporary folder, the file after being merged, and by the text after merging
Part is stored into distributed file system；
Based on the file after merging, corresponding storage location information in distributed memory system is updated.
Alternatively, each file in temporary folder merges processing, the step of the file after being merged
Suddenly include：
The service end scans each file in the temporary folder；
Composition file is obtained, and determines that the capability value after merging with the composition file is less than in the file of scanning and presets
The file of threshold values, by the Piece file mergence of determination into the composition file.
Alternatively, methods described also includes：
When receiving file polling instruction, index information corresponding to file polling instruction is determined；
The merging file pointed by the index information is searched in distributed file system；
The file that merged is reduced, with from merged restored in file corresponding to the index information it is literary
Part.
Alternatively, the file by after merging is stored to after the step in distributed file system, and methods described is also
Including：
The service end generates file identification information and file Hash letter based on the file stored in distributed file system
Breath；
By the client feedback file identification information and file Hash information to described sender, for the transmission
Side is transmitted to recipient by file identification information and file Hash information；
When receiving the file identification information that the recipient sends by the client, in distributed file system
File corresponding to the file identification information is extracted, and feeds back to the recipient, so that the recipient passes through file Hash
Information examines the file, and the file is obtained when examining successfully.
Alternatively, the number of the service end includes multiple, and the service end and client of the document handling system pass through
Gateway connects, and the mode that file is uploaded to service end from client includes：Gateway uploads client according to default strategy
File poll is uploaded in service end.
Alternatively, the file based on after merging, corresponding storage location information in distributed memory system is updated
After step, methods described also includes：
Each file in the service end scanning distributed file system, to monitor the storage duration of each file；
When documentary storage duration reaches preset duration, the file in the distributed file system is deleted,
And delete the storage location information of file described in the distributed memory system.
Alternatively, the Internet data center also includes distributed application program coordination service, the service end scanning
Each file in distributed file system, the step of to monitor the storage duration of each file before, methods described also includes：
The service end sends the solicited message for deleting lock to distributed application program coordination service；
When acquisition is locked successfully, each file in scanning distributed file system is performed, to monitor depositing for each file
The step of storing up duration.
Alternatively, the service end is located in main Internet data center, standby Internet data center in systems be present
In the case of, the file based on after merging, update distributed memory system in corresponding storage location information the step of it
Afterwards, methods described includes：
Service end of the service end by the file synchronization of storage to standby Internet data center place document handling system
In, file storage operations are performed with the service end of document handling system where incense Internet data center.
In addition, to achieve the above object, the present invention also provides a kind of service end, the service end includes memory, processing
Device and the file storage program that can be run on the memory and on the processor is stored in, the file storage program quilt
The step of file memory method as described above is realized during the computing device.
In addition, to achieve the above object, the present invention also provides a kind of computer-readable recording medium, described computer-readable
File storage program is stored with storage medium, text as described above is realized when the file storage program is executed by processor
The step of part storage method.
Technical scheme proposed by the present invention, the service end of document handling system first pass through client and receive what sender uploaded
File, then by the file cache received into temporary folder, and each file is recorded in distributed memory system
Storage location information, then processing is merged to each file in temporary folder, the file after being merged, and will merge
File afterwards is stored into distributed file system, the file being based ultimately upon after merging, and is updated corresponding in distributed memory system
Storage location information, be easy to subsequently according to the storage location information read file.In this programme, the file received is entered
Row merging treatment, then the file after merging is stored into distributed file system, the merging of file make it that system is storable
File amount increases, further, since distributed file system has scalability, can by distributed file system storage file
The quantity of documents of storage is more, and relative to existing document storage mode, the storable file amount of this programme is bigger, is more suitable for depositing
Store up substantial amounts of small documents.
Brief description of the drawings
Fig. 1 is the service end structural representation for the hardware running environment that scheme of the embodiment of the present invention is related to；
Fig. 2 is the schematic flow sheet of file memory method first embodiment of the present invention；
Fig. 3 is the file storage rack composition of the present invention；
Fig. 4 is the schematic diagram of Piece file mergence of the present invention；
Fig. 5 is the schematic flow sheet of file memory method second embodiment of the present invention；
Fig. 6 is the exemplary plot of file of the present invention transmission；
Fig. 7 is the schematic flow sheet of file memory method 3rd embodiment of the present invention；
Fig. 8 is the exemplary plot that file of the present invention is deleted.
The realization, functional characteristics and advantage of the object of the invention will be described further referring to the drawings in conjunction with the embodiments.
Embodiment
It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, it is not intended to limit the present invention.
The solution of the embodiment of the present invention is mainly：The service end of document handling system first passes through client and receives transmission
Fang Shangchuan file, then by the file cache received into temporary folder, and recorded respectively in distributed memory system
The storage location information of individual file, then processing is merged to each file in temporary folder, the file after being merged,
And the file after merging is stored into distributed file system, the file being based ultimately upon after merging, renewal distributed storage system
Corresponding storage location information in system, it is easy to subsequently read file according to the storage location information.To solve existing file
Storage mode, it is difficult to the problem of meeting storage demand.
It should be noted that existing document storage mode, also stores following several point defects：
File storage scheme does not have life cycle management function, does not support the functions such as the expired deletion of temporary file, easily leads
Cause data storage excessive；
Be not suitable for the access of a large amount of systems, installation and deployment are relatively troublesome.
The problem of being existed based on prior art, the present invention build a FPS (File Process System, file process
System), the FPS can support the storage of mass data, while store more parts of scheme across frame, computer room to ensure using data
The high availability of service.Its main application scenarios includes：
(1) exchange files that a halfpace is supplied between different system are provided, such as A systems pass through halfpace offer pair
Account file carries out reconciliation to B system；
(2) data storing platform based on file life cycle management is provided, mass data storage, and text can be supported
Part storage a period of time, the needs that expire are automatically deleted.
The technical term introduction of the present invention：
Hadoop：It is a distributed system architecture, can allows user's framework and the Distributed Computing Platform that uses,
User can develop and run the application program of processing mass data on Hadoop.
HDFS：Distributed file system (Hadoop Distributed File System).HDFS has high fault tolerance
Feature, and be designed to be deployed on cheap (low-cost) hardware；And it provides high-throughput (high
Throughput) carry out the data of access application, be adapted to those to have super large data set (large data set) application
Program.
HBase：It is a high reliability, high-performance, towards row, telescopic distributed memory system, utilizes HBase skills
Art can erect large-scale structure storage cluster on cheap PC Server.Belong to the Hadoop ecospheres.
Zookeeper：It it is one distributed, the distributed application program coordination service of open source code, is Google
Mono- realization increased income of Chubby, it is Hadoop and Hbase significant components.It is one and provides uniformity for Distributed Application
The software of service, there is provided function include：Configuring maintenance, domain name service, distributed synchronization, group service etc..Belong to Hadoop lifes
State circle.
TGW：Full name Tencent GateWay, be it is a set of realize more nets be uniformly accessed into, outer net network request forwarding, support
The system of automatic load balancing, TGW can be described as gateway.
NAS：Network attached storage (Network Attached Storage), is to be connected on network, possesses data and deposit
The device of function is stored up, therefore also referred to as " network memory ".It is a kind of exclusive data storage server.
RMB：Message bus system, for the RPC messenger services between multisystem.
As shown in figure 1, Fig. 1 is the service end structural representation for the hardware running environment that scheme of the embodiment of the present invention is related to.
Service end of the embodiment of the present invention can be PC (personal computer, personal computer) or flat board
Computer, pocket computer etc. have the terminal device of display function.
As shown in figure 1, the service end can include：Processor 1001, such as CPU, communication bus 1002, user interface
1003, network interface 1004, memory 1005.Wherein, communication bus 1002 is used to realize the connection communication between these components.
User interface 1003 can include display screen (Display), input block such as keyboard (Keyboard), optional user interface
1003 can also include the wireline interface (such as connecting wired keyboard, wire mouse etc.) of standard, wave point (such as with
In connection Wireless Keyboard, wireless mouse).The wireline interface that network interface 1004 can optionally include standard (is used to be connected with
Gauze network), wave point (such as WI-FI interfaces, blue tooth interface, infrared interface, for connecting wireless network).Memory
1005 can be high-speed RAM memory or stable memory (non-volatile memory), such as disk is deposited
Reservoir.Memory 1005 optionally can also be the storage device independently of aforementioned processor 1001.
Alternatively, service end can also include camera, RF (Radio Frequency, radio frequency) circuit, sensor, sound
Frequency circuit, WiFi module etc..
It will be understood by those skilled in the art that the service end structure shown in Fig. 1 does not form the restriction to service end, can
With including than illustrating more or less parts, either combining some parts or different parts arrangement.
As shown in figure 1, as in a kind of memory 1005 of computer-readable recording medium can include operating system, net
Network communication module, Subscriber Interface Module SIM and file storage program.Wherein, operating system is management and control service end and software
The program of resource, support network communication module, Subscriber Interface Module SIM, file storage program and other programs or the fortune of software
OK；Network communication module is used to managing and controlling network interface 1002；Subscriber Interface Module SIM is used to managing and controlling user interface
1003。
In the service end shown in Fig. 1, network interface 1004 is mainly used in connecting the service end of standby Internet data center,
Enter row data communication with the service end of standby Internet data center；User interface 1003 is mainly used in connecting client (user
End), enter row data communication with client；The service end calls the file stored in memory 1005 to deposit by processor 1001
Program is stored up, and performs following steps：
The file of sender's upload is received by client；
By the file cache received into temporary folder, and depositing for each file is recorded in distributed memory system
Store up positional information；
Processing is merged to each file in temporary folder, the file after being merged, and by the text after merging
Part is stored into distributed file system；
Based on the file after merging, corresponding storage location information in distributed memory system is updated.
Further, the service end calls the file storage program stored in memory 1005 by processor 1001,
Processing is merged to each file in temporary folder to realize, the step of file after being merged：
Scan each file in the temporary folder；
Composition file is obtained, and determines that the capability value after merging with the composition file is less than in the file of scanning and presets
The file of threshold values, by the Piece file mergence of determination into the composition file.
Further, the service end calls the file storage program stored in memory 1005 by processor 1001,
To realize following steps：
When receiving file polling instruction, index information corresponding to file polling instruction is determined；
The merging file pointed by the index information is searched in distributed file system；
The file that merged is reduced, with from merged restored in file corresponding to the index information it is literary
Part.
Further, the file by after merging is stored to after the step in distributed file system, the service
The file storage program stored in memory 1005 is called in end by processor 1001, to realize following steps：
Based on the file stored in distributed file system, file identification information and file Hash information are generated；
By the client feedback file identification information and file Hash information to described sender, for the transmission
Side is transmitted to recipient by file identification information and file Hash information；
When receiving the file identification information that the recipient sends by the client, in distributed file system
File corresponding to the file identification information is extracted, and feeds back to the recipient, so that the recipient passes through file Hash
Information examines the file, and the file is obtained when examining successfully.
Further, the number of the service end includes multiple, and the service end and client of the document handling system are led to
Gateway connection is crossed, the mode that file is uploaded to service end from client includes：Gateway uploads client according to default strategy
File poll be uploaded in service end.
Further, the file based on after merging, corresponding storage location information in distributed memory system is updated
The step of after, the service end calls the file storage program that stores in memory 1005 by processor 1001, to realize
Following steps：
Each file in distributed file system is scanned, to monitor the storage duration of each file；
When documentary storage duration reaches preset duration, the file in the distributed file system is deleted,
And delete the storage location information of file described in the distributed memory system.
Further, the Internet data center also includes distributed application program coordination service, and the service end is swept
Retouch each file in distributed file system, the step of to monitor the storage duration of each file before, the service end is led to
Cross processor 1001 and call the file storage program stored in memory 1005, to realize following steps：
The solicited message for deleting lock is sent to distributed application program coordination service；
When acquisition is locked successfully, each file in scanning distributed file system is performed, to monitor depositing for each file
The step of storing up duration.
Further, the service end is located in main Internet data center, exists in systems in standby internet data
In the case of the heart, the file based on after merging, update distributed memory system in corresponding storage location information the step of
Afterwards, the service end calls the file storage program stored in memory 1005 by processor 1001, to realize following walk
Suddenly：
It is mutual with incense by the file synchronization of storage into the service end for document handling system where Internet data center
The service end of document handling system performs file storage operations where networking data center.
Based on the hardware configuration of above-mentioned service end, each embodiment of file memory method of the present invention is proposed.
Reference picture 2, Fig. 2 are the schematic flow sheet of file memory method first embodiment of the present invention.
In the present embodiment, the file memory method is applied to Internet data center, the Internet data center
Including document handling system, distributed file system, distributed memory system, the document handling system includes service end and visitor
Family end, the file memory method include：
The service end of document handling system receives the file of sender's upload by client；The file cache that will be received
Into temporary folder, and record in distributed memory system the storage location information of each file；To in temporary folder
Each file merge processing, the file after being merged, and by after merging file storage arrive distributed file system
In；Based on the file after merging, corresponding storage location information in distributed memory system is updated.
In the present embodiment, the file memory method is applied to IDC (Internet Data Center, interconnection netting index
According to center) where in service end corresponding to document handling system FPS, the service end is chosen as the service end shown in Fig. 2.
It should be noted that the embodiment of the present invention sets FPS (File Process System, document handling system), should
FPS is that the small documents of independent research store the functions such as processing system, specific life cycle management, across computer room disaster tolerance.
During the present invention is implemented, the structure chart of the IDC can refer to Fig. 3：
IDC includes document handling system FPS, distributed file system HDFS, distributed memory system Hbase, wherein,
FPS mainly includes two parts：FPS-Client (client) and FPS-Server (service end).Business procedure is by integrating FPS-
Client come use FPS provide function.
FPS-Client：FPS externally provides java versions, c language versions, the client of python versions.
FPS-Server：FPS background server program, there is provided main business logic processing, including control of authority,
The functions such as the storage and reading, the life cycle management of file of file.
As can be seen from Figure 3, the service end of the document handling system is connected with client by gateway, and the gateway is used
TGW is represented, when the TGW (Tencent Gataway) is used for file forwarding, does load balancing.
It should be noted that FPS service end and the number of client do not limit, can be set according to actual conditions individual
Number, the service end number include it is multiple in the case of, the mode that file is uploaded to service end from client includes：Gateway
According to default strategy, the file poll that client uploads is uploaded in service end.
That is, by gateway, i.e. all flows that FPS-Client is uploaded files in FPS-Server can all pass through TGW,
File poll is sent to by each FPS-Server by TGW.After FPS-Server receives file, file content is stored in
In HDFS (distributed file system), while by the storage location information record of file into HBASE (distributed memory system).
It should be noted that HDFS is a distributed file system, using PC as storing, generic-document has multiple
Backup, such as three backups, it very easily can increase machine by dynamic linear, increase in face of Internet service exponential type, can be very
Easily accomplish not shut down dilatation, can be very good meet the needs of business.Meanwhile the storage location information of file is stored in
On HBase, the storage of hundred billion grades of file indexes can be provided, HBase as HDFS, belong to the Hadoop ecospheres into
Member, easily can also increase storage performance by increasing node.
In embodiments of the present invention, FPS-Client passes through HTTP (Hyper Text with the interaction between FPS-Server
Transfer Protocol, HTTP) agreement progress.
It is the specific steps that file storage is realized in the present embodiment below：
Step S10, the service end of document handling system receive the file of sender's upload by client；
That is, service end receives the file that client is uploaded by TGW.
Step S20, by the file cache received into temporary folder, and record in distributed memory system each
The storage location information of file；
In the present embodiment, as shown in figure 3, FPS-Server receives the file of sender's transmission by FPS-Client,
After FPS-Server receives file, first by the file cache received into temporary folder, the temporary folder is excellent
HDFS assigned catalogue is elected as, after FPS-Server is by file cache to temporary folder, first in distributed storage file
The middle storage location information for recording each file.
Step S30, processing is merged to each file in temporary folder, the file after being merged, and will close
File after and is stored into distributed file system；
Specifically, described " processing being merged to each file in temporary folder, the file after being merged "
Step includes：
Step 1, the service end scans the catalogue in the temporary folder, to obtain lock corresponding to catalogue；
Step 2, in the case where getting lock, the service end scans each file in the temporary folder；
Step 3, composition file is obtained, and determines that the capability value after merging with the composition file is small in the file of scanning
In the file of pre-set threshold value, by the Piece file mergence of determination into the composition file.
Further, after step 3, methods described also includes：
Step 4, the file being merged in the temporary folder is deleted.
That is, after file cache to the temporary folder of upload, FPS-Server is scanned in the temporary folder
Catalogue, to obtain lock corresponding to catalogue, if lock can be got, the FPS-Server scans each file in the temporary folder
Capability value, and preferred pair merge with the composition file after capability value be less than predetermined threshold value file merge processing,
In the present embodiment, the predetermined threshold value is set according to actual conditions, is not limited herein.The merging mode of file is：Obtain pre-
If composition file, scan each file in the temporary folder, in the file of scanning determine with the composition file
Capability value after merging is less than the file of predetermined threshold value, and by the Piece file mergence of determination into composition file, to Piece file mergence
Afterwards, you can the file after merging is stored into distributed file system HDFS, it is distributed further according to the file after merging, renewal
Index information corresponding to the file merged in storage system.
Can be each file in periodic scanning temporary folder in the embodiment of the present invention, or real time scan text
After each file in part folder, then the small documents to scanning merge processing, and renewal small documents merge in HBase
Storage location information.Small documents are merged with processing by FPS-Server so that distributed file system is arrived in follow-up storage
In file will not be too scattered, the space shared by storage file can be reduced, can substantially save the memory headroom of clustered node.
To be best understood from the present embodiment, reference picture 4, first, FPS-Server are first scanned to HDFS temp directory,
Then obtain corresponding to catalogue and lock, in the case where getting lock, each file under scanning directory, then in distributed storage
Small documents index information is obtained in system, then obtains composition file, small documents are merged into composition file, final updated index
Information simultaneously deletes the original small documents in distributed file system respectively.
Step S40, based on the file after merging, update corresponding storage location information in distributed memory system.
After the file after being merged, FPS-Server updates distributed memory system based on the file after merging,
To update storage location information corresponding to the file of merging in distributed memory system, that is to say, that FPS-Server will be closed
Storage location information updating of the file in distributed file system after and is easy to subsequently search into distributed memory system
During file, according to the storage location information index to corresponding information.
In addition, in the embodiment of the present invention, methods described also includes：
Step A, when receiving file polling instruction, determine index information corresponding to file polling instruction；
Step B, the merging file pointed by the index information is searched in distributed file system；
Step C, the file that merged is reduced, so that from having merged, that the index information is restored in file is corresponding
File.
The technical scheme that the present embodiment proposes, the service end of document handling system first pass through client and receive sender's upload
File, then by the file cache received into temporary folder, and each file is recorded in distributed memory system
Storage location information, then processing is merged to each file in temporary folder, the file after being merged, and closing
File after and is stored into distributed file system, the file being based ultimately upon after merging, and it is right in distributed memory system to update
The storage location information answered, it is easy to subsequently read file according to the storage location information.In this programme, to the file received
Processing is merged, then the file after merging is stored into distributed file system, the merging of file causes system to store
The increase of file amount, further, since distributed file system has scalability, by distributed file system storage file,
Storable quantity of documents is more, and relative to existing document storage mode, the storable file amount of this programme is bigger, is more suitable for
Store substantial amounts of small documents.
Further, reference picture 5, the second embodiment of file memory method of the present invention is proposed based on first embodiment.
The difference of the second embodiment of file memory method and the first embodiment of file memory method is, the step
After S30, methods described also includes：
Step S50, the service end generate file identification information and text based on the file stored in distributed file system
Part Hash information；
Step S60, by the client feedback file identification information and file Hash information to described sender, for
Described sender transmits file identification information and file Hash information to recipient；
Step S70, when receiving the file identification information that the recipient sends by the client, in distributed text
File corresponding to the file identification information is extracted in part system, and feeds back to the recipient, so that the recipient passes through
File Hash information examines the file, and the file is obtained when examining successfully.
In the present embodiment, after FPS-Server storage files in distributed file system HDFS, according to this
The file of HDFS storages, generates file identification information (File Id) and file Hash information (File corresponding to this document
Hash), after File Id and the File Hash is obtained, FPS-Server feeds back File Id and the File Hash to transmission
Side, so that sender transmits File Id and the File Hash to recipient.
It should be noted that interacted between sender and recipient by RMB messenger service buses.When recipient connects
After receiving File Id and the File Hash, using FPS-Client, File Id are sent to the FPS-Server.
When FPS-Server receives the File Id of recipient transmission by the FPS-Client, in HDFS
File corresponding to the File Id is extracted, and feeds back to the recipient, so that the recipient is examined by File Hash
The file, and the file is obtained when examining successfully.That is, recipient is entered by File Id to FPS-Server
Row download, and by File Hash verify file accuracy, download with file authentication verification all in FPS-Client it is complete
Into.
To be best understood from the present embodiment, reference picture 6, file is uploaded to FPS by sender, and after file uploads successfully,
FPS can return to the File Id of sender's this document with File Hash, after sender receives, by File Id with File
Hash is sent to after recipient recipient receives message informing by RMB messenger service buses, using FPS FPS-Client,
The FPS-Server for sending File Id to FPS carries out the download of file, and file is verified by File Hash by FPS-Client
Accuracy, after verifying successfully, FPS-Client returns to the successful message of recipient's file download.
In the present embodiment, the file between sender and recipient is realized by each system in Internet data center
Transmission, and by the verification of file identification information and file Hash information progress file, improve the accuracy that file transmits.
Further, reference picture 7, the 3rd embodiment of file memory method of the present invention is proposed based on first embodiment.
The difference of the 3rd embodiment of file memory method and the first embodiment of file memory method is, the step
After S40, methods described includes：
Step S80, the service end scans each file in distributed file system, to monitor the storage of each file
Duration；
Step S90, when documentary storage duration reaches preset duration, delete the institute in the distributed file system
File is stated, and deletes the storage location information of file described in the distributed memory system.
In the present embodiment, the mode of each file in service end scanning distributed file system is preferably regularly swept
Retouch.Therefore, after distributed file system HDFS storage files, FPS-Server backstages can be periodically executed operations described below：It is fixed
Phase scans expired file and deletes file, to save disk space.Specifically：The FPS-Server scannings are distributed
Each file in file system HDFS, to monitor the storage duration of each file, reach default in documentary storage duration
During duration, the preset duration is set according to actual conditions, is not limited, if the preset duration is 3 months.When the storage of file
When duration reaches preset duration, illustrate that the storage time of this document is longer, in order to realize the life cycle management of file storage, delete
Except the file in the distributed file system, and delete the storage location of file described in the distributed memory system
Information.
In the present embodiment, the Internet data center also includes distributed application program coordination service, the step
Before S80, methods described also includes：
The service end sends the solicited message for deleting lock to distributed application program coordination service；
When acquisition is locked successfully, each file in scanning distributed file system is performed, to monitor depositing for each file
The step of storing up duration.
As shown in figure 3, the application program coordination service is represented with Zookeeper.Before file is deleted, FPS-
Server first sends to Zookeeper and obtains the solicited message for deleting lock, if lock can successfully be got, just performs step S80.
To be best understood from the present invention, reference picture 8, FPS-Server timings please to Zookeeper transmission acquisition deletion locks
Information is sought, if lock can be got, i.e. FPS-Server, which is obtained, to be locked successfully, and now, FPS-Server is to distributed memory system
Hbase asks expired data, i.e. FPS-Server reaches to Hbase request storage durations to be deposited corresponding to the file of preset duration
Positional information is stored up, after request is arrived, deletes the storage location information that the request is arrived.Subsequently, FPS-Server is to Zookeeper
Acquisition request deletes lock, and after lock is got, when deletion storage duration reaches default in distributed file system HDFS
Long file.
In the present embodiment, by being timed deletion to expired data so that this document storage has life cycle,
Scratch file can be regularly deleted, prevents that file amount is excessive, improves the intelligent of file storage.
Further, the fourth embodiment of file memory method of the present invention is proposed based on first embodiment.
The difference of the fourth embodiment of file memory method and the first of file memory method to 3rd embodiment is, institute
Service end is stated to be located in main Internet data center, in the case of standby Internet data center being in systems present, the step
After S40, methods described includes：
Step D, the service end is by the file synchronization of storage to for document handling system where Internet data center
In service end, file storage operations are performed with the service end of document handling system where incense Internet data center.
In the present embodiment, the Internet data center IDC of deployment includes more sets, such as includes Liang Tao Internet data centers
IDC, is main IDC and by IDC respectively, network interworking between two IDC.Operation system is connected to by integrated FPS-Client
Transmitting file in TGW requests, certain FPS-Server that TGW is routed requests in main IDC according to specified strategy, afterwards formally
Start upper transmitting file.During file is temporarily stored in by FPS-Server, while the positional information of file is stored in HBASE.
After the file in main IDC uploads successfully, FPS-Server in main IDC can be by this document asynchronous upload to standby
In FPS-Server in IDC, the file consistence in two clusters is ensured with this, FPS active and standby synchronization is using patrolling
Collect backup.
In the present embodiment, by the backup of file, in the case of main IDC failures, can continue to provide clothes by standby IDC
Business, does not influence the storage and use of file, availability is higher.
In addition, the embodiment of the present invention also proposes a kind of computer-readable recording medium, the computer-readable recording medium
On be stored with file storage program, following operation is realized when the file storage program is executed by processor：
The file of sender's upload is received by client；
By the file cache received into temporary folder, and depositing for each file is recorded in distributed memory system
Store up positional information；
Processing is merged to each file in temporary folder, the file after being merged, and by the text after merging
Part is stored into distributed file system；
Based on the file after merging, corresponding storage location information in distributed memory system is updated.
Further, when the file storage program is executed by processor, also realize to each text in temporary folder
Part merges processing, the operation of the file after being merged：
Scan each file in the temporary folder；
Composition file is obtained, and determines that the capability value after merging with the composition file is less than in the file of scanning and presets
The file of threshold values, by the Piece file mergence of determination into the composition file.
Further, when the file storage program is executed by processor, following operate also is realized：
When receiving file polling instruction, index information corresponding to file polling instruction is determined；
The merging file pointed by the index information is searched in distributed file system；
The file that merged is reduced, with from merged restored in file corresponding to the index information it is literary
Part.
Further, the file by after merging is stored to after the step in distributed file system, the file
When storage program is executed by processor, following operate also is realized：
Based on the file stored in distributed file system, file identification information and file Hash information are generated；
By the client feedback file identification information and file Hash information to described sender, for the transmission
Side is transmitted to recipient by file identification information and file Hash information；
When receiving the file identification information that the recipient sends by the client, in distributed file system
File corresponding to the file identification information is extracted, and feeds back to the recipient, so that the recipient passes through file Hash
Information examines the file, and the file is obtained when examining successfully.
Further, the number of the service end includes multiple, and the service end and client of the document handling system are led to
Gateway connection is crossed, the mode that file is uploaded to service end from client includes：Gateway uploads client according to default strategy
File poll be uploaded in service end.
Further, the file based on after merging, corresponding storage location information in distributed memory system is updated
The step of after, when the file storage program is executed by processor, also realize following operate：
Each file in distributed file system is scanned, to monitor the storage duration of each file；
When documentary storage duration reaches preset duration, the file in the distributed file system is deleted,
And delete the storage location information of file described in the distributed memory system.
Further, the Internet data center also includes distributed application program coordination service, and the service end is swept
Retouch each file in distributed file system, the step of to monitor the storage duration of each file before, the file storage
When program is executed by processor, following operate also is realized：
The solicited message for deleting lock is sent to distributed application program coordination service；
When acquisition is locked successfully, each file in scanning distributed file system is performed, to monitor depositing for each file
The step of storing up duration.
Further, the service end is located in main Internet data center, exists in systems in standby internet data
In the case of the heart, the file based on after merging, update distributed memory system in corresponding storage location information the step of
Afterwards, when the file storage program is executed by processor, following operate also is realized：
It is mutual with incense by the file synchronization of storage into the service end for document handling system where Internet data center
The service end of document handling system performs file storage operations where networking data center.
It should be noted that herein, term " comprising ", "comprising" or its any other variant are intended to non-row
His property includes, so that process, method, article or device including a series of elements not only include those key elements, and
And also include the other key elements being not expressly set out, or also include for this process, method, article or device institute inherently
Key element.In the absence of more restrictions, the key element limited by sentence "including a ...", it is not excluded that including this
Other identical element also be present in the process of key element, method, article or device.
The embodiments of the present invention are for illustration only, do not represent the quality of embodiment.
Through the above description of the embodiments, those skilled in the art can be understood that above-described embodiment side
Method can add the mode of required general hardware platform to realize by software, naturally it is also possible to by hardware, but in many cases
The former is more preferably embodiment.Based on such understanding, technical scheme is substantially done to prior art in other words
Going out the part of contribution can be embodied in the form of software product, and the computer software product is stored in a storage medium
In (such as ROM/RAM, magnetic disc, CD), including some instructions to cause a station terminal equipment (can be mobile phone, computer, clothes
Be engaged in device, air conditioner, or network equipment etc.) perform method described in each embodiment of the present invention.
The preferred embodiments of the present invention are these are only, are not intended to limit the scope of the invention, it is every to utilize this hair
The equivalent structure or equivalent flow conversion that bright specification and accompanying drawing content are made, or directly or indirectly it is used in other related skills
Art field, is included within the scope of the present invention.
Claims (10)
1. a kind of file memory method, it is characterised in that applied to Internet data center, the Internet data center includes
Document handling system, distributed file system, distributed memory system, the document handling system include service end and client
End, the file memory method include：
The service end of document handling system receives the file of sender's upload by client；
By the file cache received into temporary folder, and record in distributed memory system the storage position of each file
Confidence ceases；
Processing is merged to each file in temporary folder, the file after being merged, and the file after merging is deposited
Store up in distributed file system；
Based on the file after merging, corresponding storage location information in distributed memory system is updated.
2. file memory method as claimed in claim 1, it is characterised in that each file in temporary folder enters
Row merging treatment, include the step of file after being merged：
The service end scans each file in the temporary folder；
Composition file is obtained, and determines that the capability value after merging with the composition file is less than pre-set threshold value in the file of scanning
File, by the Piece file mergence of determination into the composition file.
3. file memory method as claimed in claim 2, it is characterised in that methods described also includes：
When receiving file polling instruction, index information corresponding to file polling instruction is determined；
The merging file pointed by the index information is searched in distributed file system；
The file that merged is reduced, to restore file corresponding to the index information in file from having merged.
4. file memory method as claimed in claim 1, it is characterised in that the file storage by after merging to distribution
After step in file system, methods described also includes：
The service end generates file identification information and file Hash information based on the file stored in distributed file system；
By the client feedback file identification information and file Hash information to described sender, so that described sender will
File identification information and file Hash information are transmitted to recipient；
When receiving the file identification information that the recipient sends by the client, extracted in distributed file system
File corresponding to the file identification information, and the recipient is fed back to, so that the recipient passes through file Hash information
The file is examined, and the file is obtained when examining successfully.
5. file memory method as claimed in claim 1, it is characterised in that the number of the service end is described including multiple
The service end of document handling system is connected with client by gateway, and the mode that file is uploaded to service end from client includes：
The file poll that client uploads is uploaded in service end by gateway according to default strategy.
6. file memory method as claimed in claim 1, it is characterised in that the file based on after merging, renewal distribution
In formula storage system the step of corresponding storage location information after, methods described also includes：
Each file in the service end scanning distributed file system, to monitor the storage duration of each file；
When documentary storage duration reaches preset duration, the file in the distributed file system is deleted, and delete
Except the storage location information of file described in the distributed memory system.
7. file memory method as claimed in claim 6, it is characterised in that the Internet data center also includes distribution
Application program coordination service, the service end scans each file in distributed file system, to monitor depositing for each file
Before the step of storing up duration, methods described also includes：
The service end sends the solicited message for deleting lock to distributed application program coordination service；
Obtain lock successfully when, execution scans each file in distributed file system, during monitoring the storage of each file
Long step.
8. the file memory method as described in claim any one of 1-7, it is characterised in that the service end is located at main internet
In data center, in the case of standby Internet data center being in systems present, the file based on after merging, renewal distribution
In formula storage system the step of corresponding storage location information after, methods described includes：
The service end by the service end of document handling system where the file synchronization of storage to standby Internet data center, with
The service end of document handling system performs file storage operations where incense Internet data center.
9. a kind of service end, it is characterised in that the service end includes memory, processor and is stored on the memory simultaneously
The file storage program that can be run on the processor, the file storage program are realized as weighed during the computing device
Profit requires the step of file memory method any one of 1 to 8.
10. a kind of computer-readable recording medium, it is characterised in that be stored with file on the computer-readable recording medium and deposit
Program is stored up, the file storage as any one of claim 1 to 8 is realized when the file storage program is executed by processor
The step of method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710885384.3A CN107861686B (en) | 2017-09-26 | 2017-09-26 | File storage method, server and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710885384.3A CN107861686B (en) | 2017-09-26 | 2017-09-26 | File storage method, server and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107861686A true CN107861686A (en) | 2018-03-30 |
| CN107861686B CN107861686B (en) | 2021-01-05 |
Family
ID=61698675
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710885384.3A Active CN107861686B (en) | 2017-09-26 | 2017-09-26 | File storage method, server and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107861686B (en) |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108768948A (en) * | 2018-04-28 | 2018-11-06 | 努比亚技术有限公司 | A kind of access rights management method, server and computer readable storage medium |
| CN109033345A (en) * | 2018-07-24 | 2018-12-18 | 中国科学技术大学 | Distributed data for large scientific facilities control system achieves method for parameter configuration |
| CN109582684A (en) * | 2018-11-30 | 2019-04-05 | 深圳市盟天科技有限公司 | A kind of update method of local cache data, device, server and storage medium |
| CN109597578A (en) * | 2018-12-03 | 2019-04-09 | 郑州云海信息技术有限公司 | A kind of date storage method, system, equipment and computer readable storage medium |
| CN110012050A (en) * | 2018-12-04 | 2019-07-12 | 阿里巴巴集团控股有限公司 | Message Processing, storage method, apparatus and system |
| CN110491478A (en) * | 2019-08-22 | 2019-11-22 | 中电健康云科技有限公司 | A kind of image file distributed storage system and its implementation based on ceph |
| CN110795403A (en) * | 2019-10-31 | 2020-02-14 | 北京永亚普信科技有限责任公司 | File arrival scanning optimization method for polling mechanism |
| CN110888837A (en) * | 2019-11-15 | 2020-03-17 | 星辰天合（北京）数据科技有限公司 | Object storage small file merging method and device |
| CN111444160A (en) * | 2020-03-31 | 2020-07-24 | 南京领行科技股份有限公司 | Data storage system and uploading and storing method, device, equipment and medium |
| CN111737052A (en) * | 2020-06-19 | 2020-10-02 | 中国工商银行股份有限公司 | Distributed object storage system and method |
| CN112035057A (en) * | 2020-07-24 | 2020-12-04 | 武汉达梦数据库有限公司 | Hive file merging method and device |
| CN112231293A (en) * | 2020-09-14 | 2021-01-15 | 杭州数梦工场科技有限公司 | File reading method and device, electronic equipment and storage medium |
| WO2021031905A1 (en) * | 2019-08-19 | 2021-02-25 | 深圳前海微众银行股份有限公司 | Data management method, apparatus, device and system and computer-readable storage medium |
| CN112463191A (en) * | 2020-11-26 | 2021-03-09 | 北京沃东天骏信息技术有限公司 | File updating method and device, equipment and storage medium |
| CN112738157A (en) * | 2020-12-11 | 2021-04-30 | 苏州浪潮智能科技有限公司 | File uploading method, system and equipment |
| CN112905557A (en) * | 2021-03-03 | 2021-06-04 | 山东兆物网络技术股份有限公司 | Mass file integration storage method and system supporting asynchronous submission |
| CN112948327A (en) * | 2021-04-01 | 2021-06-11 | 北京奇艺世纪科技有限公司 | File processing method, system, electronic device and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN201726424U (en) * | 2009-08-18 | 2011-01-26 | 升东网络科技发展(上海)有限公司 | Distributed storage system |
| CN103353892A (en) * | 2013-07-05 | 2013-10-16 | 北京东方网信科技股份有限公司 | Method and system for data cleaning suitable for mass storage |
| CN104820717A (en) * | 2015-05-22 | 2015-08-05 | 国网智能电网研究院 | Massive small file storage and management method and system |
| CN105138571A (en) * | 2015-07-24 | 2015-12-09 | 四川长虹电器股份有限公司 | Distributed file system and method for storing lots of small files |
| CN105404652A (en) * | 2015-10-29 | 2016-03-16 | 河海大学 | Mass small file processing method based on HDFS |
| CN105468686A (en) * | 2015-11-17 | 2016-04-06 | 北京奇虎科技有限公司 | Method and device for reducing redundant data |
| CN105956183A (en) * | 2016-05-30 | 2016-09-21 | 广东电网有限责任公司电力调度控制中心 | Method and system for multi-stage optimization storage of a lot of small files in distributed database |
-
2017
- 2017-09-26 CN CN201710885384.3A patent/CN107861686B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN201726424U (en) * | 2009-08-18 | 2011-01-26 | 升东网络科技发展(上海)有限公司 | Distributed storage system |
| CN103353892A (en) * | 2013-07-05 | 2013-10-16 | 北京东方网信科技股份有限公司 | Method and system for data cleaning suitable for mass storage |
| CN104820717A (en) * | 2015-05-22 | 2015-08-05 | 国网智能电网研究院 | Massive small file storage and management method and system |
| CN105138571A (en) * | 2015-07-24 | 2015-12-09 | 四川长虹电器股份有限公司 | Distributed file system and method for storing lots of small files |
| CN105404652A (en) * | 2015-10-29 | 2016-03-16 | 河海大学 | Mass small file processing method based on HDFS |
| CN105468686A (en) * | 2015-11-17 | 2016-04-06 | 北京奇虎科技有限公司 | Method and device for reducing redundant data |
| CN105956183A (en) * | 2016-05-30 | 2016-09-21 | 广东电网有限责任公司电力调度控制中心 | Method and system for multi-stage optimization storage of a lot of small files in distributed database |
Cited By (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108768948A (en) * | 2018-04-28 | 2018-11-06 | 努比亚技术有限公司 | A kind of access rights management method, server and computer readable storage medium |
| CN108768948B (en) * | 2018-04-28 | 2021-04-16 | 努比亚技术有限公司 | Access right management method, server and computer readable storage medium |
| CN109033345A (en) * | 2018-07-24 | 2018-12-18 | 中国科学技术大学 | Distributed data for large scientific facilities control system achieves method for parameter configuration |
| CN109582684A (en) * | 2018-11-30 | 2019-04-05 | 深圳市盟天科技有限公司 | A kind of update method of local cache data, device, server and storage medium |
| CN109597578A (en) * | 2018-12-03 | 2019-04-09 | 郑州云海信息技术有限公司 | A kind of date storage method, system, equipment and computer readable storage medium |
| CN110012050A (en) * | 2018-12-04 | 2019-07-12 | 阿里巴巴集团控股有限公司 | Message Processing, storage method, apparatus and system |
| WO2021031905A1 (en) * | 2019-08-19 | 2021-02-25 | 深圳前海微众银行股份有限公司 | Data management method, apparatus, device and system and computer-readable storage medium |
| CN110491478A (en) * | 2019-08-22 | 2019-11-22 | 中电健康云科技有限公司 | A kind of image file distributed storage system and its implementation based on ceph |
| CN110795403A (en) * | 2019-10-31 | 2020-02-14 | 北京永亚普信科技有限责任公司 | File arrival scanning optimization method for polling mechanism |
| CN110795403B (en) * | 2019-10-31 | 2022-03-11 | 北京永亚普信科技有限责任公司 | File arrival scanning optimization method for polling mechanism |
| CN110888837B (en) * | 2019-11-15 | 2021-01-22 | 星辰天合（北京）数据科技有限公司 | Object storage small file merging method and device |
| CN110888837A (en) * | 2019-11-15 | 2020-03-17 | 星辰天合（北京）数据科技有限公司 | Object storage small file merging method and device |
| CN111444160A (en) * | 2020-03-31 | 2020-07-24 | 南京领行科技股份有限公司 | Data storage system and uploading and storing method, device, equipment and medium |
| CN111444160B (en) * | 2020-03-31 | 2022-06-07 | 南京领行科技股份有限公司 | Data storage system and uploading and storing method, device, equipment and medium |
| CN111737052A (en) * | 2020-06-19 | 2020-10-02 | 中国工商银行股份有限公司 | Distributed object storage system and method |
| CN111737052B (en) * | 2020-06-19 | 2023-07-07 | 中国工商银行股份有限公司 | Distributed object storage system and method |
| CN112035057A (en) * | 2020-07-24 | 2020-12-04 | 武汉达梦数据库有限公司 | Hive file merging method and device |
| CN112035057B (en) * | 2020-07-24 | 2022-06-21 | 武汉达梦数据库股份有限公司 | Hive file merging method and device |
| CN112231293A (en) * | 2020-09-14 | 2021-01-15 | 杭州数梦工场科技有限公司 | File reading method and device, electronic equipment and storage medium |
| CN112463191A (en) * | 2020-11-26 | 2021-03-09 | 北京沃东天骏信息技术有限公司 | File updating method and device, equipment and storage medium |
| CN112738157A (en) * | 2020-12-11 | 2021-04-30 | 苏州浪潮智能科技有限公司 | File uploading method, system and equipment |
| CN112905557A (en) * | 2021-03-03 | 2021-06-04 | 山东兆物网络技术股份有限公司 | Mass file integration storage method and system supporting asynchronous submission |
| CN112905557B (en) * | 2021-03-03 | 2023-01-24 | 山东兆物网络技术股份有限公司 | Mass file integration storage method and system supporting asynchronous submission |
| CN112948327A (en) * | 2021-04-01 | 2021-06-11 | 北京奇艺世纪科技有限公司 | File processing method, system, electronic device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107861686B (en) | 2021-01-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107861686A (en) | File memory method, service end and computer-readable recording medium | |
| US11216418B2 (en) | Method for seamless access to a cloud storage system by an endpoint device using metadata | |
| US10423496B2 (en) | Deduplicated data distribution techniques | |
| EP2706719B1 (en) | File synchronization method and device | |
| KR102095436B1 (en) | Request handling at the service layer | |
| US8931034B2 (en) | System, method, and policy engine for granting temporary access to electronic content | |
| US7698392B2 (en) | Method and system for establishing a user-friendly data transfer service application executing within a heterogeneous distributed service application execution environment | |
| US8463936B2 (en) | Method and device for distributing digital data in particular for a peer-to-peer network | |
| US7720920B2 (en) | Client side based data synchronization and storage | |
| CN113965560A (en) | Data transmission method, proxy server, storage medium, and electronic device | |
| US20090187622A1 (en) | Method, system and apparatus for data synchronization | |
| EP2466852A1 (en) | Digital content management | |
| US7970881B2 (en) | Bypassing uploading of data from a wireless device using outbound attachment caching | |
| US20130151468A1 (en) | File synchronization method and device | |
| US20080189294A1 (en) | Method and apparatus for sharing content | |
| CN101924785A (en) | Data uploading and downloading methods and system | |
| US11943260B2 (en) | Synthetic request injection to retrieve metadata for cloud policy enforcement | |
| CN103518216A (en) | Content sharing within private user group | |
| CN113342746A (en) | File management system, file management method, electronic device, and storage medium | |
| US20200257813A1 (en) | Digital Data Processing System For Efficiently Storing, Moving, And/Or Processing Data Across A Plurality of Computing Clusters | |
| CN102932443A (en) | HDFS (hadoop distributed file system) cluster based distributed cloud storage system | |
| CN103825922B (en) | A kind of data-updating method and web server | |
| Göndör et al. | The current state of interoperability in decentralized online social networking services | |
| CN110086831A (en) | Method for authenticating for gateway | |
| CN104580276B (en) | Information-pushing method, device, system and information access device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |