CN101764848A - Method and device for transmitting network files - Google Patents
Method and device for transmitting network files Download PDFInfo
- Publication number
- CN101764848A CN101764848A CN201010034025A CN201010034025A CN101764848A CN 101764848 A CN101764848 A CN 101764848A CN 201010034025 A CN201010034025 A CN 201010034025A CN 201010034025 A CN201010034025 A CN 201010034025A CN 101764848 A CN101764848 A CN 101764848A
- Authority
- CN
- China
- Prior art keywords
- network
- cluster
- network file
- files
- servers
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 29
- 230000015654 memory Effects 0.000 claims abstract description 49
- 230000008569 process Effects 0.000 claims description 5
- 230000004044 response Effects 0.000 claims description 4
- 230000003993 interaction Effects 0.000 claims description 3
- 230000005540 biological transmission Effects 0.000 abstract description 4
- 238000006467 substitution reaction Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention provides a method and a device for transmitting network files, relating to the field of computers and aiming to solve the problem of low transmission speed of the network files in the prior art. The method comprises the following steps of: receiving the request of a client to read the network file by a cache memory cluster, and then processing the request of the client according to the network file cached in the internal memory of the cluster in advance. The technical scheme of the invention can be applied to the field of data communication.
Description
Technical Field
The present invention relates to the field of computers, and in particular, to a method and an apparatus for transmitting a network file.
Background
NFS (Network File System) is an RPC (Remote Procedure Call) service that can achieve File sharing between different operating systems. When a user needs a remote file, the remote file system is hung under the file system of the user, and the remote file system is not similar to the file on a local machine.
The NFS protocol is suitable for applications between different machines, operating systems, network architectures, and transport protocols. This wide range of adaptability is obtained by using Remote Procedure Call (RPC) primitives built on top of external data descriptions (XDRs).
The NFS comprises an NFS Client terminal and an NFS Server terminal, namely a Client-Server architecture is adopted. The Client is mainly responsible for processing the operation request of the user to the remote file and sending the content of the request to the Server where the file is located from the network according to a certain packet format. And the Server receives the request of the Client, calls the function of the VFS of the local machine to perform the actual operation of the file, and returns the result to the Client according to a certain format. And after the Client obtains the return result of the Server, the Client sends the return result to the user.
In practical application, the traditional TCP/IP protocol and the low disk performance inevitably bring storage performance limitation to the NFS, so that the NFS is only suitable for being applied in a smaller network or a local area network. If the NFS is applied to an enterprise network, due to the bandwidth and disk performance of the enterprise network, it is likely that when multiple clients access the NAS file system, the performance of the NFS is greatly reduced, and finally the user's requirements cannot be met.
Disclosure of Invention
The invention provides a method and a device for transmitting network files, which can improve the transmission rate of the network files.
In order to achieve the purpose, the invention provides the following technical scheme:
a plurality of network files are cached in a memory of the cluster and used for information interaction between a client and a network file server.
Further, the cluster also has the following characteristics: the cluster comprises a plurality of cache servers configured with large memories, and the cache servers are connected through a high-speed network.
Further, the cluster also has the following characteristics: when receiving a read request of a network file sent by the client, the cache server has the following processing modes:
when the network file is cached in a local memory, the network file is obtained from the local memory and is sent to the client;
when the local memory does not have the network file, but the memories of other cache servers in the cluster sometimes inform the target cache server in the cluster, which caches the network file, of sending the network file
And when the network file does not exist in the memory of the cluster, acquiring the network file from the network file server and sending the network file to the client.
Further, the cluster also has the following characteristics: the cache server determines whether the network file exists in the memories of other servers in the cluster or not by adopting the following method:
inquiring the information recorded with the locally cached network files periodically reported by other servers in the cluster; or,
and receiving query responses returned by other servers in the cluster by querying whether the network files are cached in the other servers in the cluster.
Further, the cluster also has the following characteristics: the network files cached in the cache server comprise at least one of the following: network files with small data size, frequently accessed network files, recently written network files, and recently read network files.
A method of transferring network files, comprising:
the cache storage cluster receives a request of a client for reading a network file, and processes the request of the client according to the network file cached in the memory of the cluster in advance.
Further, the method also has the following characteristics: the cache storage cluster comprises a plurality of cache servers configured with large memories, and the cache servers are connected through a high-speed network.
Further, the method also has the following characteristics: when receiving a read request of a network file sent by the client, the cache server has the following processing modes:
when the network file is cached in a local memory, the network file is obtained from the local memory and is sent to the client;
when the local memory does not have the network file, but the memories of other cache servers in the cluster sometimes inform the target cache server in the cluster, which caches the network file, of sending the network file
And when the network file does not exist in the memory of the cluster, acquiring the network file from the network file server and sending the network file to the client.
Further, the method also has the following characteristics: the cache server determines whether the network file exists in the memories of other servers in the cluster or not by adopting the following method:
inquiring the information recorded with the locally cached network files periodically reported by other servers in the cluster; or,
and receiving query responses returned by other servers in the cluster by querying whether the network files are cached in the other servers in the cluster.
Further, the method also has the following characteristics: the network files cached in the cache server comprise at least one of the following: network files with small data size, frequently accessed network files, recently written network files, and recently read network files.
According to the technical scheme provided by the invention, the characteristic of high read/write speed of the memory is adopted, the memory of the cache storage cluster stores a plurality of network files in advance, the network files are provided for the client, the transmission speed of the network files is improved, and the access times of the client to the network file server are reduced.
Drawings
FIG. 1 is a schematic diagram of a network file system according to the present invention;
fig. 2 is a flowchart of a method for transmitting network files according to the present invention.
Detailed Description
The technical solutions provided by the embodiments of the present invention are further described below with reference to the accompanying drawings.
As shown in fig. 1, the present invention provides a network file system, which includes a client, a network file server, and a cache storage cluster, where a plurality of network files are cached in a memory of the cache storage cluster, and information interaction is performed between the client and the network file server.
The cache storage cluster comprises a plurality of cache servers, each cache server is configured with a large memory, and the servers are connected through a high-speed network, such as a 40Gbit/s high-speed infiniband network. The method comprises the following steps that a cache server in a cluster acquires a plurality of network files from a network file server in advance, wherein the files acquired by the cache server can be files of specific folders in the network file server, and the network files further have the following characteristics: small amount of data, frequent access, recent write, recent read.
According to the technical scheme provided by the invention, the characteristic of high read/write speed of the memory is adopted, the memory of the cache storage cluster stores a plurality of network files in advance, the network files are provided for the client, the transmission speed of the network files is improved, and the access times of the client to the network file server are reduced.
As shown in fig. 2, the present invention provides a method for transmitting a network file, including:
The following describes a process of reading network data by a client:
when a network file reading request sent by the client is received, whether a local memory caches a network file required by a user is judged, if yes, the network file is directly extracted from the memory of the server and sent to the client, the times of accessing the network file server by the client are reduced, and the processing pressure of the network file server is reduced.
If the network file does not exist in the memory of the server and the network files exist in the other servers, the cache server cached with the network file sends the network file to the client;
and if all the servers in the cluster do not have the network file, the server acquires the network file from the network file server and then sends the network file to the client.
After the user obtains the network file, if the user modifies the network file in the server through the client, the server obtains the disk performance of the current network file system, such as the current read/write rate, and sends the information written by the user to the network file server according to the current disk performance.
In the above process, if the network file does not exist in the memory of the server and the network file exists in the other servers, how to implement that the cache server caching the network file sends the network file to the client is described as follows:
example one
After the server acquires the network file from the network file server, the server generates a network file list for recording the network file in the memory of the server, wherein the network file list comprises the name of the network file and the address of the network file; and registering the network file list with other servers in the server cluster, so that other cache servers know which network files are stored in the server. When the network file in the memory of the server changes, such as adding or deleting the network file, the server needs to send the change information of the network file to other servers. Similarly, the operation of other servers in the cluster is the same as above, so that the server knows which network files are stored in all other servers in the cluster.
When the server does not have the network file required by the user, the server inquires whether other servers have target cache servers of the network file from a locally acquired network file list according to the name of the network file when the client accesses and the storage address of the network file, and if the other servers have the network file, the server informs the target cache servers to send the network file to the client.
And after receiving the notification sent by the server, the target cache server extracts the network file from the memory and sends the network file to the client.
Example two
When the server does not have the network file required by the user, the server queries whether the network file exists in other servers in the cluster, for example, sends a query request including the name and the storage address of the network file. If other servers in the cluster have the network file, the network file is notified, otherwise, the network file is not notified.
And the target cache server with the network file in the cluster can send the network file to the client immediately after receiving the query request.
If there are a plurality of target cache servers with the network file in the cluster, the network file is sent by adopting the above means, and the client receives a plurality of same network files. In order to avoid the phenomenon, the invention adopts the following technical scheme: and after receiving the query request, other servers in the cluster only send the query result. After the server receives the query results returned by other servers, a target cache server is determined from the target cache servers, the determined target cache server is informed to send the network file, and the network file is sent to the client side by only one target cache server.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware that is instructed to be associated with a program, which may be stored in a computer-readable storage medium, and which, when executed, includes one or a combination of the steps of the method embodiments.
In addition, each functional unit in each embodiment of the present invention may be implemented in a form of hardware, or may be implemented in a form of a software functional module. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.
The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. The cache storage cluster is characterized in that a plurality of network files are cached in a memory of the cluster and used for information interaction between a client and a network file server.
2. The cluster according to claim 1, wherein the cluster comprises a plurality of cache servers configured with large memories, and the cache servers are connected with each other through a high-speed network.
3. The cluster according to claim 1 or 2, wherein when receiving a read request of a network file sent by the client, the cache server has the following processing modes:
when the network file is cached in a local memory, the network file is obtained from the local memory and is sent to the client;
when the local memory does not have the network file, but the memories of other cache servers in the cluster sometimes inform the target cache server in the cluster, which caches the network file, of sending the network file
And when the network file does not exist in the memory of the cluster, acquiring the network file from the network file server and sending the network file to the client.
4. The cluster of claim 3, wherein the cache server determines whether the network file exists in the memories of the other servers in the cluster by:
inquiring the information recorded with the locally cached network files periodically reported by other servers in the cluster; or,
and receiving query responses returned by other servers in the cluster by querying whether the network files are cached in the other servers in the cluster.
5. The cluster of claim 4, wherein the network files cached in the cache server comprise at least one of: network files with small data size, frequently accessed network files, recently written network files, and recently read network files.
6. A method of transferring network files, comprising:
the cache storage cluster receives a request of a client for reading a network file, and processes the request of the client according to the network file cached in the memory of the cluster in advance.
7. The method of claim 6, wherein the cache storage cluster comprises a plurality of cache servers configured with large memories, and the cache servers are connected to each other through a high-speed network.
8. The method according to claim 6 or 7, wherein, when receiving a read request of the network file sent by the client, the cache server has the following processing modes:
when the network file is cached in a local memory, the network file is obtained from the local memory and is sent to the client;
when the local memory does not have the network file, but the memories of other cache servers in the cluster sometimes inform the target cache server in the cluster, which caches the network file, of sending the network file
And when the network file does not exist in the memory of the cluster, acquiring the network file from the network file server and sending the network file to the client.
9. The method of claim 8, wherein the cache server determines whether the network file exists in the memories of the other servers in the cluster by:
inquiring the information recorded with the locally cached network files periodically reported by other servers in the cluster; or,
and receiving query responses returned by other servers in the cluster by querying whether the network files are cached in the other servers in the cluster.
10. The method of claim 9, wherein the network files cached in the cache server comprise at least one of: network files with small data size, frequently accessed network files, recently written network files, and recently read network files.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201010034025A CN101764848A (en) | 2010-01-12 | 2010-01-12 | Method and device for transmitting network files |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201010034025A CN101764848A (en) | 2010-01-12 | 2010-01-12 | Method and device for transmitting network files |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN101764848A true CN101764848A (en) | 2010-06-30 |
Family
ID=42495828
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201010034025A Pending CN101764848A (en) | 2010-01-12 | 2010-01-12 | Method and device for transmitting network files |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101764848A (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102325055A (en) * | 2011-10-26 | 2012-01-18 | 迈普通信技术股份有限公司 | Simple network management protocol (SNMP) network management system and method for reducing repeated SNMP requests |
| CN103731396A (en) * | 2012-10-10 | 2014-04-16 | 中国移动通信集团江西有限公司 | Resource access method and system and cache resource information pushing device |
| WO2014067434A1 (en) * | 2012-10-31 | 2014-05-08 | 中国银联股份有限公司 | File transmission method and system thereof |
| CN104123341A (en) * | 2014-06-26 | 2014-10-29 | 小米科技有限责任公司 | Method and device for downloading file and route device |
| CN104935632A (en) * | 2015-04-24 | 2015-09-23 | 网宿科技股份有限公司 | Acceleration method and system based on CIFS protocols |
| CN105635196A (en) * | 2014-10-27 | 2016-06-01 | 中国电信股份有限公司 | Method and system of file data obtaining, and application server |
| CN106790434A (en) * | 2016-12-05 | 2017-05-31 | 上海爱数信息技术股份有限公司 | The management method of network data, network attached storage gateway and storage service system |
| WO2017092356A1 (en) * | 2015-12-01 | 2017-06-08 | 乐视控股（北京）有限公司 | Server, method and system for providing service data |
| CN107426288A (en) * | 2017-05-26 | 2017-12-01 | 郑州云海信息技术有限公司 | A kind of resource-sharing schedule method and apparatus based on storage network |
| CN112130747A (en) * | 2019-06-24 | 2020-12-25 | 北京金山云网络技术有限公司 | Distributed object storage system and data read-write method |
| CN112351109A (en) * | 2020-11-27 | 2021-02-09 | 中国农业银行股份有限公司 | Accessory processing method and device |
-
2010
- 2010-01-12 CN CN201010034025A patent/CN101764848A/en active Pending
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102325055A (en) * | 2011-10-26 | 2012-01-18 | 迈普通信技术股份有限公司 | Simple network management protocol (SNMP) network management system and method for reducing repeated SNMP requests |
| CN103731396A (en) * | 2012-10-10 | 2014-04-16 | 中国移动通信集团江西有限公司 | Resource access method and system and cache resource information pushing device |
| US11122108B2 (en) | 2012-10-31 | 2021-09-14 | China Unionpay Co., Ltd. | End-to-end file transmission method and system thereof |
| WO2014067434A1 (en) * | 2012-10-31 | 2014-05-08 | 中国银联股份有限公司 | File transmission method and system thereof |
| CN104123341A (en) * | 2014-06-26 | 2014-10-29 | 小米科技有限责任公司 | Method and device for downloading file and route device |
| CN105635196A (en) * | 2014-10-27 | 2016-06-01 | 中国电信股份有限公司 | Method and system of file data obtaining, and application server |
| CN104935632A (en) * | 2015-04-24 | 2015-09-23 | 网宿科技股份有限公司 | Acceleration method and system based on CIFS protocols |
| WO2017092356A1 (en) * | 2015-12-01 | 2017-06-08 | 乐视控股（北京）有限公司 | Server, method and system for providing service data |
| CN106790434A (en) * | 2016-12-05 | 2017-05-31 | 上海爱数信息技术股份有限公司 | The management method of network data, network attached storage gateway and storage service system |
| CN106790434B (en) * | 2016-12-05 | 2020-03-27 | 上海爱数信息技术股份有限公司 | Network data management method, network attached storage gateway and storage service system |
| CN107426288A (en) * | 2017-05-26 | 2017-12-01 | 郑州云海信息技术有限公司 | A kind of resource-sharing schedule method and apparatus based on storage network |
| CN112130747A (en) * | 2019-06-24 | 2020-12-25 | 北京金山云网络技术有限公司 | Distributed object storage system and data read-write method |
| CN112351109A (en) * | 2020-11-27 | 2021-02-09 | 中国农业银行股份有限公司 | Accessory processing method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101764848A (en) | Method and device for transmitting network files | |
| US10326853B2 (en) | Method and apparatus for reducing network resource transmission size using delta compression | |
| US10880390B2 (en) | Method and apparatus for reducing network resource transmission size using delta compression | |
| US8463846B2 (en) | File bundling for cache servers of content delivery networks | |
| US7287136B2 (en) | Cache device, and method and computer program for controlling cached data | |
| EP2791815B1 (en) | Application-driven cdn pre-caching | |
| US8996653B1 (en) | Systems and methods for client authentication | |
| US9148332B2 (en) | Content delivery network | |
| US10057368B1 (en) | Method and system for incremental cache lookup and insertion | |
| US20080114897A1 (en) | System and method for transmission of DNS beacons | |
| US20120096116A1 (en) | Content distribution network using a web browser and locally stored content to directly exchange content between users | |
| CN107179878B (en) | Data storage method and device based on application optimization | |
| CN105635196A (en) | Method and system of file data obtaining, and application server | |
| US20120054295A1 (en) | Method and apparatus for providing or acquiring the contents of a network resource for a mobile device | |
| CN105302801A (en) | Resource caching method and apparatus | |
| EP2011029B1 (en) | Managing network response buffering behavior | |
| US11843650B2 (en) | Two-level cache architecture for live video streaming through hybrid ICN | |
| CN112597039A (en) | Virtual machine access method, system, device and computer readable storage medium | |
| US7249219B1 (en) | Method and apparatus to improve buffer cache hit rate | |
| CN1908915A (en) | System and method for management of metadata | |
| CN104615597A (en) | Method, device and system for clearing cache file in browser | |
| CN103269353A (en) | Web cache and return optimization method and Web cache system | |
| CN110855786B (en) | Processing method and device for network data request and computer storage medium | |
| WO2010031297A1 (en) | Method of wireless application protocol (wap) gateway pull service and system thereof | |
| CN110247996B (en) | Wireless router device of configurable cache system based on intelligent classroom |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C12 | Rejection of a patent application after its publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20100630 |