CN108540382A - A kind of Web content storage system and its method for routing - Google Patents
A kind of Web content storage system and its method for routing Download PDFInfo
- Publication number
- CN108540382A CN108540382A CN201810168810.6A CN201810168810A CN108540382A CN 108540382 A CN108540382 A CN 108540382A CN 201810168810 A CN201810168810 A CN 201810168810A CN 108540382 A CN108540382 A CN 108540382A
- Authority
- CN
- China
- Prior art keywords
- storage
- units
- storing sub
- processing unit
- content
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L45/00—Routing or path finding of packets in data switching networks
- H04L45/14—Routing performance; Theoretical aspects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L45/00—Routing or path finding of packets in data switching networks
- H04L45/74—Address processing for routing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/12—Avoiding congestion; Recovering from congestion
- H04L47/125—Avoiding congestion; Recovering from congestion by balancing the load, e.g. traffic engineering
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The Web content storage system that control of the present invention is detached with storage, including storing sub-units, are mainly made of common storage, for storing Web content；Storage processing unit, for carrying out relevant treatment to Web content；Storage control unit is detached with storage processing unit, the storage state for monitoring storage processing unit working condition, storing sub-units, and according to the characteristic of user demand and storage medium, configuration, the storage behavior etc. of management storage processing unit.The present invention is using control and the thought that detaches of storage, storage control unit monitoring, the storage behavior for managing storage processing unit so that Web content intelligent equalization store corresponding storing sub-units.Storage control unit can dynamic collection storage processing unit and storing sub-units state.According to these status informations, user can freely write storage control unit storage algorithm software, improve the flexibility of storage device with it is intelligent.
Description
Technical field
The present invention relates to network content technical field of memory more particularly to a kind of Web content storage system and its routing sides
Method.
Background technology
There are mass data redundancies in existing internet, are badly in need of a kind of efficient cache mechanism reduction network redundancy.Intelligence
Intelligent contract network (SINET) supports data packet or data using the design philosophy of " resource is detached with position " from original design
The caching of block granularity substantially reduces network redundancy, improves link utilization and user experience.SINET is no longer sought using IP address
Location, it is new addressing space to uniquely determine to replace：SID (Service ID, service identifiers), each Web content
There are one unique SID.Meanwhile SINET introduces the concept of caching, and SINET base network devices, i.e. net is added in caching function
Network component, networking component according to the SID of network packet in its caching system into line access, to support data in network layer
Packet granularity is cached along road.SINET caching mechanisms are operated in network layer, are decoupled with application layer, and caching mechanism has flexibly, efficiently
And the advantages such as unrelated with application layer protocol.Networking component flows through the data of itself according to cache policy storage, facilitates other use
Family obtains data nearby, improves network efficiency.
Storage system in conventional network component generally directly uses the storage medium of high speed as either statically or dynamically deposited at random
Reservoir stores Web content, and to realize the storage forwarding of its linear speed, but this system memory size is limited, and expensive,
It is not easy to dispose on a large scale；Storage system in another networking component is to use distributed memory technology, using cheap
Storage medium parallel processing come realize networking component linear speed store forwarding.Distributed storage technology generally uses storage tube
Reason unit is allocated and manages to the content of access.Client accesses the forward direction MMU memory management unit inquiry access data of data
Position, determine and initiate to access to the storage medium of attribution data again after Data Position.The storage management list of this storage system
The router-level topology of the responsible storage content of member and forwarding unify routing algorithm by Web content routing forwarding to difference by Hash etc.
Storage medium in.This method enables expensive storage medium parallel processing, improves the total bandwidth of storage system, reduces
The cost of storage system.But because routing algorithm is excessively bound with forwarding capability, flexibility and autgmentability existing defects.
As storage demand is constantly increasing, this traditional technology will be unable to efficiently handle mass data storage.For by a large amount of
The large-scale storage systems of content storage apparatus composition, conventional memory device is because of the control plane for being responsible for router-level topology and responsible road
It is bundled on a hardware by the forwarding forwarded, can not perceive to smart collaboration the storage state of other equipment, therefore can not
Realize the wisdom cooperation storage of large-scale networks.
Invention content
The present invention provides a kind of Web content storage system, to promote the flexibility, intelligent of storage system；Improve storage
Equipment memory rate reduces production cost.
To achieve the goals above, this invention takes following technical solutions：
The present invention provides a kind of Web content storage system, including storage control unit, storage processing unit and storage
Unit；The storage control unit is communicated to connect with the storage processing unit, the storage processing unit and storage
Unit communication connects；
The storage control unit configures the method for routing of the storage processing unit, and the monitoring storage processing list
The storage state of the first and described storing sub-units；
The storage processing unit receives the storage content that terminal is sent, and will be in the storage according to the method for routing
It stores storage and arrives the storing sub-units.
Preferably, the storage control unit includes first communication module, control interface level of abstraction and storage control application
Layer；
The first communication module is communicated to connect with the storage processing unit, and the storage control application layer passes through described
Control interface level of abstraction is communicated to connect with the first communication module；
The method for routing that application layer configures the storage processing unit is controlled by storage, the first communication module receives
The storage state and the transmission method for routing for the storage content, the storing sub-units that the storage processing unit reports.
Preferably, the storage processing unit includes the fragment composite module communicated to connect successively, index module, caching mould
Block, second communication module, routing module and prefetch module；
The fragment composite module stores the storage content fragment that terminal is sent to the storing sub-units or polymerization
The storage content of the storing sub-units；
The index module provides global index's information for the storage content of the storing sub-units；
The cache module caches the hot spot storage content of the storing sub-units；
The second communication module is communicated to connect with the storage control unit and the storing sub-units respectively, and described the
Two communication modules receive described in the storage state that storage content, the storing sub-units that the terminal is sent report and reception
The method for routing that storage control unit is sent；
The storage content that the terminal is sent is routed to corresponding storage by the routing module according to the method for routing
Subelement；
The module that prefetches will be prefetched to the cache module with the relevant data block of current accessed data block.
Preferably, the quantity of the storing sub-units is one or more, and the storing sub-units are made of storage medium.
It is as follows the present invention also provides the method for routing of the Web content storage system：
The storage content that terminal is sent is reported to the storage control unit by the storage processing unit；
The storage control unit judges whether the difference that the storage content calls time on this and called time on last time is less than
Given threshold；
If more than given threshold, then call time in update, and abandon the storage content；
If small Mr. Yu's threshold value, according to the storage state information for the storing sub-units that storage processing unit reports, by storage
The presently written rate ascending order of unit arranges storing sub-units；
The storing sub-units in the preceding setting position of above-mentioned ascending sort are chosen, are dropped according to storing sub-units residual capacity size
Sequence arranges；
The storing sub-units in the preceding setting position of above-mentioned descending sort are selected, are dropped according to the read rate size of storing sub-units
Sequence arranges；
The minimum storing sub-units of read rate are obtained, routing forwarding entry is issued, make storage content that the read rate be written
In minimum storing sub-units.
As seen from the above technical solution provided by the invention, the thought that the present invention is detached using control with storage, will
Storage control unit is detached with storage processing unit, and different storage processing units can be with：It is not connected to, connects one or more phases
Same or different storage control unit, multiple storage devices can form storage group, wisdom collaboration storage.Storage processing unit
Software implementation：User can write particular memory control (algorithm) application in storage control unit, make storage according to concrete scene
The realization of algorithm is more flexible, convenient.In addition, the present invention is poly- using the principle of locality of network flow and the rate of storage device
Effect is closed, realizes that a kind of buffer memory rate is far above the SINET cache devices of existing caching system, to improve in networking component
Memory rate is unable to catch up with forwarding rate and the high problem of networking component production cost, to significantly improve network utilization and use
Family is experienced, and promotes SINET in the deployment of internet range.
The additional aspect of the present invention and advantage will be set forth in part in the description, these will become from the following description
Obviously, or practice through the invention is recognized.
Description of the drawings
In order to illustrate the technical solution of the embodiments of the present invention more clearly, required use in being described below to embodiment
Attached drawing be briefly described, it should be apparent that, drawings in the following description are only some embodiments of the invention, for this
For the those of ordinary skill of field, without having to pay creative labor, other are can also be obtained according to these attached drawings
Attached drawing.
Fig. 1 is the structural schematic diagram of Web content storage system described in the embodiment of the present invention；
Fig. 2 is the structural schematic diagram of storage control unit described in the embodiment of the present invention；
Fig. 3 is the structural schematic diagram of storage processing unit described in the embodiment of the present invention；
Fig. 4 is the structural schematic diagram of storing sub-units described in the embodiment of the present invention；
Fig. 5 is the method for routing flow chart of storage control unit configuration described in the embodiment of the present invention；
Fig. 6 is that a storage control unit controls at multiple storages in Web content storage system described in the embodiment of the present invention
Manage the structural schematic diagram of unit.
Specific implementation mode
Embodiments of the present invention are described below in detail, the example of the embodiment is shown in the accompanying drawings, wherein from beginning
Same or similar element or element with the same or similar functions are indicated to same or similar label eventually.Below by ginseng
The embodiment for examining attached drawing description is exemplary, and is only used for explaining the present invention, and is not construed as limiting the claims.
Those skilled in the art of the present technique are appreciated that unless expressly stated, singulative " one " used herein, " one
It is a ", " described " and "the" may also comprise plural form.It is to be further understood that is used in the specification of the present invention arranges
It refers to there are the feature, integer, step, operation, element and/or component, but it is not excluded that presence or addition to take leave " comprising "
Other one or more features, integer, step, operation, element, component and/or their group.It should be understood that when we claim member
Part is " connected " or when " coupled " to another element, it can be directly connected or coupled to other elements, or there may also be
Intermediary element.In addition, " connection " used herein or " coupling " may include being wirelessly connected or coupling.Wording used herein
"and/or" includes any cell of one or more associated list items and all combines.
It will be understood by those skilled in the art that unless otherwise defined, all terms used herein (including technical term and
Scientific terminology) there is meaning identical with the general understanding of the those of ordinary skill in fields of the present invention.It should also be appreciated that
, those terms such as defined in the general dictionary, which should be understood that, to be had and the meaning one in the context of the prior art
The meaning of cause, and unless defined as here, will not be explained with the meaning of idealization or too formal.
For ease of the understanding to the embodiment of the present invention, done further by taking several specific embodiments as an example below in conjunction with attached drawing
Explanation, and each embodiment does not constitute the restriction to the embodiment of the present invention.
The basic thought of the present invention is to detach the control of content storage apparatus with storage processing unit, to improve storage
It is the flexibility of equipment, intelligent.Storage processing unit is responsible for basic storage behavior processing, such as tables look-up, route, distributing in storage
Hold storing sub-units, the real-time status of monitoring network, storage processing unit, storing sub-units etc., as real-time storage handling capacity,
Residual capacity etc. is stored, and reports storage control unit monitoring result.Storage control unit software implementation, with storage processing unit point
From being established communication by the control port of storage processing unit (generally Ethernet interface).Storage control unit is received
The storage state that collection storage processing unit monitors, according to these status datas, the storage of dynamic configuration storage processing unit turns
It delivers, automatic collaboration storage.Different storage processing units can be with：It is not connected to, connects one or more identical or different deposit
Control unit is stored up, multiple storage devices can form storage group, wisdom collaboration storage.User can be special according to actual storage scene
Point writes particular memory algorithm application software in storage control unit, and flexible, intelligent control stores behavior, such as according to dynamic
The forwarding table memory that different storage processing units are arranged in network flow feature makes efficient, equilibrium storage of data on flows etc..It is main
Design object is listed below：
The function that can complete general distributed memory system includes equal by unified algorithm (such as hash algorithm) load
It weighs, according to storage real-time traffic and storage device characteristic load balancing；
The production cost for reducing networking component in wisdom contract network, utilizes network flow principle of locality and generic storage
The parallel processing of equipment improves memory rate；
The scalability for enhancing storage system hot can be inserted when memory rate does not catch up with the forwarding rate of networking component
The mode pulled out increases one or more of storage control unit, storage processing unit or storing sub-units；
Intelligent, the flexibility of storage system are improved, user can be according to the variation of the demand and application scenarios of oneself, dynamic
Memory management algorithm is intelligently adjusted, storage device value is maximized.
Embodiment one
The present embodiment provides a kind of control with store the high-speed content storage device that detaches, overall structure as shown in Figure 1,
Include mainly three parts：Storage control unit, storage processing unit and storing sub-units.
In the present embodiment, storage control unit is detached with storage processing unit, storing sub-units, for monitoring at storage
Manage the state of unit and storing sub-units.Different storage processing units can be with：Be not connected to, connect it is one or more identical or not
Same storage control unit, multiple storage devices can form storage group, wisdom collaboration storage.Storage processing unit software
Change, user can write particular memory control (algorithm) application in storage control unit, make storage algorithm according to concrete scene
Realization it is more flexible, convenient；
In the present embodiment, the storage processing unit is responsible for routeing storage content to storing sub-units, distribute (or it is poly-
Close) storage flow, enable each storing sub-units parallel memorizing, to improve the total memory rate of storage device；Meanwhile it storing
Processing unit collects storage characteristics, the memory rate and residual storage capacity of dynamic monitoring storing sub-units of storing sub-units,
And periodically or triggering property reports and collects result to storage control unit, for the storage algorithm decision of storage control unit provide according to
According to.
In the present embodiment, the storing sub-units be responsible for storage, read, update with deletion be stored in it is interior in submodule
Hold.
In the present embodiment, the storage control unit embodiment is as shown in Fig. 2, it includes mainly three parts：
Communication module：Ethernet communication module is responsible for communicating with storage processing unit, for receiving storage processing unit
The information such as the storage content of report, the memory rate of storing sub-units, residual capacity；Send the management and configuration of storage processing unit
Information；
Control interface level of abstraction：Bottom function package is abstracted, is provided for upper layer storage strategy application software (APP) unified
Interface call；
(APP) is applied in storage control：User is stored to be compiled according to the storage information of collection, application scenarios and oneself demand
Write specific APP, make the realization of storage scheme it is more convenient, it is flexible with it is intelligent.
In the present embodiment, the specific embodiment of the storage processing unit is as shown in figure 3, it includes mainly six big portions
Point：
Fragment composite module：Fragment (or combination) is carried out to the big storage content of storage, it will be in these continuous small storages
Hold piece and be distributed in multiple storing sub-units (or content in the multiple storing sub-units of polymerization), it is thus possible to make multiple storages
Subelement executes parallel, significantly improves storage device access performance；
Index module：Content to be stored in each storing sub-units provides global index's information.This index can be provided only and be deposited
Store up the correspondence of content and storage device；
Cache module：The content got from storing sub-units is cached, hot spot data is cached, reduces the visit to storing sub-units
It asks；
Communication module：For being communicated with storage control unit, storage state is reported, storage routing configuration information is received
Deng；
Routing module：It will be deposited according to the flow table (or equalization algorithm of acquiescence, such as hash algorithm) of storage control unit configuration
Storage content is routed to respective stored subelement；
Prefetch module：According to the principle of locality of network flow, i.e., user will horse after accessing certain data block in network
Upper access and the relevant data block of this data block, prefetch data block to the cache module of storage processing unit, make full use of and deposit
Store up the bus bandwidth of subelement.
Entire storage processing unit, by 3.0 interface communications of PCIE, can provide maximum 128Gb/s's with networking component
Bandwidth.It is communicated by 1Gb/s or 10Gb/s Ethernet interfaces between network submodular.Storage processing unit and storing sub-units it
Between loose coupling the characteristics of improve the scalability of system.
In the present embodiment, the specific embodiment of the storing sub-units is as shown in Figure 4.ARM is mainly for the treatment of depositing
The operative algorithm for storing up the content in solid state disk such as writing, reading, update and delete operation algorithm, and is realized to SATA 3.0
The management of interface bus agreement, Ethernet protocol and DDR memories.The storage of some program in machine codes when ROM is used for offline.Solid-state is hard
Disk is for storing network data content.
In the present embodiment, content deposit system flow is as follows：When write request arrives, storage content passes through PCI E
3.0 interface protocols pass to storage processing unit, and the fragment composite module of storage processing unit divides larger storage content
(or smaller storage content is buffered in cache module, until being combined into particular size at the content blocks of particular size
Hold block), routing module these content blocks are distributed to by 1Gb/s or 10Gb/s Ethernet interfaces in different storing sub-units into
Row storage, makes storing sub-units parallel memorizing content, improves system memory bandwidth.What the routing forwarding entry of routing module preserved
It is the correspondence of storage content and storing sub-units number, when new storage content arrives, routing module is without this content
Corresponding entry, then storage processing unit report storage control unit, storage control unit to be issued by the control application of operation
Routing table route this content to respective stored subelement.Meanwhile index module records the storage device number of this content, for
Read requests use.Finally, storing sub-units store Web content into solid state disk, and will be related under the control of ARM
Index information storage in memory DDR.
In the present embodiment, it is as follows to read system flow for content：When read requests arrive, connect by the communications of PCIE 3.0
Mouth obtains read requests content identification SID；Then cache module is searched by SID, if there are in respective request in cache module
Hold, then directly returns；If cache module does not have request content, index module is searched, judges whether Mr. Yu's storing sub-units
In；If not indexed in index module, return communication interface lookup result；If the index of content is found, by prefetching
Module prefetches request data and specific quantity related data to cache module from storing sub-units, and passes through 3.0 interfaces of PCIE
Return to the content of request.
In the present embodiment, (APP) is applied in the storage control unit operation storage, including Web content stores algorithm,
As shown in figure 4, being a kind of Web content method for routing, or it is storage algorithm.In the system of the present invention, user can be as writing
Computer applied algorithm equally writes the storage application in storage control unit.Any moment user disposes new storage algorithm,
Speed deployment is can be carried out, need to do is only to call storage control unit API, writes storage application, to big to increase
The flexibility of storage system with it is intelligent；
Storage algorithm embodiment flow chart shown in Fig. 4 is described when storage forwarding entry is not configured in storage processing unit
When, the storage routing procedure of content is as follows：
Storage processing unit reports the content identification of this storage content to storage control unit；
Storage algorithm in storage control unit judges the difference for calling time and calling time on last time in this content identification this time
Whether a certain threshold value is less than；
If more than certain threshold value, then more new content reports the time, then abandons this content；
If small Mr. Yu's threshold value, the storing sub-units status information that storage algorithm is reported according to storage processing unit, by storage
The presently written rate ascending order of subelement arranges storing sub-units；
The storing sub-units in former positions of above-mentioned sequence are selected, then remain these memory modules according to storing sub-units
Remaining amount of capacity descending arrangement；
The storing sub-units in former positions of above-mentioned sequence are selected, then by these memory modules according to storing sub-units
Read rate size descending arranges；
The minimum storing sub-units of above-mentioned read rate are obtained, routing forwarding entry is issued, make storage content that above-mentioned reading be written
In the minimum storing sub-units of rate.
In the present embodiment, the algorithm is a simple storage algorithm embodiment, but can absolutely prove this storage
The advantage of system.In this embodiment, solid state disk is normally used as storage medium.The service life of solid state disk is limited, and one
As its erasing times in 10K-100K ten thousand times.Therefore, in storing algorithm, it is contemplated that the service life of solid state disk limits, above-mentioned
Step 2, it increases content and reports frequency filter.System only stores the Web content of request frequency in a certain range, less
The erasing times of solid state disk.In addition, this algorithm considers the factors such as writing rate, residual capacity, read-out speed, to maximum
While changing the service efficiency of storing sub-units, the factors such as the wear leveling of solid state disk have also been fully considered.Compared to simple
Hash distributed storage algorithm, this system consider it is more multifactor, can be more intelligent.
In addition, the framework that the storage system of the present invention is detached using storage with control, multiple storage processing units can connect
Identical (or multiple) controller is connect, to realize that multiple storage system global collaboratives store Web content.Such as Fig. 6 institutes
Show, basic control flow journey is as follows：
Corresponding contents are returned to client by client request server content, server；
The content that server-side returns will pass through networking component 1 and networking component 3 (or networking component 1 and networking component 2)
Forwarding storage is to client；
Situations such as storage control unit can be loaded according to the storage of the storage system along road network component, server is returned
Content be stored in one or more networking components of above-mentioned networking component, to make the performance of multiple storage systems reach most
It is excellent.
Embodiment two
The present embodiment provides a kind of Web content storage systems, including storage control unit, storage processing unit and storage
Subelement；The storage control unit is communicated to connect with the storage processing unit, the storage processing unit and the storage
Subelement communicates to connect；
The storage control unit configures the method for routing of the storage processing unit, and the monitoring storage processing list
The storage state of the first and described storing sub-units；
The storage processing unit receives the storage content that terminal is sent, and will be in the storage according to the method for routing
It stores storage and arrives the storing sub-units.
In the present embodiment, the storage control unit includes first communication module, control interface level of abstraction and storage control
Application layer processed；
The first communication module is communicated to connect with the storage processing unit, and the storage control application layer passes through described
Control interface level of abstraction is communicated to connect with the first communication module；
The method for routing that application layer configures the storage processing unit is controlled by storage, the first communication module receives
The storage state and the transmission method for routing for the storage content, the storing sub-units that the storage processing unit reports.
In the present embodiment, the storage processing unit include the fragment composite module communicated to connect successively, index module,
Cache module, second communication module, routing module and prefetch module；
The fragment composite module stores the storage content fragment that terminal is sent to the storing sub-units or polymerization
The storage content of the storing sub-units；
The index module provides global index's information for the storage content of the storing sub-units；
The cache module caches the hot spot storage content of the storing sub-units；
The second communication module is communicated to connect with the storage control unit and the storing sub-units respectively, and described the
Two communication modules receive described in the storage state that storage content, the storing sub-units that the terminal is sent report and reception
The method for routing that storage control unit is sent；
The storage content that the terminal is sent is routed to corresponding storage by the routing module according to the method for routing
Subelement；
The module that prefetches will be prefetched to the cache module with the relevant data block of current accessed data block.
In the present embodiment, the quantity of the storing sub-units is one or more, and the storing sub-units are situated between by storage
Matter forms.
In conclusion the content storage system described in the embodiment of the present invention, one and processing unit stored above can deposited
It is balanced, flexibly, cooperate with to wisdom according to storing sub-units characteristic, network topology and flow feature under the control for storing up control unit
Storage.This system uses cheap storage medium, but reaches the memory rate of expensive professional storage medium；This system can be improved
Storage device stores collaboration capabilities, reduces storage system production cost；Meanwhile promoting the flexibility, intelligent of storage system.
One of ordinary skill in the art will appreciate that：Attached drawing is the schematic diagram of one embodiment, module in attached drawing or
Flow is not necessarily implemented necessary to the present invention.
As seen through the above description of the embodiments, those skilled in the art can be understood that the present invention can
It is realized by the mode of software plus required general hardware platform.Based on this understanding, technical scheme of the present invention essence
On in other words the part that contributes to existing technology can be expressed in the form of software products, the computer software product
It can be stored in a storage medium, such as ROM/RAM, magnetic disc, CD, including some instructions are used so that a computer equipment
(can be personal computer, server either network equipment etc.) executes the certain of each embodiment of the present invention or embodiment
Method described in part.
Each embodiment in this specification is described in a progressive manner, identical similar portion between each embodiment
Point just to refer each other, and each embodiment focuses on the differences from other embodiments.Especially for device or
For system embodiment, since it is substantially similar to the method embodiment, so describing fairly simple, related place is referring to method
The part of embodiment illustrates.Apparatus and system embodiment described above is only schematical, wherein the conduct
The unit that separating component illustrates may or may not be physically separated, the component shown as unit can be or
Person may not be physical unit, you can be located at a place, or may be distributed over multiple network units.It can root
According to actual need that some or all of module therein is selected to achieve the purpose of the solution of this embodiment.Ordinary skill
Personnel are without creative efforts, you can to understand and implement.
The foregoing is only a preferred embodiment of the present invention, but scope of protection of the present invention is not limited thereto,
Any one skilled in the art in the technical scope disclosed by the present invention, the change or replacement that can be readily occurred in,
It should be covered by the protection scope of the present invention.Therefore, protection scope of the present invention should be with scope of the claims
Subject to.
Claims (5)
1. a kind of Web content storage system, which is characterized in that single including storage control unit, storage processing unit and storage
Member；The storage control unit is communicated to connect with the storage processing unit, and the storage processing unit and storage are single
Member communication connection；
The storage control unit configures the method for routing of the storage processing unit, and the monitoring storage processing unit and
The storage state of the storing sub-units；
The storage processing unit receives the storage content that terminal is sent, and deposits the storage content according to the method for routing
Store up the storing sub-units.
2. Web content storage system according to claim 1, which is characterized in that the storage control unit includes first
Communication module, control interface level of abstraction and storage control application layer；
The first communication module is communicated to connect with the storage processing unit, and the storage control application layer passes through the control
Interface abstraction layer is communicated to connect with the first communication module；
Application layer is controlled by storage and configures the method for routing of the storage processing unit, described in the first communication module receives
The storage state and the transmission method for routing for the storage content, the storing sub-units that storage processing unit reports.
3. Web content storage system according to claim 1, which is characterized in that the storage processing unit includes successively
The fragment composite module of communication connection, index module, cache module, second communication module, routing module and prefetch module；
The fragment composite module stores the storage content fragment that terminal is sent to described in the storing sub-units or polymerization
The storage content of storing sub-units；
The index module provides global index's information for the storage content of the storing sub-units；
The cache module caches the hot spot storage content of the storing sub-units；
The second communication module is communicated to connect with the storage control unit and the storing sub-units respectively, and described second is logical
Letter module receives the storage state that storage content, the storing sub-units that the terminal is sent report and receives the storage
The method for routing that control unit is sent；
The storage content that the terminal is sent is routed to corresponding storage list by the routing module according to the method for routing
Member；
The module that prefetches will be prefetched to the cache module with the relevant data block of current accessed data block.
4. Web content storage system according to claim 1, which is characterized in that the quantity of the storing sub-units is one
A or multiple, the storing sub-units are made of storage medium.
5. a kind of method for routing of the Web content storage system as described in claim any one of 1-4, which is characterized in that
The storage content that terminal is sent is reported to the storage control unit by the storage processing unit；
The storage control unit judges that the storage content calls time on this and whether the difference that called time on last time is less than setting
Threshold value；
If more than given threshold, then call time in update, and abandon the storage content；
If small Mr. Yu's threshold value, according to the storage state information for the storing sub-units that storage processing unit reports, by storing sub-units
Presently written rate ascending order arrange storing sub-units；
The storing sub-units in the preceding setting position of above-mentioned ascending sort are chosen, are arranged according to storing sub-units residual capacity size descending
Row；
The storing sub-units in the preceding setting position of above-mentioned descending sort are selected, are arranged according to the read rate size descending of storing sub-units
Row；
The minimum storing sub-units of read rate are obtained, routing forwarding entry is issued, make storage content that the read rate be written minimum
Storing sub-units in.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810168810.6A CN108540382B (en) | 2018-02-28 | 2018-02-28 | Network content storage system and routing method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810168810.6A CN108540382B (en) | 2018-02-28 | 2018-02-28 | Network content storage system and routing method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108540382A true CN108540382A (en) | 2018-09-14 |
| CN108540382B CN108540382B (en) | 2020-08-18 |
Family
ID=63486371
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810168810.6A Active CN108540382B (en) | 2018-02-28 | 2018-02-28 | Network content storage system and routing method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108540382B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102882983A (en) * | 2012-10-22 | 2013-01-16 | 南京云创存储科技有限公司 | Rapid data memory method for improving concurrent visiting performance in cloud memory system |
| CN103813336A (en) * | 2012-11-07 | 2014-05-21 | 华为技术有限公司 | Wireless local area network transmission control method, wireless local area network device and wireless local area network system |
| CN104135539A (en) * | 2014-08-15 | 2014-11-05 | 华为技术有限公司 | Data storage method, SDN controller and distributed network storage system |
| US20150350154A1 (en) * | 2014-06-03 | 2015-12-03 | John Myla | Using Distributed Network Elements to Send Authoritative DNS Responses |
| CN105516263A (en) * | 2015-11-28 | 2016-04-20 | 华为技术有限公司 | Data distribution method, device in storage system, calculation nodes and storage system |
| CN107147507A (en) * | 2016-03-01 | 2017-09-08 | 中卫大河云联网络技术有限公司 | The control plane framework and control method of a kind of software defined network |
| CN107682411A (en) * | 2017-09-14 | 2018-02-09 | 广州西麦科技股份有限公司 | A kind of extensive SDN controllers cluster and network system |
-
2018
- 2018-02-28 CN CN201810168810.6A patent/CN108540382B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102882983A (en) * | 2012-10-22 | 2013-01-16 | 南京云创存储科技有限公司 | Rapid data memory method for improving concurrent visiting performance in cloud memory system |
| CN103813336A (en) * | 2012-11-07 | 2014-05-21 | 华为技术有限公司 | Wireless local area network transmission control method, wireless local area network device and wireless local area network system |
| US20150350154A1 (en) * | 2014-06-03 | 2015-12-03 | John Myla | Using Distributed Network Elements to Send Authoritative DNS Responses |
| CN104135539A (en) * | 2014-08-15 | 2014-11-05 | 华为技术有限公司 | Data storage method, SDN controller and distributed network storage system |
| CN105516263A (en) * | 2015-11-28 | 2016-04-20 | 华为技术有限公司 | Data distribution method, device in storage system, calculation nodes and storage system |
| CN107147507A (en) * | 2016-03-01 | 2017-09-08 | 中卫大河云联网络技术有限公司 | The control plane framework and control method of a kind of software defined network |
| CN107682411A (en) * | 2017-09-14 | 2018-02-09 | 广州西麦科技股份有限公司 | A kind of extensive SDN controllers cluster and network system |
Non-Patent Citations (1)
| Title |
|---|
| 张宏科等: "智慧协同标识网络", 《中兴通讯技术》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108540382B (en) | 2020-08-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106131182B (en) | Name a kind of cooperation caching method based on Popularity prediction in data network | |
| US8068512B2 (en) | Efficient utilization of cache servers in mobile communication system | |
| US8397010B1 (en) | Convenient, flexible, and efficient management of memory space and bandwidth | |
| JP5745169B2 (en) | Content processing method, content processing device, and content processing system | |
| US20160132541A1 (en) | Efficient implementations for mapreduce systems | |
| US8307170B2 (en) | Information processing method and system | |
| CN102118433A (en) | Multiple-tier distributed cluster system | |
| CN103607312B (en) | Data request processing method and system for server system | |
| CN107231395A (en) | Date storage method, device and system | |
| CN108900570B (en) | Cache replacement method based on content value | |
| EP4057579A1 (en) | Data forwarding method, data buffering method, device, and related apparatus | |
| US7818505B2 (en) | Method and apparatus for managing a cache memory in a mass-storage system | |
| CN102439580A (en) | Systems and methods for network acceleration and efficient indexing for caching file systems | |
| CN103036994A (en) | Cloud storage system capable of achieving load balance | |
| CN102439934A (en) | Method and system for managing multilevel caches of edge server in cdn | |
| CN102833352A (en) | Distributed cache management system and method for implementing distributed cache management | |
| CN107018172A (en) | System and method for the adaptive partition in distributed caching memory | |
| CN107368608A (en) | The HDFS small documents buffer memory management methods of algorithm are replaced based on ARC | |
| CN102104617A (en) | Method for storing massive picture data by website operating system | |
| US8402198B1 (en) | Mapping engine for a storage device | |
| WO2020199760A1 (en) | Data storage method, memory and server | |
| CN108965479A (en) | A kind of domain collaboration caching method and device based on content center network | |
| CN1595910A (en) | A data packet receiving interface component of network processor and storage management method thereof | |
| CN101141482B (en) | Network resource management system and method | |
| CN109002260A (en) | A kind of data cached processing method and processing system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |