CN106302659A - A kind of based on cloud storage system promotes access data quick storage method - Google Patents
A kind of based on cloud storage system promotes access data quick storage method Download PDFInfo
- Publication number
- CN106302659A CN106302659A CN201610624940.7A CN201610624940A CN106302659A CN 106302659 A CN106302659 A CN 106302659A CN 201610624940 A CN201610624940 A CN 201610624940A CN 106302659 A CN106302659 A CN 106302659A
- Authority
- CN
- China
- Prior art keywords
- data
- storage server
- server
- data storage
- client
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/561—Adding application-functional data or data for application control, e.g. adding metadata
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
Abstract
The invention discloses a kind of based on cloud storage system promotes access data quick storage method.Described cloud storage system includes meta data server, data storage server and client, is only controlled the transmission of signal between meta data server and client, does not carry out storing the transmission of data stream；Carry out between client and data storage server storing data stream transmitting；Individual data file read-write is concurrently carried out；When client writes data, data are divided into each data block, write each data block and operate according to following steps, and parallel writes data block to data storage server.The high speed concurrent access performance aspect of cloud storage system that the present invention supports as the storage of a large number of users data is set out, and proposes a kind of to meet multi-user, the method for the high demand concurrently accessed, calls offer storage convenient, high performance to real world applications and support.
Description
Technical field
The invention belongs to computer memory technical field, particularly relate to a kind of based on cloud storage system promotes access number
According to quick storage method.
Background technology
Along with the fast development of Internet technology, the scope of business of the Internet is also constantly expanding, and the Internet is own through becoming
Important component part indispensable in people's daily life.The Internet is used demand own through from traditional portal by user
Station has been increasingly turned to the Internet service of heavy traffic.Meanwhile, data present volatile growth, take for the Internet
For business side, in the business of new extension, the cost of required storage resource and computer hardware carries constantly, in data
Construction and the maintenance cost of the heart are raising year by year, and the consumption of personal management and resource is also being continuously increased.Solve this series of
Problem, it is necessary to set up a new platform and treatment mechanism coordinated and dispatches limited resource, make the collection of data, place
Reason and calculating become more to imitate, fast.
While the continuous research and development of cloud computing, the technology cloud storage of its core is also closed by people more and more
Note.Cloud storage is that different types of storages a large amount of in network are set lattice by cluster application, grid and distributed file system etc.
Gathering collaborative work by application software, common externally offer data storage and Operational Visit function, it is saved for user
Great amount of cost, has therefore obtained supporting widely and apply.
Summary of the invention
It is an object of the invention to provide a kind of based on cloud storage system promotes access data quick storage method, pass through
Use and access the mode that cloud data carry out splitting, carry out single process, the cloud storage system supported as the storage of a large number of users data
The high speed concurrent access performance aspect of system is set out.
The present invention is achieved by the following technical solutions:
The present invention is a kind of based on promoting access data quick storage method, described cloud storage system bag in cloud storage system
Include meta data server, data storage server and client, it is characterised in that comprise the steps:
Only it is controlled the transmission of signal between meta data server and client, does not carry out storing the transmission of data stream；
Carry out between client and data storage server storing data stream transmitting；
The data file even partition that will be greater than threshold value is each data block, and distributed storage is in each data storage service
On device；
Individual data file read-write is concurrently carried out；
When client writes data, data are divided into each data block, write each data block and operate according to following steps,
Parallel write data block to data storage server:
A client initiates data write request to meta data server；
B meta data server calculates purpose data according to the method for the load balancing of its data storage server managed and deposits
Storage server, creates new data block on purpose data storage server；
The data block of the data storage server of c mesh creates successfully, returns result to meta data server；
D meta data server backs up and synchronizes this data message, and purpose data storage server information is returned to visitor
Family end；
E client, according to the purpose data storage server information obtained, sends to corresponding purpose data storage server
Data write request, and send data to purpose data storage server；
The data storage server of f mesh receives data and stores corresponding data block, the most also to other data storage clothes
Business device backup；
G is after local data write and having backed up, and purpose data storage server will complete information and return to client
End；
After client has received information, i.e. complete the write of data.
Preferably, described in will be greater than the data file even partition of threshold value be each data block, distributed storage is each
Concrete steps on data storage server:
S1 client obtains data block according to request side-play amount when reading data file divided by the size of data block and indexes, and
Data read request is initiated to meta data server；
S2 meta data server returns the list of the data storage server at this data block place, and data storage server arranges
Table includes address and the sequence number of each data storage server；
S3 client sends to the data storage server that read-write requests number is minimum reads data block request；
S4 data storage server reads data block from local file system, and returns data block to client.
Preferably, the method for described load balancing is: travel through all data storage server lists, data storage server
List includes the address of each data storage server, depositing divided by maximum data with the space of each data storage server respectively
The space of storage server as the dynamic weighting factor of each data storage server, weighter factor span 0.0～1.0,
Weighter factor constitutes new weights with the current weight sum of data storage server, enters all data storage server weights
Row sequence, is arranged in the data storage server that load is minimum before list, is then back to data available storage server
Address, its weights subtract 1.
The method have the advantages that
1, the high speed concurrent access performance aspect of the cloud storage system that the present invention supports as the storage of a large number of users data goes out
Send out, propose a kind of to meet multi-user, the method for the high demand concurrently accessed, call to real world applications that offer is convenient, high-performance
Storage support.
2, the high speed concurrent access performance aspect of the cloud storage system that the present invention supports as the storage of a large number of users data goes out
Send out, use and access the mode that cloud data carry out splitting, carry out single process.
Certainly, the arbitrary product implementing the present invention it is not absolutely required to reach all the above advantage simultaneously.
Accompanying drawing explanation
In order to be illustrated more clearly that the technical scheme of the embodiment of the present invention, embodiment will be described required use below
Accompanying drawing is briefly described, it should be apparent that, the accompanying drawing in describing below is only some embodiments of the present invention, for ability
From the point of view of the those of ordinary skill of territory, on the premise of not paying creative work, it is also possible to obtain the attached of other according to these accompanying drawings
Figure.
Fig. 1 is a kind of based on promoting the frame diagram accessing data quick storage method in cloud storage system of the present invention.
Detailed description of the invention
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete
Describe, it is clear that described embodiment is only a part of embodiment of the present invention rather than whole embodiments wholely.Based on
Embodiment in the present invention, those of ordinary skill in the art obtained under not making creative work premise all other
Embodiment, broadly falls into the scope of protection of the invention.
Referring to shown in Fig. 1, the present invention is a kind of based on promoting access data quick storage method, institute in cloud storage system
State cloud storage system and include meta data server, data storage server and client, comprise the steps:
Only it is controlled the transmission of signal between meta data server and client, does not carry out storing the transmission of data stream；
Carry out between client and data storage server storing data stream transmitting；
The data file even partition that will be greater than threshold value is each data block, and distributed storage is in each data storage service
On device；
Individual data file read-write is concurrently carried out；
When client writes data, data are divided into each data block, write each data block and operate according to following steps,
Parallel write data block to data storage server:
A client initiates data write request to meta data server；
B meta data server calculates purpose data according to the method for the load balancing of its data storage server managed and deposits
Storage server, creates new data block on purpose data storage server；
The data block of the data storage server of c mesh creates successfully, returns result to meta data server；
D meta data server backs up and synchronizes this data message, and purpose data storage server information is returned to visitor
Family end；
E client, according to the purpose data storage server information obtained, sends to corresponding purpose data storage server
Data write request, and send data to purpose data storage server；
The data storage server of f mesh receives data and stores corresponding data block, the most also to other data storage clothes
Business device backup；
G is after local data write and having backed up, and purpose data storage server will complete information and return to client
End；
After client has received information, i.e. complete the write of data.
Preferably, described in will be greater than the data file even partition of threshold value be each data block, distributed storage is each
Concrete steps on data storage server:
S1 client obtains data block according to request side-play amount when reading data file divided by the size of data block and indexes, and
Data read request is initiated to meta data server；
S2 meta data server returns the list of the data storage server at this data block place, and data storage server arranges
Table includes address and the sequence number of each data storage server；
S3 client sends to the data storage server that read-write requests number is minimum reads data block request；
S4 data storage server reads data block from local file system, and returns data block to client.
Wherein, the method for load balancing is: travel through all data storage server lists, data storage server list bag
Include the address of each data storage server, service divided by maximum data storage with the space of each data storage server respectively
The space of device as the dynamic weighting factor of each data storage server, weighter factor span 0.0～1.0, weight because of
Son constitutes new weights with the current weight sum of data storage server, arranges all data storage server weights
Sequence, is arranged in the data storage server that load is minimum before list, is then back to the address of data available storage server,
Its weights subtract 1.
It should be noted that in said system embodiment, included unit simply carries out drawing according to function logic
Point, but it is not limited to above-mentioned division, as long as being capable of corresponding function；It addition, each functional unit is concrete
Title also only to facilitate mutually distinguish, is not limited to protection scope of the present invention.
It addition, one of ordinary skill in the art will appreciate that all or part of step realizing in the various embodiments described above method
The program that can be by completes to instruct relevant hardware, and corresponding program can be stored in an embodied on computer readable storage and be situated between
In matter, described storage medium, such as ROM/RAM, disk or CD etc..
Present invention disclosed above preferred embodiment is only intended to help to illustrate the present invention.Preferred embodiment is the most detailed
Describe all of details, be also not intended to the detailed description of the invention that this invention is only described.Obviously, according to the content of this specification,
Can make many modifications and variations.These embodiments are chosen and specifically described to this specification, is to preferably explain the present invention
Principle and actual application so that skilled artisan can be best understood by and utilize the present invention.The present invention is only
Limited by claims and four corner thereof and equivalent.
Claims (3)
1. accessing a data quick storage method based on promoting in cloud storage system, described cloud storage system includes that metadata takes
Business device, data storage server and client, it is characterised in that comprise the steps:
Only it is controlled the transmission of signal between meta data server and client, does not carry out storing the transmission of data stream；
Carry out between client and data storage server storing data stream transmitting；
The data file even partition that will be greater than threshold value is each data block, and distributed storage is at each data storage server
On；
Individual data file read-write is concurrently carried out；
When client writes data, data are divided into each data block, write each data block and operate according to following steps, parallel
Write data block to data storage server:
A client initiates data write request to meta data server；
B meta data server calculates purpose data storage clothes according to the method for the load balancing of its data storage server managed
Business device, creates new data block on purpose data storage server；
The data block of the data storage server of c mesh creates successfully, returns result to meta data server；
D meta data server backs up and synchronizes this data message, and purpose data storage server information is returned to client；
E client, according to the purpose data storage server information obtained, sends data to corresponding purpose data storage server
Write request, and send data to purpose data storage server；
The data storage server of f mesh receives data and stores corresponding data block, simultaneously also to other data storage servers
Backup；
G is after local data write and having backed up, and purpose data storage server will complete information and return to client；
After client has received information, i.e. complete the write of data.
One the most according to claim 1 accesses data quick storage method, its feature based on promoting in cloud storage system
Be, described in will be greater than the data file even partition of threshold value be each data block, distributed storage is at each data storage clothes
Concrete steps on business device:
S1 client obtains data block according to request side-play amount when reading data file divided by the size of data block and indexes, and to unit
Data server initiates data read request；
S2 meta data server returns the list of the data storage server at this data block place, data storage server list bag
Include address and the sequence number of each data storage server；
S3 client sends to the data storage server that read-write requests number is minimum reads data block request；
S4 data storage server reads data block from local file system, and returns data block to client.
One the most according to claim 1 accesses data quick storage method, its feature based on promoting in cloud storage system
Being, the method for described load balancing is: travel through all data storage server lists, and data storage server list includes often
The address of individual data storage server, stores server with the space of each data storage server divided by maximum data respectively
Space as the dynamic weighting factor of each data storage server, weighter factor span 0.0～1.0, weighter factor with
The current weight sum of data storage server constitutes new weights, is ranked up all data storage server weights,
The data storage server of load minimum is arranged in before list, is then back to the address of data available storage server, its power
Value subtracts 1.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610624940.7A CN106302659A (en) | 2016-08-02 | 2016-08-02 | A kind of based on cloud storage system promotes access data quick storage method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610624940.7A CN106302659A (en) | 2016-08-02 | 2016-08-02 | A kind of based on cloud storage system promotes access data quick storage method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106302659A true CN106302659A (en) | 2017-01-04 |
Family
ID=57664212
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610624940.7A Pending CN106302659A (en) | 2016-08-02 | 2016-08-02 | A kind of based on cloud storage system promotes access data quick storage method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106302659A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108052284A (en) * | 2017-12-08 | 2018-05-18 | 北京奇虎科技有限公司 | A kind of distributed data storage method and device |
| CN108268344A (en) * | 2017-12-26 | 2018-07-10 | 华为技术有限公司 | A kind of data processing method and device |
| CN109240624A (en) * | 2018-09-29 | 2019-01-18 | 郑州云海信息技术有限公司 | A kind of data processing method and device |
| CN110620802A (en) * | 2018-06-20 | 2019-12-27 | 华为技术有限公司 | Load balancing method and device |
| CN111124280A (en) * | 2019-11-29 | 2020-05-08 | 浪潮电子信息产业股份有限公司 | Data additional writing method and device, electronic equipment and storage medium |
| CN112667150A (en) * | 2020-12-07 | 2021-04-16 | 沈阳飞机设计研究所扬州协同创新研究院有限公司 | Avionics system data remote storage method |
| CN112882647A (en) * | 2019-11-29 | 2021-06-01 | 伊姆西Ip控股有限责任公司 | Method, electronic device and computer program product for storing and accessing data |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2020802A2 (en) * | 2007-08-03 | 2009-02-04 | Nintendo Co., Ltd. | Handheld wireless game device server, handheld wireless device client and system using same |
| CN102185910A (en) * | 2011-04-22 | 2011-09-14 | 湖南大学 | Data distribution method for diskless network server based on solid state disk (SSD) and hard disk drive (HDD) hybrid storage |
| CN102307221A (en) * | 2011-03-25 | 2012-01-04 | 国云科技股份有限公司 | Cloud storage system and implementation method thereof |
| CN102523258A (en) * | 2011-11-30 | 2012-06-27 | 广东电子工业研究院有限公司 | Data storage framework facing cloud operation system and load balancing method thereof |
| CN102882983A (en) * | 2012-10-22 | 2013-01-16 | 南京云创存储科技有限公司 | Rapid data memory method for improving concurrent visiting performance in cloud memory system |
-
2016
- 2016-08-02 CN CN201610624940.7A patent/CN106302659A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2020802A2 (en) * | 2007-08-03 | 2009-02-04 | Nintendo Co., Ltd. | Handheld wireless game device server, handheld wireless device client and system using same |
| CN102307221A (en) * | 2011-03-25 | 2012-01-04 | 国云科技股份有限公司 | Cloud storage system and implementation method thereof |
| CN102185910A (en) * | 2011-04-22 | 2011-09-14 | 湖南大学 | Data distribution method for diskless network server based on solid state disk (SSD) and hard disk drive (HDD) hybrid storage |
| CN102523258A (en) * | 2011-11-30 | 2012-06-27 | 广东电子工业研究院有限公司 | Data storage framework facing cloud operation system and load balancing method thereof |
| CN102882983A (en) * | 2012-10-22 | 2013-01-16 | 南京云创存储科技有限公司 | Rapid data memory method for improving concurrent visiting performance in cloud memory system |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108052284A (en) * | 2017-12-08 | 2018-05-18 | 北京奇虎科技有限公司 | A kind of distributed data storage method and device |
| WO2019109538A1 (en) * | 2017-12-08 | 2019-06-13 | 北京奇虎科技有限公司 | Distributed data storage method and device |
| CN108052284B (en) * | 2017-12-08 | 2020-11-06 | 北京奇虎科技有限公司 | Distributed data storage method and device |
| CN108268344A (en) * | 2017-12-26 | 2018-07-10 | 华为技术有限公司 | A kind of data processing method and device |
| CN110620802A (en) * | 2018-06-20 | 2019-12-27 | 华为技术有限公司 | Load balancing method and device |
| CN110620802B (en) * | 2018-06-20 | 2021-04-09 | 华为技术有限公司 | Load balancing method and device |
| CN109240624A (en) * | 2018-09-29 | 2019-01-18 | 郑州云海信息技术有限公司 | A kind of data processing method and device |
| CN111124280A (en) * | 2019-11-29 | 2020-05-08 | 浪潮电子信息产业股份有限公司 | Data additional writing method and device, electronic equipment and storage medium |
| CN112882647A (en) * | 2019-11-29 | 2021-06-01 | 伊姆西Ip控股有限责任公司 | Method, electronic device and computer program product for storing and accessing data |
| CN112667150A (en) * | 2020-12-07 | 2021-04-16 | 沈阳飞机设计研究所扬州协同创新研究院有限公司 | Avionics system data remote storage method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106302659A (en) | A kind of based on cloud storage system promotes access data quick storage method | |
| CN105144121B (en) | Cache content addressable data block is for Storage Virtualization | |
| US20140173232A1 (en) | Method and Apparatus for Automated Migration of Data Among Storage Centers | |
| CN106775446A (en) | Based on the distributed file system small documents access method that solid state hard disc accelerates | |
| CN102523158A (en) | Metadata server cluster load balancing method based on weight | |
| CN101916289B (en) | Method for establishing digital library storage system supporting mass small files and dynamic backup number | |
| CN102882983A (en) | Rapid data memory method for improving concurrent visiting performance in cloud memory system | |
| CN103491155B (en) | Cloud computing method and system for achieving mobile computing and obtaining mobile data | |
| CN103530388A (en) | Performance improving data processing method in cloud storage system | |
| CN104133882A (en) | HDFS (Hadoop Distributed File System)-based old file processing method | |
| CN107241319A (en) | Distributed network crawler system and dispatching method based on VPN | |
| CN104270412A (en) | Three-level caching method based on Hadoop distributed file system | |
| CN102013991B (en) | Method, management equipment and system for automatically expanding capacity | |
| CN103442070A (en) | Private cloud storage resource allocation method based on statistical prediction | |
| Ye et al. | A novel blocks placement strategy for Hadoop | |
| CN108519856B (en) | Data block copy placement method based on heterogeneous Hadoop cluster environment | |
| CN103049574B (en) | Realize key assignments file system and the method for file dynamic copies | |
| CN105141656B (en) | A kind of implementation of load balancing of the internet lightweight application based on cloud platform | |
| CN106446099A (en) | Distributed cloud storage method and system and uploading and downloading method thereof | |
| CN103067479A (en) | Network disk synchronized method and system based on file coldness and hotness | |
| CN104182487A (en) | Unified storage method supporting various storage modes | |
| CN107203623B (en) | Load balancing and adjusting method of web crawler system | |
| Irie et al. | A novel automated tiered storage architecture for achieving both cost saving and qoe | |
| CN106919696A (en) | SEO websites construction method and the response method of SEO requests | |
| CN106201921A (en) | The method of adjustment of a kind of cache partitions capacity and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170104 |
|
| RJ01 | Rejection of invention patent application after publication |