CN110166542A - Distributed memory system and data read-write method based on ICP/IP protocol - Google Patents
Distributed memory system and data read-write method based on ICP/IP protocol Download PDFInfo
- Publication number
- CN110166542A CN110166542A CN201910406187.8A CN201910406187A CN110166542A CN 110166542 A CN110166542 A CN 110166542A CN 201910406187 A CN201910406187 A CN 201910406187A CN 110166542 A CN110166542 A CN 110166542A
- Authority
- CN
- China
- Prior art keywords
- data
- storage
- node
- management module
- request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/16—Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
- H04L69/161—Implementation details of TCP/IP or UDP/IP stack architecture; Specification of modified or new header fields
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of distributed memory systems based on ICP/IP protocol, including client, application server, front-end server and storage end；Storage end includes memory node management module, storage parent node and storage child node；Memory node management module includes the information list of data attribute, and the data that the storage parent node and the storage child node are stored can be grouped by memory node management module according to data attribute；It stores parent node for storing data, stores child node copy for storing data.The invention also discloses a kind of data read-write methods of distributed memory system.The efficiency that distributed memory system and data read-write method provided by the invention improves data storage and read.
Description
Technical field
The present invention relates to computer data storages, and in particular to a kind of distributed memory system based on ICP/IP protocol and
Data read-write method.
Background technique
Distributed memory system is that data dispersion is stored in more independent equipment.With distributed memory system
Extensive use in each field, user are also higher and higher to the performance requirement of storage system, improve distributed block storage system
Performance, which seems, to become more and more important.ICP/IP protocol has become most common Networking protocol between computer, and different manufacturers is allowed to produce
The entirely different operating system of computer of various models interconnected by TCP/IP, it be " Global Internet " or " because
The basis of special net " becomes a kind of actual industrial standard.
Existing distributed management system is that management function is distributed on network using the network platform, and management system is most
To be managed data by file name, such as system can put all types of files for containing the same word in title
It sets together.On the one hand, take the relatively complicated complexity of file so that looking into, cause data file to transfer speed slower；On the other hand
Data, which are looked into take, also needs secondary screening, after data screening for the first time is carried out on network, then screens number in original database again
According to data, a large amount of time is not only wasted, while called data is slower, readwrite performance is poor.
Summary of the invention
Goal of the invention: the purpose of the present invention is to provide a kind of distributed memory systems based on ICP/IP protocol, data
Reading/writing method improves the readwrite performance of storage system.
Technical solution: it the invention discloses a kind of distributed memory system based on ICP/IP protocol, including client, answers
With server, front-end server and storage end；Client is connect with application server, for uploading data, generating reading and writing data
Request, and data and reading and writing data request are sent to application server；Application server is connect with front-end server, and being used for will
Data and reading and writing data request are sent to front-end server；Front-end server is connect with storage end, can carry out data with storage end
Interaction；Storage end includes storage parent node and storage child node, and each storage parent node is connected with multiple storage child nodes；Storage
Parent node read-write requests and data for receiving data are sent in response to received data read-write requests and data to client
Or storing data, and reading and writing data request and data are sent to multiple storage child nodes；Store child node for receiving data
Read-write requests and data, and according to reading and writing data request to client send or storing data copy；Storage end peripheral hardware physics
Equipment connectivity port, so that external physical equipment is connect with storage end and uploads data to storage end.
Further, storage end further includes memory node management module, and memory node management module includes data attribute
Information list, memory node management module can will store parent node and store the data that be stored of child node according to data attribute into
Row grouping.
The invention also discloses a kind of method for writing data carried out using distributed memory system, this method comprises: (1)
External physical equipment or client generate data write request, and data write request is sent out via application server and front-end server
It send to storage end；(2) in response to data write request, storage end stores the data received into storage parent node, and simultaneously
The copy of data is stored into multiple storage child nodes.
It is further limited as to the above method, storage end further includes memory node management module, memory node management
Module includes the information list of data attribute, and step (2) further includes that, in response to data write request, memory node management module will
Data are matched with the information list of data attribute, and are grouped according to the match attribute of data, and then will be after grouping
Data are stored into storage parent node, and simultaneously store its copy into multiple storage child nodes.
Invention additionally discloses a kind of method for reading data carried out using distributed memory system, this method comprises: (1) is objective
Family end generates data read request, and it is transmitted to front-end server by application server；(2) front-end server reads data
Request is sent to storage end；(3) in response to data read request, memory node management module extracts the data category in data read request
Property, and search corresponding attribute packet data in storage parent node and storage child node simultaneously according to data read request；It (4) will be with
Front-end server is transferred and are sent to the matched data of data read request or data copy, and passes through application clothes by front-end server
Business device feeds back to client.
The invention also discloses a kind of storage end, which includes memory node management module, storage parent node and deposits
Child node is stored up, each storage parent node is connected with multiple storage child nodes, and memory node management module connects the female section of each storage
Point and each storage child node；Memory node management module includes the information list of data attribute, is used for data and its copy
It is grouped according to data attribute；Store parent node for storing data；Store the copy of child node for storing data.
The utility model has the advantages that compared with prior art, distributed management system of the invention can effectively utilize distributed management,
Data resource is integrated, stored and is transferred.After outer data is uploaded, system can be by outer data and Installed System Memory
It stores up database and carries out data comparison screening, the identical data grouping of attribute is stored.Storage parent node and sub- memory node reflect
Setting is penetrated so that operating lookup simultaneously to the data copy after former data and node mapping in data read process.Such system
It has been equivalent to mostly sub- entrance, can more quickly have been inquired in internal system database by this little entrance, called data, simultaneously
Each data manipulation is needed the multiple User behavior carried out to be reduced to once by system, i.e., operation reduces opening for a memory access every time
Pin, improves the efficiency of system.
In addition, distributed management system of the invention is framed in storage end by TCP/IP sets multiple storage parent nodes,
Each storage parent node connects multiple sub- memory nodes.In system storage parent node failure, sub- memory node can be connect rapidly
For storage parent node work, so that effectively lifting system availability and reliability, not only makes checking for file more succinct, together
When called data fast speed, can be used family find required data file in a short time；And in terms of safety, it is based on
The data transfer rate of ICP/IP protocol is very fast, saves time and resource, while reducing the case where loss of data damages, drop
Low work difficulty, ensure that the safety of data, improves work efficiency.
Detailed description of the invention
Fig. 1 is the structural schematic diagram of distributed memory system of the invention；
Fig. 2 is method for writing data flow chart of the invention；
Fig. 3 is method for reading data flow chart of the invention；
Fig. 4 is storage end structural schematic diagram of the invention.
Specific embodiment
The present invention is described further in the following with reference to the drawings and specific embodiments:
As shown in Figure 1, the application first proposed a kind of distributed memory system based on ICP/IP protocol, including client
End 101, application server 102, front-end server 103 and storage end 104；Client 101 includes multiple client operation port,
It connect with application server 102, and data are uploaded by client operation port, generate reading and writing data request, and by data
And reading and writing data request is sent to application server 102.Application server 102 is connect with front-end server 103, for number of transferring
According to, by data and reading and writing data request be sent to front-end server 103；Front-end server 103 is connect with storage end 104, and preceding
Holding includes node Application service element in server 103, can correspond to storage parent node and storage child node in storage end, mention
High read-write efficiency；Front-end server 103 can carry out data interaction with storage end 104.
As shown in figure 4, storage end 104 includes memory node management module 401, multiple storage parent nodes 402 and multiple
Child node 403 is stored, each storage parent node 402 is connected with multiple storage child nodes 403；Memory node management module 401 is wrapped
The information list of data attribute is included, received data can be grouped by memory node management module according to data attribute classification.
Include multiple mode classifications in information list, such as according to data type include numeric type, text-type and byte type, according to data
Format includes audio, video, picture etc., includes the modes such as data-link, USB flash disk, network account login according to upload mode.Storage is female
Node 402 for receiving data read-write requests and grouping after data, in response to received data read-write requests and packet count
According to the transmission of client 101 or storing data, and by reading and writing data request and after being grouped, data are sent to multiple storage child nodes
403；Storage child node 403 be used for receive storing data read-write requests and grouping after data, and according to reading and writing data request to
Client 101 is sent or the copy of storing data.Data are uploaded to storage end 104, storage end 104 for convenience of external physical equipment
Peripheral hardware physical equipment connectivity port, is connected with physical equipment node 105, and external equipment can pass through data-link, USB flash disk, network account
Number log in etc. modes data are uploaded to storage end 104.
The application also additionally provides a kind of method for writing data carried out using distributed memory system, as shown in Fig. 2, should
Method the following steps are included:
Step S201: external physical equipment or client generate data write request, and data write request is taken via application
Business device and front-end server are sent to storage end.
Step S202: in response to data write request, memory node management module is by the information list of data and data attribute
It is matched, and is grouped according to the match attribute of data.
Step S203: the data after grouping are stored into storage parent node, and its copy is stored to multiple deposit simultaneously
It stores up in child node.
Above-mentioned method for writing data can guarantee the safety of data storage, when storage parent node can not work normally,
The storage of data can be completed by storage child node.
Present invention also provides a kind of method for reading data carried out using distributed memory system, as shown in figure 3, the party
Method the following steps are included:
Step 301: client generates data read request, and it is transmitted to front-end server by application server.
Step 302: data read request is sent to storage end by front-end server.
Step 303: in response to the data read request, memory node management module extracts the data category in data read request
Property, and search corresponding attribute packet data in storage parent node and storage child node simultaneously according to data read request.
Step 304: it will transfer and be sent to front-end server with the matched data of data read request or data copy, and by
Front-end server feeds back to client by application server.
The mapping of storage parent node and multiple storage child nodes is arranged in storage end, to former number in above-mentioned method for reading data
According to the data copy after mapping with node while lookup is operated, is equivalent to and opens up multiple subquery entrances, pass through these subqueries
Entrance more rapid can be inquired in storage end in conjunction with the inquiry channel of female memory node and read data.And each channel
Inquiry read operation it is independent mutually, when a channel can not be inquired, other channels still ensure that the normal reading of data.Separately
On the one hand, the grouping that memory node management module carries out data can further increase the efficiency of reading data.
Disclosed herein as well is a kind of storage ends, as shown in figure 4, the storage end includes memory node management module 401, deposits
Store up parent node 402, storage child node 403；Each storage parent node 402 is connected with multiple storage child nodes 403, memory node pipe
Reason module 401 connects each storage parent node 402 and each storage child node 403.Memory node management module 401 includes data
The information list of attribute for being grouped data and its copy according to data attribute, and is sent to storage 402 He of parent node
Store child node 403.Store parent node 402 for storing data；Store the copy of child node 403 for storing data.
Claims (6)
1. a kind of distributed memory system based on ICP/IP protocol, it is characterised in that: including client, application server, preceding
Hold server and storage end；
The client is connect with application server, for uploading data, generating reading and writing data request, and data and data is read
Write request is sent to application server；
The application server is connect with front-end server, for the data and reading and writing data request to be sent to front end
Server；
The front-end server is connect with storage end, can carry out data interaction with storage end；
The storage end includes storage parent node and storage child node, and each storage parent node is connected with multiple storage child nodes；
The storage parent node is asked for receiving the reading and writing data request and the data in response to the received reading and writing data of institute
Ask and the data send to client or store the data, and by the reading and writing data request and the data be sent to it is more
A storage child node；The storage child node is used to receive the reading and writing data request and the data, and according to the data
Read-write requests send or store the copy of the data to client；
The storage end peripheral hardware physical equipment connectivity port, so that external physical equipment is connect with storage end and uploads data to depositing
Chu Duan.
2. distributed memory system according to claim 1, which is characterized in that the storage end further includes memory node pipe
Module is managed, the memory node management module includes the information list of data attribute, and memory node management module can be deposited described
The data that storage parent node and the storage child node are stored are grouped according to data attribute.
3. a kind of method for writing data carried out using distributed memory system described in claim 1, which is characterized in that described
Method includes:
(1) the external physical equipment or the client generate data write request, and by data write request via application service
Device and front-end server are sent to the storage end；
(2) in response to the data write request, the storage end stores the data received into storage parent node, and
The copy of the data is stored into multiple storage child nodes simultaneously.
4. method for writing data according to claim 3, which is characterized in that the storage end further includes memory node management
Module, the memory node management module include the information list of data attribute, and the step (2) further includes, in response to described
Data write request, the memory node management module match the data with the information list of data attribute, and according to
The match attribute of the data is grouped, and then the data after grouping are stored into storage parent node, and simultaneously will
Its copy is stored into multiple storage child nodes.
5. a kind of method for reading data carried out using distributed memory system as claimed in claim 2, which is characterized in that described
Method for reading data includes:
(1) client generates data read request, and it is transmitted to front-end server by application server；
(2) data read request is sent to storage end by the front-end server；
(3) in response to the data read request, the memory node management module extracts the data category in the data read request
Property, and corresponding attribute packet count in the storage parent node and the storage child node is searched according to data read request simultaneously
According to；
(4) front-end server will be transferred and be sent to the matched data of data read request or data copy, and by front end services
Device feeds back to client by application server.
6. a kind of storage end, which is characterized in that the storage end includes memory node management module, storage parent node and storage
Node, each storage parent node are connected with multiple storage child nodes, and the memory node management module connection is each
Store parent node and each storage child node；
The memory node management module includes the information list of data attribute, is used for the data and its copy according to data
Attribute is grouped；The storage parent node is for storing data；The storage child node is used to store the copy of the data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910406187.8A CN110166542A (en) | 2019-05-16 | 2019-05-16 | Distributed memory system and data read-write method based on ICP/IP protocol |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910406187.8A CN110166542A (en) | 2019-05-16 | 2019-05-16 | Distributed memory system and data read-write method based on ICP/IP protocol |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110166542A true CN110166542A (en) | 2019-08-23 |
Family
ID=67634783
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910406187.8A Pending CN110166542A (en) | 2019-05-16 | 2019-05-16 | Distributed memory system and data read-write method based on ICP/IP protocol |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110166542A (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1941736A (en) * | 2005-09-30 | 2007-04-04 | 西门子通信技术（北京）有限公司 | Content distributing system and method for re-directing user request |
| US20090019007A1 (en) * | 2007-03-28 | 2009-01-15 | Kabushiki Kaisha Toshiba | System, method, and apparatus for searching information across distributed databases |
| CN102752381A (en) * | 2012-06-28 | 2012-10-24 | 北京邮电大学 | Multi-movable-duplicate mechanism applied to distributed storage and access method thereof |
| CN103412897A (en) * | 2013-07-25 | 2013-11-27 | 中国科学院软件研究所 | Parallel data processing method based on distributed structure |
| CN105760556A (en) * | 2016-04-19 | 2016-07-13 | 江苏物联网研究发展中心 | Low-time delay high-throughput multi-copy file read-write optimization method |
| CN106790563A (en) * | 2016-12-27 | 2017-05-31 | 浙江省公众信息产业有限公司 | Distributed memory system and method |
| CN108182263A (en) * | 2018-01-05 | 2018-06-19 | 郑州云海信息技术有限公司 | A kind of date storage method of data center's total management system |
-
2019
- 2019-05-16 CN CN201910406187.8A patent/CN110166542A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1941736A (en) * | 2005-09-30 | 2007-04-04 | 西门子通信技术（北京）有限公司 | Content distributing system and method for re-directing user request |
| US20090019007A1 (en) * | 2007-03-28 | 2009-01-15 | Kabushiki Kaisha Toshiba | System, method, and apparatus for searching information across distributed databases |
| CN102752381A (en) * | 2012-06-28 | 2012-10-24 | 北京邮电大学 | Multi-movable-duplicate mechanism applied to distributed storage and access method thereof |
| CN103412897A (en) * | 2013-07-25 | 2013-11-27 | 中国科学院软件研究所 | Parallel data processing method based on distributed structure |
| CN105760556A (en) * | 2016-04-19 | 2016-07-13 | 江苏物联网研究发展中心 | Low-time delay high-throughput multi-copy file read-write optimization method |
| CN106790563A (en) * | 2016-12-27 | 2017-05-31 | 浙江省公众信息产业有限公司 | Distributed memory system and method |
| CN108182263A (en) * | 2018-01-05 | 2018-06-19 | 郑州云海信息技术有限公司 | A kind of date storage method of data center's total management system |
Non-Patent Citations (2)
| Title |
|---|
| DINKARSITARAM 等: "《迁移到云端——在云计算的新世界开发应用》", 30 June 2015 * |
| 李洪心: "《电子商务安全 第2版》", 30 April 2012 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20060015529A1 (en) | Method and apparatus of hierarchical storage management based on data value | |
| CN105447046A (en) | Distributed system data consistency processing method, device and system | |
| CN104699718A (en) | Method and device for rapidly introducing business data | |
| US11429566B2 (en) | Approach for a controllable trade-off between cost and availability of indexed data in a cloud log aggregation solution such as splunk or sumo | |
| CN101694626A (en) | Script execution system and method | |
| CN107818120A (en) | Data processing method and device based on big data | |
| CN108108476A (en) | The method of work of highly reliable distributed information log system | |
| CN107911249A (en) | Order line sending method, device and the equipment of a kind of network equipment | |
| CN105592178B (en) | A kind of back end method for determining position and device | |
| CN108427728A (en) | Management method, equipment and the computer-readable medium of metadata | |
| CN104239353B (en) | WEB classification control and log audit method | |
| CN110287201A (en) | Data access method, device, equipment and storage medium | |
| CN109542861A (en) | File management method, device and system | |
| CN106649164A (en) | Hardware interface management method | |
| CN111767327A (en) | Data warehouse component method and system with dependency relationship among data streams | |
| CN111159176A (en) | Method and system for storing and reading mass stream data | |
| CN107580032A (en) | Data processing method, device and equipment | |
| CN111435286A (en) | Data storage method, device and system | |
| CN107943412A (en) | A kind of subregion division, the method, apparatus and system for deleting data file in subregion | |
| CN101789963A (en) | Data synchronization system | |
| WO2017000592A1 (en) | Data processing method, apparatus and system | |
| CN113934713A (en) | Order data indexing method, system, computer equipment and storage medium | |
| CN106933497B (en) | Management scheduling device, system and method based on SAS | |
| US20100174762A1 (en) | Apparatus, System, and Method for Maintaining A Context Stack | |
| CN110166542A (en) | Distributed memory system and data read-write method based on ICP/IP protocol |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190823 |