CN111708601A - Cloud computing-oriented intelligent storage gateway, system and control method thereof - Google Patents

Cloud computing-oriented intelligent storage gateway, system and control method thereof Download PDF

Info

Publication number
CN111708601A
CN111708601A CN202010348526.4A CN202010348526A CN111708601A CN 111708601 A CN111708601 A CN 111708601A CN 202010348526 A CN202010348526 A CN 202010348526A CN 111708601 A CN111708601 A CN 111708601A
Authority
CN
China
Prior art keywords
intelligent storage
storage gateway
file system
distributed file
service data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010348526.4A
Other languages
Chinese (zh)
Inventor
郭彦辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Udsafe Technology Co ltd
Original Assignee
Beijing Udsafe Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Udsafe Technology Co ltd filed Critical Beijing Udsafe Technology Co ltd
Priority to CN202010348526.4A priority Critical patent/CN111708601A/en
Publication of CN111708601A publication Critical patent/CN111708601A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/18File system types
    • G06F16/182Distributed file systems
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00Data switching networks
    • H04L12/66Arrangements for connecting between networks having differing types of switching systems, e.g. gateways
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1001Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1097Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/08Protocols for interworking; Protocol conversion
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/18Multiprotocol handlers, e.g. single devices capable of handling multiple protocols
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • G06F2009/45595Network integration; Enabling network access in virtual machine instances

Abstract

The invention relates to an intelligent storage gateway, a system and a control method thereof facing cloud computing, wherein the intelligent storage gateway comprises: the virtual machine access interface is used for receiving the service data sent by the virtual machine through a network protocol and writing the service data to the disk equipment; the disk device is formed by mapping block devices of the distributed file system and is used for writing the service data of the virtual machine into the distributed file system; the technical scheme provided by the invention aims to solve the problem that the same service access file in a service pool is generated by storing according to an object under a cloud environment, all virtual machines needing to access the file and the object can access an intelligent storage gateway under the same service pool, and the intelligent storage gateway is responsible for docking a distributed file system.

Description

Cloud computing-oriented intelligent storage gateway, system and control method thereof
Technical Field
The invention relates to the field of cloud storage, in particular to an intelligent storage gateway and system for cloud computing and a control method thereof.
Background
At present, an enterprise is in a serious revolution of data center IT infrastructure which is not seen for decades, and meanwhile, along with the expansion of the scale of the enterprise and the growth of self business, the enterprise needs to uniformly plan, manage and measure storage resources from different dimensions of user business, data, service and the like.
In the face of the PB-level mass data storage requirement brought by the continuous improvement of the informatization degree, the traditional storage system has a bottleneck in the expansion of capacity and performance. Cloud storage has gained wide acceptance in the industry with its advantages of strong scalability, high cost performance, good fault tolerance, etc. The distributed file system and the distributed block storage are used as important technologies in cloud storage, and become important cornerstones for establishing cloud storage development.
In the existing cloud environment, the virtual machines have to rely on a distributed file system for file and object storage, and for different service pools, the network problem is the biggest problem of limiting different service pools from accessing the distributed file system.
Disclosure of Invention
Aiming at the defects of the prior art, the invention aims to solve the problem that the same service access file in a service pool is generated by storing according to an object under the cloud environment, all virtual machines needing to access the file and the object can access an intelligent storage gateway under the same service pool, and the intelligent storage gateway is responsible for docking a distributed file system.
The purpose of the invention is realized by adopting the following technical scheme:
in a cloud computing oriented smart storage gateway, the improvement comprising:
the virtual machine access interface is used for receiving the service data sent by the virtual machine through a network protocol and writing the service data to the disk equipment;
the disk device is formed by mapping block devices of the distributed file system and is used for writing the service data of the virtual machine into the distributed file system;
and the file management system is used for managing the disk equipment.
Preferably, the file management system is specifically configured to:
integrating the disk devices into a resource pool;
and dividing the volume for storing the file and the volume for storing the object on the resource pool, and respectively building a network protocol on the volume for storing the file and the volume for storing the object.
Further, the network protocol includes: NFS protocol, CIFS protocol and/or S3 protocol.
The invention also provides an intelligent storage system facing cloud computing, which is improved in that the intelligent storage system comprises the intelligent storage gateway; further comprising:
the virtual machine is used for writing the service data to the intelligent storage gateway through a user network file protocol;
the KVM virtualization module is used for mapping the block devices of the distributed file system to the intelligent storage gateway;
and the distributed file system is used for storing the service data.
Preferably, the KVM virtualization module is specifically configured to: and mapping the block devices of the distributed file system to the intelligent storage gateway in a virtio, scsi or ide hard disk drive mode.
Based on the same inventive concept, the invention also provides a control method based on the intelligent storage system facing the cloud computing, and the improvement is that the control method comprises the following steps:
the virtual machine sends the service data to the intelligent storage gateway through a user network file protocol;
KVM virtualization module maps block devices of distributed file system to intelligent storage gateway
The distributed file system stores the service data.
Compared with the closest prior art, the invention has the following beneficial effects:
the invention provides an intelligent storage system facing cloud computing, which comprises: the virtual machine access interface is used for receiving the service data sent by the virtual machine through a network protocol and writing the service data to the disk equipment; the disk device is formed by mapping block devices of the distributed file system and is used for writing the service data of the virtual machine into the distributed file system; and the file management system is used for managing the disk equipment. The technical scheme provided by the invention has no excessive requirements on a bottom storage system, can flexibly use any distributed file system, integrates block storage resources of the distributed file system, and provides file and object services to the outside uniformly.
The further intelligent storage gateway is embedded into cloud computing, and has the characteristics of high availability, cold and hot migration, configuration adjustment, affinity, counter affinity and the like related to the virtual machine.
The intelligent storage gateway can be embedded into a plurality of tenants, so that the network path is greatly reduced, the network delay is reduced, the performance is improved, meanwhile, the problem that the tenant network is communicated with the storage network is reduced, and the operation cost is greatly reduced.
The intelligent storage gateway can be embedded into any distributed file, single-point faults and performance bottlenecks are avoided, and the intelligent storage gateway has the advantages of any distributed file system, is high in data reliability, realizes I/O load balance among different nodes, supports online transverse expansion and the like.
The intelligent storage gateway uses an advanced file system, has the technologies of dynamic striping, multiple security levels, automatic verification processing of disk faults, rapid data recovery, data compression, repeated data deletion and the like, and utilizes the storage space to the maximum extent.
By the advantages, efficient and reliable operation of user services can be guaranteed.
Drawings
Fig. 1 is a schematic view of an application scenario of the cloud computing-oriented intelligent storage system provided by the invention.
Detailed Description
The following describes embodiments of the present invention in further detail with reference to the accompanying drawings.
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As is well known, in a conventional cloud platform, a distributed file system is used at a bottom layer, the distributed file system is built on a storage network, one project is a tenant, one tenant has one or more user service virtual machines, the user service virtual machines are isolated from the storage network by vlans, and it is very convenient for a virtual machine in one tenant to access a block storage, for example, an rbd provided by ceph can be directly used by a virtual machine in the tenant through a virtual, scsi, ide and other hard disk drives without passing through a network. However, it is inconvenient to access the file and object storage, and the storage network and the tenant network must be opened through the route between vlans and the virtual machine network is configured to access the file and object storage.
The invention provides an intelligent storage gateway facing cloud computing, which comprises:
the virtual machine access interface is used for receiving the service data sent by the virtual machine through a network protocol and writing the service data to the disk equipment;
the disk device is formed by mapping block devices of the distributed file system and is used for writing the service data of the virtual machine into the distributed file system;
and the file management system is used for managing the disk equipment.
The file management system is specifically configured to:
integrating the disk devices into a resource pool;
and dividing the volume for storing the file and the volume for storing the object on the resource pool, and respectively building a network protocol on the volume for storing the file and the volume for storing the object.
Wherein the network file protocol comprises: NFS protocol, CIFS protocol and/or S3 protocol.
Based on the cloud computing-oriented intelligent storage gateway, the invention also provides a cloud computing-oriented intelligent storage system, which comprises:
the virtual machine is used for writing the service data to the intelligent storage gateway through a user network file protocol;
the KVM virtualization module is used for mapping the block devices of the distributed file system to the intelligent storage gateway;
and the distributed file system is used for storing the service data.
The KVM virtualization module is specifically configured to: and mapping the block devices of the distributed file system to the intelligent storage gateway in a virtio, scsi or ide hard disk drive mode.
In the preferred embodiment provided by the invention, as shown in an application scenario shown in fig. 1, the cloud computing-oriented intelligent storage gateway plays a role in connecting files, object service data and a distributed file system in a virtual machine, and is composed of a protocol, a file system and a disk device, because the cloud computing-oriented intelligent storage gateway is embedded into a tenant network, a tenant can access the intelligent storage gateway through the tenant network without walking a storage network, so that the network path is shorter and the performance is higher, and secondly, the intelligent storage gateway accesses a distributed file system block device through virtio-sci, and the access concurrency is increased through striping of files, so that the performance is improved.
The disk device is provided by a distributed file system block device, and the distributed file system maps the distributed file system block device to the intelligent storage gateway through a virtio-scsi mode.
The method comprises the steps of using a file system to manage intelligent storage gateway disk devices, namely distributed file system block devices, constructing all the disk devices into a storage pool, and then dividing different volumes on the storage pool to respectively store files and object data. The intelligent storage gateway has various characteristics of various security levels, copy-on-write, snapshot and clone, read-write cache, dynamic striping, compression and deduplication, dynamic adjustment of file and object storage resources and the like through a file system.
The protocol plays a role in mapping and outputting storage space in the intelligent storage gateway, and according to the above, it is known that different volumes are divided on a storage pool to store files and object data respectively, and the different volumes are mapped and output through NFS, CIFS and S3 respectively for other virtual machines in tenants to access.
The distributed file system block device is the most common data storage method in a storage interface, provides integration for a cloud computing system through libvirt or qemu, and is a medium for an intelligent storage gateway to access the distributed file system.
The distributed file system is the final data carrier, and the data is finally processed by the distributed file system through the block device.
Further, the invention also provides a control method based on the cloud computing-oriented intelligent storage system, and the method comprises the following steps:
the virtual machine sends the service data to the intelligent storage gateway through a user network file protocol;
KVM virtualization module maps block devices of distributed file system to intelligent storage gateway
The distributed file system stores the service data.
Based on this, in a cloud environment, the intelligent storage gateway receives user service data through NFS, CIFS and S3 file and object storage protocols, manages distributed file block equipment through a file system, writes the distributed file block equipment into the distributed file system through virtio-scsi, realizes protocol conversion, reduces network paths and operation cost, has all advantages of the distributed file system, provides requirements of user service for accessing files and objects, and improves performance of accessing files and object storage.
The intelligent storage gateway and other virtual machines use the same network, namely a tenant network, user services access the intelligent storage gateway through the tenant network, and the intelligent storage gateway directly writes data of files and objects in the tenant network into the distributed file system, so that the storage network is skipped, network paths are reduced, network delay is reduced, and performance is improved.
The intelligent storage gateway uses the block device of the distributed file system, provides various security levels, copy-on-write, snapshot and clone, read-write cache, dynamic striping, compression deduplication, dynamic adjustment of file and object storage resources and other characteristics through file system management, finally provides file and object storage through a protocol, realizes protocol conversion, and simultaneously enables the intelligent storage gateway to use a plurality of strips to increase the performance of file and object storage without considering the security of the strips due to the security of the distributed file system block, and finally provides a new mode with performance superior to that of the traditional mode for accessing the file object storage in the cloud environment.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting the same, and although the present invention is described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: modifications and equivalents may be made to the embodiments of the invention without departing from the spirit and scope of the invention, which is to be covered by the claims.

Claims (6)

1. A cloud computing-oriented intelligent storage gateway, comprising:
the virtual machine access interface is used for receiving the service data sent by the virtual machine through a network protocol and writing the service data to the disk equipment;
the disk device is formed by mapping block devices of the distributed file system and is used for writing the service data of the virtual machine into the distributed file system;
and the file management system is used for managing the disk equipment.
2. The intelligent storage gateway of claim 1, wherein the file management system is specifically configured to:
integrating the disk devices into a resource pool;
and dividing the volume for storing the file and the volume for storing the object on the resource pool, and respectively building a network protocol on the volume for storing the file and the volume for storing the object.
3. The intelligent storage gateway of claim 1 or 2, wherein the network protocol comprises: NFS protocol, CIFS protocol and/or S3 protocol.
4. A cloud computing-oriented smart storage system comprising the smart storage gateway of any one of claims 1-3; further comprising:
the virtual machine is used for writing the service data to the intelligent storage gateway through a user network file protocol;
the KVM virtualization module is used for mapping the block devices of the distributed file system to the intelligent storage gateway;
and the distributed file system is used for storing the service data.
5. The intelligent storage system according to claim 1, wherein the KVM virtualization module is specifically configured to: and mapping the block devices of the distributed file system to the intelligent storage gateway in a virtio, scsi or ide hard disk drive mode.
6. A control method for a cloud computing-oriented intelligent storage system based on any one of claims 4 to 5, characterized by comprising:
the virtual machine sends the service data to the intelligent storage gateway through a user network file protocol;
KVM virtualization module maps block devices of distributed file system to intelligent storage gateway
The distributed file system stores the service data.
CN202010348526.4A 2020-04-28 2020-04-28 Cloud computing-oriented intelligent storage gateway, system and control method thereof Pending CN111708601A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010348526.4A CN111708601A (en) 2020-04-28 2020-04-28 Cloud computing-oriented intelligent storage gateway, system and control method thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010348526.4A CN111708601A (en) 2020-04-28 2020-04-28 Cloud computing-oriented intelligent storage gateway, system and control method thereof

Publications (1)

Publication Number Publication Date
CN111708601A true CN111708601A (en) 2020-09-25

Family

ID=72536856

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010348526.4A Pending CN111708601A (en) 2020-04-28 2020-04-28 Cloud computing-oriented intelligent storage gateway, system and control method thereof

Country Status (1)

Country Link
CN (1) CN111708601A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113805943A (en) * 2021-09-18 2021-12-17 上海中通吉网络技术有限公司 Protocol system of integrated intelligent device
CN113821170A (en) * 2021-08-31 2021-12-21 济南浪潮数据技术有限公司 Distributed storage system, access method and component

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107241426A (en) * 2017-06-27 2017-10-10 西安京华科讯软件科技有限公司 A kind of distributed memory system of the home server hard disk based on cloud computing

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107241426A (en) * 2017-06-27 2017-10-10 西安京华科讯软件科技有限公司 A kind of distributed memory system of the home server hard disk based on cloud computing

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113821170A (en) * 2021-08-31 2021-12-21 济南浪潮数据技术有限公司 Distributed storage system, access method and component
CN113805943A (en) * 2021-09-18 2021-12-17 上海中通吉网络技术有限公司 Protocol system of integrated intelligent device

Similar Documents

Publication Publication Date Title
US11379285B1 (en) Mediation for synchronous replication
US11520720B1 (en) Weighted resource allocation for workload scheduling
US10296258B1 (en) Offloading data storage to a decentralized storage network
CA2978889C (en) Opportunistic resource migration to optimize resource placement
CN103748545B (en) Data storage system and data storage control method
US10831398B2 (en) Storage device efficiency during data replication
US10871911B2 (en) Reducing data amplification when replicating objects across different sites
JP2019212330A (en) Scalable distributed storage architecture
JP2020052730A (en) Method to determine disposition for vm/container and volume in hci environment, and storage system
US11146564B1 (en) Login authentication in a cloud storage platform
CN103763383A (en) Integrated cloud storage system and storage method thereof
US10609174B2 (en) Parallel prefetching log/meta stream sub-portions to recreate partition states in a distributed computing system
US20190238412A1 (en) Policy based path management
US20170123675A1 (en) Method for layered storage of enterprise data
US10616134B1 (en) Prioritizing resource hosts for resource placement
US10521151B1 (en) Determining effective space utilization in a storage system
CN111949210A (en) Metadata storage method, system and storage medium in distributed storage system
CN103595799A (en) Method for achieving distributed shared data bank
CN111708601A (en) Cloud computing-oriented intelligent storage gateway, system and control method thereof
US11809379B2 (en) Storage tiering for deduplicated storage environments
US10812408B1 (en) Preventing concentrated selection of resource hosts for placing resources
US11068192B1 (en) Utilizing mutiple snapshot sources for creating new copy of volume in a networked environment wherein additional snapshot sources are reserved with lower performance levels than a primary snapshot source
CN117311593A (en) Data processing method, device and system
US11336519B1 (en) Evaluating placement configurations for distributed resource placement
US11121981B1 (en) Optimistically granting permission to host computing resources

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination