Disclosure of Invention
In view of this, the present invention provides a heterogeneous resource standardized encapsulation system based on a cloud environment, which not only can simplify the overall structure of the encapsulation system, but also can improve the utilization rate of resources.
Based on the above purpose, the present invention provides a heterogeneous resource standardized encapsulation system based on a cloud environment, which includes:
the physical resource encapsulation module is used for realizing the life cycle management of physical resources by encapsulating the physical machine resources and providing a series of API interfaces for managing the physical machine for the encapsulation system;
the centralized storage resource encapsulation module is used for realizing uniform data storage resource encapsulation by carrying out resource encapsulation on the centralized storage resource;
the distributed storage resource packaging module is used for packaging the local storage resources of the server into a virtual storage resource pool based on distributed storage software to form extensible virtual resources;
the container resource packaging module is used for enabling the application system to realize rapid deployment through the characteristics of light weight and rapid deployment of resource packaging of the container virtual sub-body;
the application encapsulation module is used for being compatible with multiple development languages and encapsulating the application and the corresponding specific operating environment so as to rapidly deploy and operate the application;
a network resource encapsulation module, configured to implement an internal SDN (software defined network) environment based on network resource encapsulation, or interface with an external SDN controller; thereby being capable of supporting virtual network management; the method is used for quickly establishing a stable, reliable and completely isolated private network and a virtual router based on a firewall and a router of an operating system kernel to realize network topology;
each packaging module in the packaging system abstracts heterogeneous resources in the system through a preset rule, so that a standard multilayer resource packaging model is formed, and a lower-layer mode is strictly called by an upper layer, so that the packaging of the heterogeneous resources maintains a uniform model.
Optionally, the packaging system classifies resources in the system, and designs three dimensions of abstraction, concrete and feature; the dimensional principle of abstract design is as follows: the resource is used as a physical or virtual child; the resources are shared and each resource is considered as a resource pool; the resource has a life cycle; the corresponding semantics of the resources are kept consistent.
Optionally, when the encapsulation system encapsulates resources, the encapsulation system further includes a two-layer resource management module including a conductor (constructor) module and an agent (client agent) module; the controller module is used for providing uniform semantics upwards, and realizing allocation, scheduling, monitoring, updating and destruction of specific resources in the resource pool downwards or towards the resource pool; the agent module is used for providing function encapsulation of resource entities, binding the resource entities, defining concrete action realization of cloud computing scheduling on resource requests, realizing different driving drivers according to resource characteristics and completing differentiation realization according to different manufacturers and equipment.
Optionally, the physical resource encapsulation module is further configured to manage a bare operating system, mount the operating system on a hardware shelf and maintain the operating system on the hardware shelf, solve addition and deletion of a physical machine, power management and operating system deployment, and provide a resource encapsulation service for a related application requiring a physical machine environment.
Optionally, the centralized storage resource encapsulation module is further configured to support coexistence of multiple backend storages, implement a commercial storage SAN, an NAS storage, and a distributed storage, and simultaneously enable storage backend to define a volume type respectively.
Optionally, the centralized storage resource encapsulation module is further configured to provide block storage, file storage, and file system functions, so that the extension is not limited by any hypervisor, and the storage logic is separated from the virtualization platform to implement storage fault domain isolation.
Optionally, the network resource encapsulation module is further configured to bind a public network IP to the router, and set a port mapping access virtual host.
From the above, the heterogeneous resource standardized encapsulation system based on the cloud environment provided by the invention classifies all resources of the system according to certain characteristics, and then encapsulates the resources based on different resources to form a standard multilayer encapsulation model, so that the structural design of the whole system is simplified, and all the resources are kept in a unified model; meanwhile, the standardized encapsulation based on the resources can also expand the type of resource encapsulation. Therefore, the cloud environment-based heterogeneous resource standardized packaging system can simplify the overall structure of the packaging system and improve the utilization rate of resources.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to specific embodiments and the accompanying drawings.
It should be noted that all expressions using "first" and "second" in the embodiments of the present invention are used for distinguishing two entities with the same name but different names or different parameters, and it should be noted that "first" and "second" are merely for convenience of description and should not be construed as limitations of the embodiments of the present invention, and they are not described in any more detail in the following embodiments.
First, for ease of understanding, the related concepts designed by this application are explained:
the cloud environment related in the application comprises a cloud operating system, wherein the cloud operating system is a data center operating system which takes a resource packaging technology as a core and takes a distributed cloud computing technology and a distributed cloud storage technology as supports. The cloud operating system manages massive basic hardware and software resources, enables a plurality of physical devices to be displayed outwards as logically packaged entities, drives large-scale hardware resources uniformly for the next time, manages a large number of distributed application programs for the last time, provides computing capacity, storage space and network throughput bandwidth which cannot be compared with a single operating system, is a comprehensive cloud platform management system supporting distributed architecture and centralized architecture and supporting traditional application and emerging internet application, and is an overall management operation system of a cloud computing data center.
The cloud operating system comprises OpenStack, and the OpenStack is a cloud infrastructure platform which is rich in functions, easy to deploy, capable of being expanded in a large scale and unified in standard, and meanwhile, services such as a container, a database, an SDN, big data, an Internet of things and an application market can be integrated to form complete cloud computing service capacity. At present, more than half of the 100 strong enterprises in the world adopt the OpenStack platform, and a large number of traditional enterprises are embracing the OpenStack open source framework.
The container resources related in the cloud operating system refer to a lightweight virtualization solution based on a container technology, the concept of 'build, ship, run' is provided, and development, testing and operation and maintenance processes are opened. By now the container image has been downloaded more than 40 hundred million times, 43 million applications are fully containerized globally. Eight banks in the first ten world have introduced the Docker technology, and leading telecom operators such as foreign AT & T are all using containers.
The cloud operating system also relates to Power server management technology, x86 server management technology, virtualization technology and VMware virtualization technology.
The present application is directed to an improved packaging system designed to optimize the utilization of structures and resources in the event that the existing packaging structure for a resource handling system is defective.
Taking the system design of an enterprise as an example, for an enterprise-level data center, in order to simplify management and improve resource efficiency, a heterogeneous resource scheduling engine covering all kinds of computing resources is designed as a data center operating system, and becomes the core of an enterprise information infrastructure. Such a system requires that the resource requirements of all enterprise applications, such as Power computing resources, x86 physical server resources, KVM virtual hosts, business virtual hosts, containers, etc., can be scheduled in parallel. The distributed computing framework has evolved from the traditional centralized computing framework, and thus has a clear analogy relationship with the traditional computing in concept. The distributed computing framework is divided into several layers including the encapsulation of resources, the orchestration of services, and the encapsulation of applications. According to the invention, by researching the distributed cross-domain computing framework driven by application requirements, the distributed framework with resources adjusted according to application running states and uniformly adapted according to needs is realized, and resource sharing and efficient cross-domain resource encapsulation are realized. Enterprise-level enhancement and independent research and development are carried out by relying on open source technologies such as Openstack, Docker and meso. By standardized encapsulation of data center mixed resources (physical resources such as servers, disk arrays and network equipment and virtual resources such as virtual machines and containers), an 'all-in-one' IT operation environment is provided for business application, and unified management and flexible allocation of basic resources are realized. The invention has the following important significance:
(1) and the resources are adjusted according to the application running state as required. For example: with the diversification of the applications carried by some enterprises, especially the distributed application with large consumption resources and obvious tide performance, the flexible expansion and contraction requirements of the resources are strong. Although the underlying resource can be replanned by technologies such as virtualization, the resource can not be transferred in the application running process, and the resource can only be re-partitioned and deployed after all tasks are completed. The resource partitioning of the application is basically estimated according to the resource peak value, and is adjusted periodically by combining the data observed in the operation. With the increase of the application of the enterprise internet, the resource demand of part of services shows obvious peaks and troughs, or sudden increase occurs occasionally, and the traditional estimation method causes huge resource waste. By researching the distributed cross-domain computing framework driven by application requirements, the resource reallocation under the application running state can be realized, the resource sharing is realized under the uniformly-adaptive distributed framework, the resource waste is avoided, and the use cost is reduced.
(2) The uniformly adaptive distributed framework realizes resource sharing. By researching the uniformly-adaptive distributed framework, the resource sharing can be carried out on various application frameworks, the formation of resource vertical shafts on the cloud by the various application frameworks is broken, the resource utilization rate is improved, the resource use cost is reduced, and the resource waste is avoided. Meanwhile, the uniformly-adaptive distributed framework can also provide a guidance function for the design of an application framework of a newly-built application system in the future, so that resource sharing is better performed among various user applications with different calculation types, the quick and flexible delivery of basic resources is supported, and the change of basic resource supply from demand configuration to demand supply is promoted.
(3) Efficient resource encapsulation. All resources of the system are classified according to certain characteristics, and then a standard multilayer packaging model is formed based on different resource packages, so that the structural design of the whole system is simplified, and all resources are kept in a unified model; meanwhile, the standardized encapsulation based on the resources can also expand the type of resource encapsulation. The design simplifies the development amount of resource encapsulation aiming at different devices, ensures that each device can customize and optimize the drive, and provides efficient resource access.
Specifically, the standardized encapsulation framework of heterogeneous resources defines a software architecture design for heterogeneous resource sealing in a cloud environment. In the framework, resources are abstracted firstly to form a standard multi-layer resource packaging model, a lower-layer mode is called strictly by an upper layer, a uniform model is kept on the packaging design of resources such as calculation, storage, network and the like, and the system design is simplified. Meanwhile, the standardized encapsulation of data center heterogeneous IT resources (physical resources such as servers, disk arrays and network equipment and virtual resources such as virtual machines and containers) is realized.
In an enterprise data center, an application system is not a software design architecture generally, and may be large enterprise-level software with an early C/S structure, layered architecture software with a B/S structure, and applications such as internet +, big data, internet of things and the like with a popular distributed software architecture. Different software architectures require different load bearing environments, and thus there are different times of minicomputers, physical servers, virtualization servers, centralized storage devices, distributed storage devices, managed switches, SDN switching devices, firewalls, routers, NFS devices, and the like, typically found in enterprise data centers. With the deep development of cloud computing technology, a large number of businesses in an enterprise have a cloud entering requirement, and a corresponding resource architecture is also a requirement that resources are managed and controlled.
The technical solutions for adding resources and devices into the cloud environment are various, such as PowerKVM technology of Power servers, IPMI technology of physical servers, API interface of x86 virtualization technology, command interface of storage devices, configuration management protocol of network devices, and so on. Under the same technical standard, the implementation of different manufacturers is very different. Therefore, if a peer-to-peer design is adopted, resource management of the enterprise cloud environment can be trapped in the slip which is a mass of repeated work.
The standardized encapsulation framework of the heterogeneous resources performs methodology analysis on resource encapsulation requirements, then classifies enterprise resources according to the methodology, and performs design of three dimensions of abstraction, concrete and feature under classification, so that encapsulation of the heterogeneous resources is greatly reduced by a simple framework, and resource admission of the whole resource type is possible. Meanwhile, the standardized resource encapsulation also expands the capability of a cloud environment, multiple physical resources and equipment are supported, virtual resources and equipment can be supported, and the type and flexibility of resource encapsulation are greatly expanded.
The heterogeneous resource standardized encapsulation framework abstracts the encapsulation of the resources into the following dimensions: 1. the resource is a physical or virtual child; 2. the resources are shared, each resource being treated as a resource pool; 3. resources all have a life cycle; 4. the semantics of the resources should be consistent. Therefore, in resource encapsulation, two layers of resource management, i.e., the Conductor and the Agent, need to be designed. Wherein the factor provides uniform semantics upwards, and is responsible for allocation, scheduling, monitoring, updating and destruction of specific resources in the resource pool downwards or towards the resource pool. The Agent provides the realization function encapsulation of the resource entity; the Agent binds a resource entity, defines the specific action realization of the cloud computing scheduling on the resource request, realizes different Driver according to the resource characteristics, and each Driver defines the specific action realization of the resource request of specific resources and completes the differentiation realization according to the difference of manufacturers and equipment. The differentiated implementation includes two types, one is inheritance and implementation of the driver, and the other is a specific function set written according to the characteristics of the manufacturer.
Referring to fig. 1, a schematic structural diagram of an embodiment of a heterogeneous resource standardized encapsulation system based on a cloud environment provided by the present invention is shown. The heterogeneous resource standardized encapsulation system based on the cloud environment comprises:
the physical resource encapsulation module 101 is configured to implement lifecycle management by encapsulating physical machine resources, and provide a series of API interfaces for managing physical machines for an encapsulation system. The physical resource encapsulation module 101 may manage a bare operating system, and maintain the bare operating system from hardware on-shelf, operating system installation to hardware off-shelf, thereby solving the problems of addition, deletion, power management, operating system deployment, and the like of a physical machine, and providing resource encapsulation service for related applications requiring a physical machine environment.
The centralized storage resource encapsulation module 102 is configured to implement uniform data storage resource encapsulation by performing resource encapsulation on the centralized storage under the dual drive of technical development. Wherein the storage resource encapsulation supports coexistence of multiple backend storages. Not only can commercial storage SAN, NAS storage (NFS protocol), and distributed storage be implemented, but also the storage backend can separately define volume types.
The distributed storage resource encapsulation module 103 is configured to encapsulate a local hard disk of the x86 server into a virtual storage resource pool based on distributed storage software, and provide functions of block storage, file storage, and a file system, so that the extension is not limited by performance and scale of any management service, that is, can be extended to thousands of nodes; meanwhile, based on the separation of the storage logic and the virtualization platform, the storage fault domain isolation can be realized, and the safety is improved. The platform is more stable due to the independent fault domain, the system is more suitable for application scenes with intensive IO or high reliability requirements, and cold and hot data separation, easy expansion, super-large capacity and rich service types are realized. In addition, a variety of cloud storage services may be acquired through a variety of types of storage pools.
The container resource encapsulation module 104 is configured to enable the application system to implement fast deployment and fast iteration of versions through the characteristics of light weight and fast deployment of the resource encapsulation technology of the container virtual child, improve the distribution strategy in the container-based gray release, and enable the container resource encapsulation to encapsulate resources at a finer granularity.
The application packaging module 105 is used for being compatible with multiple development languages on one hand and packaging the application and a specific running environment so as to rapidly deploy and run the application. Application encapsulation and fast delivery are achieved.
The network resource encapsulation module 106 is configured to implement an internal SDN environment based on network resource encapsulation, or interface with an external SDN controller, support virtual network management, and establish a stable, reliable, and completely isolated private network and virtual router quickly based on a firewall and a router of an operating system kernel, so as to implement a network topology. And can bind the IP of the public network for the router, set up the port to map and visit the fictitious host computer, in order to save IP resources of the public network. The network virtualization aspect can support various network forms, such as shared networks, private networks, subnets, virtual network cards, external networks, SNAT/DNAT, routers, firewalls, QoS and other software-defined network functions; meanwhile, the method has the capability of being compatible with SDN products of mainstream manufacturers, and supports the virtual network IP address repeating function of multiple tenants.
According to the embodiment, the heterogeneous resource standardized packaging system based on the cloud environment classifies all resources of the system according to certain characteristics, and then forms a standard multilayer packaging model based on different resource packages, so that the structural design of the whole system is simplified, and all the resources keep a uniform model; meanwhile, the standardized encapsulation based on the resources can also expand the type and flexibility of the resource encapsulation. Therefore, the cloud environment-based heterogeneous resource standardized packaging system can simplify the overall structure of the packaging system and improve the utilization rate of resources.
Optionally, the application relates to a cloud environment-based heterogeneous resource standardized encapsulation framework design, and finally, a standardized encapsulation system is constructed, so that a standardized environment of heterogeneous resources is provided for an enterprise-level application system, and diversified requirements of different business systems on computing, storage and networks are met. In order to meet the service scenario of cloud computing, heterogeneous resources are represented by computing resources of different processor architectures, centralized and distributed storage resources, handover networks such as management and SDN, software and hardware network elements, and the like. The standardized closure of heterogeneous resources provides rich scheduling resources and complex resource orchestration for resource scheduling in a cloud environment. For enterprise applications, the application architectures and scales are different, and the support in a simple resource environment cannot be normalized, so that the standardized encapsulation of heterogeneous resources is an important demand index for private cloud construction. The patent relates to cloud resource packaging for purchasing various IT equipment in nearly ten years in the center of an enterprise, and ensures that the guaranteed resources of the enterprise can be brought into a cloud environment by adopting a proper resource packaging technology according to business planning.
The method also supports the condition that the hardware resources are not satisfied, and meets the general requirements of enterprises for specific resources in a software form. Namely, the resources of the software simulation hardware can meet the deployment requirements of enterprise application on specific resources, such as typical resources of a virtual firewall, virtual load balancing and the like; on one hand, the resources of the software simulation hardware support flexible configuration and change, fine-grained resource supply in the cloud can be configured, and the characteristics of more flexibility and configurability compared with physical resources are realized.
Therefore, the protection-required idea of the present application includes at least the following:
(1) standardized encapsulation with heterogeneous resources: power computing resources, x86 servers, mainstream x86 virtualization technology, NAS storage, SAN storage, distributed storage, managed switching networks, SDN networks, NFV devices, containers, and all types of resources that occur in an enterprise cloud environment.
(2) A multi-layer resource packaging model is designed, a mode of a lower layer is strictly called by an upper layer, a unified model is kept on the packaging design of resources such as calculation, storage, network and the like, and the system design is simplified.
(3) The design realizes a plug-in resource driving system, and can flexibly add new resource types to the maximum extent, such as a server with a new model, network equipment in a software form and the like, without carrying out code reconstruction on the whole system.
(4) The standardized remote calling model is realized, and the remote calling can be realized by using a high-speed stable remote calling mode. The standardized communication protocol design can simplify the calling of an external system and realize high-speed request processing through an asynchronous interface.
Those of ordinary skill in the art will understand that: the discussion of any embodiment above is meant to be exemplary only, and is not intended to intimate that the scope of the disclosure, including the claims, is limited to these examples; within the idea of the invention, also features in the above embodiments or in different embodiments may be combined, steps may be implemented in any order, and there are many other variations of the different aspects of the invention as described above, which are not provided in detail for the sake of brevity.
In addition, well known power/ground connections to Integrated Circuit (IC) chips and other components may or may not be shown within the provided figures for simplicity of illustration and discussion, and so as not to obscure the invention. Furthermore, devices may be shown in block diagram form in order to avoid obscuring the invention, and also in view of the fact that specifics with respect to implementation of such block diagram devices are highly dependent upon the platform within which the present invention is to be implemented (i.e., specifics should be well within purview of one skilled in the art). Where specific details (e.g., circuits) are set forth in order to describe example embodiments of the invention, it should be apparent to one skilled in the art that the invention can be practiced without, or with variation of, these specific details. Accordingly, the description is to be regarded as illustrative instead of restrictive.
While the present invention has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those of ordinary skill in the art in light of the foregoing description. For example, other memory architectures (e.g., dynamic ram (dram)) may use the discussed embodiments.
The embodiments of the invention are intended to embrace all such alternatives, modifications and variances that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, substitutions, improvements and the like that may be made without departing from the spirit and principles of the invention are intended to be included within the scope of the invention.