CN110489203A - A kind of container Scheduling Framework system - Google Patents
A kind of container Scheduling Framework system Download PDFInfo
- Publication number
- CN110489203A CN110489203A CN201910585502.8A CN201910585502A CN110489203A CN 110489203 A CN110489203 A CN 110489203A CN 201910585502 A CN201910585502 A CN 201910585502A CN 110489203 A CN110489203 A CN 110489203A
- Authority
- CN
- China
- Prior art keywords
- container
- node
- message
- scheduling framework
- framework system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L61/00—Network arrangements, protocols or services for addressing or naming
- H04L61/45—Network directories; Name-to-address mapping
- H04L61/4505—Network directories; Name-to-address mapping using standardised directories; using standardised directory access protocols
- H04L61/4511—Network directories; Name-to-address mapping using standardised directories; using standardised directory access protocols using domain name system [DNS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L61/00—Network arrangements, protocols or services for addressing or naming
- H04L61/50—Address allocation
- H04L61/5007—Internet protocol [IP] addresses
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45562—Creating, deleting, cloning virtual machine instances
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45595—Network integration; Enabling network access in virtual machine instances
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a kind of container Scheduling Framework systems, including main controlled node, container node and storage unit；The container node and main controlled node are communicated to connect with storage unit, and for multiple containers combination of nodes to form container clustered node, the storage unit is used to store the status information of each container clustered node；The main controlled node is used to execute following steps: obtaining the status information at storage unit and practical cluster state information；Judge whether status information and the practical cluster state information of storage are consistent, if it is not, then executing rate-determining steps；Communication creation container instance is carried out with the control process of container clustered node, so that status information is consistent with practical cluster state.Container Scheduling Framework system of the invention passes through the difference for constantly comparing status information and practical cluster environment in cluster, once finding differences will be adjusted, so that cluster state information is consistent with practical cluster environment.
Description
Technical field
The present invention relates to a kind of field of computer technology more particularly to a kind of container Scheduling Framework systems.
Background technique
Currently, the data of big data era 59% be all invalid data, 70% data are excessively complicated, 85% enterprise's number
It can not timely and accurately be provided correctly for business according to the enterprise that framework can not adapt to demand, 98% that data volume and complexity increase
Information.Nowadays, it big data challenge and opportunity and deposits, big data service platform construction is just experiencing from simple to complex, by list
One arrives development process polynary, by part to whole omnibearing integrated.However support the cloud data center of big data service
Often there is following challenge: 1, being difficult to start large-scale calculations resource in a short time to cope with a large amount of distributed computational nodes
Demand；2, elastic calculation cost performance is low, and each virtual machine needs to consume a large amount of additional system resources, based on the batch processing from
In line analysis, these resources are even more the most of the time all in idle state.And how to allow to carry out change status information with
And realize that the main-machine communication across container becomes those skilled in the art's technical problem urgently to be resolved.
Summary of the invention
For overcome the deficiencies in the prior art, the purpose of the present invention is to provide a kind of container Scheduling Framework system, energy
Keep cluster state information consistent with practical cluster environment.
The purpose of the present invention adopts the following technical scheme that realization:
A kind of container Scheduling Framework system, including main controlled node, container node and storage unit；The container node and master
It controls node to communicate to connect with storage unit, multiple containers combination of nodes to form container clustered node, use by the storage unit
In the status information for storing each container clustered node；The main controlled node is for executing following steps:
Obtaining step: the status information at storage unit and practical cluster state information are obtained；
Judgment step: judge whether the status information of storage and practical cluster state information are consistent, if it is not, then executing control
Step processed；
Rate-determining steps: communication creation container instance is carried out with the control process of container clustered node, so that status information
It is consistent with practical cluster state.
Further, the storage unit is specially High Availabitity key assignments storage system ETCD.
Further, the container clustered node is in communication with each other by multiple containers node is constituted, for realizing whole right
Outer service.
Further, each container node in same container clustered node is stored with complete service data.
Further, each container node in same container clustered node guarantees the service number of its maintenance by Raft agreement
According to being consistent.
It further, further include virtual DNS component, the virtual DNS component is for container in monitoring of containers clustered node
The creation of node.
Further, the virtual DNS component is also used to search in the memory unit corresponding with the host name of container node
IP, and IP is distributed into corresponding container node and is used.
Further, the container node includes container, bridge and network extension element, the first message packet of the bridge
Loaded in the second message with routing iinformation, the second message is forwarded to target according to routing iinformation by the network extension element
Physical host；Network process on the target physical host obtains the first message for being responsible for the second message of decoding, and will
First message is forwarded at the bridge at target physical host.
Further, first message is TCP message.
Further, second message is UDP message.
Compared with prior art, the beneficial effects of the present invention are:
Container Scheduling Framework system of the invention passes through the status information and practical cluster environment constantly compared in cluster
Difference, so that cluster state information is consistent with practical cluster environment, is able to achieve unified once finding differences will be adjusted
Shared resource distribution in cluster between different data frame.
Detailed description of the invention
Fig. 1 is the control flow chart of the main controlled node in container Scheduling Framework system of the invention；
Fig. 2 is the schematic illustration of the working mechanism of container scheduling；
Fig. 3 is the schematic illustration of virtual DNS working mechanism；
Fig. 4 is the concrete principle schematic diagram of virtual network working mechanism.
Specific embodiment
In the following, being described further in conjunction with attached drawing and specific embodiment to the present invention, it should be noted that not
Under the premise of conflicting, new implementation can be formed between various embodiments described below or between each technical characteristic in any combination
Example.
As depicted in figs. 1 and 2, a kind of container Scheduling Framework system, including main controlled node, container section are present embodiments provided
Point and storage unit；The container node and main controlled node are communicated to connect with storage unit, and multiple containers combination of nodes is with shape
At container clustered node, the storage unit is used to store the status information of each container clustered node；The storage unit is specific
For High Availabitity key assignments storage system ETCD.The container clustered node is in communication with each other by multiple containers node to be constituted, and reality is used for
Now whole externally service；Each container node in same container clustered node is stored with complete service data, same container
Each container node in clustered node guarantees that the service data of its maintenance is consistent by Raft agreement.
The main controlled node is for executing following steps:
Step S1: the status information at storage unit and practical cluster state information are obtained；
Step S2: judge whether the status information of storage and practical cluster state information are consistent, if it is not, then executing control
Step；
Step S3: with the control process of container clustered node carry out communication creation container instance so that status information with
Practical cluster state is consistent.
The present embodiment uses the container Scheduling Framework layer built based on Kubernetes to monitor all container cluster services
The state of device plays great buffer function to the performance of cluster, effectively improves the handling capacity of entire cluster.The container tune
Degree ccf layer, which passes through, constantly compares status information in cluster and whether practical cluster environment is consistent, when the state for finding cluster is believed
When breath is inconsistent with virtual condition, carries out communicating creation container instance with the control process of container clustered node, make status information
It is again consistent with actual environment.The present embodiment further includes High Availabitity key assignments storage system ETCD, High Availabitity key assignments storage system
The ETCD that unites stores various cluster state informations, is in communication with each other by multiple nodes and constitutes whole external service, and each node stores
Complete data, and by Raft agreement guarantee that the data of each node maintenance are consistent.
Kubernetes, abbreviation K8s are to replace abridging made of 8 characters " ubernete " with 8.It is an open source,
For managing the application of the containerization in cloud platform in multiple main frames, the target of Kubernetes is to allow the application of deployment container
Simple and efficient, Kubernetes is provided using deployment, planning, a kind of mechanism for updating and safeguarding.
Traditional application deployment way is to install application by plug-in unit or script.Disadvantage of this is that the fortune of application
Row, configuration, management, all life cycles will bind with current operation system, do so the upgrading for being unfavorable for applying and update/return
The operation such as rolling, naturally it is also possible to certain functions are realized by way of creating virtual machine, but virtual machine is very heavy, and unfavorable
In portability.
New mode is realized by deployment container mode, is mutually isolated between each container, and each container has oneself
File system, process will not influence each other between container, can distinguish computing resource.Relative to virtual machine, container energy rapid deployment,
Since container and bottom facilities, machine file system decouple, so it can be carried out between different clouds, different editions operating system
Migration.
Container takes up less resources, disposes fastly, and each application can be packaged into a container mirror image, each application and container
Between in a pair a relationship also container is made to have greater advantages, can be in the stage of build or release, for application wound using container
Container mirror image is built, because each application does not need to combine with remaining application stack, also not dependent on production environment foundation structure,
This makes from researching and developing to testing, produces and can provide consistent environment.Similarly, container than virtual machine light weight, more " transparent ", this is more just
In monitoring and management.
As shown in Figure 3 and Figure 4, the container Scheduling Framework system of the present embodiment further includes virtual DNS component, described virtual
Creation of the DNS component for container node in monitoring of containers clustered node.The virtual DNS component is also used in the memory unit
IP corresponding with the host name of container node is searched, and IP is distributed into corresponding container node and is used.
The container node includes container, bridge and network extension element, and the first message of the bridge, which is packaged in, to be had
In second message of routing iinformation, the second message is forwarded to target physical master according to routing iinformation by the network extension element
Machine；Network process on the target physical host obtains the first message for being responsible for the second message of decoding, and first is reported
Text is forwarded at the bridge at target physical host.First message is TCP message.Second message is UDP message.
IP caused by virtual DNS component solution dynamic creation container is unpredictable, is responsible for monitoring the wound of container in cluster
Build situation, virtual DNS, which is responsible for searching host name in ETCD storage and correspond to IP and distributes to container, to be used, and passes through network expanded set
Part realize container across host communication, the TCP message of container bridge is packaged in the UDP message with routing iinformation, net
Message is forwarded to target physical host by routing iinformation by network extension element, and the network process on target physical host is responsible for solution
Packet.TCP message is forwarded on the container bridge of the machine, to be realized in the case where not influencing container legacy network mechanism
Container across host communication.DNS (domain name system), provides domain name resolution service, solves the IP for being difficult to remember
Address problem, by more human nature it is readable remember can identify in a manner of map corresponding IP address.Domain name mapping is that domain name is directed toward net
Stage space IP allows people that a kind of service of website is accessed by the domain name registered with can be convenient.IP address is network subscript
Know the numeric address of website, remember for convenience, IP address is replaced to identify site address using domain name.Domain name mapping is exactly domain
Name arrives the conversion process of IP address.The parsing work of domain name is completed by DNS component.
Domain name mapping also makes domain name be directed toward, server setting, domain name configures and reversely IP registration etc..Say to obtain simple point
It is exactly by domain name mapping easy to remember into IP, it is then domain name mapping is existed to an IP address that service is completed by dns server
A subdirectory and domain name are bound on the host of this IP address.Dynamic-dns (English: Dynamic DNS, abbreviation DDNS) are domains
One of name system (DNS) automatically updates the technology of name server (Name server) content.According to the domain name of internet
Rule is concluded, domain name must follow fixed IP address.But dynamic DNS system provides a fixed title clothes for dynamic domain
Business device (Name server) enables external user to connect the network address of dynamic subscriber through immediate updating.
The shared resource point of container Scheduling Framework system of the invention between big data frames different in the same cluster
Match, completes across the fine-grained resource-sharing between frame；By on different framework integration to unified basic framework, allow to use
Family manages cluster resource across frame, can not only increase resource be for and handling capacity, improve the flexibility of platform, but also
It is able to achieve the effective use of faster data calculating and resource, the complexity of architecture can be substantially reduced and improves it reliably
Property.
The above embodiment is only the preferred embodiment of the present invention, and the scope of protection of the present invention is not limited thereto,
The variation and replacement for any unsubstantiality that those skilled in the art is done on the basis of the present invention belong to institute of the present invention
Claimed range.
Claims (10)
1. a kind of container Scheduling Framework system, which is characterized in that including main controlled node, container node and storage unit；The appearance
Device node and main controlled node are communicated to connect with storage unit, and multiple containers combination of nodes is described to form container clustered node
Storage unit is used to store the status information of each container clustered node；The main controlled node is for executing following steps:
Obtaining step: the status information at storage unit and practical cluster state information are obtained；
Judgment step: judging whether the status information of storage and practical cluster state information are consistent, if it is not, then executing control step
Suddenly；
Rate-determining steps: communication creation container instance is carried out with the control process of container clustered node, so that status information and reality
Border cluster state is consistent.
2. a kind of container Scheduling Framework system as described in claim 1, which is characterized in that the storage unit is specially that height can
With key assignments storage system ETCD.
3. a kind of container Scheduling Framework system as described in claim 1, which is characterized in that the container clustered node is by multiple
Container node is in communication with each other composition, for realizing whole externally service.
4. a kind of container Scheduling Framework system as claimed in claim 3, which is characterized in that each in same container clustered node
Container node is stored with complete service data.
5. a kind of container Scheduling Framework system as claimed in claim 4, which is characterized in that each in same container clustered node
Container node guarantees that the service data of its maintenance is consistent by Raft agreement.
6. a kind of container Scheduling Framework system as described in claim 1, which is characterized in that it further include virtual DNS component, it is described
Creation of the virtual DNS component for container node in monitoring of containers clustered node.
7. a kind of container Scheduling Framework system as claimed in claim 6, which is characterized in that the virtual DNS component is also used to
IP corresponding with the host name of container node is searched in the memory unit, and IP is distributed into corresponding container node and is used.
8. a kind of container Scheduling Framework system as described in claim 1, which is characterized in that the container node include container,
Bridge and network extension element, the first message of the bridge are packaged in the second message with routing iinformation, the network
Second message is forwarded to target physical host according to routing iinformation by extension element；Network process on the target physical host
For being responsible for the second message of decoding to obtain the first message, and the first message is forwarded at the bridge at target physical host.
9. a kind of container Scheduling Framework system as claimed in claim 8, which is characterized in that first message is TCP message.
10. a kind of container Scheduling Framework system as claimed in claim 8, which is characterized in that second message is UDP report
Text.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910585502.8A CN110489203A (en) | 2019-07-01 | 2019-07-01 | A kind of container Scheduling Framework system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910585502.8A CN110489203A (en) | 2019-07-01 | 2019-07-01 | A kind of container Scheduling Framework system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110489203A true CN110489203A (en) | 2019-11-22 |
Family
ID=68546410
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910585502.8A Pending CN110489203A (en) | 2019-07-01 | 2019-07-01 | A kind of container Scheduling Framework system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110489203A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111767145A (en) * | 2020-06-24 | 2020-10-13 | 中国工商银行股份有限公司 | Container scheduling system, method, device and equipment |
| CN113676471A (en) * | 2021-08-17 | 2021-11-19 | 上海道客网络科技有限公司 | Cross-node communication method, system, medium and electronic device based on container cloud platform |
| CN116034576A (en) * | 2020-08-13 | 2023-04-28 | 易享信息技术有限公司 | Container Orchestration System (COS) service discovery across multiple COS clusters based on COS cluster domain name system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106302771A (en) * | 2016-08-23 | 2017-01-04 | 浪潮电子信息产业股份有限公司 | A kind of method for configuring domain name of the application created based on Docker container |
| CN106790483A (en) * | 2016-12-13 | 2017-05-31 | 武汉邮电科学研究院 | Hadoop group systems and fast construction method based on container technique |
| CN107766157A (en) * | 2017-11-02 | 2018-03-06 | 山东浪潮云服务信息科技有限公司 | Distributed container cluster framework implementation method based on domestic CPU and OS |
| US20180173512A1 (en) * | 2016-12-19 | 2018-06-21 | International Business Machines Corporation | Optimized Creation of Distributed Storage and Distributed Processing Clusters on Demand |
| CN108829509A (en) * | 2018-05-03 | 2018-11-16 | 山东汇贸电子口岸有限公司 | Distributed container cluster framework resources management method based on domestic CPU and operating system |
| CN109582441A (en) * | 2018-11-30 | 2019-04-05 | 北京百度网讯科技有限公司 | For providing system, the method and apparatus of container service |
-
2019
- 2019-07-01 CN CN201910585502.8A patent/CN110489203A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106302771A (en) * | 2016-08-23 | 2017-01-04 | 浪潮电子信息产业股份有限公司 | A kind of method for configuring domain name of the application created based on Docker container |
| CN106790483A (en) * | 2016-12-13 | 2017-05-31 | 武汉邮电科学研究院 | Hadoop group systems and fast construction method based on container technique |
| US20180173512A1 (en) * | 2016-12-19 | 2018-06-21 | International Business Machines Corporation | Optimized Creation of Distributed Storage and Distributed Processing Clusters on Demand |
| CN107766157A (en) * | 2017-11-02 | 2018-03-06 | 山东浪潮云服务信息科技有限公司 | Distributed container cluster framework implementation method based on domestic CPU and OS |
| CN108829509A (en) * | 2018-05-03 | 2018-11-16 | 山东汇贸电子口岸有限公司 | Distributed container cluster framework resources management method based on domestic CPU and operating system |
| CN109582441A (en) * | 2018-11-30 | 2019-04-05 | 北京百度网讯科技有限公司 | For providing system, the method and apparatus of container service |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111767145A (en) * | 2020-06-24 | 2020-10-13 | 中国工商银行股份有限公司 | Container scheduling system, method, device and equipment |
| CN116034576A (en) * | 2020-08-13 | 2023-04-28 | 易享信息技术有限公司 | Container Orchestration System (COS) service discovery across multiple COS clusters based on COS cluster domain name system |
| CN116034576B (en) * | 2020-08-13 | 2023-10-03 | 硕动力公司 | Container Orchestration System (COS) service discovery across multiple COS clusters based on COS cluster domain name system |
| CN113676471A (en) * | 2021-08-17 | 2021-11-19 | 上海道客网络科技有限公司 | Cross-node communication method, system, medium and electronic device based on container cloud platform |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104486445B (en) | Distributed extendable resource monitoring system based on cloud platform | |
| US11172022B2 (en) | Migrating cloud resources | |
| US10148757B2 (en) | Migrating cloud resources | |
| CN108989091A (en) | Based on the tenant network partition method of Kubernetes network, storage medium, electronic equipment | |
| CN110489203A (en) | A kind of container Scheduling Framework system | |
| US9830333B1 (en) | Deterministic data replication with conflict resolution | |
| CN110489205A (en) | A kind of education services architecture system based on container cluster | |
| CN110489204A (en) | A kind of big data platform architecture system based on container cluster | |
| CN111970337B (en) | P2P network communication construction method, system, medium and terminal under cross-cloud environment | |
| CN113626128B (en) | Audio-visual media micro-service third-party module access method, system and electronic equipment | |
| US20180131605A1 (en) | Floating internet protocol for private networks | |
| CN113630479B (en) | Domain name resolution method and related products | |
| Zhang et al. | Design and implementation of task scheduling strategies for massive remote sensing data processing across multiple data centers | |
| CN116010027A (en) | Method for managing task processing cluster, method for executing task and container cluster | |
| CN107249038A (en) | Business datum retransmission method and system | |
| CN109309581A (en) | Container management system crossing hardware architecture | |
| Yang et al. | High-performance docker integration scheme based on OpenStack | |
| CN109218378A (en) | A kind of small-sized logistics management platform designing method based on cloud platform | |
| Liu et al. | KubFBS: A fine‐grained and balance‐aware scheduling system for deep learning tasks based on kubernetes | |
| CN108228272A (en) | WEB containers generation processing method, equipment and server | |
| US20220383219A1 (en) | Access processing method, device, storage medium and program product | |
| CN111131449A (en) | Method for constructing service clustering framework of water resource management system | |
| John et al. | Making Cloud Easy: Design Considerations and First Components of a Distributed Operating System for Cloud. | |
| CN110365743A (en) | A kind of a variety of load balancer implementation methods that can customize loading algorithm of support realized based on Zookeeper | |
| CN115622976A (en) | Domain name management system, domain name registration and resolution method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191122 |
|
| RJ01 | Rejection of invention patent application after publication |