CN114443069A - Method and system for dynamically expanding cluster to construct operating system mirror image - Google Patents
Method and system for dynamically expanding cluster to construct operating system mirror image Download PDFInfo
- Publication number
- CN114443069A CN114443069A CN202111669242.6A CN202111669242A CN114443069A CN 114443069 A CN114443069 A CN 114443069A CN 202111669242 A CN202111669242 A CN 202111669242A CN 114443069 A CN114443069 A CN 114443069A
- Authority
- CN
- China
- Prior art keywords
- construction
- task
- type
- machine
- construction machine
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000010276 construction Methods 0.000 claims abstract description 280
- 230000008569 process Effects 0.000 claims description 15
- 238000004458 analytical method Methods 0.000 claims description 9
- 238000003860 storage Methods 0.000 claims description 9
- 230000001502 supplementing effect Effects 0.000 claims description 7
- 238000004891 communication Methods 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 238000006467 substitution reaction Methods 0.000 description 3
- 238000009826 distribution Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 101000888786 Gloeobacter violaceus (strain ATCC 29082 / PCC 7421) Proton-gated ion channel Proteins 0.000 description 1
- DIWRORZWFLOCLC-UHFFFAOYSA-N Lorazepam Chemical compound C12=CC(Cl)=CC=C2NC(=O)C(O)N=C1C1=CC=CC=C1Cl DIWRORZWFLOCLC-UHFFFAOYSA-N 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005265 energy consumption Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 238000007670 refining Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/60—Software deployment
- G06F8/61—Installation
- G06F8/63—Image based installation; Cloning; Build to order
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/70—Software maintenance or management
- G06F8/71—Version control; Configuration management
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention belongs to the technical field of operating system mirror image construction, and particularly provides a method and a system for constructing an operating system mirror image by a dynamically expanded cluster, wherein the method comprises the following steps: the server side constructs template information when receiving a creation requirement, generates a corresponding construction task based on the template information and stores the construction task in a database; receiving a request sent by a construction machine, and judging whether the construction machine is idle or not according to request information; if so, issuing the construction tasks to corresponding construction machines to execute the construction tasks; if not, inquiring whether an uninitiated construction task exists in the database; when the un-started construction tasks exist and the number of the un-started construction tasks is larger than a set first threshold value, selecting a host of the host cluster, deploying the selected host as a new construction machine according to the task type, and adding the construction machine into a corresponding local area network; and the server receives a request sent by the newly deployed construction machine, and issues a corresponding construction task to the construction machine to execute the construction task. And a building machine is rapidly added, and the compiling speed is improved.
Description
Technical Field
The invention relates to the technical field of operating system mirror image construction, in particular to a method and a system for constructing an operating system mirror image by a dynamically expanded cluster.
Background
The operating system serves as a basic platform of modern computer technology and is responsible for interaction of application programs and users. The application program is provided with a uniform interface with the user. And shielding the difference of different hardware at the bottom layer. And finally, the process of the user can be run efficiently under the condition that the user does not feel. However, in the current environment, hardware devices are more and more complex, and hardware scenes are more and more extensive. Taking Linux operating system as an example, as an operating system, the Linux operating system can run on a server as a service operating system to provide services. Meanwhile, the method can run on a mobile phone platform with an ARM architecture, and provides bottom-layer interface support for an Android operating system. The intelligent vehicle-mounted intelligent device can also run on a vehicle-mounted operating system, and can provide intelligent device support for the vehicle. In the field of edge computing, the Linux operating system can support edge devices.
Therefore, it becomes important to flexibly customize the operating system according to the characteristics of the hardware platform for different hardware platforms. The same operating system, if it is desired to adapt to different operating environments. Different custom generation schemes for the installation image of the operating system are required. In a different architectural environment. If the expansion is needed, the different architectures need to be expanded horizontally. And simultaneously, the application scenes are considered to be different. Each application scenario may be for a different architecture. Therefore, the application scene needs to be longitudinally expanded. The longitudinal extension is divided into four scenes in the scheme: server scene/mobile device scene/edge device scene/cloud computing scene.
The existing Linux distribution version is manufactured by the following steps: 1. the source of the software package is ready to be included. Including all software packages that need to be packaged. 2. The pungi configuration file is modified. Mainly comprising a Comps file and a Varients file. Where the comp file includes a collection of defined software packages. The Varients file includes a list of additional software packages. After configuration is complete, the pungi command is used. The parameters are increased. And Pungi reads the corresponding configuration file. And generating corresponding output. The defects of the manufacturing mode of the Linux distribution are mainly as follows: 1. human intervention is required. Comps and other profiles need to be written manually. If the configuration file was written incorrectly or not in compliance with the configuration source. An error condition may occur. 2. Automated construction cannot be achieved. Requiring human involvement for configuration each time. Configuration source and output paths, formats, etc. 3. The expansion is complex. If a horizontally-extended architecture or a vertically-extended application scene is needed, a corresponding configuration file and a server need to be manually maintained. Is performed manually. The error is easy to occur, and the unified management cannot be performed.
Disclosure of Invention
The invention provides a method and a system for constructing an operating system mirror image by a dynamically expanded cluster, aiming at the problems that the existing operating system hair style can not be automatically constructed by human intervention, if an architecture or a longitudinal expansion application scene needs to be transversely expanded, a corresponding configuration file and a server need to be manually maintained, errors are easy to occur, and meanwhile unified management can not be carried out.
The technical scheme of the invention is as follows:
in a first aspect, the technical solution of the present invention provides a method for dynamically expanding a cluster to construct an operating system image, wherein based on a host cluster, the host cluster includes a server and a plurality of builders communicating in the same local area network; the method comprises the following steps:
the server side constructs template information when receiving a creation requirement, generates a corresponding construction task based on the template information and stores the construction task in a database;
the server receives a request sent by the construction machine and judges whether the construction machine is idle or not according to the request information;
when the building machine is idle, the building tasks are issued to the corresponding building machine to execute the building tasks;
when all the building machines are busy, inquiring whether an unfixed building task exists in the database;
when the un-started construction tasks exist and the number of the un-started construction tasks is larger than a set first threshold value, selecting a host of the host cluster, deploying the selected host as a new construction machine according to the task type, and adding the construction machine into a corresponding local area network;
and the server receives a request sent by the newly deployed construction machine, and issues a corresponding construction task to the construction machine to execute the construction task.
An ordinary host can be rapidly deployed into a building machine. The method can dynamically and transversely and rapidly increase the construction machine without any configuration modification on the original cluster, thereby improving the compiling speed.
Further, the step of the server receiving the request sent by the builder and judging whether the builder is idle according to the request information includes:
the server receives the request sent by the construction machine, refreshes the state of the construction machine and sets the online state;
judging whether the construction machine is idle or not according to the request information;
if yes, judging whether the state is a first idle state or a second idle state; the first idle state is idle and has no task to be executed; the second idle state is a current idle state.
Further, when the building machine is idle, the step of issuing the building task to the corresponding building machine to execute the building task includes:
when the building machine is idle, acquiring the type information of the idle building machine according to the request information;
inquiring whether a corresponding construction task exists in a database or not according to the type information of the construction machine;
if the corresponding construction task exists, the inquired construction task is issued to the corresponding idle construction machine to execute the construction task;
and if the corresponding construction tasks do not exist, inquiring whether the construction tasks which are not started exist in the database, and if the construction tasks which are not started do not exist, closing the first idle-state construction machine.
When the tasks which are executed periodically exist, the request information of the construction machine also comprises the request information of whether a new task is needed, and the tasks which are executed periodically can be executed and completed within the specified time.
Further, if there is no corresponding construction task, the step of querying whether there is an unextarted construction task in the database further includes:
if the construction tasks which do not start exist, the number of the construction tasks of the same type in the database is obtained;
judging whether a first type of construction machine exists or not; defining a building machine in a first idle state, wherein the building machine with the same architecture type as a first type of building task in the building machines in the first idle state is a first type of building machine; defining the construction tasks with the same type and the largest quantity in the database as a first type construction task;
if so, configuring the scene type of the first type of construction machine to be the same as the scene type of the first type of construction task;
after configuration is completed, the server receives a request sent by the first type construction machine, and issues the first type construction task to the first type construction machine to execute the construction task.
The construction machine can be longitudinally expanded, and the configuration of the construction machine is transferred under different scenes of the same structure. Tasks between the building machines do not interfere with each other. The expansion can be performed quickly.
Further, the step of judging whether the first type of building machine exists further includes:
if the first type of building machine does not exist, judging whether a second type of building machine exists or not; defining the building machines in the second idle state, wherein the building machines with the number of tasks executed periodically and less than a second threshold value exist in the second idle state as second type building machines;
if the second type of building machine exists, stopping the service of the second type of building machine, and configuring the scene type of the second type of building machine to be the same as the scene type of the first type of building task;
after configuration is completed, the server receives a request sent by a second type of construction machine, and issues a first type of construction task to the second type of construction machine to execute the construction task;
if the second type of building machine does not exist, judging whether the number of the first type of building tasks is larger than a first threshold value; if yes, executing the following steps: selecting a host of the host cluster, deploying the selected host as a new construction machine according to the task type, and adding the construction machine into a corresponding local area network; if not, executing the steps: and the server receives the request sent by the construction machine and judges whether the construction machine is idle or not according to the request information. The construction machine can be longitudinally expanded, and the configuration of the construction machine is transferred under different scenes of the same structure. Tasks between the building machines do not interfere with each other. The expansion can be performed quickly.
Further, the method comprises:
the construction machine sends a request to the server once every set first interval time;
and when the server side does not receive the request sent by the construction machine at the set second time interval, refreshing the state of the construction machine and setting the offline state.
Further, the template information refers to all information required by the construction machine in the construction process, and includes the framework of the template, the name of the template, the type of the template, all software packages of the template, and configuration files of the template;
in the step of receiving the issued construction task and executing the construction task, the step of executing the construction task includes:
accessing all software package data through a software package source, downloading a corresponding software package in the source through a configuration file, and generating a corresponding configuration file;
analyzing the dependence of the software package to obtain all software packages capable of supporting configuration and dependence packages and generating a software package list;
classifying the software package list and the construction type of the construction machine, supplementing the software package according to the customized information, and finally generating and storing a task template, wherein the task template is used for the constructed configuration file;
and selecting a task template to execute the construction task to complete the construction of the mirror image before the set time based on the priority and the task generation time.
Further, the request information in the step of sending the request to the server end every time the builder sets a first interval time comprises the architecture of the builder, the type of the builder, the state of the builder and the number of task templates to be executed in the builder;
the builder determines whether the current request requires a new task based on its state, the priority of the task template, and the task generation time.
In a second aspect, the technical solution of the present invention provides a system for dynamically expanding a cluster to construct an operating system image, including a host cluster, where each host of the host cluster is provided with a deployment execution end; configuring a plurality of hosts of a host cluster in the same local area network, wherein the hosts can be mutually communicated with each other;
the host of the same local area network comprises a server and a plurality of construction machines;
the server is used for receiving the configuration information and the requirements of the user, constructing template information according to the received information, generating a corresponding construction task based on the template information and storing the construction task in a database; receiving a request sent by a construction machine, and issuing a construction task to the corresponding construction machine when judging that the construction machine is idle; when no construction machine is judged to be idle, inquiring whether an uninitiated construction task exists in a database and outputting information to a deployment execution end;
the construction machine is used for sending a request to the server once every set first interval time; the system is also used for receiving and executing the construction task;
and the deployment execution end is used for deploying the host in which the execution end is positioned as a new construction machine according to the task type and adding the construction machine into the corresponding local area network when the server has the construction tasks which are not started and the number of the construction tasks is greater than a set first threshold value. And is also used to reconfigure the scene type of the builder according to the received information.
Further, the template information refers to all information required by the construction machine in the construction process, and includes the framework of the template, the name of the template, the type of the template, all software packages of the template, and configuration files of the template;
the construction machine comprises a configuration and analysis module, a customization module, a task storage module, a task scheduling module and a task execution module;
the configuration and analysis module is used for accessing all software package data through a software package source, downloading a corresponding software package in the source through a configuration file and generating a corresponding configuration file; analyzing the dependence of the software package to obtain all software packages capable of supporting configuration and dependence packages and generating a software package list;
the customizing module is used for classifying the construction types of the construction machine based on the software package list and supplementing the software packages according to customizing information, finally generating a task template and sending the task template to the task storage module, wherein the task template is used for constructing a configuration file;
the task storage module is used for storing the task template generated by the customization module;
the task scheduling module is used for selecting a task template and sending the task template to the task execution module based on the priority and the task generation time; the task scheduling module is also used for judging whether a new task is needed or not based on the priority and the time of task generation;
and the task execution module is used for executing the construction task to finish the construction of the mirror image before the set time.
According to the technical scheme, the invention has the following advantages: 1: the software package for constructing the related files and the foundation can be automatically generated. Since during the build process, the software package is the input source for the build. The builder accesses all package data through the package source. And downloading the corresponding software package in the source through the configuration file. This process requires support by the configuration file and the software package source. The system can automatically select a proper source and generate a corresponding configuration file. The configuration file is not required to be written by manual work, and the software package source is not required to be considered to participate in organization and configuration. The complexity of manual operation is reduced. Time is saved. And reduces the chance of human error. 2: the construction machine can be expanded horizontally and vertically based on the requirement dynamic. Tasks between the building machines do not interfere with each other. Under the condition that the original cluster is not required to be modified in any configuration, the construction machine can be quickly added, and the expansion can be quickly carried out, so that the compiling speed is improved.
In addition, the invention has reliable design principle, simple structure and very wide application prospect.
Therefore, compared with the prior art, the invention has prominent substantive features and remarkable progress, and the beneficial effects of the implementation are also obvious.
Drawings
In order to more clearly illustrate the embodiments or technical solutions in the prior art of the present invention, the drawings used in the description of the embodiments or prior art will be briefly described below, and it is obvious for those skilled in the art that other drawings can be obtained based on these drawings without creative efforts.
FIG. 1 is a schematic flow diagram of a method of one embodiment of the invention.
Fig. 2 is a schematic flow diagram of a method of another embodiment of the invention.
FIG. 3 is a flow chart illustrating the process performed by the building machine in the method of one embodiment of the present invention.
FIG. 4 is a schematic block diagram of a system of one embodiment of the present invention.
Detailed Description
The operating system on the server platform often has powerful CPU, memory, hard disk, and other devices. The speed and efficiency of executing commands is sought after in hardware. The computing power can be enhanced. And thus at the operating system level. Efficiency of operating system execution is much sought after. Some customization to the operating system of the server is required. The operating system of the adaptation server needs to carry the basic operating environment required by the application. And install a specific software library. The efficiency requirements of the server application can be supported.
At the same time, the operating system may also need to adapt to some mobile device scenarios. For example, hardware devices such as Android mobile phones and tablet computers. This type of device is distinct from a server device. First, from the architecture perspective, the server architecture mainly adopts the X86 architecture or the Power architecture. And mobile devices often employ an ARM architecture CPU. At the same time. As for mobile devices, the power consumption of the CPU affects the efficiency and lifetime of devices such as application software and batteries, and therefore, the power consumption of the CPU needs to be considered when operating. At the same time, the device often requires a large amount of user interaction and operation. User interaction requires a fast response in real time. Thus in the context of a mobile device. Some unnecessary development software and unnecessary runtime files are cut down first. Meanwhile, the related application of energy consumption monitoring on the mobile phone operating system needs to be increased.
Compared with the two application scenes, the method is common. Still other types of scenarios require the operating system to respond in real time. For example, the scene of the edge device has a high requirement on real-time performance. It is required that the response to a specific signal can be realized in the microsecond level. Efficiency and operating peaks are relatively abandoned to improve the response time of the operating system to certain signals and operations.
In current cloud computing scenarios, the operating system does not run on any hardware platform. But rather runs on a software environment that is emulated by the host. In the cloud computing scenario, in order to adapt to the environment more quickly, the operating system may directly provide a basic disk image without constructing a hardware boot device. And reading the disk image of the operating system by the host machine to directly guide and start the virtual machine.
It can be appreciated from the above description that the same operating system, if desired, may be adapted to different operating environments. Different custom generation schemes for the installation image of the operating system are required. The current mainstream architectures are X86, Arm and Power architectures. And thus in different architectural environments. If the expansion is needed, the different architectures need to be expanded horizontally. And simultaneously, the application scenes are considered to be different. Each application scenario may be for a different architecture. Therefore, the application scene needs to be expanded longitudinally. The longitudinal expansion is divided into four scenarios in the scheme: server scene/mobile device scene/edge device scene/cloud computing scene. The scenarios supported by each architecture may differ: for example, the X86 and Power architectures are server and virtualization scenarios, which are not typically the case with mobile devices. The Arm architecture is mainly based on mobile devices and edge device scenes, but the server scene needs to be partially involved as well as the virtualized scene. Therefore, the scenarios that need to be adapted in view of the above operating systems are different. In order to adapt to the dynamic expansion requirement, the invention provides an operating system image construction method capable of supporting dynamic expansion.
In order to make those skilled in the art better understand the technical solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention. It should be noted that the server is a center of the management task, and the building machine is a host that actually performs the building work; RPM is the build source data.
As shown in fig. 1, an embodiment of the present invention provides a method for dynamically expanding a cluster to construct an operating system image, where based on a host cluster, the host cluster includes a server and a plurality of builders that are in the same local area network for communication; the method comprises the following steps:
step 1: the server side constructs template information when receiving a creation requirement, generates a corresponding construction task based on the template information and stores the construction task in a database;
step 2: the server receives a request sent by the construction machine and judges whether the construction machine is idle or not according to the request information; when the building machine is idle, executing step 6; when all the building machines are busy, executing the step 3;
in fact, the process of sending a request to a server by a builder is divided into two processes, one is a process of requesting a task, and the other is a process of monitoring the builder by using a heartbeat mechanism; and the server opens the http port, and can be called by any construction machine after opening. As long as the corresponding interface is called successfully, the server registers the basic information of the construction machine to a database of the server;
and step 3: inquiring whether an uninitiated construction task exists in a database;
and 4, step 4: when the un-started construction tasks exist and the number of the un-started construction tasks is larger than a set first threshold value, selecting a host of the host cluster, deploying the selected host as a new construction machine according to the task type, and adding the construction machine into a corresponding local area network; here, the task type includes an architecture type and a scenario type;
it should be noted that if all the building machines are building. But the construction tasks of the server side are excessive, so that the tasks are overstocked. The embodiment provides the one-key deployment script, and a common host can be rapidly deployed into the building machine. The building machine can be quickly added under the condition that the original cluster is not required to be modified in any configuration, so that the compiling speed is improved.
And 5: the server receives a request sent by a newly deployed construction machine, and executes the step 6;
step 6: and issuing the construction tasks to corresponding construction machines to execute the construction tasks.
In some embodiments, the step 2, in which the server receives the request sent by the builder, and the step of determining whether the builder is idle according to the request information includes:
step 21: the server receives the request sent by the construction machine, refreshes the state of the construction machine and sets the online state;
step 22: judging whether the construction machine is idle or not according to the request information; if yes, go to step 23; if not, executing the step 3;
step 23: judging whether the state is a first idle state or a second idle state; the first idle state is idle and has no task to be executed; the second idle state is a current idle state in which there are tasks that are periodically executed.
As shown in FIG. 2, in some embodiments, the step of obtaining type information of an idle builder from the request information when the builder is idle in step 6 includes:
s161: when the building machine is idle, acquiring the type information of the idle building machine according to the request information;
s162: inquiring whether a corresponding construction task exists in a database or not according to the type information of the construction machine; if the corresponding construction task exists, executing S163, and if the corresponding construction task does not exist, executing S164;
s163: sending the inquired construction tasks to corresponding idle construction machines to execute the construction tasks;
s164: inquiring whether an un-started construction task exists in the database, and if the un-started construction task does not exist, executing S165; if there is an un-started construction task, executing S166;
s165: the first idle state builder is shut down.
S166: acquiring the quantity of construction tasks of the same type in a database;
s167: judging whether a first type of construction machine exists or not; if yes, executing S168, otherwise, executing S170;
s168: configuring the scene type of a first type of construction machine to be the same as the scene type of a first type of construction task;
s169: after configuration is completed, the server receives a request sent by a first type of construction machine, and issues a first type of construction task to the first type of construction machine to execute the construction task;
s170: judging whether a second type of building machine exists or not; if so, perform S171; if not, go to S173;
s171: stopping the service of the second type of building machine, and configuring the scene type of the second type of building machine to be the same as the scene type of the first type of building task;
s172: after configuration is completed, the server receives a request sent by a second type of construction machine, and issues a first type of construction task to the second type of construction machine to execute the construction task;
s173: judging whether the number of the first type of construction tasks is larger than a first threshold value; if yes, executing step 4: selecting a host of the host cluster, deploying the selected host as a new construction machine according to the task type, and adding the construction machine into a corresponding local area network; if not, executing the step 2.
For example, when the building requirement of the server operating system image is less, but the building requirement of the edge device is more, the building machine can be longitudinally extended, and the configuration of the building machine is transferred between the server scene and the edge device scene. Tasks between the building machines do not interfere with each other. The expansion can be performed quickly.
It should be noted that, in the embodiment of the present application, the construction machine sends a request to the server once every time the first interval is set; and when the server side does not receive the request sent by the construction machine at the set second time interval, refreshing the state of the construction machine and setting the offline state. The request information comprises the framework of the construction machine, the type of the construction machine, the state of the construction machine and the number of task templates to be executed in the construction machine;
here, the builder sends a request every 3 seconds. The server considers the construction machine to be online as long as the server can receive the request of the construction machine within 9 seconds. And if the server does not receive the request information of the construction machine within 9 seconds, the construction machine is considered to be offline.
The builder determines whether the current request requires a new task based on its state, the priority of the task template, and the task generation time.
The template information refers to all information required by the construction machine in the construction process, and comprises the framework of the template, the name of the template, the type of the template, all software packages of the template and configuration files of the template;
as shown in FIG. 3, in some embodiments, in step 6, the step of receiving the issued build task and executing the build task, the step of executing the build task includes:
s261: accessing all software package data through a software package source, downloading a corresponding software package in the source through a configuration file, and generating a corresponding configuration file;
s262: analyzing the dependence of the software package to obtain all software packages capable of supporting configuration and dependence packages and generating a software package list;
s263: classifying the software package list and the construction type of the construction machine, supplementing the software package according to the customized information, and finally generating and storing a task template, wherein the task template is used for the constructed configuration file;
s264: and selecting a task template to execute the construction task to complete the construction of the mirror image before the set time based on the priority and the task generation time.
If there is no special configuration, the system has a default source of software packages. The default software package source comprises a default compiled software package, common application software and the like. Common application software is a software package compiled by other compiling systems (e.g. koji).
1. Take the example of generating a server image under the X86 architecture.
The source configuration is not required, and the default source is used completely. The user desires to select the default source because the default source is the source provided by the authority. And configuring task architecture and type, and configuring the template into X86 architecture server type. After configuration is complete, the existing RPM packet list is checked. The template information may be generated by adding a special RPM packet or deleting an unnecessary RPM packet therein. Based on the selected server type, a server dependent RPM pack is added. Including development tools (GLIC, Python) and related software packages for server optimization. The entire software package is generated. A build task is generated based on the template information. The X86 architecture builder requests the server every 3 seconds. The server queries for a build task with X86. The task is issued to the builder. And starting to construct the task after the construction machine obtains the task.
2. The extensibility is described by taking the example of generating a server image under the X86 architecture.
First assume that all the builders are busy. No new builder is available. But the task has an unextarted task. Using a one-key deployment script, add an X86 builder. And adding the construction machine into the corresponding local area network.
The construction machine requests construction tasks; the build task is started by the request.
A one-key deployment script can be used to quickly deploy a new build machine. Manual intervention is not required. And starting to construct the task after the construction machine obtains the task. The backlog task is released. The current system has no additional construction task and directly closes the construction machine. The server will automatically determine the 9 second timeout. After a timeout the task will assume that the builder is offline. The building machines are dynamically reduced without additional operations.
As shown in fig. 4, an embodiment of the present invention provides a system for dynamically expanding a cluster to construct an operating system image, where the system includes a host cluster, and each host of the host cluster is provided with a deployment execution end; configuring a plurality of hosts of a host cluster in the same local area network, wherein the hosts can be mutually communicated with each other;
the host of the same local area network comprises a server and a plurality of construction machines;
the server is used for receiving the configuration information and the requirements of the user, constructing template information according to the received information, generating a corresponding construction task based on the template information and storing the corresponding construction task in a database; receiving a request sent by a construction machine, and issuing a construction task to the corresponding construction machine when judging that the construction machine is idle; when no construction machine is judged to be idle, inquiring whether an un-started construction task exists in the database and outputting information to a deployment execution end;
the construction machine is used for sending a request to the server once every set first interval time; the system is also used for receiving and executing the construction task;
and the deployment execution end is used for deploying the host in which the execution end is positioned as a new construction machine according to the task type and adding the construction machine into the corresponding local area network when the server has the construction tasks which are not started and the number of the construction tasks is greater than a set first threshold value. And is also used to reconfigure the scene type of the builder according to the received information.
In some embodiments, the template information refers to all information required by the building machine in the building process, including the framework of the template, the name of the template, the type of the template, all software packages of the template, and the configuration file of the template;
the construction machine comprises a configuration and analysis module, a customization module, a task storage module, a task scheduling module and a task execution module;
the configuration and analysis module is used for accessing all software package data through a software package source, downloading a corresponding software package in the source through a configuration file and generating a corresponding configuration file; analyzing the dependence of the software package to obtain all software packages capable of supporting configuration and dependence packages and generating a software package list;
it should be noted that the module supports two software package sources. A base packet source and an extension packet source, respectively. The basic package source is a software package indispensable to the operating system, and the software package source mainly comprises a core software package of the operating system. Including basic software packages such as bash, perl, python, yum, etc. The operating system cannot be built properly without these software packages. The underlying packet source may not be deleted or modified. Yet another software package source is an extended software package source. The extension software package source is a software package source with different types and customized, and the source supports the type configuration and generates a mirror image by using a software package in the source after the source is manufactured. After the custom software package source is added, the module will first analyze the source. After determining that the source configuration is correct. And caching the software package information of the source. If the configuration is incorrect, the source is not in effect. No service can be provided for the subsequent modules. The analysis function mainly analyzes the dependency function of the RPM package. There is a dependency due to the RPM package. The software package needs to perform dependency analysis. For example, the user configures the software package A, the software package A depends on the software package B, the software package B depends on the software packages C and D, and the like. In the analysis process, the software packages A, B, C and D are all used as software packages needing to be input into the mirror image. Due to the actual configuration. A user may configure many software packages. Therefore, all software packages and dependency packages capable of supporting the user configuration software package need to be analyzed based on the user configuration software package.
The customizing module is used for classifying the construction types of the construction machine based on the software package list and supplementing the software packages according to customizing information, finally generating a task template and sending the task template to the task storage module, wherein the task template is used for constructing a configuration file;
source configured by the last module. The output is the complete support software package. The module distinguishes the construction type of the construction machine and generates corresponding template configuration. And refining the software package requirements corresponding to the mirror image type into a list of software packages. The types of mirroring are divided into four scene types: server scene/mobile device scene/edge device scene/cloud computing scene. Different image types correspond to different software package configurations. And can configure special requirements of users. For example, the user needs to type in a designated version of docker or a version of the user's own source. And generates a specific profile after the user selects based on the characteristics when saving. Mainly comprises a comps file (RPM grouping file), a varients file (hierarchical definition file) and a lorax configuration file.
The user profile is based on the user-selected profile, sorting and supplementing software packages based on the entire list of software packages. For example, in the edge device scenario, a driver support of the edge device is required, and therefore, a driver software package of the edge device is added on the basis of the previous step. Finally, the configuration file for construction is generated.
The task storage module is used for storing the task template generated by the customization module; and storing the saved specific tasks. Storing the basic information and the generated configuration file content; after receiving the construction task, the construction machine does not necessarily execute the task immediately, and needs to execute the task according to the generation time of the task, so that the received task needs to be stored after being analyzed and processed;
the task scheduling module is used for selecting a task template and sending the task template to the task execution module based on the priority and the task generation time; the task priority is divided into three priorities, normal, priority and urgent. Default to normal priority. The task scheduling module controls the number of compilation tasks that are executed simultaneously. The number may be configured by a configuration file (/ etc/img-builder/sch. 0 is any number. If the number is limited, the task with high priority is selected to be executed in advance. The task generation time has two configurations, one is generated periodically. The tasks are periodically timed. The selectable period is week, 14 days, month. Alternatively, a single generation may be performed. A single generation may select the latest generation time. The system ensures that the task is completed before the generation time when scheduling. The builder can only perform the same task at the same time. Therefore, the task scheduling module needs to analyze the tasks of the building machine when the building machine requests the tasks. And determining whether the current builder requesting the task has a new task or not based on the priority and the periodic task.
And the task execution module is used for executing the construction task to finish the construction of the mirror image before the set time.
Although the present invention has been described in detail by referring to the drawings in connection with the preferred embodiments, the present invention is not limited thereto. Various equivalent modifications or substitutions can be made on the embodiments of the present invention by those skilled in the art without departing from the spirit and scope of the present invention, and these modifications or substitutions are within the scope of the present invention/any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.
Claims (10)
1. A method for constructing an operating system mirror image by a dynamically expanded cluster is characterized in that based on a host cluster, the host cluster comprises a server side and a plurality of construction machines which are positioned in the same local area network for communication; the method comprises the following steps:
the server side constructs template information when receiving a creation requirement, generates a corresponding construction task based on the template information and stores the construction task in a database;
the server receives a request sent by the construction machine and judges whether the construction machine is idle or not according to the request information;
when the building machine is idle, the building tasks are issued to the corresponding building machine to execute the building tasks;
when all the building machines are busy, inquiring whether an unfixed building task exists in the database;
when the un-started construction tasks exist and the number of the un-started construction tasks is larger than a set first threshold value, selecting a host of the host cluster, deploying the selected host as a new construction machine according to the task type, and adding the construction machine into a corresponding local area network;
and the server receives a request sent by the newly deployed construction machine, and issues a corresponding construction task to the construction machine to execute the construction task.
2. The method for dynamically expanding a cluster to construct an operating system image according to claim 1, wherein the step of receiving a request sent by a builder by a server and judging whether the builder is idle according to the request information comprises:
the server receives a request sent by the construction machine, refreshes the state of the construction machine and sets the online state;
judging whether the construction machine is idle or not according to the request information;
if yes, judging whether the state is a first idle state or a second idle state; the first idle state is idle and has no task to be executed; the second idle state is a current idle state in which there are tasks that are periodically executed.
3. The method of claim 2, wherein the step of issuing the build task to the corresponding build machine to execute the build task when the build machine is idle comprises:
when the building machine is idle, acquiring the type information of the idle building machine according to the request information;
inquiring whether a corresponding construction task exists in a database or not according to the type information of the construction machine;
if the corresponding construction task exists, the inquired construction task is issued to the corresponding idle construction machine to execute the construction task;
and if the corresponding construction tasks do not exist, inquiring whether the construction tasks which are not started exist in the database, and if the construction tasks which are not started do not exist, closing the first idle-state construction machine.
4. The method of claim 3, wherein if there is no corresponding build task, the step of querying the database for the presence of an unexecuted build task further comprises:
if the construction tasks which do not start exist, the number of the construction tasks of the same type in the database is obtained;
judging whether a first type of construction machine exists or not; defining a building machine in a first idle state, wherein the building machine with the same architecture type as a first type of building task in the building machines in the first idle state is a first type of building machine; defining the construction tasks with the same type and the largest quantity in the database as a first type construction task;
if so, configuring the scene type of the first type of construction machine to be the same as the scene type of the first type of construction task;
after configuration is completed, the server receives a request sent by the first type construction machine, and issues the first type construction task to the first type construction machine to execute the construction task.
5. The method of claim 4, wherein the step of determining whether the first type of builder exists further comprises:
if the first type of building machine does not exist, judging whether a second type of building machine exists or not; defining the building machines in the second idle state, wherein the building machines with the number of tasks executed periodically and less than a second threshold value exist in the second idle state as second type building machines;
if the second type of building machine exists, stopping the service of the second type of building machine, and configuring the scene type of the second type of building machine to be the same as the scene type of the first type of building task;
after configuration is completed, the server receives a request sent by a second type of construction machine, and issues a first type of construction task to the second type of construction machine to execute the construction task;
if the second type of building machine does not exist, judging whether the number of the first type of building tasks is larger than a first threshold value; if yes, executing the following steps: selecting a host of the host cluster, deploying the selected host as a new construction machine according to the task type, and adding the construction machine into a corresponding local area network; if not, executing the following steps: and the server receives the request sent by the construction machine and judges whether the construction machine is idle or not according to the request information.
6. The method for dynamically extending a cluster build operating system image of claim 1, comprising:
the construction machine sends a request to the server once every set first interval time;
and when the server side does not receive the request sent by the construction machine at the set second time interval, refreshing the state of the construction machine and setting the offline state.
7. The method for constructing an operating system image by using a dynamically expanded cluster according to claim 1, wherein the template information refers to all information required by a construction machine in a construction process, and comprises a template architecture, a template name, a template type, all software packages of the template and a configuration file of the template;
in the step of receiving the issued construction task and executing the construction task, the step of executing the construction task includes:
accessing all software package data through a software package source, downloading a corresponding software package in the source through a configuration file, and generating a corresponding configuration file;
analyzing the dependence of the software package to obtain all software packages capable of supporting configuration and dependence packages and generating a software package list;
classifying the software package list and the construction type of the construction machine, supplementing the software package according to the customized information, and finally generating and storing a task template, wherein the task template is used for the constructed configuration file;
and selecting a task template to execute the construction task to complete the construction of the mirror image before the set time based on the priority and the task generation time.
8. The method for constructing an operating system image by using a dynamically expanded cluster as claimed in claim 7, wherein the request information in the step of sending a request to the server every time the builder sets the first interval comprises the architecture of the builder, the type of the builder, the state of the builder, and the number of task templates to be executed in the builder;
the builder determines whether the current request requires a new task based on its state, the priority of the task template, and the task generation time.
9. A system for constructing an operating system mirror image by a dynamically expanded cluster is characterized by comprising a host cluster, wherein each host of the host cluster is provided with a deployment execution end; configuring a plurality of hosts of a host cluster in the same local area network, wherein the hosts can be mutually communicated with each other;
the host of the same local area network comprises a server and a plurality of construction machines;
the server is used for receiving the configuration information and the requirements of the user, constructing template information according to the received information, generating a corresponding construction task based on the template information and storing the construction task in a database; receiving a request sent by a construction machine, and issuing a construction task to the corresponding construction machine when judging that the construction machine is idle; when no construction machine is judged to be idle, inquiring whether an un-started construction task exists in the database and outputting information to a deployment execution end;
the construction machine is used for sending a request to the server once every set first interval time; the system is also used for receiving and executing the construction task;
and the deployment execution end is used for deploying the host in which the execution end is positioned as a new construction machine according to the task type and adding the construction machine into the corresponding local area network when the server has the construction tasks which are not started and the number of the construction tasks is greater than a set first threshold value.
10. The system for dynamically expanding a cluster to construct an operating system image according to claim 9, wherein the template information refers to all information required by a construction machine in a construction process, and includes a template architecture, a template name, a template type, all software packages of the template, and a configuration file of the template;
the construction machine comprises a configuration and analysis module, a customization module, a task storage module, a task scheduling module and a task execution module;
the configuration and analysis module is used for accessing all software package data through a software package source, downloading a corresponding software package in the source through a configuration file and generating a corresponding configuration file; analyzing the dependence of the software package to obtain all software packages capable of supporting configuration and dependence packages and generating a software package list;
the customizing module is used for classifying the construction types of the construction machine based on the software package list and supplementing the software packages according to customizing information, finally generating a task template and sending the task template to the task storage module, wherein the task template is used for constructing a configuration file;
the task storage module is used for storing the task template generated by the customization module;
the task scheduling module is used for selecting a task template and sending the task template to the task execution module based on the priority and the task generation time; the task scheduling module is also used for judging whether a new task is needed or not based on the priority and the time of task generation;
and the task execution module is used for executing the construction task to finish the construction of the mirror image before the set time.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111669242.6A CN114443069B (en) | 2021-12-30 | 2021-12-30 | Method and system for constructing operating system mirror image by dynamically expanding cluster |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111669242.6A CN114443069B (en) | 2021-12-30 | 2021-12-30 | Method and system for constructing operating system mirror image by dynamically expanding cluster |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114443069A true CN114443069A (en) | 2022-05-06 |
| CN114443069B CN114443069B (en) | 2023-09-08 |
Family
ID=81366752
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111669242.6A Active CN114443069B (en) | 2021-12-30 | 2021-12-30 | Method and system for constructing operating system mirror image by dynamically expanding cluster |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114443069B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170093964A1 (en) * | 2015-09-25 | 2017-03-30 | International Business Machines Corporation | Self-expanding software defined computing cluster |
| CN106888254A (en) * | 2017-01-20 | 2017-06-23 | 华南理工大学 | A kind of exchange method between container cloud framework based on Kubernetes and its each module |
| CN108388472A (en) * | 2018-03-01 | 2018-08-10 | 吉林大学 | A kind of elastic task scheduling system and method based on Docker clusters |
| CN109240653A (en) * | 2018-07-06 | 2019-01-18 | 广州小鹏汽车科技有限公司 | A kind of method and system constructing internet O&M resource associations system |
| US20190087161A1 (en) * | 2017-09-18 | 2019-03-21 | Rapyuta Robotics Co., Ltd. | Build and deploy cloud device application on cloud and devices |
| CN110460670A (en) * | 2019-08-19 | 2019-11-15 | 浪潮云信息技术有限公司 | A kind of application cluster framework Automation arranging method based on Terraform and Ansible |
| CN112486513A (en) * | 2020-11-25 | 2021-03-12 | 湖南麒麟信安科技股份有限公司 | Container-based cluster management method and system |
-
2021
- 2021-12-30 CN CN202111669242.6A patent/CN114443069B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170093964A1 (en) * | 2015-09-25 | 2017-03-30 | International Business Machines Corporation | Self-expanding software defined computing cluster |
| CN106888254A (en) * | 2017-01-20 | 2017-06-23 | 华南理工大学 | A kind of exchange method between container cloud framework based on Kubernetes and its each module |
| US20190087161A1 (en) * | 2017-09-18 | 2019-03-21 | Rapyuta Robotics Co., Ltd. | Build and deploy cloud device application on cloud and devices |
| CN108388472A (en) * | 2018-03-01 | 2018-08-10 | 吉林大学 | A kind of elastic task scheduling system and method based on Docker clusters |
| CN109240653A (en) * | 2018-07-06 | 2019-01-18 | 广州小鹏汽车科技有限公司 | A kind of method and system constructing internet O&M resource associations system |
| CN110460670A (en) * | 2019-08-19 | 2019-11-15 | 浪潮云信息技术有限公司 | A kind of application cluster framework Automation arranging method based on Terraform and Ansible |
| CN112486513A (en) * | 2020-11-25 | 2021-03-12 | 湖南麒麟信安科技股份有限公司 | Container-based cluster management method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114443069B (en) | 2023-09-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP5926864B2 (en) | System and method for configuring a cloud computing system | |
| JP6373840B2 (en) | System and method for tuning a cloud computing system | |
| US9262231B2 (en) | System and method for modifying a hardware configuration of a cloud computing system | |
| US9152532B2 (en) | System and method for configuring a cloud computing system with a synthetic test workload | |
| US9043770B2 (en) | Program module applicability analyzer for software development and testing for multi-processor environments | |
| US9658895B2 (en) | System and method for configuring boot-time parameters of nodes of a cloud computing system | |
| US7437706B2 (en) | Automating the life cycle of a distributed computing application | |
| US20140006751A1 (en) | Source Code Level Multistage Scheduling Approach for Software Development and Testing for Multi-Processor Environments | |
| US20140047095A1 (en) | System and method for tuning a cloud computing system | |
| US20140047079A1 (en) | System and method for emulating a desired network configuration in a cloud computing system | |
| US20140007044A1 (en) | Source Code Generator for Software Development and Testing for Multi-Processor Environments | |
| CN107612962B (en) | Distributed simulation evaluation test management system | |
| CN112199196B (en) | Resource allocation method, medium and server | |
| CN113946323A (en) | Online development method, device, equipment and storage medium based on micro-service system | |
| CN113434131A (en) | Programming method and device, computer equipment and storage medium | |
| CN107493200B (en) | Optical disc image file creating method, virtual machine deploying method and device | |
| CN117034821B (en) | Regression verification method and medium for chip design front-end simulation verification | |
| US20210389994A1 (en) | Automated performance tuning using workload profiling in a distributed computing environment | |
| CN114443069B (en) | Method and system for constructing operating system mirror image by dynamically expanding cluster | |
| WO2022257303A1 (en) | Method and apparatus for deploying deep learning system | |
| US9262141B1 (en) | Distributed computations of graphical programs having a pattern | |
| Yoshinaka et al. | Model-based development considering self-driving systems for many-core processors | |
| CN111475998B (en) | Hybrid execution of Electronic Design Automation (EDA) process to delay acquisition of remote resources | |
| CN115373696B (en) | Low code configuration method, system, equipment and storage medium for software resource generation | |
| KR20190049059A (en) | Apparatus and method for cloud service brokerage |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |