CN114443069B - Method and system for constructing operating system mirror image by dynamically expanding cluster - Google Patents

Method and system for constructing operating system mirror image by dynamically expanding cluster Download PDF

Info

Publication number
CN114443069B
CN114443069B CN202111669242.6A CN202111669242A CN114443069B CN 114443069 B CN114443069 B CN 114443069B CN 202111669242 A CN202111669242 A CN 202111669242A CN 114443069 B CN114443069 B CN 114443069B
Authority
CN
China
Prior art keywords
construction
task
machine
template
type
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202111669242.6A
Other languages
Chinese (zh)
Other versions
CN114443069A (en
Inventor
张宇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Suzhou Inspur Intelligent Technology Co Ltd
Original Assignee
Suzhou Inspur Intelligent Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Suzhou Inspur Intelligent Technology Co Ltd filed Critical Suzhou Inspur Intelligent Technology Co Ltd
Priority to CN202111669242.6A priority Critical patent/CN114443069B/en
Publication of CN114443069A publication Critical patent/CN114443069A/en
Application granted granted Critical
Publication of CN114443069B publication Critical patent/CN114443069B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00Arrangements for software engineering
    • G06F8/60Software deployment
    • G06F8/61Installation
    • G06F8/63Image based installation; Cloning; Build to order
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00Arrangements for software engineering
    • G06F8/70Software maintenance or management
    • G06F8/71Version control; Configuration management
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Security & Cryptography (AREA)
  • Stored Programmes (AREA)

Abstract

The application belongs to the technical field of operating system image construction, and particularly provides a method and a system for constructing an operating system image by a dynamically-expanded cluster, wherein the method comprises the following steps: the server receives the construction template information when the creation requirement is received, generates a corresponding construction task based on the template information and stores the construction task in a database; receiving a request sent by a constructor, and judging whether the constructor is idle according to request information; if yes, issuing the construction task to a corresponding construction machine to execute the construction task; if not, inquiring whether an uninitiated construction task exists in the database; when the construction tasks which are not started exist and the number is larger than a set first threshold value, selecting a host of a host cluster, deploying the selected host as a new construction machine according to the task type, and adding the construction machine into a corresponding local area network; the server receives a request sent by a newly deployed construction machine, and issues a corresponding construction task to the construction machine to execute the construction task. And the building machine is rapidly increased, and the compiling speed is improved.

Description

Method and system for constructing operating system mirror image by dynamically expanding cluster
Technical Field
The application relates to the technical field of operating system image construction, in particular to a method and a system for constructing an operating system image by a dynamically-expanded cluster.
Background
The operating system serves as a basic platform for modern computer technology and is responsible for the interaction of applications with users. Providing a unified interface for the application and the user. Shielding the difference of different hardware at the bottom layer. And finally, the process of the user can be efficiently operated under the condition that the user does not feel. However, in the current environment, hardware devices are more and more complex, and hardware scenes are more and more extensive. Taking Linux operating system as an example, the Linux operating system can run on a server to provide services as a service operating system. Meanwhile, the mobile phone platform can run on an ARM architecture, and provides interface support of a bottom layer for an Android operating system. The intelligent device can also run on a vehicle-mounted operating system, and intelligent device support can be provided for the vehicle. In the field of edge computing, linux operating systems can support edge devices.
It is therefore of particular importance to be able to flexibly customize an operating system conforming to the characteristics of a hardware platform for different hardware platforms. The same operating system if it is desired to adapt to different operating environments. Different custom generation schemes are required for the installation image of the operating system. Under different architectural environments. If expansion is required, lateral expansion is required for different architectures. And meanwhile, different application scenes are considered. Each application scenario may be for a different architecture. Therefore, the application scene needs to be longitudinally expanded. The longitudinal expansion is divided into four scenarios in this scheme: server scenario/mobile device scenario/edge device scenario/cloud computing scenario.
The existing manufacturing process of the Linux release plate comprises the following steps: 1. a source containing the software package is prepared. Including all software packages that need to be packaged. 2. Modify pungi profile. Mainly comprises Comps files and Varients files. Wherein the Comps file includes a set of defining packages. The Varients file includes an additional list of software packages. After configuration is completed, the pungi command is used. The parameters are increased. Pungi reads the corresponding configuration file. The corresponding output is produced. The manufacturing method of the Linux release plate has the following main defects: 1. human intervention is required. The Comps and other configuration files need to be written manually. If the configuration file is written in error or inconsistent with the configuration source. An error reporting situation may occur. 2. Automatic construction cannot be achieved. Each time requiring human participation in the configuration. Configuration source and output paths, formats, etc. 3. The expansion is more complex. If the application scene needs to be expanded horizontally or longitudinally, the corresponding configuration file and the server need to be maintained manually. Manually. Error is easy, and unified management cannot be performed.
Disclosure of Invention
Aiming at the problems that the existing operating system hairstyle construction needs manual intervention and cannot realize automatic construction, and if a transverse expansion framework or a longitudinal expansion application scene is needed, corresponding configuration files and servers are needed to be manually maintained, errors are easy to occur and unified management cannot be carried out, the application provides a method and a system for constructing an operating system mirror image by a dynamic expansion cluster.
The technical scheme of the application is as follows:
in a first aspect, the present application provides a method for dynamically expanding a cluster to construct an operating system image, where the method is based on a host cluster, and the host cluster includes a server and a plurality of constructors that are in the same local area network for communication; the method comprises the following steps:
the server receives the construction template information when the creation requirement is received, generates a corresponding construction task based on the template information and stores the construction task in a database;
the server receives a request sent by the construction machine and judges whether the construction machine is idle or not according to the request information;
when the construction machine is idle, issuing a construction task to a corresponding construction machine to execute the construction task;
when all the construction machines are busy, inquiring whether an uninitiated construction task exists in the database;
when the construction tasks which are not started exist and the number is larger than a set first threshold value, selecting a host of a host cluster, deploying the selected host as a new construction machine according to the task type, and adding the construction machine into a corresponding local area network;
the server receives a request sent by a newly deployed construction machine, and issues a corresponding construction task to the construction machine to execute the construction task.
An ordinary host can be quickly deployed into a constructor. Under the condition that the original cluster does not need to be subjected to any configuration modification, the building machine is dynamically and transversely and rapidly increased, so that the compiling speed is improved.
Further, the step of receiving, by the server, a request sent by the builder, and determining whether the builder is idle according to the request information includes:
the server receives a request sent by the construction machine, and refreshes and sets the state of the construction machine in an on-line state;
judging whether the building machine is idle according to the request information;
if yes, judging whether the first idle state or the second idle state is adopted; the first idle state is idle and has no task to be executed; the second idle state is the current idle state.
Further, when the building machine is idle, the step of issuing the building task to the corresponding building machine to execute the building task includes:
when the building machine is idle, acquiring type information of the idle building machine according to the request information;
inquiring whether a corresponding construction task exists in a database according to the type information of the construction machine;
if the corresponding construction task exists, the queried construction task is issued to a corresponding idle construction machine to execute the construction task;
if the corresponding construction task does not exist, inquiring whether the construction task which does not exist in the database, and if the construction task which does not exist exists, closing the construction machine in the first idle state.
When the periodically executed task exists, the request information of the construction machine also comprises request information of whether a new task is needed or not, so that the periodically executed task can be executed and completed in a specified time.
Further, if no corresponding construction task exists, the step of querying whether an uninitiated construction task exists in the database further includes:
if the construction tasks which are not started exist, acquiring the number of the construction tasks of each type in the database;
judging whether a first type of construction machine exists or not; defining a building machine with the same architecture type as a first type building task in a building machine in a first idle state as a first type building machine; defining the construction tasks with the same type and the largest quantity in the database as a first type construction task;
if yes, configuring the scene type of the first type building machine to be the same as the scene type of the first type building task;
after configuration is completed, the server receives a request sent by the first type building machine, and issues a first type building task to the first type building machine to execute the building task.
The construction machine can be longitudinally expanded, and the configuration of the construction machine is transferred under different scenes of the same architecture. The tasks between the building machines do not interfere with each other. The expansion can be performed quickly.
Further, the step of determining whether the first class build machine exists further includes:
if the first type of building machine does not exist, judging whether the second type of building machine exists or not; wherein, the construction machine with the number of periodically executed tasks less than a second threshold value in the construction machines in the second idle state is defined as a second class of construction machines;
if the second class of construction machines exist, stopping the service of the second class of construction machines, and configuring the scene type of the second class of construction machines to be the same as the scene type of the first class of construction tasks;
after configuration is completed, the server receives a request sent by the second class building machine, and issues a first class building task to the second class building machine to execute the building task;
if the second type of building machine does not exist, judging whether the number of the first type of building tasks is larger than a first threshold value; if yes, executing the steps: selecting a host of a host cluster, deploying the selected host as a new building machine according to the task type, and adding the building machine into a corresponding local area network; if not, executing the steps of: the server receives the request sent by the constructor and judges whether the constructor is idle according to the request information. The construction machine can be longitudinally expanded, and the configuration of the construction machine is transferred under different scenes of the same architecture. The tasks between the building machines do not interfere with each other. The expansion can be performed quickly.
Further, the method comprises the steps of:
the construction machine sends a request to the server once every first interval time is set;
and when the server side does not receive the request sent by the constructor in the set second time interval, refreshing the state of the constructor and setting the offline state.
Further, the template information refers to all information required by the construction machine in the construction process, including the architecture of the template, the name of the template, the type of the template, all software packages of the template and configuration files of the template;
in the step of receiving the issued construction task and executing the construction task, the step of executing the construction task includes:
accessing all software package data through a software package source, downloading corresponding software packages in the source through configuration files and generating corresponding configuration files;
performing dependency analysis on the software packages, analyzing all the software packages and the dependency packages capable of supporting configuration, and generating a software package list;
classifying based on the software package list and the construction type of the construction machine, supplementing the software package according to the customized information, and finally generating and storing a task template, wherein the task template is used for constructing a configuration file;
and selecting a task template to execute a construction task to complete the construction of the mirror image before the set time based on the priority and the task generation time.
Further, the request information in the step that the construction machine sends a request to the server once every set first interval time comprises the construction machine architecture, the construction machine type, the construction machine state and the number of task templates to be executed in the construction machine;
the builder determines whether the current request requires a new task based on its own state, the priority of the task template, and the task generation time.
In a second aspect, the present application provides a system for dynamically expanding a cluster to construct an operating system image, including a host cluster, where each host of the host cluster is provided with a deployment execution end; a plurality of hosts of a host cluster are configured in the same local area network, and the hosts can be mutually communicated with the network;
the host of the same local area network comprises a server and a plurality of construction machines;
the server is used for receiving configuration information and requirements of a user, constructing template information according to the received information, generating corresponding construction tasks based on the template information and storing the construction tasks in a database; receiving a request sent by a construction machine, and when judging that the construction machine is idle, issuing a construction task to the corresponding construction machine; when judging that the non-construction machine is idle, inquiring whether an uninitiated construction task exists in the database and outputting information to a deployment execution end;
the building machine is used for sending a request to the server once every set first interval time; and is also used for receiving and executing construction tasks;
the deployment execution end is used for deploying the host computer where the task is located as a new construction machine according to the task type when the server end has the construction tasks which are not started and the number of the construction tasks is larger than a set first threshold value, and adding the construction machine into a corresponding local area network. And is also used to reconfigure the scene type of the build machine based on the received information.
Further, the template information refers to all information required by the construction machine in the construction process, including the architecture of the template, the name of the template, the type of the template, all software packages of the template and configuration files of the template;
the building machine comprises a configuration and analysis module, a customization module, a task storage module, a task scheduling module and a task execution module;
the configuration and analysis module is used for accessing all software package data through the software package source, downloading corresponding software packages in the source through the configuration files and generating corresponding configuration files; performing dependency analysis on the software packages, analyzing all the software packages and the dependency packages capable of supporting configuration, and generating a software package list;
the customization module is used for classifying the construction types of the construction machine based on the software package list and supplementing the software package according to customization information, and finally generating a task template and sending the task template to the task storage module, wherein the task template is used for constructing a configuration file;
the task storage module is used for storing the task template generated by the customization module;
the task scheduling module is used for selecting a task template to send to the task execution module based on the priority and the task generation time; the method is also used for judging whether a new task is needed or not based on the priority and the time of task generation;
and the task execution module is used for executing the construction task to complete the construction of the mirror image before the set time.
From the above technical scheme, the application has the following advantages: 1: a software package can be automatically generated that builds related files and foundations. Since the software package is the source of input for the build during the build process. The build machine accesses all the package data through the package source. And downloading the corresponding software package in the source through the configuration file. This process requires configuration files and support for the source of the software package. The system can automatically select a proper source and generate a corresponding configuration file. The configuration file is not required to be manually written, and the organization and configuration of the software package source are not required to be considered to be participated. The complexity of manual operation is reduced. Time is saved. And reduces the probability of human involvement errors. 2: the build machine can be dynamically expanded laterally and longitudinally based on demand. The tasks between the building machines do not interfere with each other. Under the condition that the original cluster does not need to be subjected to any configuration modification, the construction machine is rapidly increased and rapidly expanded, so that the compiling speed is improved.
In addition, the application has reliable design principle, simple structure and very wide application prospect.
It can be seen that the present application has outstanding substantial features and significant advances over the prior art, as well as its practical advantages.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required to be used in the description of the embodiments or the prior art will be briefly described below, and it will be obvious to those skilled in the art that other drawings can be obtained from these drawings without inventive effort.
FIG. 1 is a schematic flow chart of a method of one embodiment of the application.
Fig. 2 is a schematic flow chart of a method of another embodiment of the application.
FIG. 3 is a schematic flow chart of a build machine execution process in a method of one embodiment of the application.
Fig. 4 is a schematic block diagram of a system of one embodiment of the present application.
Detailed Description
The operating system on the server platform often has powerful CPU, memory, hard disk, etc. The speed and efficiency of executing commands is pursued in hardware. The calculation force can be enhanced. Thus at the operating system level. The efficiency of operating system execution is more sought after. A certain customization is required for the operating system of the server. The operating system of the adaptation server needs to carry the basic operating environment needed by the application. And installs a specific software library. Can support the efficiency requirements of the server application.
At the same time, the operating system may also need to adapt to the context of some mobile devices. Such as Android handsets and tablet computers. This type of device is distinct from a server device. First, from the architecture perspective, the current mainstream of server architecture mainly adopts an X86 architecture or a Power architecture. Whereas mobile devices often employ an ARM architecture CPU. At the same time. As a mobile device, the power consumption of the CPU affects the efficiency and lifetime of the application software and the battery, and therefore the power consumption of the CPU needs to be considered during operation. At the same time, since the device often requires a lot of user interaction and manipulation. User interactions require quick responses in real time. Thus in the context of a mobile device. Some unnecessary development software and unnecessary runtime files are first cut off. While there is a need to increase the relevant applications for energy consumption monitoring of the operating system of a mobile phone.
Compared with the two application scenes, the method is common. There are also other types of scenarios that require the operating system to respond in real-time. For example, the scene of the edge device has high real-time requirements. It is required to be able to achieve a response to a specific signal at the microsecond level. Some efficiency and operating peaks are relatively abandoned to increase the response time of the operating system to specific signals and operations.
In the current cloud computing scenario, the operating system does not run on any hardware platform. But rather on a software environment emulated by the host machine. In the cloud computing scenario, in order to adapt to the environment more quickly, the operating system may directly provide a basic disk image without building a hardware boot device. And the host machine reads the disk mirror image of the operating system to directly boot and start the virtual machine.
From the above description it will be appreciated that the same operating system, if it is desired to adapt to different operating environments. Different custom generation schemes are required for the installation image of the operating system. Since the current mainstream architecture is X86 and three architectures of Arm and Power. Thus in different architectural environments. If expansion is required, lateral expansion is required for different architectures. And meanwhile, different application scenes are considered. Each application scenario may be for a different architecture. Therefore, the application scene needs to be longitudinally expanded. The longitudinal expansion is divided into four scenarios in this scheme: server scenario/mobile device scenario/edge device scenario/cloud computing scenario. The scenarios supported by each architecture may be different: for example, the X86 and Power architecture is based on server and virtualization scenarios, which are typically not the case with mobile devices. The Arm architecture is mainly mobile devices, and the edge device scene is mainly, but the server scene and the virtualization scene also need to be partially related. Thus, consider the scenario above where the operating system needs to adapt. In order to meet the requirement of dynamic expansion, the application provides an operating system image construction method capable of supporting dynamic expansion.
In order to make the technical solution of the present application better understood by those skilled in the art, the technical solution of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present application without making any inventive effort, shall fall within the scope of the present application. The server side is the center of the management task, and the construction machine is a host machine for actually carrying out construction work; RPM is build source data.
As shown in fig. 1, an embodiment of the present application provides a method for dynamically expanding a cluster to construct an operating system image, where the host cluster includes a server and a plurality of constructors that are in the same lan for communication, based on the host cluster; the method comprises the following steps:
step 1: the server receives the construction template information when the creation requirement is received, generates a corresponding construction task based on the template information and stores the construction task in a database;
step 2: the server receives a request sent by the construction machine and judges whether the construction machine is idle or not according to the request information; when the building machine is idle, executing the step 6; when all the building machines are busy, executing the step 3;
in fact, the process of sending the request to the server by the builder is divided into two processes, one is a process of requesting the task, and the other is a process of monitoring the builder by using a heartbeat mechanism; the server opens the http port, and can be called by any constructor after opening. The server registers the basic information of the constructor to the database of the server as long as the corresponding interface is successfully called;
step 3: inquiring whether an uninitiated construction task exists in a database;
step 4: when the construction tasks which are not started exist and the number is larger than a set first threshold value, selecting a host of a host cluster, deploying the selected host as a new construction machine according to the task type, and adding the construction machine into a corresponding local area network; here, task types include architecture types and scene types;
it should be noted that if all build machines are building. However, when too many construction tasks of the server end result in backlog of the tasks. In this embodiment, a one-key deployment script is provided, so that a common host can be quickly deployed into a constructor. Under the condition that the original cluster does not need to be subjected to any configuration modification, the building machine is rapidly increased, so that the compiling speed is improved.
Step 5: the server receives a request sent by a newly deployed construction machine, and step 6 is executed;
step 6: and issuing the construction task to a corresponding construction machine to execute the construction task.
In some embodiments, the step of receiving, by the server, the request sent by the builder in step 2, and determining whether the builder is idle according to the request information includes:
step 21: the server receives a request sent by the construction machine, and refreshes and sets the state of the construction machine in an on-line state;
step 22: judging whether the building machine is idle according to the request information; if yes, go to step 23; if not, executing the step 3;
step 23: judging whether the first idle state or the second idle state; wherein the first idle state is idle and has no task to be executed; the second idle state is the current idle state in which there are periodically executing tasks.
As shown in fig. 2, in some embodiments, when the building machine is idle in step 6, the step of acquiring the type information of the idle building machine according to the request information includes:
s161: when the building machine is idle, acquiring type information of the idle building machine according to the request information;
s162: inquiring whether a corresponding construction task exists in a database according to the type information of the construction machine; if there is a corresponding construction task, S163 is executed, and if there is no corresponding construction task, S164 is executed;
s163: issuing the queried construction task to a corresponding idle construction machine to execute the construction task;
s164: inquiring whether an unexplained construction task exists in the database, and executing S165 if the unexplained construction task does not exist; if there is an unexplained build task, S166 is performed;
s165: and closing the building machine in the first idle state.
S166: acquiring the number of construction tasks of each type in a database;
s167: judging whether a first type of construction machine exists or not; if yes, executing S168, if not, executing S170;
s168: configuring the scene type of the first type building machine to be the same as the scene type of the first type building task;
s169: after configuration is completed, the server receives a request sent by a first type building machine, and issues a first type building task to the first type building machine to execute the building task;
s170: judging whether a second class of building machines exist or not; if so, S171 is performed; if not, executing S173;
s171: stopping the service of the second class building machine, and configuring the scene type of the second class building machine to be the same as the scene type of the first class building task;
s172: after configuration is completed, the server receives a request sent by the second class building machine, and issues a first class building task to the second class building machine to execute the building task;
s173: judging whether the number of the first type of construction tasks is larger than a first threshold value; if yes, executing step 4: selecting a host of a host cluster, deploying the selected host as a new building machine according to the task type, and adding the building machine into a corresponding local area network; if not, executing the step 2.
For example, when the construction requirement of the server operating system image is less, but the construction requirement of the edge device is more, the construction machine can be longitudinally expanded, and the configuration of the construction machine is transferred between a server scene and an edge device scene. The tasks between the building machines do not interfere with each other. The expansion can be performed quickly.
It should be noted that, in the embodiment of the present application, the builder sends a request to the server once every first interval time is set; and when the server side does not receive the request sent by the constructor in the set second time interval, refreshing the state of the constructor and setting the offline state. The request information comprises the architecture of a construction machine, the type of the construction machine, the state of the construction machine and the number of task templates to be executed in the construction machine;
here, the build machine sends a request every 3 seconds. The server considers the building machine to be online as long as the server can receive the request of the building machine within 9 seconds. If the server side does not receive the request information of the constructor within 9 seconds, the constructor is considered to be offline.
The builder determines whether the current request requires a new task based on its own state, the priority of the task template, and the task generation time.
The template information refers to all information required by the construction machine in the construction process, including the framework of the template, the name of the template, the type of the template, all software packages of the template and configuration files of the template;
as shown in fig. 3, in some embodiments, in step 6, the step of executing the build task in the step of receiving the issued build task by the build machine includes:
s261: accessing all software package data through a software package source, downloading corresponding software packages in the source through configuration files and generating corresponding configuration files;
s262: performing dependency analysis on the software packages, analyzing all the software packages and the dependency packages capable of supporting configuration, and generating a software package list;
s263: classifying based on the software package list and the construction type of the construction machine, supplementing the software package according to the customized information, and finally generating and storing a task template, wherein the task template is used for constructing a configuration file;
s264: and selecting a task template to execute a construction task to complete the construction of the mirror image before the set time based on the priority and the task generation time.
If there is no special configuration, the system has a default source of the package. The default package source includes default compiled packages, commonly used application software, and the like. A commonly used application software is a software package compiled by other compilation systems (e.g. koji).
1. Take the example of generating a server mirror image under the X86 architecture.
Configuration of the source is not required, and default sources are all used. After the user needs to select the default source, the default source is the official provided source. The task architecture and type are configured, and the template is configured as the type of the X86 architecture server. After configuration is completed, the existing RPM packet list is checked. Special RPM packages may be added or unnecessary RPM packages may be deleted here to generate template information. Based on the selected server type, the RPM dependent packets for the server are increased. Including development tools (GLIC, python) and related software packages for server optimization. All software packages are generated. A build task is generated based on the template information. The X86 architecture builder requests the server every 3 seconds. The server inquires that the construction task of X86 exists. The task is issued to the build machine. And after the construction machine obtains the task, starting to construct the task.
2. Extensibility is described by taking the example of generating a server-side mirror under the X86 architecture.
First assume that all build machines are busy. No new build machines are available. But the task has an unexposed task. One-touch deployment script is used, adding one X86 builder. And adding the building machine into a corresponding local area network.
Requesting a construction task by a construction machine; the build task is started after the request.
New build machines can be deployed quickly with one-touch deployment scripts. No manual intervention is required. And after the construction machine obtains the task, starting to construct the task. The backlog task is released. The current system has no additional build tasks and shuts down the build machine directly. The server automatically determines a 9 second timeout. After timeout, the task considers the build machine offline. The build machine can be dynamically reduced without additional operations.
As shown in fig. 4, an embodiment of the present application provides a system for dynamically expanding a cluster to construct an operating system image, including a host cluster, where each host of the host cluster is provided with a deployment execution end; a plurality of hosts of a host cluster are configured in the same local area network, and the hosts can be mutually communicated with the network;
the host of the same local area network comprises a server and a plurality of construction machines;
the server is used for receiving configuration information and requirements of a user, constructing template information according to the received information, generating corresponding construction tasks based on the template information and storing the construction tasks in a database; receiving a request sent by a construction machine, and when judging that the construction machine is idle, issuing a construction task to the corresponding construction machine; when judging that the non-construction machine is idle, inquiring whether an uninitiated construction task exists in the database and outputting information to a deployment execution end;
the building machine is used for sending a request to the server once every set first interval time; and is also used for receiving and executing construction tasks;
the deployment execution end is used for deploying the host computer where the task is located as a new construction machine according to the task type when the server end has the construction tasks which are not started and the number of the construction tasks is larger than a set first threshold value, and adding the construction machine into a corresponding local area network. And is also used to reconfigure the scene type of the build machine based on the received information.
In some embodiments, the template information refers to information all required by the building machine in the building process, including the architecture of the template, the name of the template, the type of the template, all software packages of the template, and the configuration file of the template;
the building machine comprises a configuration and analysis module, a customization module, a task storage module, a task scheduling module and a task execution module;
the configuration and analysis module is used for accessing all software package data through the software package source, downloading corresponding software packages in the source through the configuration files and generating corresponding configuration files; performing dependency analysis on the software packages, analyzing all the software packages and the dependency packages capable of supporting configuration, and generating a software package list;
it should be noted that this module supports two software package sources. Respectively a base packet source and an extension packet source. The basic package source is a software package essential to an operating system, and mainly comprises a core software package of the operating system. Including base packages such as bash, perl, python, yum, etc. The operating system cannot be built properly without these software packages. The underlying packet source cannot be deleted or modified. And one software package source is an extension software package source. The extension software package source is a software package source customized by type distinction, and the source support type configures the self-made source and then uses the software package in the self-source to generate an image. After adding the custom package source, the module will analyze the source first. After determining that the source configuration is correct. Caching the source software package information. If the configuration is incorrect, the source is not validated. The subsequent module cannot be serviced. The analysis function mainly analyzes the dependency function of the RPM package. There is a dependency due to RPM packets. The software package therefore requires dependency analysis. For example, the user has configured an A package, an A package depends on a B package, a B package depends on a C, D package, etc. In the analysis process, all the software packages A, B, C and D are required to be used as software packages needing to be mirrored. Due to the actual configuration. The user may configure many software packages. It is therefore desirable to analyze all software packages and dependency packages based on user-configured software packages that can support user-configured software packages.
The customization module is used for classifying the construction types of the construction machine based on the software package list and supplementing the software package according to customization information, and finally generating a task template and sending the task template to the task storage module, wherein the task template is used for constructing a configuration file;
sources configured by the last module. The output is the entire supporting software package. The module distinguishes the construction type of the construction machine and generates corresponding template configuration. The software package requirements corresponding to the image types are refined to a list of software packages. The types of images are divided into four scene types: server scenario/mobile device scenario/edge device scenario/cloud computing scenario. Different image types correspond to the configuration of different software packages. While the special needs of the user can be configured. For example, the user needs to strike a specified version of the docker or the version of the user's own source. And generates a specific profile after the user selects based on the characteristics at the time of saving. Mainly includes combs file (RPM grouping file), varients file (grading definition file) and lorax configuration file.
The user characteristics are classified based on the characteristics selected by the user, based on the entire software package list, and supplement the software packages. For example, in the edge device scenario, a lower edge device driving support is required, so that the driving software package of the edge device is added on the basis of the previous step. And finally generating a configuration file for construction.
The task storage module is used for storing the task template generated by the customization module; and storing the saved specific tasks. Storing the basic information and the generated configuration file content; after receiving the construction task, the construction machine does not need to execute the task immediately, and the task needs to be executed according to the generation time of the task, so that the received task needs to be stored after being analyzed and processed;
the task scheduling module is used for selecting a task template to send to the task execution module based on the priority and the task generation time; task priorities are three priorities, normal, priority and urgent. Default to normal priority. The task scheduling module controls the number of simultaneous execution of compiled tasks. The quantity may be configured by a configuration file (/ etc/img-builder/sch. 0 is any number. If the number is limited, a high priority task is selected to be executed in advance. The task generation time has two configurations, one is periodic generation. Tasks are timed for periods. The optional period is week, 14 days, and month. Or alternatively, may be generated in a single pass. A single generation may select the latest generation time. The system will ensure that the task is completed before the time of generation when it is scheduled. The build machines can only perform the same task at the same time. The task scheduling module therefore needs to analyze the tasks of the build machine when the build machine requests the tasks. Based on the priority and the periodic tasks, it is determined whether the build machine currently requesting the task has a new task.
And the task execution module is used for executing the construction task to complete the construction of the mirror image before the set time.
Although the present application has been described in detail by way of preferred embodiments with reference to the accompanying drawings, the present application is not limited thereto. Various equivalent modifications and substitutions may be made in the embodiments of the present application by those skilled in the art without departing from the spirit and scope of the present application, and it is intended that all such modifications and substitutions be within the scope of the present application/be within the scope of the present application as defined by the appended claims. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (10)

1. A method for constructing an operating system image by a dynamically expanded cluster is characterized in that the method is based on a host cluster, wherein the host cluster comprises a server side and a plurality of constructors which are in communication with one another in the same local area network; the method comprises the following steps:
the server receives the construction template information when the creation requirement is received, generates a corresponding construction task based on the template information and stores the construction task in a database;
the server receives a request sent by the construction machine and judges whether the construction machine is idle or not according to the request information;
when the construction machine is idle, issuing a construction task to a corresponding construction machine to execute the construction task;
when all the construction machines are busy, inquiring whether an uninitiated construction task exists in the database;
when the construction tasks which are not started exist and the number is larger than a set first threshold value, selecting a host of a host cluster, deploying the selected host as a new construction machine according to the task type, and adding the construction machine into a corresponding local area network;
the server receives a request sent by a newly deployed construction machine, and issues a corresponding construction task to the construction machine to execute the construction task.
2. The method for dynamically expanding clusters to construct operating system images according to claim 1, wherein the step of the server receiving the request sent by the constructor and determining whether the constructor is idle according to the request information comprises:
the server receives a request sent by a constructor, refreshes the state of the constructor and sets the state of the constructor in an on-line state;
judging whether the building machine is idle according to the request information;
if yes, judging whether the first idle state or the second idle state is adopted; the first idle state is idle and has no task to be executed; the second idle state is the current idle state in which there are periodically executing tasks.
3. The method for dynamically expanding clusters to construct operating system images according to claim 2, wherein the step of issuing the construction task to the corresponding construction machine to execute the construction task when the construction machine is idle comprises:
when the building machine is idle, acquiring type information of the idle building machine according to the request information;
inquiring whether a corresponding construction task exists in a database according to the type information of the construction machine;
if the corresponding construction task exists, the queried construction task is issued to a corresponding idle construction machine to execute the construction task;
if the corresponding construction task does not exist, inquiring whether the construction task which does not exist in the database, and if the construction task which does not exist exists, closing the construction machine in the first idle state.
4. The method for dynamically expanding a cluster to build an operating system image according to claim 3, wherein the step of querying the database for the presence of an unexplained build task if there is no corresponding build task further comprises:
if the construction tasks which are not started exist, acquiring the number of the construction tasks of each type in the database;
judging whether a first type of construction machine exists or not; defining a building machine with the same architecture type as a first type building task in a building machine in a first idle state as a first type building machine; defining the construction tasks with the same type and the largest quantity in the database as a first type construction task;
if yes, configuring the scene type of the first type building machine to be the same as the scene type of the first type building task;
after configuration is completed, the server receives a request sent by the first type building machine, and issues a first type building task to the first type building machine to execute the building task.
5. The method for dynamically expanding a cluster to build an operating system image according to claim 4, wherein the step of determining whether a first class of build machines exists further comprises:
if the first type of building machine does not exist, judging whether the second type of building machine exists or not; wherein, the construction machine with the number of periodically executed tasks less than a second threshold value in the construction machines in the second idle state is defined as a second class of construction machines;
if the second class of construction machines exist, stopping the service of the second class of construction machines, and configuring the scene type of the second class of construction machines to be the same as the scene type of the first class of construction tasks;
after configuration is completed, the server receives a request sent by the second class building machine, and issues a first class building task to the second class building machine to execute the building task;
if the second type of building machine does not exist, judging whether the number of the first type of building tasks is larger than a first threshold value; if yes, executing the steps: selecting a host of a host cluster, deploying the selected host as a new building machine according to the task type, and adding the building machine into a corresponding local area network; if not, executing the steps of: the server receives the request sent by the constructor and judges whether the constructor is idle according to the request information.
6. The method for dynamically expanding clusters to construct an operating system image of claim 1, comprising:
the construction machine sends a request to the server once every first interval time is set;
and when the server side does not receive the request sent by the constructor in the set second time interval, refreshing the state of the constructor and setting the offline state.
7. The method for dynamically expanding clusters to construct operating system images according to claim 1, wherein the template information refers to all information required by a construction machine in the construction process, including the architecture of a template, the name of the template, the type of the template, all software packages of the template and configuration files of the template;
in the step of receiving the issued construction task and executing the construction task, the step of executing the construction task includes:
accessing all software package data through a software package source, downloading corresponding software packages in the source through configuration files and generating corresponding configuration files;
performing dependency analysis on the software packages, analyzing all the software packages and the dependency packages capable of supporting configuration, and generating a software package list;
classifying based on the software package list and the construction type of the construction machine, supplementing the software package according to the customized information, and finally generating and storing a task template, wherein the task template is used for constructing a configuration file;
and selecting a task template to execute a construction task to complete the construction of the mirror image before the set time based on the priority and the task generation time.
8. The method for dynamically expanding a cluster to construct an operating system image according to claim 7, wherein the step of the constructor sending a request to the server once every set first interval time requests information including architecture of the constructor, type of the constructor, state of the constructor, number of task templates to be executed in the constructor;
the builder determines whether the current request requires a new task based on its own state, the priority of the task template, and the task generation time.
9. The system for constructing the operating system mirror image by the dynamically expanded cluster is characterized by comprising a host cluster, wherein each host of the host cluster is provided with a deployment execution end; a plurality of hosts of a host cluster are configured in the same local area network, and the hosts can be mutually communicated with the network;
the host of the same local area network comprises a server and a plurality of construction machines;
the server is used for receiving configuration information and requirements of a user, constructing template information according to the received information, generating corresponding construction tasks based on the template information and storing the construction tasks in a database; receiving a request sent by a construction machine, and when judging that the construction machine is idle, issuing a construction task to the corresponding construction machine; when judging that the non-construction machine is idle, inquiring whether an uninitiated construction task exists in the database and outputting information to a deployment execution end;
the building machine is used for sending a request to the server once every set first interval time; and is also used for receiving and executing construction tasks;
the deployment execution end is used for deploying the host computer where the task is located as a new construction machine according to the task type when the server end has the construction tasks which are not started and the number of the construction tasks is larger than a set first threshold value, and adding the construction machine into a corresponding local area network.
10. The system for dynamically expanding clusters to construct operating system images according to claim 9, wherein the template information is information required by all the construction machines in the construction process, and comprises the architecture of the template, the name of the template, the type of the template, all software packages of the template and configuration files of the template;
the building machine comprises a configuration and analysis module, a customization module, a task storage module, a task scheduling module and a task execution module;
the configuration and analysis module is used for accessing all software package data through the software package source, downloading corresponding software packages in the source through the configuration files and generating corresponding configuration files; performing dependency analysis on the software packages, analyzing all the software packages and the dependency packages capable of supporting configuration, and generating a software package list;
the customization module is used for classifying the construction types of the construction machine based on the software package list and supplementing the software package according to customization information, and finally generating a task template and sending the task template to the task storage module, wherein the task template is used for constructing a configuration file;
the task storage module is used for storing the task template generated by the customization module;
the task scheduling module is used for selecting a task template to send to the task execution module based on the priority and the task generation time; the method is also used for judging whether a new task is needed or not based on the priority and the time of task generation;
and the task execution module is used for executing the construction task to complete the construction of the mirror image before the set time.
CN202111669242.6A 2021-12-30 2021-12-30 Method and system for constructing operating system mirror image by dynamically expanding cluster Active CN114443069B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111669242.6A CN114443069B (en) 2021-12-30 2021-12-30 Method and system for constructing operating system mirror image by dynamically expanding cluster

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111669242.6A CN114443069B (en) 2021-12-30 2021-12-30 Method and system for constructing operating system mirror image by dynamically expanding cluster

Publications (2)

Publication Number Publication Date
CN114443069A CN114443069A (en) 2022-05-06
CN114443069B true CN114443069B (en) 2023-09-08

Family

ID=81366752

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111669242.6A Active CN114443069B (en) 2021-12-30 2021-12-30 Method and system for constructing operating system mirror image by dynamically expanding cluster

Country Status (1)

Country Link
CN (1) CN114443069B (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106888254A (en) * 2017-01-20 2017-06-23 华南理工大学 A kind of exchange method between container cloud framework based on Kubernetes and its each module
CN108388472A (en) * 2018-03-01 2018-08-10 吉林大学 A kind of elastic task scheduling system and method based on Docker clusters
CN109240653A (en) * 2018-07-06 2019-01-18 广州小鹏汽车科技有限公司 A kind of method and system constructing internet O&M resource associations system
CN110460670A (en) * 2019-08-19 2019-11-15 浪潮云信息技术有限公司 A kind of application cluster framework Automation arranging method based on Terraform and Ansible
CN112486513A (en) * 2020-11-25 2021-03-12 湖南麒麟信安科技股份有限公司 Container-based cluster management method and system

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9992276B2 (en) * 2015-09-25 2018-06-05 International Business Machines Corporation Self-expanding software defined computing cluster
US10901812B2 (en) * 2017-09-18 2021-01-26 Rapyuta Robotics Co., Ltd. Managing communication between cloud and heterogeneous devices across networks

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106888254A (en) * 2017-01-20 2017-06-23 华南理工大学 A kind of exchange method between container cloud framework based on Kubernetes and its each module
CN108388472A (en) * 2018-03-01 2018-08-10 吉林大学 A kind of elastic task scheduling system and method based on Docker clusters
CN109240653A (en) * 2018-07-06 2019-01-18 广州小鹏汽车科技有限公司 A kind of method and system constructing internet O&M resource associations system
CN110460670A (en) * 2019-08-19 2019-11-15 浪潮云信息技术有限公司 A kind of application cluster framework Automation arranging method based on Terraform and Ansible
CN112486513A (en) * 2020-11-25 2021-03-12 湖南麒麟信安科技股份有限公司 Container-based cluster management method and system

Also Published As

Publication number Publication date
CN114443069A (en) 2022-05-06

Similar Documents

Publication Publication Date Title
KR102501992B1 (en) Data processing method and related products
JP6373840B2 (en) System and method for tuning a cloud computing system
JP5926864B2 (en) System and method for configuring a cloud computing system
US8621419B2 (en) Automating the life cycle of a distributed computing application
US9262231B2 (en) System and method for modifying a hardware configuration of a cloud computing system
US9152532B2 (en) System and method for configuring a cloud computing system with a synthetic test workload
US20140047079A1 (en) System and method for emulating a desired network configuration in a cloud computing system
CN113946323A (en) Online development method, device, equipment and storage medium based on micro-service system
CN117170803A (en) Method for generating signal-creation operating system pipeline, electronic equipment and storage medium
CN112395736A (en) Parallel simulation job scheduling method of distributed interactive simulation system
CN116414521A (en) Virtual machine remote command interaction method, device and medium for distributed heterogeneous shooting ranges
WO2023039741A1 (en) Online development method and apparatus based on microservice system, device, and storage medium
CN107493200B (en) Optical disc image file creating method, virtual machine deploying method and device
CN114443069B (en) Method and system for constructing operating system mirror image by dynamically expanding cluster
CN117041111A (en) Vehicle cloud function test method and device, electronic equipment and storage medium
US20230224212A1 (en) Automated Deployment of Control Nodes at Remote Locations
WO2022257303A1 (en) Method and apparatus for deploying deep learning system
CN112685051A (en) Method, device, platform and storage medium for automatically executing shell script
CN115373696B (en) Low code configuration method, system, equipment and storage medium for software resource generation
CN113590589A (en) Database building method and device and electronic equipment
CN117873490A (en) Multi-edge management framework system based on OpenFaaS
CN116760654A (en) Intelligent AI construction operation method, device, equipment and medium
CN117493192A (en) Semi-physical simulation method and platform based on QEMU
CN116302412A (en) Big data platform task scheduling processing method and device
TW202347178A (en) Artificial intelligence model control system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant