CN113806035B

CN113806035B - Distributed scheduling method and service server

Info

Publication number: CN113806035B
Application number: CN202110256988.8A
Authority: CN
Inventors: 周智
Original assignee: Jingdong Technology Holding Co Ltd
Current assignee: Jingdong Technology Holding Co Ltd
Priority date: 2021-03-09
Filing date: 2021-03-09
Publication date: 2024-06-18
Anticipated expiration: 2041-03-09
Also published as: CN113806035A

Abstract

The application provides a distributed scheduling method and a service server, wherein the method comprises the following steps: acquiring a task request, wherein the task request has a task attribute; sending a construction request to a scheduling server according to the task request, wherein the construction request has task attributes, and the scheduling server selects a target construction server from a plurality of construction servers according to the task attributes and a health state mapping table; and receiving the address of the target construction server fed back by the scheduling server, and calling the target construction server to execute the task corresponding to the task request according to the address, so that the scheduling can be performed according to the health state of the construction server, the scheduling efficiency is improved, and the task execution efficiency is improved.

Description

Distributed scheduling method and service server

Technical Field

The present application relates to the field of data processing technologies, and in particular, to a distributed scheduling method and a service server.

Background

At present, with the development of the mobile internet, a large internet company can maintain different APP (including Android and IOS) by more than ten or more than ten teams at the same time, so that in order to reduce cost and increase efficiency, the development and test efficiency is improved, continuous delivery tasks are completed rapidly, and a set of high-performance construction service platform is needed.

In the related art, a Jenkins continuous integration tool is adopted in a construction service platform, a Master server is responsible for dispatching tasks and sending requests through MASTER SLAVE mechanisms provided by the tool, and a plurality of Slave servers can execute construction tasks concurrently.

The service platform is constructed, the Master server is fixed in scheduling strategies for the plurality of Slave servers, the scheduling strategies are difficult to dynamically adjust according to the health states of the plurality of Slave servers, the scheduling efficiency is poor, and the task execution efficiency is poor.

Disclosure of Invention

The present application aims to solve at least one of the technical problems in the related art to some extent.

The application provides a distributed scheduling method and a service server, which are used for realizing scheduling according to the health state of a constructed server, so that the scheduling efficiency is improved, and the task execution efficiency is improved.

An embodiment of a first aspect of the present application provides a distributed scheduling method, including:

Acquiring a task request, wherein the task request has a task attribute;

sending a construction request to a scheduling server according to the task request, wherein the construction request has the task attribute, and the scheduling server selects a target construction server from a plurality of construction servers according to the task attribute and a health state mapping table;

And receiving the address of the target construction server fed back by the scheduling server, and calling the target construction server to execute the task corresponding to the task request according to the address.

In the technical scheme, the scheduling server can select the target construction server according to the health state of the construction server, so that the scheduling efficiency is improved; the business server can schedule the target construction server to execute the task, so that the task execution efficiency is improved.

In the above technical solution, the setting of the health status mapping table enables the scheduling server to know the health status of each building server, and delete the health status information of a building server in the health status mapping table when a certain building server is disconnected or fails; if a certain construction server is reconnected or a certain construction server is newly added, the health state information of the construction server can be added in the health state mapping table, so that the pluggable of each construction server is realized, the capacity can be automatically reduced and expanded, and the influence on other construction servers is avoided.

Optionally, the method further comprises: if the address of the target construction server fed back by the dispatching server is not received, the target construction server is determined according to a construction server configuration file, so that the service server can not receive the address of the target construction server fed back by the dispatching server under the condition that the dispatching server is down or a dispatching strategy is wrong, and the service server can select the target construction server by polling the addresses of a plurality of construction servers, thereby achieving disaster recovery and automatic degradation of the dispatching service without influencing task execution.

Optionally, when the scheduling policy is a source code compiling scheduling policy or a source code dependency analysis scheduling policy, the corresponding task number is preferentially selected to not reach the corresponding maximum value, and the first construction server with the downloaded codes is used as the target construction server, so that the code is prevented from being re-downloaded, the time occupied by the code re-downloading is reduced, the scheduling efficiency is further improved, the task execution time is reduced, and the task execution efficiency is further improved.

Optionally, when the scheduling policy is a source code compiling scheduling policy or a source code dependency analysis scheduling policy, and the corresponding task number does not reach the corresponding maximum value, and when the first building server of the downloaded code exists, the corresponding task number is preferentially selected to not reach the corresponding maximum value, and the second server of the downloaded code does not exist, and the predicted building time is smaller than the preset time threshold value, as the target building server, the predicted building time can be reduced as much as possible, that is, the task execution time is reduced, so that the task execution efficiency is further improved.

An embodiment of a second aspect of the present application provides a service server, including: the acquisition module is used for acquiring a task request, wherein the task request has task attributes;

The sending module is used for sending a construction request to a scheduling server according to the task request, wherein the construction request has the task attribute, and the scheduling server selects a target construction server from a plurality of construction servers according to the task attribute and a health state mapping table;

And the processing module is used for receiving the address of the target construction server fed back by the scheduling server, and calling the target construction server to execute the task corresponding to the task request according to the address.

The service server of the embodiment of the application obtains the task request, wherein the task request has task attributes; sending a construction request to a scheduling server according to the task request, wherein the construction request has task attributes, and the scheduling server selects a target construction server from a plurality of construction servers according to the task attributes and a health state mapping table; and receiving the address of the target construction server fed back by the scheduling server, and calling the target construction server to execute the task corresponding to the task request according to the address, so that the scheduling can be performed according to the health state of the construction server, the scheduling efficiency is improved, and the task execution efficiency is improved.

An embodiment of a third aspect of the present application provides a service platform, including:

The system comprises a service server, a scheduling server and a plurality of construction servers, wherein the scheduling server is respectively connected with the service server and the construction servers;

The service server is configured to perform the distributed scheduling method according to the embodiment of the first aspect.

An embodiment of a fourth aspect of the present application proposes a computer device comprising: the system comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor is used for realizing the distributed scheduling method as set forth in the embodiment of the first aspect of the application or the distributed scheduling method as set forth in the embodiment of the second aspect of the application when the processor executes the program.

An embodiment of a fifth aspect of the present application proposes a computer program product which, when executed by an instruction processor in the computer program product, performs the distributed scheduling method proposed by the embodiment of the first aspect of the present application, or as proposed by the embodiment of the second aspect of the present application.

Additional aspects and advantages of the application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the application.

Drawings

The foregoing and/or additional aspects and advantages of the application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:

FIG. 1 is a schematic diagram of an architecture for building a service platform;

FIG. 2 is a flowchart of a distributed scheduling method according to an embodiment of the present application;

Fig. 3 is a flow chart of a distributed scheduling method according to a second embodiment of the present application;

fig. 4 is a schematic structural diagram of a service server according to a third embodiment of the present application;

fig. 5 is a schematic structural diagram of a scheduling server according to a fourth embodiment of the present application;

FIG. 6 illustrates a block diagram of an exemplary computer device suitable for use in implementing embodiments of the present application.

Detailed Description

Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative and intended to explain the present application and should not be construed as limiting the application.

In the traditional construction service platform, a Jenkins continuous integration tool is adopted, a MASTER SLAVE mechanism provided by the tool is used for enabling a Master server to be responsible for dispatching tasks and sending requests, a plurality of Slave servers can execute construction tasks concurrently, but dispatching strategies of the Master server to the plurality of Slave servers are fixed, dynamic adjustment of the dispatching strategies according to states of the plurality of Slave servers is difficult, dispatching efficiency is poor, and task execution efficiency is poor.

Therefore, the application provides a distributed scheduling method mainly aiming at the technical problems of poor scheduling efficiency and poor task execution efficiency in the prior art.

According to the distributed scheduling method, the task request is obtained, wherein the task request has the task attribute; sending a construction request to a scheduling server according to the task request, wherein the construction request has task attributes, and the scheduling server selects a target construction server from a plurality of construction servers according to the task attributes and a health state mapping table; and receiving the address of the target construction server fed back by the scheduling server, and calling the target construction server to execute the task corresponding to the task request according to the address, so that the scheduling can be performed according to the health state of the construction server, the scheduling efficiency is improved, and the task execution efficiency is improved.

In the embodiment of the application, the construction of the service platform comprises the following steps: the system comprises a service Server (WebServer), a scheduling Server (Distributed Server) and a plurality of building servers (building Server), wherein the scheduling Server is respectively connected with the service Server and the plurality of building servers. The service server is used for acquiring a task request of a front-end user, sending a construction request to the scheduling server according to the task request so as to acquire an address of a target construction server, and calling the target construction server to execute a task corresponding to the task request according to the address. Fig. 1 is a schematic diagram of an architecture for constructing a service platform.

The scheduling server maintains a health state mapping table (HEALTHY MAP) for scheduling the target construction server for a specific construction request in combination with the health state mapping table, and manages the construction server in combination with the health state mapping table. The construction server maintains a memory queue, wherein the memory queue comprises the tasks which are being executed, and the construction server executes each task in the memory queue according to the sequence of the tasks in the memory queue.

In the embodiment of the application, the dispatch server refers to a server deployed with dispatch services. The build server refers to a server deployed with a build service. One building service can be deployed on the same server, and the building service and the scheduling service can be deployed on the same server at the same time. Therefore, the scheduling server and the construction server in the application can be the same server or different servers, and are determined according to the deployment position of the service, but the corresponding relation between the service and the server is determined.

The distributed scheduling method and the service server according to the embodiments of the present application are described below with reference to the accompanying drawings.

Fig. 2 is a flowchart of a distributed scheduling method according to an embodiment of the present application.

The embodiment of the application is exemplified by the distributed scheduling method being configured in a service server, and the service server can be applied to any computer equipment so that the computer equipment can execute the distributed scheduling function.

As shown in fig. 2, the distributed scheduling method may include the steps of:

step 201, a task request is obtained, wherein the task request has a task attribute.

In the embodiment of the application, the service server receives the task request of the front-end user through the front-end. Wherein the front end is provided with an interface for the front end user to create the task request. Wherein the task attributes include at least one of the following parameters: the type of the platform, the service scene and the construction task identifier of the software to be constructed.

Wherein the platform type is, for example, an IOS platform or an Android platform. The business scenario may be, for example, creating APP (CLONE), creating APP version (BRANCH), building APP (business), analyzing dependency of APP source code (anaiyze), packing channel package (channel) for Android platform, and the like.

Step 202, sending a construction request to a scheduling server according to the task request, wherein the construction request has task attributes, and the scheduling server selects a target construction server from a plurality of construction servers according to the task attributes and the health status mapping table.

In the embodiment of the application, the service server maintains a message queue for storing task requests waiting for no idle construction server, and once the idle construction server exists, the construction request with the task attribute in the task request is reinitiated to the dispatching server. Therefore, in the embodiment of the application, the triggering condition of the service server for sending the construction request to the scheduling server according to the task request is that the idle construction server exists, and the message queue is empty; if the message queue is not empty, the earliest task request is fetched from the message queue, the construction request is sent according to the task attribute in the task request, and the task request just received is cached in the message queue.

In the embodiment of the application, the health state mapping table comprises a plurality of health state indexes of a plurality of construction servers. The health status index includes, for example, a disk remaining space, a CPU remaining space, a memory queue size, a maximum number of parallel execution tasks of a build service deployed on the build server, and the like.

In the embodiment of the present application, the health status mapping table may be obtained, for example, by the scheduling server receiving health status feedback messages sent by a plurality of building servers, where the health status feedback messages include health status information corresponding to the building servers; and the scheduling server maintains a health state mapping table according to the health state information of the plurality of construction servers. The construction server also stores at least one of the following data in the memory queue: total capacity of disk (total), remaining space of disk, total number of cores of CPU, remaining space of CPU, total size of memory, remaining space of memory, size of memory queue, number of tasks executed in parallel to maximum extent by build service deployed on build server, etc., the build server can obtain health status information from the memory queue and provide to the scheduling server to maintain health status mapping table. The way in which the construction server provides the health status information to the dispatch server may be, for example, pushing the health status information to the dispatch server through WebSocket timing.

In the application, the health state mapping table is set on the dispatching server, so that the dispatching server can know the health state of each construction server, and when a certain construction server is disconnected or fails, the health state information of the construction server in the health state mapping table is deleted; if a certain construction server is reconnected or a certain construction server is newly added, the health state information of the construction server can be added in the health state mapping table, so that the pluggable of each construction server is realized, the capacity can be automatically reduced and expanded, and the influence on other construction servers is avoided.

In the embodiment of the application, the scheduling server can select the target building server from a plurality of building servers through the following steps: acquiring a service scene in a task attribute; acquiring a scheduling strategy corresponding to a service scene; and selecting a target construction server from among the plurality of construction servers according to the scheduling policy according to the health status mapping table.

The service scenario may be, for example, creating APP (CLONE), creating APP version (BRANCH), constructing APP (business), analyzing dependency of APP source code (ANALYZE), packing channel package (channel) for Android platform, and the like. Creating a scheduling strategy corresponding to the APP as a source code clone scheduling strategy; creating a scheduling strategy corresponding to the APP version as a branch scheduling strategy; constructing a scheduling strategy corresponding to the APP as a source code compiling scheduling strategy; analyzing a scheduling policy corresponding to the dependence of the APP source code as a source code dependence analysis scheduling policy; the scheduling strategy corresponding to the Android platform package channel package is a multi-channel package scheduling strategy.

In the embodiment of the application, (1) in the source code clone scheduling policy and the branch scheduling policy creation, the plurality of health status indexes are in descending order of priority and are a disk health status index, a CPU health status index and a memory health status index. (2) In the source code compiling and scheduling strategy and the source code dependency analysis and scheduling strategy, the order of the plurality of health state indexes in descending order of priority is that the number of tasks in a memory queue of a building server does not reach the maximum value of the queue, the building server has downloaded codes, and the estimated building time of the building server is smaller than a preset time threshold, a disk health state index, a CPU health state index and a memory health state index. (3) In the multi-channel packet scheduling strategy, the health status indexes are in descending order according to the priority, and the health status indexes of the CPU and the health status indexes of the memory are in descending order.

When the scheduling policy is a source code clone scheduling policy, a branch scheduling policy or a multi-channel packet scheduling policy is created, the scheduling server selects a target construction server in such a way that a construction server with the maximum corresponding health state index is obtained from a plurality of construction servers as the target construction server according to the sequence corresponding to the scheduling policy; if the construction server with the maximum corresponding health state index is not obtained, randomly selecting one construction server from a plurality of construction servers as a target construction server.

In the embodiment of the application, when the scheduling policy is a source code clone scheduling policy, a branch scheduling policy or a multi-channel packet scheduling policy is created, the construction server with the largest corresponding health state index is preferentially selected as the target server, so that the resources during task execution can be increased, the task execution time is reduced, and the task execution efficiency is further improved.

When the scheduling policy is a source code compiling scheduling policy or a source code dependency analysis scheduling policy, the scheduling server selects a target construction server in such a way that the number of corresponding tasks is obtained from a plurality of construction servers and does not reach a corresponding maximum value, and a first construction server with downloaded codes exists, and the first construction server is used as the target construction server; if the first construction server is not acquired, acquiring a plurality of second construction servers, of which the corresponding task numbers do not reach the corresponding maximum value, the downloaded codes do not exist, and the estimated construction time is smaller than a preset time threshold, from the plurality of construction servers; sequentially obtaining a construction server with the maximum corresponding health state index from a plurality of second construction servers according to the health state index of the magnetic disk, the health state index of the CPU and the health state index of the memory in the sequence corresponding to the scheduling strategy, and taking the construction server as a target construction server; if the construction server with the maximum corresponding health state index is not obtained, randomly selecting one construction server from a plurality of construction servers as a target construction server.

In the embodiment of the application, when the scheduling policy is a source code compiling scheduling policy or a source code dependency analysis scheduling policy, the corresponding task number is preferentially selected to not reach the corresponding maximum value, and the first construction server with the downloaded codes is used as the target construction server, so that the code is prevented from being re-downloaded, the time occupied by the code re-downloading is reduced, the scheduling efficiency is further improved, the task execution time is reduced, and the task execution efficiency is further improved.

In the embodiment of the application, when the corresponding task number does not reach the corresponding maximum value and the first construction server of the downloaded code exists, the second server which does not reach the corresponding maximum value and has no downloaded code and has the estimated construction time smaller than the preset time threshold value is preferentially selected as the target construction server, so that the estimated construction time can be reduced as much as possible, namely the task execution time is reduced, and the task execution efficiency is further improved.

And 203, receiving the address of the target construction server fed back by the scheduling server, and calling the target construction server to execute the task corresponding to the task request according to the address.

In the embodiment of the present application, the process of executing step 203 by the service server may be, for example, acquiring task parameters from a database according to a task request; and sending the task parameters to the target construction server, and executing the task parameters through the target construction server.

In the embodiment of the application, the business server particularly can call the address of the target construction server through the remote procedure call RPC (Remote Procedure Call) so as to send the task parameters to the target construction server, so that the target construction server executes the task according to the task parameters. It should be noted that, each building server maintains a memory queue, and the building server may add the executed task to the memory queue.

In the embodiment of the application, the service server and the scheduling server are isolated from the functions of the construction server, namely the service server is used for sending a construction request to acquire a target construction server, and then the target construction server is called to execute tasks; the scheduling server is used for selecting a target construction server according to the construction request; the build server is used to perform tasks. The 3 servers have single functions, have no function cross, do not influence other servers when one of the servers is expanded or updated, realize low coupling and high cohesion between the servers, and further improve the scheduling efficiency and the task execution efficiency.

In the embodiment of the present application, after step 203, the service server may further execute the following steps: receiving task state information fed back by a plurality of construction servers; and updating the task state in the database according to the task state information so as to ensure the accuracy of the task state in the database.

In addition, it should be noted that, if the service server does not receive the address of the target building server fed back by the scheduling server, the target building server is determined according to the building server configuration file. The service server may determine the target build server according to the build server configuration file, for example, by polling addresses of a plurality of build servers, and selecting an available build server from the plurality of build servers as the target build server.

In the embodiment of the application, under the condition that the dispatching server is down or the dispatching strategy is in error, the service server can not receive the addresses of the target construction servers fed back by the dispatching server, and the service server can select the target construction server by polling the addresses of the plurality of construction servers, so that disaster recovery and automatic degradation of the dispatching service can be realized without influencing the execution of tasks.

According to the distributed scheduling method, the task request is obtained, wherein the task request has the task attribute; sending a construction request to a scheduling server according to the task request, wherein the construction request has task attributes, and the scheduling server selects a target construction server from a plurality of construction servers according to the task attributes and a health state mapping table; and receiving the address of the target construction server fed back by the scheduling server, and calling the target construction server to execute the task request according to the address, so that the scheduling can be performed according to the health state of the construction server, the scheduling efficiency is improved, and the task execution efficiency is improved.

Fig. 3 is a flowchart of a distributed scheduling method according to a second embodiment of the present application.

The embodiment of the application is exemplified by the distributed scheduling method being configured in a scheduling server, and the scheduling server can be applied to any computer device so that the computer device can execute the distributed scheduling function.

As shown in fig. 3, the distributed scheduling method may include the steps of:

step 301, a construction request of a service server is received, wherein the construction request has task attributes.

In the embodiment of the application, the construction request of the service server is a construction request sent to the scheduling server according to the task attribute in the task request after the service server acquires the task request of the front-end user.

Step 302, selecting a target building server from among a plurality of building servers according to the task attribute and the health status mapping table.

In the embodiment of the present application, the health status mapping table may be obtained, for example, by the scheduling server receiving health status feedback messages sent by a plurality of building servers, where the health status feedback messages include health status information corresponding to the building servers; and the scheduling server maintains a health state mapping table according to the health state information of the plurality of construction servers. The method for providing the health status information to the scheduling server by the construction server may be, for example, pushing the health status information to the scheduling server at regular time by WebSocket.

In the embodiment of the application, when the scheduling policy is a source code clone scheduling policy, a branch scheduling policy or a multi-channel packet scheduling policy is created, a scheduling server selects a target construction server in such a way that a construction server with the largest corresponding health state index is obtained from a plurality of construction servers as the target construction server according to the sequence corresponding to the scheduling policy; if the construction server with the maximum corresponding health state index is not obtained, randomly selecting one construction server from a plurality of construction servers as a target construction server.

Specifically, taking a scheduling policy as a source code clone scheduling policy as an example, (1) a scheduling server may first obtain a construction server with a corresponding maximum disk health status index from a plurality of construction servers, that is, a construction server with a maximum disk remaining space, and if the construction server is obtained, take the construction server as a target construction server; (2) If the magnetic disk health state indexes of all the construction servers are consistent, acquiring the construction server with the largest corresponding CPU health state index from a plurality of construction servers, namely, the construction server with the largest CPU residual space, and taking the construction server as a target construction server if the construction server is acquired; (3) If the CPU health status indexes of all the construction servers are consistent, acquiring the construction server with the largest corresponding memory health status index from a plurality of construction servers, namely, the construction server with the largest memory residual space, and if the construction server is acquired, taking the construction server as a target construction server; (4) And if the memory health state indexes of all the construction servers are consistent, randomly selecting the construction server from the plurality of construction servers as a target construction server.

In the embodiment of the application, when a scheduling policy is a source code compiling scheduling policy or a source code dependency analysis scheduling policy, a scheduling server selects a target construction server in such a way that the number of corresponding tasks is obtained from a plurality of construction servers and does not reach a corresponding maximum value, and a first construction server with downloaded codes exists, and the first construction server is used as the target construction server; if the first construction server is not acquired, acquiring a plurality of second construction servers, of which the corresponding task numbers do not reach the corresponding maximum value, the downloaded codes do not exist, and the estimated construction time is smaller than a preset time threshold, from the plurality of construction servers; sequentially obtaining a construction server with the maximum corresponding health state index from a plurality of second construction servers according to the health state index of the magnetic disk, the health state index of the CPU and the health state index of the memory in the sequence corresponding to the scheduling strategy, and taking the construction server as a target construction server; if the construction server with the maximum corresponding health state index is not obtained, randomly selecting one construction server from a plurality of construction servers as a target construction server. The corresponding maximum value refers to the maximum task number of the memory queue corresponding to the construction server.

Specifically, taking the scheduling policy as a source code compiling scheduling policy as an example, (1) the scheduling server may obtain a building server with the number of corresponding tasks not reaching the corresponding maximum value from a plurality of building servers, and if the number of the corresponding tasks is not obtained, the building server does not receive the building request; (2) If so, acquiring a first construction server with downloaded codes from construction servers with the corresponding task numbers not reaching the corresponding maximum value, and if so, taking the first construction server as a target construction server; (3) If not, acquiring a second construction server with predicted construction time smaller than a preset time threshold from construction servers with the corresponding task number not reaching the corresponding maximum value; if the number of the second construction servers is single, taking the single construction server as a target construction server; (4) If the number of the second construction servers is a plurality of, acquiring a construction server with the largest corresponding disk health state index from the plurality of second construction servers, namely, a construction server with the largest disk residual space, and if the construction server is acquired, taking the construction server as a target construction server; (5) If the magnetic disk health state indexes of all the construction servers are consistent, acquiring the construction server with the largest corresponding CPU health state index from a plurality of construction servers, namely, the construction server with the largest CPU residual space, and taking the construction server as a target construction server if the construction server is acquired; (6) If the CPU health status indexes of all the construction servers are consistent, acquiring the construction server with the largest corresponding memory health status index from a plurality of construction servers, namely, the construction server with the largest memory residual space, and if the construction server is acquired, taking the construction server as a target construction server; (7) And if the memory health state indexes of all the construction servers are consistent, randomly selecting the construction server from the plurality of construction servers as a target construction server.

And 303, feeding back the address of the target construction server to the service server so that the service server calls the target construction server to execute the task corresponding to the task attribute according to the address.

In the embodiment of the application, after the scheduling server feeds back the address of the target construction server to the service server, the service server can acquire task parameters from the database according to the task request corresponding to the task attribute; and sending the task parameters to the target construction server, and executing the task parameters through the target construction server. The business server can update the task state in the database according to the task state information fed back by the plurality of construction servers so as to ensure the accuracy of the task state in the database.

The distributed scheduling method of the embodiment of the application receives the construction request of the service server, wherein the construction request has task attributes; selecting a target construction server from among a plurality of construction servers according to the task attribute and the health state mapping table; the address of the target construction server is fed back to the service server, so that the service server calls a task request corresponding to the task attribute executed by the target construction server according to the address, and therefore scheduling can be performed according to the health state of the construction server, scheduling efficiency is improved, and task execution efficiency is improved.

Fig. 4 is a schematic structural diagram of a service server according to a third embodiment of the present application.

As shown in fig. 4, the service server 400 may include: an acquisition module 410, a transmission module 420, and a processing module 430.

The acquiring module 410 is configured to acquire a task request, where the task request has a task attribute;

a sending module 420, configured to send a construction request to a scheduling server according to the task request, where the construction request has the task attribute, and the scheduling server selects a target construction server from a plurality of construction servers according to the task attribute and a health status mapping table;

and the processing module 430 is configured to receive the address of the target building server fed back by the scheduling server, and call the target building server to execute the task corresponding to the task request according to the address.

Further, in a possible implementation manner of the embodiment of the present application, the processing module 430 is further configured to determine, when the address of the target build server fed back by the scheduling server is not received, the target build server according to a build server configuration file.

Further, in a possible implementation manner of the embodiment of the present application, the processing module 430 is specifically configured to poll addresses of the plurality of building servers, and select an available building server from the plurality of building servers as the target building server.

Further, in one possible implementation manner of the embodiment of the present application, the processing module 430 is specifically configured to obtain the task parameters from the database according to the task request; and sending the task parameters to the target construction server, and executing the task parameters through the target construction server.

Further, in one possible implementation manner of the embodiment of the present application, the health status mapping table is obtained by the scheduling server receiving health status feedback messages sent by the plurality of building servers, where the health status feedback messages include health status information of the corresponding building servers; and the scheduling server maintains the health state mapping table according to the health state messages of the plurality of construction servers.

Further, in a possible implementation manner of the embodiment of the present application, the processing module 430 is further configured to receive task state information fed back by the plurality of construction servers; and updating the task state in the database according to the task state information.

Further, in a possible implementation of the embodiment of the present application, the scheduling server selects a target build server from among the plurality of build servers by: acquiring a service scene in the task attribute; acquiring a scheduling strategy corresponding to the service scene; and selecting a target construction server from the plurality of construction servers according to the scheduling policy according to the health state mapping table.

Further, in one possible implementation manner of the embodiment of the present application, the health status mapping table includes a plurality of health status indicators, and the target building server is selected in such a manner that priorities of the plurality of health status indicators are determined according to the scheduling policy; and selecting a target construction server from among the candidate construction servers according to the priorities of the health status indexes.

Further, in a possible implementation manner of the embodiment of the present application, the scheduling policy includes at least one of the following policies: a source code clone scheduling strategy, a branch scheduling strategy, a source code compiling scheduling strategy, a source code dependency analysis scheduling strategy and a multi-channel packet scheduling strategy are established;

in the source code clone scheduling strategy and the branch scheduling strategy, the health state indexes are in descending order according to the priority, wherein the descending order is a disk health state index, a CPU health state index and a memory health state index;

In the source code compiling and scheduling strategy and the source code dependency analysis and scheduling strategy, the order of the plurality of health state indexes ordered according to the descending priority is that the number of tasks in a memory queue of a construction server does not reach the maximum value of the queue, the construction server has downloaded codes, the predicted construction time of the construction server is smaller than a preset time threshold, a disk health state index, a CPU health state index and a memory health state index;

In the multi-channel packet scheduling strategy, the plurality of health status indexes are in descending order of priority and are CPU health status indexes and memory health status indexes.

Further, in one possible implementation manner of the embodiment of the present application, when the scheduling policy is a source code clone scheduling policy, a branch scheduling policy is created, or a multi-channel packet scheduling policy, a target building server is selected in a manner that, according to an order corresponding to the scheduling policy, a building server with a maximum corresponding health status index is obtained from the plurality of building servers, and is used as the target building server; and if the construction server with the maximum corresponding health state index is not acquired, randomly selecting one construction server from the plurality of construction servers as the target construction server.

Further, in one possible implementation manner of the embodiment of the present application, when the scheduling policy is a source code compiling scheduling policy or a source code dependency analysis scheduling policy, the target building server is selected in such a manner that the number of tasks obtained from the plurality of building servers does not reach a corresponding maximum value, and a first building server with downloaded codes exists, and the first building server is used as the target building server;

If the first construction server is not acquired, acquiring a plurality of second construction servers, of which the corresponding task numbers do not reach the corresponding maximum value, and the downloaded codes do not exist, and the estimated construction time is smaller than the preset time threshold value, from among the plurality of construction servers;

Sequentially obtaining a construction server with the maximum corresponding health state index from the plurality of second construction servers according to the health state index of the magnetic disk, the health state index of the CPU and the health state index of the internal memory in the sequence corresponding to the scheduling strategy, and taking the construction server as the target construction server;

And if the construction server with the maximum corresponding health state index is not acquired, randomly selecting one construction server from the plurality of construction servers as the target construction server.

The service server of the embodiment of the application obtains the task request, wherein the task request has task attributes; sending a construction request to a scheduling server according to the task request, wherein the construction request has task attributes, and the scheduling server selects a target construction server from a plurality of construction servers according to the task attributes and a health state mapping table; and receiving the address of the target construction server fed back by the scheduling server, and calling the target construction server to execute the task request according to the address, so that the scheduling can be performed according to the health state of the construction server, the scheduling efficiency is improved, and the task execution efficiency is improved.

Fig. 5 is a schematic structural diagram of a scheduling server according to a fourth embodiment of the present application.

As shown in fig. 5, the scheduling server 500 may include: a receiving module 510, a selecting module 520 and a feedback module 530.

The receiving module 510 is configured to receive a construction request of a service server, where the construction request has a task attribute;

A selection module 520, configured to select a target building server from among a plurality of building servers according to the task attribute and the health status mapping table;

And a feedback module 530, configured to feed back an address of the target building server to the service server, so that the service server invokes the target building server to execute the task corresponding to the task attribute according to the address.

Further, in a possible implementation manner of the embodiment of the present application, the scheduling server further includes: a processing module; the receiving module 510 is further configured to receive health status feedback messages sent by the plurality of building servers, where the health status feedback messages include health status information of the corresponding building servers; the processing module is used for maintaining the health state mapping table according to the health state information of the plurality of construction servers.

Further, in one possible implementation manner of the embodiment of the present application, the processing module is specifically configured to obtain a service scenario in the task attribute; acquiring a scheduling strategy corresponding to the service scene; and selecting the target construction server from the plurality of construction servers according to the scheduling policy according to the health state mapping table.

Further, in a possible implementation manner of the embodiment of the present application, the health status mapping table includes a plurality of health status indicators, where the selecting module 520 is specifically configured to determine priorities of the plurality of health status indicators according to the scheduling policy; and selecting the target construction server from among the construction servers according to the health status indexes and the corresponding priorities.

in the source code clone scheduling strategy and the creation branch scheduling strategy, the plurality of health state indexes are in descending order of priority and are in the order of disk health state indexes, CPU health state indexes and memory health state indexes;

In the source code compiling and scheduling strategy and the source code dependency analysis and scheduling strategy, the order of the plurality of health state indexes ordered according to the descending priority is that the number of tasks in a memory queue of a construction server does not reach the maximum value of the queue, the construction server has downloaded codes, the estimated construction time of the construction server is smaller than a preset time threshold, the health state index of a disk, the health state index of a CPU and the health state index of a memory;

Further, in a possible implementation manner of the embodiment of the present application, when the scheduling policy is a source code clone scheduling policy, a branch scheduling policy is created, or a multi-channel packet scheduling policy, the selecting module 520 is specifically configured to obtain, as a target building server, a building server with a corresponding maximum health status index from among a plurality of building servers according to an order corresponding to the scheduling policy; if the construction server with the maximum corresponding health state index is not obtained, randomly selecting one construction server from a plurality of construction servers as a target construction server.

Further, in one possible implementation manner of the embodiment of the present application, when the scheduling policy is a source code compiling scheduling policy or a source code dependency analysis scheduling policy, the selecting module 520 is specifically configured to obtain, from among a plurality of building servers, that the number of tasks corresponding to the first building server does not reach a corresponding maximum value, and that there is a first building server with downloaded codes, and use the first building server as the target building server; if the first construction server is not acquired, acquiring a plurality of second construction servers, of which the corresponding task numbers do not reach the corresponding maximum value, the downloaded codes do not exist, and the estimated construction time is smaller than a preset time threshold, from the plurality of construction servers; sequentially obtaining a construction server with the maximum corresponding health state index from a plurality of second construction servers according to the health state index of the magnetic disk, the health state index of the CPU and the health state index of the memory in the sequence corresponding to the scheduling strategy, and taking the construction server as a target construction server; if the construction server with the maximum corresponding health state index is not obtained, randomly selecting one construction server from a plurality of construction servers as a target construction server.

The scheduling server receives the construction request of the service server, wherein the construction request has task attributes; selecting a target construction server from among a plurality of construction servers according to the task attribute and the health state mapping table; the address of the target construction server is fed back to the service server, so that the service server calls a task request corresponding to the task attribute executed by the target construction server according to the address, and therefore scheduling can be performed according to the health state of the construction server, scheduling efficiency is improved, and task execution efficiency is improved.

In order to implement the above embodiment, the present application further provides a service platform, including: the system comprises a service server, a scheduling server and a plurality of construction servers, wherein the scheduling server is respectively connected with the service server and the construction servers; the service server is configured to perform the distributed scheduling method described in the embodiment shown in fig. 2.

In order to implement the above embodiment, the present application further proposes a computer device including: the distributed scheduling method according to the foregoing embodiment of the present application is implemented by a memory, a processor, and a computer program stored in the memory and executable on the processor, when the processor executes the program.

In order to implement the above embodiments, the present application also proposes a computer program product which, when executed by an instruction processor in the computer program product, performs a distributed scheduling method as proposed in the previous embodiments of the present application.

FIG. 6 illustrates a block diagram of an exemplary computer device suitable for use in implementing embodiments of the present application. The computer device 12 shown in fig. 6 is merely an example and should not be construed as limiting the functionality and scope of use of embodiments of the present application.

As shown in FIG. 6, the computer device 12 is in the form of a general purpose computing device. Components of computer device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, a bus 18 that connects the various system components, including the system memory 28 and the processing units 16.

Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, and a local bus using any of a variety of bus architectures. By way of example, and not limitation, such architectures include industry Standard architecture (Industry Standard Architecture; hereinafter ISA) bus, micro channel architecture (Micro Channel Architecture; hereinafter MAC) bus, enhanced ISA bus, video electronics standards Association (Video Electronics Standards Association; hereinafter VESA) local bus, and peripheral component interconnect (PERIPHERAL COMPONENT INTERCONNECTION; hereinafter PCI) bus.

Computer device 12 typically includes a variety of computer system readable media. Such media can be any available media that is accessible by computer device 12 and includes both volatile and nonvolatile media, removable and non-removable media.

Memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (Random Access Memory; hereinafter: RAM) 30 and/or cache memory 32. The computer device 12 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 34 may be used to read from or write to non-removable, nonvolatile magnetic media (not shown in FIG. 6, commonly referred to as a "hard disk drive"). Although not shown in fig. 6, a disk drive for reading from and writing to a removable nonvolatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive for reading from or writing to a removable nonvolatile optical disk (e.g., a compact disk read only memory (Compact Disc Read Only Memory; hereinafter CD-ROM), digital versatile read only optical disk (Digital Video Disc Read Only Memory; hereinafter DVD-ROM), or other optical media) may be provided. In such cases, each drive may be coupled to bus 18 through one or more data medium interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to carry out the functions of embodiments of the application.

A program/utility 40 having a set (at least one) of program modules 42 may be stored in, for example, memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment. Program modules 42 generally perform the functions and/or methods of the embodiments described herein.

The computer device 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), one or more devices that enable a user to interact with the computer device 12, and/or any devices (e.g., network card, modem, etc.) that enable the computer device 12 to communicate with one or more other computing devices. Such communication may occur through an input/output (I/O) interface 22. Moreover, the computer device 12 may also communicate with one or more networks such as a local area network (Local Area Network; hereinafter: LAN), a wide area network (Wide Area Network; hereinafter: WAN) and/or a public network such as the Internet via the network adapter 20. As shown, network adapter 20 communicates with other modules of computer device 12 via bus 18. It should be appreciated that although not shown, other hardware and/or software modules may be used in connection with computer device 12, including, but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, data backup storage systems, and the like.

The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, for example, implementing the methods mentioned in the foregoing embodiments.

In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, the different embodiments or examples described in this specification and the features of the different embodiments or examples may be combined and combined by those skilled in the art without contradiction.

Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In the description of the present application, the meaning of "plurality" means at least two, for example, two, three, etc., unless specifically defined otherwise.

Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and additional implementations are included within the scope of the preferred embodiment of the present application in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order from that shown or discussed, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the embodiments of the present application.

Logic and/or steps represented in the flowcharts or otherwise described herein, e.g., a ordered listing of executable instructions for implementing logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program is printed, as the program may be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.

It is to be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. As with the other embodiments, if implemented in hardware, may be implemented using any one or combination of the following techniques, as is well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.

Those of ordinary skill in the art will appreciate that all or a portion of the steps carried out in the method of the above-described embodiments may be implemented by a program to instruct related hardware, where the program may be stored in a computer readable storage medium, and where the program, when executed, includes one or a combination of the steps of the method embodiments.

In addition, each functional unit in the embodiments of the present application may be integrated in one processing module, or each unit may exist alone physically, or two or more units may be integrated in one module. The integrated modules may be implemented in hardware or in software functional modules. The integrated modules may also be stored in a computer readable storage medium if implemented in the form of software functional modules and sold or used as a stand-alone product.

The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, or the like. While embodiments of the present application have been shown and described above, it will be understood that the above embodiments are illustrative and not to be construed as limiting the application, and that variations, modifications, alternatives and variations may be made to the above embodiments by one of ordinary skill in the art within the scope of the application.

Claims

1. A distributed scheduling method, comprising:

Acquiring a task request, wherein the task request has a task attribute;

Sending a construction request to a scheduling server according to the task request, wherein the construction request has the task attribute, the scheduling server selects a target construction server from a plurality of construction servers according to the task attribute and a health state mapping table, and the health state mapping table comprises a plurality of health state indexes of the construction servers;

receiving an address of the target construction server fed back by the scheduling server, and calling the target construction server to execute a task corresponding to the task request according to the address;

wherein the dispatch server selects a target build server from among the plurality of build servers by:

Acquiring a service scene in the task attribute;

Acquiring a scheduling strategy corresponding to the service scene;

Determining priorities of the plurality of health status indicators according to the scheduling policy;

Selecting a target building server from among the plurality of candidate building servers according to the priorities of the plurality of health status indicators;

When the scheduling policy is a source code compiled scheduling policy or a source code dependent analysis scheduling policy, the selecting the target build server from among the plurality of build servers includes:

Acquiring a first construction server with downloaded codes from the plurality of construction servers, wherein the number of the corresponding tasks does not reach a corresponding maximum value, and the first construction server is used as the target construction server;

If the first build server is not acquired, acquiring a second build server, of which the number of tasks is not up to a corresponding maximum value and downloaded codes are not available and the build time is expected to be smaller than a preset time threshold, from among the plurality of build servers as a target build server.

2. The distributed scheduling method of claim 1, further comprising:

And if the address of the target construction server fed back by the scheduling server is not received, determining the target construction server according to a construction server configuration file.

3. The distributed scheduling method of claim 2, wherein the build server profile includes addresses of a plurality of build servers, and wherein determining the target build server from the build server profile comprises:

Polling addresses of the plurality of build servers, and selecting an available build server from among the plurality of build servers as the target build server.

4. The distributed scheduling method of claim 1, wherein invoking the target build server to execute the task corresponding to the task request according to the address comprises:

acquiring task parameters from a database according to the task request;

and sending the task parameters to the target construction server, and executing the task parameters through the target construction server.

5. The distributed scheduling method of claim 1, wherein the health status mapping table is obtained by,

The scheduling server receives health state feedback messages sent by the plurality of construction servers, wherein the health state feedback messages comprise health state information of the corresponding construction servers;

and the scheduling server maintains the health state mapping table according to the health state information of the plurality of construction servers.

6. The distributed scheduling method of claim 1, further comprising:

receiving task state information fed back by the plurality of construction servers;

And updating the task state in the database according to the task state information.

7. The distributed scheduling method of claim 1, wherein the scheduling policy comprises at least one of: a source code clone scheduling strategy, a branch scheduling strategy, a source code compiling scheduling strategy, a source code dependency analysis scheduling strategy and a multi-channel packet scheduling strategy are established;

8. The distributed scheduling method of claim 7, wherein when the scheduling policy is a source code clone scheduling policy, a create branch scheduling policy, or a multi-channel packet scheduling policy, the selecting the target build server from among the plurality of build servers according to the plurality of health indicators and corresponding priorities comprises:

according to the sequence corresponding to the scheduling strategy, acquiring a construction server with the maximum corresponding health state index from the plurality of construction servers as the target construction server;

9. The distributed scheduling method of claim 7, wherein the second build server is a plurality of build servers, the obtaining a corresponding number of tasks from among the plurality of build servers does not reach a corresponding maximum value, and no downloaded code exists, and the second build server whose build time is expected to be less than the preset time threshold is a target build server, comprising:

Sequentially obtaining a construction server with the maximum corresponding health state index from a plurality of second construction servers according to the health state index of the magnetic disk, the health state index of the CPU and the health state index of the internal memory in the sequence corresponding to the scheduling strategy, and taking the construction server with the maximum corresponding health state index as the target construction server;

10. The distributed scheduling method of any one of claims 1-9, wherein the traffic scenario is at least one of: creating APP, creating APP version, constructing APP, analyzing dependence of APP source codes, and packaging channel packages aiming at Android platforms.

11. A service server, comprising:

The acquisition module is used for acquiring a task request, wherein the task request has task attributes;

The sending module is used for sending a construction request to a scheduling server according to the task request, wherein the construction request has the task attribute, the scheduling server selects a target construction server from a plurality of construction servers according to the task attribute and a health state mapping table, and the health state mapping table comprises a plurality of health state indexes of the construction servers;

the processing module is used for receiving the address of the target construction server fed back by the scheduling server and calling the target construction server to execute the task corresponding to the task request according to the address;

Acquiring a service scene in the task attribute;

Acquiring a scheduling strategy corresponding to the service scene;

12. A build service platform, comprising:

the service server is configured to perform the distributed scheduling method according to any one of claims 1-10.

13. An electronic device, comprising:

At least one processor; and

A memory communicatively coupled to the at least one processor; wherein,

The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-10.

14. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-10.