CN109753300B

CN109753300B - Algorithm upgrading method, calculation task sending method and related device

Info

Publication number: CN109753300B
Application number: CN201711071592.6A
Authority: CN
Inventors: 姜晓燕; 蔡宁; 王少萌; 杨旭
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2017-11-03
Filing date: 2017-11-03
Publication date: 2022-05-06
Anticipated expiration: 2037-11-03
Also published as: CN109753300A

Abstract

The embodiment of the application discloses an algorithm upgrading method, which is applied to upgrading of algorithm packages deployed on computing equipment in a computing cluster and comprises the steps of obtaining an algorithm description file of an algorithm to be upgraded, determining the algorithm package corresponding to the algorithm description file according to the algorithm description file, and updating the algorithm to be upgraded into the corresponding algorithm package; the updated algorithm package is deployed in the computing equipment, so that the computing equipment processes the task of the algorithm to be upgraded through the updated algorithm package, the situation that a large data processing platform cannot provide data processing service to the outside when the algorithm service is upgraded originally is avoided, and the system efficiency is improved. The embodiment of the application also discloses an algorithm upgrading device, computing equipment, an algorithm client, a computing task sending method, a device, an algorithm client, a relevant storage medium, a computer program product and the like.

Description

Algorithm upgrading method, calculation task sending method and related device

Technical Field

The present application relates to the field of data processing, and in particular, to an algorithm upgrading method, a calculation task sending method, and a related product.

Background

Various data need to be stored in the big data processing process, and the stored data can be calculated according to the calculation requirement to obtain the required calculation result. The big data processing platform can realize the storage and the calculation service of big data.

The big data processing platform mainly comprises a control cluster and a computing cluster, wherein the computing cluster comprises a plurality of computing nodes (or computing devices), algorithm services are deployed in the control cluster, the algorithm services mainly comprise algorithm control related logic, and the control cluster can submit computing tasks to the computing devices in the computing cluster according to the algorithm services and monitor the execution condition of the computing tasks in the computing devices. An algorithm package required by big data calculation is deployed in the computing device, and calculation of the calculation tasks submitted by the control cluster can be realized through the algorithm package.

In order to meet different big data calculation requirements, the algorithm for big data needs to be updated iteratively frequently, for example, new algorithm logic, algorithm combinations and the like are added.

Disclosure of Invention

In the traditional mode, algorithms are uniformly managed through algorithm services, so that the updating of the algorithms can be realized only by upgrading the algorithm services in the control cluster. If the algorithm service needs to be upgraded, the algorithm service deployed in the control cluster needs to be stopped, the algorithm service is recompiled, and after the upgrade algorithm service is compiled, the upgraded algorithm service needs to be redeployed in the control cluster to complete the updating of the algorithm. In the process of upgrading the algorithm service, the large data processing platform cannot provide data processing service to the outside, and the system efficiency is reduced.

In order to solve the technical problems, the application provides an algorithm upgrading method, a calculation task sending method and a related product.

The embodiment of the application discloses the following technical scheme:

in a first aspect, an embodiment of the present application discloses an algorithm upgrading method, which is applied to upgrade of an algorithm package deployed on a computing device in a computing cluster, and the method includes:

acquiring an algorithm description file of an algorithm to be upgraded;

determining an algorithm package corresponding to the algorithm to be upgraded according to the algorithm description file, and updating the algorithm to be upgraded into the algorithm package corresponding to the algorithm to be upgraded;

and deploying the updated algorithm package in the computing equipment, so that the computing equipment performs processing aiming at the task of the algorithm to be upgraded through the updated algorithm package.

Optionally, the algorithm description file includes an output description of the algorithm to be upgraded, an algorithm parameter required for implementing the algorithm to be upgraded, and an operation logic description of the algorithm to be upgraded.

Optionally, the algorithm description file further includes an input description of an algorithm to be upgraded.

Optionally, obtaining an algorithm description file of an algorithm to be upgraded includes:

and acquiring an algorithm description file of the algorithm to be upgraded from a control cluster of the big data processing platform.

Optionally, after the updated algorithm package is deployed in the computing device, the method further includes:

acquiring a calculation task, wherein the calculation task comprises data to be calculated and an algorithm description file;

and calculating the data to be calculated according to the algorithm description file to obtain a calculation result.

In a second aspect, an embodiment of the present application discloses an algorithm upgrading apparatus, which is applied to upgrading an algorithm package deployed on a computing device in a computing cluster, and the apparatus includes a first obtaining unit, an updating unit, and a deploying unit, where:

the first acquisition unit is used for acquiring an algorithm description file of the algorithm to be upgraded;

the updating unit is used for determining the algorithm package corresponding to the algorithm to be upgraded according to the algorithm description file and updating the algorithm to be upgraded into the algorithm package corresponding to the algorithm to be upgraded;

and the deployment unit is used for deploying the updated algorithm package in the computing equipment so that the computing equipment performs processing aiming at the task of the algorithm to be upgraded through the updated algorithm package.

Optionally, the first obtaining unit is specifically configured to:

Optionally, the apparatus further comprises:

the computing task acquisition unit is used for acquiring computing tasks, and the computing tasks comprise data to be computed and algorithm description files;

and the calculating unit is used for calculating the data to be calculated according to the algorithm description file to obtain a calculation result.

In a third aspect, an embodiment of the present application discloses a computing device for algorithm upgrade, which is applied to upgrade of an algorithm package deployed on a computing device in a computing cluster, where the computing device includes a processor and a memory, where:

a memory for storing the program code and transmitting the program code to the processor;

a processor for performing the following method according to instructions in program code:

acquiring an algorithm description file of an algorithm to be upgraded;

Optionally, the algorithm description file includes an output description of the algorithm to be upgraded, algorithm parameters required for implementing the algorithm to be upgraded, and an operation logic description of the algorithm to be upgraded.

Optionally, the processor is further configured to execute the following method according to the instructions in the program code:

Optionally, the processor is further configured to execute the following method according to instructions in the program code:

In a fourth aspect, an embodiment of the present application discloses an algorithm upgrading method, which is applied to upgrade an algorithm package deployed on a computing device in a computing cluster, and the method includes:

acquiring an algorithm to be upgraded;

acquiring an algorithm description file of an algorithm to be upgraded;

and initiating an algorithm upgrading request aiming at the algorithm to be upgraded according to the algorithm description file.

Optionally, the method further includes updating the algorithm client for the algorithm to be upgraded.

Optionally, the obtaining of the algorithm description file of the algorithm to be upgraded includes:

acquiring the output description of the algorithm to be upgraded;

and acquiring algorithm parameters required by the algorithm to be upgraded and the operation logic description of the algorithm to be upgraded.

And obtaining an algorithm description file according to the output description, the algorithm parameters required for realizing the algorithm to be upgraded and the operation logic description of the algorithm to be upgraded.

Optionally, the algorithm description file further includes an input description of the algorithm to be upgraded, and before acquiring the output description of the algorithm to be upgraded, the method further includes:

and acquiring the input description of the algorithm to be upgraded according to the algorithm description format.

Optionally, initiating an algorithm upgrading request for an algorithm to be upgraded according to the algorithm description file, including:

and sending an algorithm upgrading request to a control cluster of the big data processing platform.

In a fifth aspect, an embodiment of the present application discloses an algorithm upgrading apparatus, which is applied to upgrading an algorithm package deployed on a computing device in a computing cluster, and the apparatus includes a second obtaining unit, a third obtaining unit, and an initiating unit, where:

the second acquisition unit is used for acquiring the algorithm to be upgraded;

the third acquisition unit is used for acquiring an algorithm description file of the algorithm to be upgraded;

and the initiating unit is used for initiating an algorithm upgrading request aiming at the algorithm to be upgraded according to the algorithm description file.

Optionally, the apparatus further includes an algorithm client updating unit, configured to update the algorithm client according to the algorithm to be upgraded.

Optionally, the third obtaining unit is specifically configured to:

acquiring the output description of the algorithm to be upgraded;

acquiring algorithm parameters required by the algorithm to be upgraded and operation logic description of the algorithm to be upgraded;

Optionally, the third obtaining unit is further configured to:

Optionally, the initiating unit is specifically configured to:

In a sixth aspect, an embodiment of the present application discloses an algorithm client for algorithm upgrade, which is applied to upgrade of an algorithm package deployed on a computing device in a computing cluster, and includes a processor and a memory, where:

acquiring an algorithm to be upgraded;

acquiring an algorithm description file of an algorithm to be upgraded;

and updating the algorithm client aiming at the algorithm to be upgraded.

acquiring the output description of the algorithm to be upgraded;

In a seventh aspect, an embodiment of the present application discloses an algorithm upgrading system, where a uniform algorithm description format is set for an algorithm applied to a large data processing platform, the system includes a computing device and an algorithm client, where:

the algorithm client is used for acquiring the algorithm to be upgraded; acquiring an algorithm description file of an algorithm to be upgraded; initiating an algorithm upgrading request aiming at the algorithm to be upgraded according to the algorithm description file;

the computing equipment is used for acquiring an algorithm description file of the algorithm to be upgraded; determining an algorithm package corresponding to the algorithm to be upgraded according to the algorithm description file, and updating the algorithm to be upgraded into the algorithm package corresponding to the algorithm to be upgraded; and deploying the updated algorithm package into the computing equipment, so that the computing equipment performs processing aiming at the task of the algorithm to be upgraded through the updated algorithm package.

In an eighth aspect, an embodiment of the present application discloses a method for sending a computing task, where the method includes:

acquiring a calculation demand;

generating a computing task according to the computing requirement, wherein the computing task comprises a task algorithm description file which is used for describing an algorithm required for processing the computing task;

and sending the computing task to the control node so that the control node distributes the computing task to the computing device for processing.

In a ninth aspect, an embodiment of the present application discloses a computing task sending device, including a computing requirement obtaining unit, a computing task generating unit, and a computing task sending unit, where:

a calculation demand acquisition unit for acquiring a calculation demand;

the computing task generating unit is used for generating a computing task according to the computing requirement, the computing task comprises a task algorithm description file, and the task algorithm description file is used for describing an algorithm required by processing the computing task;

and the computing task sending unit is used for sending the computing task to the control node so that the control node distributes the computing task to the computing equipment for processing.

In a tenth aspect, an embodiment of the present application discloses an algorithm client for sending a computation task, where the algorithm client includes a memory and a processor, where:

acquiring a calculation demand;

In an eleventh aspect, an embodiment of the present application discloses a storage medium, on which a program is stored, where the program is executed by a processor, and the method for upgrading an algorithm provided in the first aspect or the fourth aspect of the embodiment of the present application or the method for sending a computing task provided in the eighth aspect of the embodiment of the present application.

In a twelfth aspect, embodiments of the present application further disclose a computer program product comprising instructions adapted to execute a program initialized with the above method steps when run on a computing device or an algorithm client.

It can be seen from the above technical solutions that, for the upgrade of the algorithm package deployed on the computing device in the computing cluster, when the algorithm upgrade is needed, the new algorithm to be upgraded can be described according to the algorithm description format, and the algorithm description file describing the algorithm to be upgraded can be identified by the control cluster and the computing device in the big data processing platform, so when the algorithm upgrade is needed, the corresponding algorithm description file can be generated without correspondingly changing the core algorithm service of the big data processing platform as in the conventional manner, when the computing device receives the algorithm description file of the algorithm to be upgraded, the algorithm to be upgraded can be updated to the algorithm package corresponding to the algorithm to be upgraded that needs to be deployed on the computing device, and when the updated algorithm package is deployed on the computing cluster, the computing device can perform the processing for the task of the algorithm to be upgraded through the updated algorithm package, the situation that a large data processing platform cannot provide data processing service to the outside when the algorithm service is originally upgraded is avoided, and the system efficiency is improved.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive exercise.

Fig. 1 is a schematic system diagram of a big data processing platform during algorithm upgrade according to an embodiment of the present disclosure;

fig. 2 is a flowchart of a method of an algorithm upgrading method according to an embodiment of the present application;

fig. 3 is a flowchart of a method of an algorithm upgrading method according to an embodiment of the present application;

fig. 4 is a device structure diagram of an algorithm upgrading device according to an embodiment of the present application;

fig. 5 is a device structure diagram of an algorithm upgrading device according to an embodiment of the present application;

FIG. 6 is a block diagram of a computing device for algorithm upgrade provided by an embodiment of the present application;

fig. 7 is a structural diagram of an algorithm client for algorithm upgrade according to an embodiment of the present disclosure;

FIG. 8 is a block diagram of an algorithm upgrade system provided in an embodiment of the present application;

fig. 9 is a flowchart of a method for sending a computing task according to an embodiment of the present application;

fig. 10 is a device structure diagram of a computing task sending device according to an embodiment of the present application;

fig. 11 is a structural diagram of an algorithm client for sending a computation task according to an embodiment of the present application.

Detailed Description

Embodiments of the present application are described below with reference to the accompanying drawings.

The big data processing platform can provide storage and computing services of mass data. A big data processing platform may mainly include a control cluster and a computing cluster, and a plurality of computing nodes (or computing devices) may be included in the computing cluster. The control cluster is deployed with algorithm service, the algorithm service mainly comprises algorithm control related logic, an algorithm package is deployed on computing equipment of the computing cluster, the control cluster can be distributed to the computing equipment in the computing cluster for processing when acquiring computing tasks, and the computing equipment can call the computing logic required by the computing tasks through the algorithm package to process the computing tasks.

With the rapid development of big data processing services, in order to meet emerging computing requirements, algorithms for big data need to be updated iteratively and rapidly. However, in the conventional method, the algorithm is uniformly managed through the algorithm service, so that if the algorithm needs to be upgraded, the updated algorithm needs to be added to the algorithm service, otherwise, the algorithm service cannot identify what the newly added algorithm is, and how to calculate, so that the calculation task which needs to be realized through the algorithm cannot be executed. After the content of the algorithm service is updated, the updated content is installed in the control cluster in an updating mode, namely the version of the algorithm service needs to be updated, so that the capacity of the big data processing platform for specifically processing a new algorithm is realized. It can be seen that in the conventional manner, the updating of the new algorithm and the upgrading of the algorithm service of the control cluster have a coupling relationship, and the updating of the algorithm must be realized only by upgrading the algorithm service.

However, if the algorithm service of a control cluster is to be upgraded, the algorithm service used by the control cluster needs to be suspended, and after the upgraded or new algorithm service is obtained by means of recompilation or the like, the algorithm service can be reused by the control cluster after the control cluster is redeployed to the control cluster. Therefore, in the process of upgrading the algorithm service, the control cluster cannot provide the service which can be provided only by using the algorithm service to the outside. Even a skilled technician needs more than half an hour to upgrade the algorithm service for the control cluster, and the control cluster and the computing equipment connected with the control cluster cannot provide the computing service of big data in the period of time, thereby wasting the system resources of a big data processing platform.

Moreover, in addition to the above problems, the upgrading of the algorithm service requires a high level of skill of the technician, and the technician must be very familiar with the mechanism of the algorithm service, but in practice, how to update the algorithm and how to upgrade the algorithm service has little relation per se, which results in that the technician must master additional complex techniques, increasing the cost of the algorithm upgrading. Moreover, once an algorithm error or a compiling error occurs, the use of the algorithm service by the control cluster is directly influenced, and even the normal operation of the whole big data processing platform is influenced.

Furthermore, the algorithm service is used as a core program supporting the operation of the big data processing platform, and in order to provide stable service, the upgrade period of the algorithm service is not frequent, and the algorithm service is basically upgraded about every two months. However, the algorithm updating is generally required frequently, new algorithms may need to be used every few days, but the algorithms can only be updated until the algorithm service needs to be upgraded, and thus, the updating period with excessively long algorithm service obviously cannot meet the frequency and the requirement of the algorithm updating.

Therefore, the embodiment of the present application provides an algorithm upgrading method and apparatus, which are applied to upgrading an algorithm package deployed on computing devices in a computing cluster, when algorithm upgrading is required, the new algorithm to be upgraded can be described according to an algorithm description format, an algorithm description file describing the algorithm to be upgraded can be identified by each cluster in the big data processing platform, so when algorithm upgrading is required, corresponding modification on core algorithm services of the big data processing platform in a conventional manner is not required, and a corresponding algorithm description file is generated, when the algorithm description file describing the algorithm to be upgraded is received, the algorithm package to be deployed on the computing devices in the computing cluster can be updated according to the algorithm description file, and after the updated algorithm package is deployed on the computing devices, the computing devices can perform processing on tasks of the algorithm to be upgraded through the updated algorithm package, the situation that a large data processing platform cannot provide data processing service to the outside when the algorithm service is originally upgraded is avoided, and the system efficiency is improved.

Moreover, the algorithm upgrading scheme does not need to know the mechanism of algorithm service, so that the technical requirement on technical personnel is low, and the cost of algorithm upgrading is reduced.

Furthermore, because the updating period of the algorithm package is short and can generally reach about twice a week, the new algorithm described by the algorithm description file can be upgraded into a large data processing platform more quickly and put into use more quickly, and compared with the updating period of the traditional mode, the updating frequency and the updating requirement of the algorithm can be better met.

In the embodiment of the application, a uniform algorithm description format is introduced for a big data processing platform, and various algorithm types, combinations, logics and the like can be described through the algorithm description format to obtain a corresponding algorithm description file. Moreover, because the algorithm description format is uniform, the description contents of the algorithm are the same in the algorithm description file obtained by the algorithm description format for the same algorithm type, combination and logic. Moreover, through the introduction of the algorithm description format, the algorithm service can identify the content in the algorithm description file generated according to the algorithm description format, so that how to implement the algorithm described by the algorithm description file can be clear, and the coupling relation between the algorithm service upgrade and the algorithm update is released.

In addition to employing a uniform algorithm description format in a big data processing system, the algorithm description format can be deployed at an algorithm client. For example, as shown in fig. 1, the algorithm client 100 is connected to the control cluster 200, the algorithm interface is written in the algorithm client 100, the control cluster 200 may be connected to a plurality of computing devices 300, and the plurality of computing devices 300 form a computing cluster, which may cooperatively execute computing tasks. Algorithm packages are deployed in the computing device 300, and the number and types of algorithm packages may be related to the type of algorithm, for example, algorithms related to matrix calculations may be packaged into one algorithm package, and algorithms related to basic statistics may be packaged into another algorithm package. In the present application, the algorithm packages deployed by computing device 300 may be a collective term for the various algorithm types used by large data processing platforms.

The algorithm client 100 may be a client segment program installed with an algorithm Software Development Kit (SDK), and may be provided in an intelligent terminal for use by a user, a technician, or the like.

The algorithm client 100 may generate a computation task including a corresponding algorithm description file according to a computation requirement through the set algorithm description format, and when receiving the computation task, the control cluster 200 may identify content carried by the algorithm description file, such as a described algorithm, so that the control cluster 200 may send the computation task to a certain computing device 300 capable of processing the algorithm for processing, and the computing device 300 that receives the computation task may also identify the algorithm description file, call up required data and algorithm, and process the computation task according to a computation logic described in the algorithm description file.

Next, how to perform algorithm upgrade in the big data processing platform according to the algorithm description format will be described. Fig. 2 is a flowchart of a method for upgrading an algorithm according to an embodiment of the present application, where the method includes:

s201: and acquiring an algorithm description file of the algorithm to be upgraded.

For example, a computing device in a big data processing system may be a main body for obtaining the algorithm description file, and thus the embodiment illustrated in fig. 2 may be a description of the embodiment of the present application from the perspective of the computing device. The computing device can make clear the content of the algorithm to be upgraded through the algorithm description file, and adds the algorithm to be upgraded to the corresponding algorithm package during upgrading.

The algorithm to be upgraded may be a new algorithm different from the existing algorithm of the large data processing platform, and the new algorithm may be understood to include a new algorithm, a new combination of algorithms, or a new computational logic.

The algorithm description file is used for describing an algorithm to be upgraded, and the control cluster and the computing device can identify the algorithm to be upgraded through the description of the algorithm description file so as to achieve the degree of implementing the algorithm to be upgraded, for example, how the algorithm to be upgraded is calculated, specific calculation logic or flow and the like can be determined. The algorithm description file can be generated by an algorithm client, or can be generated by other terminals or servers. It should be noted that, if the algorithm description file is generated by the algorithm client, the algorithm description file may be sent to the control cluster of the big data processing platform by the algorithm client, and after the control cluster completes the permission check and the content analysis of the algorithm description file, the algorithm description file may be sent to the computing device, and the computing device updates the algorithm package for the algorithm to be upgraded. That is, if the algorithm description file is generated by the algorithm client, the algorithm description file acquired by the computing device in this step may be sent by the control cluster of the big data processing platform.

In order to enable the control cluster and the computing device in the big data processing platform to recognize the algorithm to be upgraded so as to implement the algorithm to be upgraded, the algorithm description file may include an output description of the algorithm to be upgraded, algorithm parameters required for implementing the algorithm to be upgraded, and an operation logic description of the algorithm to be upgraded.

The output description of the algorithm to be upgraded is used to clarify the information such as the type and the position output by the calculation and the calculation result of the algorithm to be upgraded, and for example, in a big data processing platform which basically adopts a table format for storage and calculation, the output description of the algorithm to be upgraded may include information such as a table name and a partition where the table name and the partition are located.

The algorithm parameters required for implementing the algorithm to be upgraded are used for defining the parameters related to the algorithm required for implementing the algorithm to be upgraded, such as the cycle number of a certain step, the iteration number of the logistic regression algorithm, and the like.

The operation logic of the algorithm to be upgraded describes a flow for explicitly implementing the algorithm to be upgraded, for example, which computation step is executed in what computation situation, and the like, and may specifically be a distributed Directed Acyclic Graph (DAG), and the order and flow of computation may be indicated by an arrow direction in the DAG Graph.

In some cases, the input content required for implementing the algorithm to be upgraded needs to be defined or provided, rather than being automatically generated by the system, so the algorithm description file in some cases also needs to include an input description for the algorithm to be upgraded, that is, information related to input parameters for explicitly implementing the algorithm to be upgraded, for example, in a large data processing platform that basically uses a table format for storage and calculation, the input description for the algorithm to be upgraded may include information such as a table name, a partition where the table name is located, and when the input parameters are matrices, information such as a number of rows and a number of columns may be further included.

S202: and determining an algorithm package corresponding to the algorithm to be upgraded according to the algorithm description file, and updating the algorithm to be upgraded into the algorithm package corresponding to the algorithm to be upgraded.

Because the computing device can clearly implement the algorithm to be upgraded through the algorithm description file, the algorithm to be upgraded can be updated to the corresponding algorithm package according to the algorithm description file, for example, if the algorithm to be upgraded is a matrix algorithm, the algorithm to be upgraded can be updated to the algorithm package corresponding to the matrix algorithm.

S203: and deploying the updated algorithm package in the computing equipment so that the computing equipment performs processing aiming at the task of the algorithm to be upgraded.

By redeploying the updated algorithm package to the computing device, the computing device can identify the algorithm to be upgraded and clearly determine how to use the algorithm to be upgraded when receiving a computing task using the algorithm to be upgraded. The updated algorithm package can be deployed by deleting the algorithm package which is not updated on the computing equipment and deploying the updated algorithm package.

After the algorithm to be upgraded is upgraded by the big data processing platform, the ability of processing the algorithm to be upgraded is achieved, and if the computing equipment acquires the computing task needing to use the algorithm to be upgraded, the computing equipment can process the computing task. For example, if the computing device obtains a computing task, where the computing task includes data to be computed and an algorithm description file corresponding to the algorithm to be upgraded, the computing device may compute the data to be computed according to the algorithm description file to obtain a computing result.

The application does not limit the time for deploying the updated algorithm package on the computing device again, for example, the algorithm package can be deployed directly after being updated, or the algorithm package can be deployed when the uniform update period of the algorithm package arrives, for example, the update frequency of twice per week is adopted by some big data processing platforms aiming at the algorithm package of the computing device and is much higher than the update frequency (once for one month and once for two months) of the computing service, so that the new algorithm described by the algorithm description file can be updated into the big data processing platform more quickly and put into use more quickly, and compared with the update period of the traditional mode, the update frequency and the requirement of the algorithm can be better met.

It can be seen that, the algorithms in the big data processing platform are all described in a uniform algorithm description format, so that when the algorithm is needed to be upgraded, the new algorithm to be upgraded can be described according to the algorithm description format, the algorithm description file describing the algorithm to be upgraded can be identified by each cluster in the big data processing platform, so that when the algorithm is needed to be upgraded, the corresponding algorithm description file can be generated without correspondingly changing the core algorithm service of the big data processing platform in a traditional way, when the algorithm description file describing the algorithm to be upgraded is received, the algorithm package needed to be deployed on the computing equipment can be updated according to the algorithm description file, after the updated algorithm package is deployed on the computing equipment, the computing equipment can perform processing aiming at the algorithm to be upgraded through the updated algorithm package, and the situation that the big data processing platform cannot provide data processing service to the outside when the algorithm service is originally upgraded is avoided, the system efficiency is improved. Moreover, because algorithm upgrading and algorithm service upgrading are decoupled, the algorithm upgrading scheme of the application is implemented without knowing the mechanism of the algorithm service, so that the technical requirements on technical personnel are low, and the cost of the algorithm upgrading is reduced.

Next, from the perspective of an algorithm client connected to a big data processing platform, the algorithm upgrading method provided in the embodiment of the present application is analyzed, and fig. 3 is a flowchart of a method of the algorithm upgrading method provided in the embodiment of the present application, where the method is applied to upgrade an algorithm package deployed on a computing device in a computing cluster, please refer to fig. 3, where the method includes:

s301: and acquiring an algorithm to be upgraded.

The method for obtaining the algorithm to be upgraded is not limited, for example, the algorithm to be upgraded may be provided by a user having a calculation requirement for the algorithm to be upgraded, may also be provided by a technician according to a possible calculation requirement, and may even be obtained by the user according to a requirement of a specific calculation task, and is provided to the algorithm client together with the calculation requirement.

S302: and acquiring an algorithm description file of the algorithm to be upgraded.

The algorithm description file is used for describing an algorithm to be upgraded, and the control cluster and the computing device can identify the algorithm to be upgraded through the description of the algorithm description file so as to achieve the degree of implementing the algorithm to be upgraded, for example, how the algorithm to be upgraded is calculated, specific calculation logic or flow and the like can be determined.

In order to enable the control cluster and the computing device in the big data processing platform to recognize the algorithm to be upgraded so as to implement the algorithm to be upgraded, the algorithm description file may include an output description of the algorithm to be upgraded, algorithm parameters required for implementing the algorithm to be upgraded, and an operation logic description of the algorithm to be upgraded. As a possible implementation manner, the algorithm parameters required by the algorithm to be upgraded and the operation logic description of the algorithm to be upgraded can be obtained by obtaining the output description of the algorithm to be upgraded, and the algorithm description file can be obtained according to the output description, the algorithm parameters required by the algorithm to be upgraded and the operation logic description of the algorithm to be upgraded.

In some cases, the input content required for implementing the algorithm to be upgraded needs to be defined or provided, rather than being automatically generated by the system, so the algorithm description file also needs to include an input description for the algorithm to be upgraded, i.e. information about input parameters for explicitly implementing the algorithm to be upgraded.

Optionally, a logistic regression algorithm is taken as an example to explain how the algorithm client generates the algorithm description file in the embodiment of the present application, and in the following steps, the generated description is obtained according to the algorithm description format.

The algorithm client may obtain an input description of the logistic regression algorithm according to the analytic logistic regression algorithm (algorithm to be upgraded), which may include a table name, a stored partition, a number of rows, a number of columns, and the like.

After the input description is determined, an output description of the logistic regression algorithm, such as a table name of the output calculation result, a stored partition, and the like, may also be determined accordingly.

And then determining that the logistic regression algorithm obtains algorithm parameters required for realizing the logistic regression algorithm and the operation logic description of the algorithm to be upgraded.

S303: and initiating an algorithm upgrading request aiming at the algorithm to be upgraded according to the algorithm description file.

After the algorithm description file of the algorithm to be upgraded is determined, an algorithm upgrading request of the algorithm to be upgraded can be initiated in the big data processing platform according to the algorithm description file, so that the algorithm to be upgraded is updated and upgraded.

According to the connection relation between the algorithm client and the big data processing platform, the algorithm client can send the algorithm upgrading request to the equipment in the big data processing platform connected with the algorithm client so as to complete the upgrading of the algorithm to be upgraded in the big data processing platform. Optionally, when the algorithm client is connected to the control cluster in the big data processing platform, the algorithm update request may be sent to the control cluster of the big data processing platform.

After the upgrade algorithm is updated into the algorithm package, the corresponding upgrade algorithm client may be used to store the relevant information of the algorithm to be upgraded in the algorithm client, so that optionally, the algorithm client may also be upgraded and updated for the operation of storing the relevant information of the algorithm to be upgraded in the algorithm client.

It can be seen that, the algorithms in the big data processing platform are all described in a uniform algorithm description format, so that when the algorithm is needed to be upgraded, the new algorithm to be upgraded can be described according to the algorithm description format, the algorithm description file describing the algorithm to be upgraded can be identified by each cluster in the big data processing platform, so that when the algorithm is needed to be upgraded, the corresponding algorithm description file can be generated without correspondingly changing the core algorithm service of the big data processing platform in a traditional way, when the algorithm description file describing the algorithm to be upgraded is received, the algorithm package needed to be deployed on the computing equipment can be updated according to the algorithm description file, after the updated algorithm package is deployed on the computing equipment, the computing equipment can perform processing aiming at the algorithm to be upgraded through the updated algorithm package, and the situation that the big data processing platform cannot provide data processing service to the outside when the algorithm service is originally upgraded is avoided, the system efficiency is improved.

Based on the above specific implementation manner of the algorithm upgrading method provided in the embodiment of the present application, the embodiment of the present application further provides an algorithm upgrading device, and the embodiment may be regarded as describing the device provided in the embodiment of the present application from the perspective of a computing device, and specifically refer to the following embodiments.

Fig. 4 is a structural diagram of an algorithm upgrading apparatus provided in an embodiment of the present application, please refer to fig. 4, where the apparatus includes a first obtaining unit 401, an updating unit 402, and a deploying unit 403, where:

a first obtaining unit 401, configured to obtain an algorithm description file of an algorithm to be upgraded;

an updating unit 402, configured to determine an algorithm package of an algorithm to be updated according to the algorithm description file, and update the algorithm to be updated to the algorithm package corresponding to the algorithm to be updated;

a deployment unit 403, configured to deploy the updated algorithm package in the computing device, so that the computing device performs processing on a task of an algorithm to be upgraded through the updated algorithm package.

Optionally, the first obtaining unit 401 is specifically configured to:

Optionally, the apparatus further comprises:

the computing task obtaining unit is used for obtaining computing tasks, and the computing tasks comprise data to be computed and algorithm description files;

Therefore, the embodiment of the application provides an algorithm upgrading device, which comprises a first obtaining unit, an updating unit and a deploying unit, wherein the first obtaining unit obtains an algorithm description file of an algorithm to be upgraded, the updating unit determines an algorithm package corresponding to the algorithm to be upgraded according to the algorithm description file, updates the algorithm to be upgraded into the algorithm package corresponding to the algorithm to be upgraded, and the deploying unit deploys the updated algorithm package in computing equipment. Moreover, because algorithm upgrading and algorithm service upgrading are decoupled, the algorithm upgrading scheme of the application is implemented without knowing the mechanism of the algorithm service, so that the technical requirements on technical personnel are low, and the cost of the algorithm upgrading is reduced.

The embodiment of the present application further provides an algorithm upgrading apparatus, and the embodiment may be regarded as describing the apparatus provided in the embodiment of the present application from the perspective of an algorithm client, specifically referring to the following embodiments.

Fig. 5 is a structural diagram of an algorithm upgrading apparatus provided in an embodiment of the present application, please refer to fig. 5, where the apparatus includes a second obtaining unit 501, a second obtaining unit 502, and an initiating unit 503, where:

a second obtaining unit 501, configured to obtain an algorithm to be upgraded;

a third obtaining unit 502, configured to obtain an algorithm description file of an algorithm to be upgraded;

the initiating unit 503 is configured to initiate an algorithm upgrade request for an algorithm to be upgraded according to the algorithm description file.

Optionally, the third obtaining unit 502 is specifically configured to:

obtaining the output description of the algorithm to be upgraded according to the algorithm description format;

and obtaining algorithm parameters required by realizing the algorithm to be upgraded and the operation logic description of the algorithm to be upgraded according to the algorithm description format.

Optionally, the third obtaining unit 502 is further configured to:

Optionally, the initiating unit 503 is specifically configured to:

Therefore, the embodiment of the application provides an algorithm upgrading device, which comprises a first obtaining unit, a second obtaining unit and an initiating unit, wherein the first obtaining unit obtains an algorithm to be upgraded, the second obtaining unit obtains an algorithm description file of the algorithm to be upgraded, and the initiating unit initiates an algorithm upgrading request aiming at the algorithm to be upgraded according to the algorithm description file, so that the algorithm to be upgraded is updated and upgraded, the phenomenon that a large data processing platform cannot provide data processing service externally when the algorithm service is upgraded originally is avoided, and the system efficiency is improved.

The apparatus in the embodiment of the present application is described above from the perspective of a modular functional entity, and the embodiment of the present application further provides a computing device for algorithm upgrade, and the computing device in the embodiment of the present application is described below from the perspective of hardware processing.

Fig. 6 is a block diagram of a computing device for algorithm upgrade according to an embodiment of the present application, referring to fig. 6, the computing device includes a processor 601 and a memory 602, where:

a memory 602 for storing and transmitting program codes to the processor;

a processor 601 for executing the following method according to instructions in the program code:

acquiring an algorithm description file of an algorithm to be upgraded;

Optionally, the processor 601 is further configured to execute the following method according to the instructions in the program code:

Therefore, the computing equipment for algorithm upgrading can obtain the algorithm description file of the algorithm to be upgraded, determine the algorithm package corresponding to the algorithm to be upgraded according to the algorithm description file, update the algorithm to be upgraded into the algorithm package corresponding to the algorithm to be upgraded, and deploy the updated algorithm package in the computing equipment, so that the computing equipment performs processing on the task of the algorithm to be upgraded through the updated algorithm package, the phenomenon that a large data processing platform cannot provide data processing service externally when the algorithm service is upgraded originally is avoided, and the system efficiency is improved. Moreover, because algorithm upgrading and algorithm service upgrading are decoupled, the algorithm upgrading scheme of the application is implemented without knowing the mechanism of the algorithm service, so that the technical requirements on technical personnel are low, and the cost of the algorithm upgrading is reduced.

The embodiment of the present application further provides an algorithm client for algorithm upgrade, and the algorithm client in the embodiment of the present application is described below from the perspective of hardware processing.

Fig. 7 is a structural diagram of an algorithm client for algorithm upgrade according to an embodiment of the present application, please refer to fig. 7, where the algorithm client includes a processor 701 and a memory 702, where:

a memory 702 for storing and transferring program code to the processor;

a processor 701 for performing the following method according to instructions in the program code:

acquiring an algorithm to be upgraded;

acquiring an algorithm description file of an algorithm to be upgraded;

Optionally, the processor 701 is further configured to execute the following method according to an instruction in the program code:

and updating the algorithm client aiming at the algorithm to be upgraded.

Optionally, the processor 701 is configured to execute the following method according to an instruction in the program code:

Therefore, the algorithm client for algorithm upgrading is provided in the embodiments of the present application, and the algorithm client obtains the algorithm to be upgraded by obtaining the algorithm to be upgraded and obtaining the algorithm description file of the algorithm to be upgraded according to the algorithm description format, and initiates an algorithm upgrading request for the algorithm to be upgraded, so as to update and upgrade the algorithm to be upgraded, thereby avoiding the occurrence of the situation that a large data processing platform cannot provide data processing service to the outside when the algorithm service is upgraded originally, and improving the system efficiency.

Based on the computing device and the algorithm client provided by the embodiment, the embodiment of the application further provides an algorithm upgrading system.

Fig. 8 is a structural diagram of an algorithm upgrading system provided in an embodiment of the present application, please refer to fig. 8, where the system includes an algorithm client 801 and a computing device 802, where:

the algorithm client 801 is used for acquiring an algorithm to be upgraded; acquiring an algorithm description file of an algorithm to be upgraded; initiating an algorithm upgrading request aiming at the algorithm to be upgraded according to the algorithm description file;

the computing device 802 is configured to obtain an algorithm description file of an algorithm to be upgraded; determining an algorithm package corresponding to the algorithm to be upgraded according to the algorithm description file, and updating the algorithm to be upgraded into the algorithm package corresponding to the algorithm to be upgraded; and deploying the updated algorithm package in the computing equipment, so that the computing equipment performs processing aiming at the task of the algorithm to be upgraded through the updated algorithm package.

Therefore, the embodiment of the application provides an algorithm upgrading system, an algorithm client acquires an algorithm to be upgraded, acquires a description file of the algorithm to be upgraded according to an algorithm description format, issues an algorithm upgrading request for the algorithm to be upgraded to a big data processing platform, the algorithm description file describing the algorithm to be upgraded can be identified by each cluster in the big data processing platform, and can update an algorithm package required to be deployed on computing equipment according to the algorithm description file when the algorithm description file of the algorithm to be upgraded is received.

In the above specific implementation manner of algorithm upgrading provided by the embodiment of the present application, in a big data processing system, in addition to the need of maintaining the algorithm, a calculation task needs to be executed according to the algorithm. Based on the above, the embodiment of the application also provides a method for sending the calculation task, so as to realize calculation of big data.

Fig. 9 is a flowchart illustrating a method for sending a computing task according to an embodiment of the present application. The embodiment corresponding to fig. 9 may be regarded as a method for sending a computing task from the perspective of an algorithm client.

Referring to fig. 9, the method includes:

s901: and acquiring a computing requirement.

Computational requirements may be understood as a requirement for computational processing of data. The embodiment of the present application is not limited to how the computing requirement is obtained, for example, the computing requirement may be provided by a user having the computing requirement, or may be provided by a technician according to a possible computing requirement. As a possible implementation manner, the algorithm client may provide a calculation requirement input interface through which a user may input a calculation requirement, so that the algorithm client may obtain the calculation requirement through the input interface. As a possible implementation manner, the calculation requirement may specifically include information such as data to be calculated.

S902: and generating a computing task according to the computing requirement, wherein the computing task comprises a task algorithm description file, and the task algorithm description file is used for describing an algorithm required for processing the computing task.

A computational task may be understood as a computational task assigned by an algorithm client according to computational requirements. The computing task may direct the computing device to execute relevant algorithms to process the data to meet the user's needs. Algorithm information that can be recognized by the computing device can be included in the computing task in order to direct the computing device to perform the associated algorithm to process the data. As a possible implementation manner, the computing task may include a task algorithm description file, and the task algorithm description file is mainly used for describing an algorithm required for processing the computing task.

For ease of understanding, the following description is made with reference to specific examples.

The user calculation requirement is to process a certain data, the processing mode includes multiple complex calculations, including at least matrix calculation and logistic regression calculation, in order to improve the calculation efficiency, multiple calculation tasks may be generated according to the calculation requirement, for example, the matrix calculation task and the logistic regression calculation task may be included, it may be understood that the matrix calculation task may include an algorithm description file of a matrix algorithm, the logistic regression calculation task may include an algorithm description file of a logistic regression algorithm, and the algorithm description file may be identified by each computing device in the calculation cluster as a task algorithm description file of each calculation task, so that the computing device may respectively execute corresponding calculation tasks according to the algorithm description file. In other implementation manners of the embodiments of the present application, the computing task may also correspond to a plurality of task algorithm description files, or the task algorithm description files may also correspond to a plurality of algorithms, that is, when the computing device executes the computing task, the computing device may also execute the computing task including the plurality of algorithms.

S903: and sending the computing task to the control node so that the control node distributes the computing task to the computing device for processing.

The algorithm client is typically connected to the control cluster. It will be appreciated that a plurality of control nodes may be included in a control cluster. The algorithm client can send the generated calculation tasks to the control node, and the control node can distribute the calculation tasks to the calculation equipment for processing.

Because the computing task comprises the task algorithm description file, the algorithm description file can be identified by each computing device in the computing cluster, and the computing device can execute the computing task by adopting the corresponding algorithm package according to the algorithm description file. It can be understood that, in the embodiment of the present application, when there is a computing requirement, the algorithm client may determine a task algorithm description file in the process of generating a computing task, where the task algorithm description file is used to describe an algorithm that needs to be adopted by the computing task, and does not need to determine by an algorithm service in the control cluster, and the control cluster is mainly used to segment the task and determine to which computing devices to distribute the task, which is equivalent to weakening the effect of the algorithm service in the control cluster on the algorithm, reducing the limitation of the algorithm service on the computing process, and improving the processing efficiency of the big data processing system.

As can be seen from the above, the calculation task sending method provided in the embodiment of the present application generates a calculation task according to a calculation requirement by obtaining the calculation requirement, and then sends the calculation task to the control node, so that the control node sends the calculation task to the computing device for processing.

Based on the method for sending the computation task provided by the embodiment, the embodiment of the application further provides a device for sending the computation task. Fig. 10 is a structural diagram of a computing task sending device according to an embodiment of the present application, and please refer to fig. 10, the device includes a computing requirement obtaining unit 1001, a computing task generating unit 1002, and a computing task sending unit 1003, where:

a calculation demand acquisition unit 1001 for acquiring a calculation demand;

the calculation task generation unit 1002 is configured to generate a calculation task according to a calculation requirement, where the calculation task includes a task algorithm description file, and the task algorithm description file is used to describe an algorithm required for processing the calculation task;

and a calculation task sending unit 1003 that sends the calculation task to the control node so that the control node distributes the calculation task to the calculation device for processing.

The above describes the computing task sending device in the embodiment of the present application from the perspective of a modular functional entity, and the embodiment of the present application also provides an algorithm client for sending a computing task, and the algorithm client in the embodiment of the present application is described below from the perspective of hardware processing.

Fig. 11 is a structural diagram of an algorithm client according to an embodiment of the present application, where the algorithm client 1100 is configured to send a computation task, and includes a processor 1102 and a memory 1101, where:

a memory 1101 for storing the program code and transmitting the program code to the processor;

a processor 1102 for performing the following method according to instructions in program code:

acquiring a calculation demand;

In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product.

The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can store or a data storage device, such as a server, a data center, etc., that is integrated with one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.

Those of ordinary skill in the art will understand that: all or part of the steps for realizing the method embodiments can be completed by hardware related to program instructions, the program can be stored in a computer readable storage medium, and the program executes the steps comprising the method embodiments when executed; and the aforementioned storage medium may be at least one of the following media: various media that can store program codes, such as read-only memory (ROM), RAM, magnetic disk, or optical disk.

It should be noted that, in the present specification, all the embodiments are described in a progressive manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the apparatus and system embodiments, since they are substantially similar to the method embodiments, they are described in a relatively simple manner, and reference may be made to some of the descriptions of the method embodiments for related points. The above-described embodiments of the apparatus and system are merely illustrative, and units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one position, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.

The above description is only one specific embodiment of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present application should be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An algorithm upgrade method for upgrading an algorithm package deployed on computing devices in a computing cluster, the method comprising:

acquiring an algorithm description file of an algorithm to be upgraded; the algorithm description file comprises the output description of the algorithm to be upgraded, the algorithm parameters required for realizing the algorithm to be upgraded and the operation logic description of the algorithm to be upgraded;

deploying the updated algorithm package in the computing equipment, so that the computing equipment performs processing aiming at the task of the algorithm to be upgraded through the updated algorithm package.

2. The method of claim 1, wherein the algorithm description file further comprises an input description of the algorithm to be upgraded.

3. The method according to claim 1, wherein the obtaining of the algorithm description file of the algorithm to be upgraded comprises:

4. The method of any of claims 1 to 3, further comprising, after said deploying the updated algorithm package in the computing device:

acquiring a computing task, wherein the computing task comprises data to be computed and the algorithm description file;

5. An algorithm upgrading device, which is applied to upgrading of algorithm packages deployed on computing devices in a computing cluster, the device comprising a first obtaining unit, an updating unit and a deploying unit, wherein:

the first obtaining unit is used for obtaining an algorithm description file of the algorithm to be upgraded; the algorithm description file comprises the output description of the algorithm to be upgraded, the algorithm parameters required for realizing the algorithm to be upgraded and the operation logic description of the algorithm to be upgraded;

the deployment unit is configured to deploy the updated algorithm package in the computing device, so that the computing device performs processing on the task of the algorithm to be upgraded through the updated algorithm package.

6. A computing device for algorithm upgrade, for use in upgrading of algorithm packages deployed on computing devices in a computing cluster, the computing device comprising a processor and a memory, wherein:

the memory is used for storing program codes and transmitting the program codes to the processor;

the processor is used for executing the following method according to the instructions in the program code:

acquiring an algorithm description file of an algorithm to be upgraded; the algorithm description file comprises the output description of the algorithm to be upgraded, the algorithm parameters required by the algorithm to be upgraded and the operation logic description of the algorithm to be upgraded;

7. The computing device of claim 6, wherein the algorithm description file further comprises an input description of the algorithm to be upgraded.

8. The computing device of claim 6, wherein the processor is further configured to perform the following method according to instructions in the program code:

9. The computing device of any of claims 6 to 8, wherein the processor is further configured to perform the following method according to instructions in the program code:

10. An algorithm upgrade method for upgrading algorithm packages deployed on computing devices in a computing cluster, the method comprising:

acquiring an algorithm to be upgraded;

acquiring an algorithm description file of the algorithm to be upgraded; the algorithm description file comprises the output description of the algorithm to be upgraded, the algorithm parameters required for realizing the algorithm to be upgraded and the operation logic description of the algorithm to be upgraded;

11. The method of claim 10, further comprising:

and updating the algorithm client aiming at the algorithm to be upgraded.

12. The method according to claim 10, wherein the obtaining of the algorithm description file of the algorithm to be upgraded comprises:

acquiring the output description of the algorithm to be upgraded;

acquiring algorithm parameters required by the algorithm to be upgraded and the operation logic description of the algorithm to be upgraded;

and obtaining the algorithm description file according to the output description, the algorithm parameters required for realizing the algorithm to be upgraded and the operation logic description of the algorithm to be upgraded.

13. The method according to claim 12, wherein the algorithm description file further includes an input description of the algorithm to be upgraded, and before obtaining an output description of the algorithm to be upgraded, the method further includes:

14. The method according to any one of claims 10 to 13, wherein the initiating an algorithm upgrade request for the algorithm to be upgraded according to the algorithm description file comprises:

and sending the algorithm upgrading request to a control cluster of the big data processing platform.

15. An algorithm upgrading apparatus, applied to upgrading of an algorithm package deployed on a computing device in a computing cluster, the apparatus comprising a second obtaining unit, a third obtaining unit and an initiating unit, wherein:

the second obtaining unit is used for obtaining the algorithm to be upgraded;

the third obtaining unit is used for obtaining the algorithm description file of the algorithm to be upgraded; the algorithm description file comprises the output description of the algorithm to be upgraded, the algorithm parameters required for realizing the algorithm to be upgraded and the operation logic description of the algorithm to be upgraded;

the initiating unit is used for initiating an algorithm upgrading request aiming at the algorithm to be upgraded according to the algorithm description file.

16. An algorithm client for algorithm upgrade, for application to upgrade of algorithm packages deployed on computing devices in a computing cluster, the algorithm client comprising a memory and a processor, wherein:

acquiring an algorithm to be upgraded;

17. The algorithm client of claim 16, wherein the processor is further configured to perform the following method according to instructions in the program code:

and updating the algorithm client aiming at the algorithm to be upgraded.

18. The algorithm client of claim 16, wherein the processor is configured to perform the following method according to instructions in the program code:

acquiring the output description of the algorithm to be upgraded;

19. The algorithm client of claim 18, wherein the processor is further configured to perform the following method according to instructions in the program code:

20. The algorithm client according to any one of claims 16 to 19, wherein the processor is configured to perform the following method according to instructions in the program code:

21. An algorithm upgrading system, characterized in that a uniform algorithm description format is set for algorithms applied to a big data processing platform, the system comprises a computing device and an algorithm client, wherein:

the algorithm client is used for acquiring the algorithm to be upgraded; acquiring an algorithm description file of the algorithm to be upgraded, wherein the algorithm description file comprises an output description of the algorithm to be upgraded, algorithm parameters required by the algorithm to be upgraded and an operation logic description of the algorithm to be upgraded; initiating an algorithm upgrading request aiming at the algorithm to be upgraded according to the algorithm description file; the computing equipment is used for acquiring an algorithm description file of the algorithm to be upgraded; determining an algorithm package corresponding to the algorithm to be upgraded according to the algorithm description file, and updating the algorithm to be upgraded into the algorithm package corresponding to the algorithm to be upgraded; deploying the updated algorithm package in the computing equipment, so that the computing equipment performs processing aiming at the task of the algorithm to be upgraded through the updated algorithm package.

22. A method for sending a computing task, the method comprising:

acquiring a calculation demand;

generating a computing task according to the computing requirement, wherein the computing task comprises a task algorithm description file which is used for describing an algorithm required for processing the computing task; the task algorithm description file comprises an output description of an algorithm required for processing the computing task, algorithm parameters required for realizing the algorithm required for processing the computing task and an operation logic description of the algorithm required for processing the computing task;

and sending the computing task to a control node so that the control node distributes the computing task to a computing device for processing.

23. A calculation task transmission device is characterized by comprising a calculation requirement acquisition unit, a calculation task generation unit and a calculation task transmission unit, wherein:

the calculation requirement acquisition unit is used for acquiring calculation requirements;

the computing task generating unit is used for generating a computing task according to the computing requirement, wherein the computing task comprises a task algorithm description file, and the task algorithm description file is used for describing an algorithm required for processing the computing task; the task algorithm description file comprises an output description of an algorithm required for processing the computing task, algorithm parameters required for realizing the algorithm required for processing the computing task and an operation logic description of the algorithm required for processing the computing task;

the computing task sending unit is used for sending the computing task to a control node so that the control node can distribute the computing task to a computing device for processing.

24. An algorithmic client for sending computational tasks, the algorithmic client comprising a memory and a processor, wherein:

acquiring a calculation demand;

25. A storage medium for storing a program code for executing the algorithm upgrading method of any one of claims 1 to 4 or claims 10 to 14 or the calculation task sending method of claim 22.

26. A computer program product comprising instructions which, when run on a computer, cause the computer to perform the algorithm upgrading method of any of claims 1 to 4 or claims 10 to 14 or the computing task sending method of claim 22.