CN116431343A

CN116431343A - Federal computing method, apparatus, device, and storage medium

Info

Publication number: CN116431343A
Application number: CN202310369577.9A
Authority: CN
Inventors: 叶玮彬; 鲁嘉俊; 刘涛
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2023-04-07
Filing date: 2023-04-07
Publication date: 2023-07-14

Abstract

The disclosure provides a federal computing method, a federal computing device, federal computing equipment and a federal computing storage medium, relates to the technical field of artificial intelligence, and particularly relates to the technical field of data processing. The specific implementation scheme is as follows: responding to a processing request of the federal computing task, and sending an information acquisition request to a control server; receiving configuration information returned by a control server; according to the configuration information, loading data to be processed of the federal computing task from a storage server to a computing server; and in the calculation server, performing federation calculation on the data to be processed of the federation calculation task to obtain a federation calculation result. Thus, the efficiency of acquiring the configuration information is improved, and the efficiency of federal calculation is also improved.

Description

Federal computing method, apparatus, device, and storage medium

Technical Field

The present disclosure relates to the field of artificial intelligence, and in particular, to the field of data processing technologies, and in particular, to a federal computing method, apparatus, device, and storage medium.

Background

With the advent of the big data age, the data volume maintained by many business systems is also increasing, and because each business system is independent, the data cannot be shared and fused, and data islands are gradually formed in each business system. In order to realize data sharing and ensure the privacy security of data, data fusion is generally performed by adopting a federal computing mode.

Disclosure of Invention

The present disclosure provides a federal computing method, apparatus, device, and storage medium.

According to an aspect of the present disclosure, there is provided a federal computing method applied to a computing server, the method including:

responding to a processing request of the federal computing task, and sending an information acquisition request to a control server; the federation computing task is used for fusing data from different service systems; the control server provides a control function for configuration information of different service systems; the information acquisition request is used for requesting to acquire configuration information of a business system associated with the federal computing task;

receiving the configuration information returned by the control server; according to the configuration information, loading the data to be processed of the federal computing task from a storage server to the computing server;

and in the calculation server, performing federation calculation on the data to be processed of the federation calculation task to obtain a federation calculation result.

According to another aspect of the present disclosure, there is provided a federal computing method applied to a control server that provides a management function of configuration information of different service systems; the method comprises the following steps:

Receiving an information acquisition request from a computing server; triggering the information acquisition request based on a processing request of the federal computing task; the federation computing task is used for fusing data from different service systems; the information acquisition request is used for requesting to acquire configuration information of a business system associated with the federal computing task;

acquiring configuration information of a service system associated with the federal computing task;

transmitting the configuration information to the computing server; the configuration information is used for loading the data to be processed of the federal computing task from a storage server to the computing server.

According to another aspect of the present disclosure, there is provided a federal computing device for use in a computing server, the device comprising:

the sending module is used for responding to the processing request of the federal computing task and sending an information acquisition request to the control server; the federation computing task is used for fusing data from different service systems; the control server provides a control function for configuration information of different service systems; the information acquisition request is used for requesting to acquire configuration information of a business system associated with the federal computing task;

The loading module is used for receiving the configuration information returned by the control server; according to the configuration information, loading the data to be processed of the federal computing task from a storage server to the computing server;

and the computing module is used for performing federation computation on the data to be processed of the federation computing task in the computing server to obtain a federation computing result.

According to another aspect of the present disclosure, there is provided a federal computing device for use in a control server that provides control functionality for configuration information for different business systems; the device comprises:

the receiving module is used for receiving an information acquisition request from the computing server; triggering the information acquisition request based on a processing request of the federal computing task; the federation computing task is used for fusing data from different service systems; the information acquisition request is used for requesting to acquire configuration information of a business system associated with the federal computing task;

the acquisition module is used for acquiring configuration information of a service system associated with the federal computing task;

the sending module is used for sending the configuration information to the computing server; the configuration information is used for loading the data to be processed of the federal computing task from a storage server to the computing server.

According to another aspect of the present disclosure, there is provided an electronic device including:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the federal computing method provided by the present disclosure.

According to another aspect of the present disclosure, there is provided a non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the federal computing method provided by the present disclosure.

According to another aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the federal calculation method provided by the present disclosure.

The technical scheme provided by the disclosure improves the efficiency of acquiring the configuration information, and also improves the efficiency of federal calculation.

It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.

Drawings

The drawings are for a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

FIG. 1 is a schematic illustration of an environment in which a federal computing method is implemented, as shown in an embodiment of the present disclosure;

FIG. 2 is a flow diagram of a federal computing method according to an embodiment of the present disclosure;

FIG. 3 is a flow chart of a federal computing method according to an embodiment of the present disclosure;

FIG. 4 is a flow chart diagram of a federal computing method according to an embodiment of the present disclosure;

FIG. 5 is a schematic diagram of an operator orchestration result shown in an embodiment of the present disclosure;

FIG. 6 is a schematic diagram of the architecture of one federal computing shown in an embodiment of the present disclosure;

FIG. 7 is a schematic diagram of the internal architecture of a computing server shown in an embodiment of the disclosure;

FIG. 8 is a schematic diagram of an interaction flow of federal computing according to an embodiment of the present disclosure;

FIG. 9 is a block diagram of the architecture of a federated computing device as illustrated in an embodiment of the present disclosure;

FIG. 10 is a block diagram of the architecture of a federated computing device as illustrated in an embodiment of the present disclosure;

FIG. 11 is a block diagram of an electronic device for implementing the federal computing method according to an embodiment of the present disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

In the technical scheme of the disclosure, the related processes of collecting, storing, using, processing, transmitting, providing, disclosing and the like of the personal information of the user accord with the regulations of related laws and regulations, and the public order colloquial is not violated.

Firstly, describing an application scenario related to an embodiment of the present disclosure, the federal computing method provided by the embodiment of the present disclosure may be applied to a scenario of big data processing, and in particular, may be applied to a data fusion scenario of service data in different service systems. In some embodiments, the federal computing method provided by the embodiments of the present disclosure can be applied to data analysis scenarios, machine learning scenarios, model training scenarios, or the like of different business systems.

At present, with the advent of the big data age, the data volume maintained by many business systems is also increasing, and because each business system is independent, the data can not be shared and fused, and a data island is gradually formed in each business system. In order to realize data sharing and ensure the privacy security of data, data fusion is generally performed by adopting a federal computing mode. The federation calculation is a data calculation mode for performing multi-party joint calculation under the condition that original data does not exist locally. The method adopts a new mode of data immobility calculation, and realizes the final calculation requirement through the interaction of the intermediate calculation results of multiple parties under the condition that the original data does not show local constraint conditions.

In related art, in some embodiments, federal computation of data may be implemented using structured query language (Structured Query Language, SQL) based execution plans, and the corresponding process may be: and acquiring service data required by the federation calculation from each service system, and further, connecting different service data in series by rewriting the SQL execution plan so as to realize fusion of the different service data. The execution plan of SQL refers to the process of completing SQL query by organizing different physical operators in an execution tree according to a certain sequencing order.

In other embodiments, a distributed file system protocol (Hadoop Distributed File System, HDFS) based implementation of federal computing of data may be employed, and the corresponding process may be: and acquiring service data required by the federation calculation from each service system, and further, adopting a file management system compatible with the HDFS to carry out data arrangement on different service data so as to realize fusion of the different service data. The file management system compatible with the HDFS may be an Alluxio management system or a JindoFS management system, etc.

However, both the federation calculation method based on the SQL execution plan and the federation calculation method based on the HDFS need to interact with each service system to obtain service data required by federation calculation, for example, taking the case of performing federation calculation on service data of three service systems, multiple data reading operations need to be performed with underlying programs of each service system respectively, so that service data of the three service systems can be obtained. Thus, the efficiency of acquiring the service data required by the federal calculation is low, and the efficiency of the federal calculation is correspondingly reduced.

Based on this, the embodiment of the disclosure provides a federal computing method, by controlling interaction between a server and a computing server, configuration information of a service system associated with a federal computing task is obtained, cloud management and control of the configuration information of the service system are achieved, and the computing server does not need to rely on a bottom program of each service system involved in the federal computing task, that is, the computing server does not need to obtain the configuration information of the service system associated with the federal computing task through multiple interactions with the bottom program of each service system, so that efficiency of obtaining the configuration information of the service system is improved, and influence on the service system caused by multiple data reading operations is avoided. Furthermore, the configuration information issued by the control server is utilized to load the data to be processed from the storage server to the calculation server so as to execute the subsequent federation calculation process, and the federation calculation efficiency can be further ensured.

FIG. 1 is a schematic diagram of an environment in which a federal computing method is implemented, as shown in an embodiment of the present disclosure. Referring to fig. 1, the implementation environment includes a control server 101, a computing server 102, and a storage server 103.

Wherein the control server 101 is provided with a management function for configuration information of different service systems. In the embodiment of the present disclosure, the control server 101 is configured to receive an information acquisition request from the computing server 102, acquire, according to the information acquisition request, configuration information of a service system associated with the federal computing task, and send the configuration information to the computing server 102 to trigger the computing server 102 to execute a subsequent federal computing process.

The computing server 102 is provided with federal computing functionality. In this embodiment of the disclosure, the computing server 102 is configured to respond to a processing request of a federation computing task, send an information acquisition request to the control server 101 to request to acquire configuration information of a service system associated with the federation computing task, further receive the configuration information returned by the control server 101, and load data to be processed of the federation computing task from the storage server 103 to the computing server 102 according to the configuration information, so as to perform federation computing on the data to be processed of the federation computing task in the computing server 102, thereby obtaining a federation computing result.

The storage server 103 is used for storing data. In the embodiment of the present disclosure, the storage server 103 is used to store service data of a plurality of service systems.

In some embodiments, the server is an independent physical server, or a server cluster or a distributed file system formed by a plurality of physical servers, or at least one of cloud servers that provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content distribution networks, and basic cloud computing services such as big data or artificial intelligence platforms, which are not limited in the embodiments of the present disclosure. In some embodiments, the number of servers described above can be greater or fewer, and embodiments of the present disclosure are not limited in this regard.

In some embodiments, the computing server 102 may be communicatively coupled to the user terminal 104. The user terminal 104 is at least one of a smart phone, a smart watch, a desktop computer, a laptop computer, a virtual reality terminal, an augmented reality terminal, a wireless terminal, a laptop portable computer, and the like. The user terminal 104 may refer broadly to one of a plurality of terminals, with the disclosed embodiments being illustrated only by the user terminal 104. Those skilled in the art will recognize that the number of terminals may be greater or lesser. In the embodiment of the present disclosure, the user terminal 104 is configured to send, to the computing server 102, a processing request of the federation computing task in response to a triggering operation of the user on the federation computing task, so as to trigger the computing server 102 to execute a subsequent federation computing process.

The method provided by the embodiment of the present disclosure is described below based on the implementation environment shown in fig. 1.

FIG. 2 is a flow chart of a federal computing method according to an embodiment of the present disclosure. In some embodiments, the federal computing method is performed by an electronic device. The electronic device may be, for example, a computing server as shown in fig. 1 described above. As shown in fig. 2, the method includes the following steps.

S201, responding to a processing request of a federal computing task, and sending an information acquisition request to a control server; the federation computing task is used for fusing data from different service systems; the control server provides a control function for configuration information of different service systems; the information acquisition request is for requesting acquisition of configuration information of a business system associated with the federated computing task.

In the embodiment of the disclosure, the federal computing task refers to a task of performing data fusion on different service data. Illustratively, taking a financial business as an example, the federal computing task may be data fusion between banking, securities, and insurance businesses. Illustratively, taking advertising business as an example, the federal computing task may be data fusion between advertising material, advertising logs, and user logs.

The processing request is used for requesting federation calculation of the federation calculation task so as to realize data fusion of different business data. In some embodiments, the configuration information of the business system associated with the federated computing task may include task configuration information and storage configuration information. The task configuration information is used for indicating task content of the federation computing task. The storage configuration information is used to indicate metadata information of the storage server.

S202, receiving the configuration information returned by the control server; and loading the data to be processed of the federal computing task from a storage server to the computing server according to the configuration information.

S203, in the calculation server, performing federation calculation on the data to be processed of the federation calculation task to obtain a federation calculation result.

According to the technical scheme provided by the embodiment of the disclosure, the configuration information of the service system associated with the federal computing task is acquired through the interaction between the control server and the computing server, so that the cloud management and control of the configuration information of the service system are realized, and the bottom programs of the service systems related to the federal computing task are not needed to be relied on, namely, the computing server does not need to acquire the configuration information of the service system associated with the federal computing task through multiple interactions with the bottom programs of the service systems, and therefore, the efficiency of acquiring the configuration information of the service system is improved, and the influence on the service system caused by multiple data reading operations is avoided. Furthermore, the configuration information issued by the control server is utilized to load the data to be processed from the storage server to the calculation server so as to execute the subsequent federation calculation process, and the federation calculation efficiency can be further ensured.

FIG. 3 is a flow chart of a federal computing method according to an embodiment of the present disclosure. In some embodiments, the federal computing method is performed by an electronic device. The electronic device may be, for example, the control server shown in fig. 1 described above. As shown in fig. 3, the method includes the following steps.

S301, receiving an information acquisition request from a computing server; triggering the information acquisition request based on a processing request of the federal computing task; the federation computing task is used for fusing data from different service systems; the information acquisition request is for requesting acquisition of configuration information of a business system associated with the federated computing task.

S302, acquiring configuration information of a business system associated with the federal computing task.

S303, sending the configuration information to the computing server; the configuration information is used for loading the data to be processed of the federal computing task from a storage server to the computing server.

FIG. 4 is a flow chart of a federal computing method according to an embodiment of the present disclosure. In some embodiments, the federal computing method is performed by an electronic device. As shown in fig. 4, the description will be given by taking an interaction procedure among the control server, the calculation server, and the storage server as an example, and the method includes the following steps.

S401, the computing server responds to a processing request of the federal computing task and sends an information acquisition request to the control server, wherein the information acquisition request is used for requesting to acquire configuration information of a service system associated with the federal computing task.

In the embodiment of the disclosure, the federal computing task is used for fusing data from different business systems. Illustratively, taking a financial business as an example, the federal computing task may be data fusion between banking, securities, and insurance businesses. Illustratively, taking advertising business as an example, the federal computing task may be data fusion between advertising material, advertising logs, and user logs.

The processing request is used for requesting federation calculation of the federation calculation task so as to realize data fusion of different business data. In some embodiments, the processing request may be triggered based on a processing operation performed by a user at the user terminal, such as a click operation or a selection operation by the user for the federated computing task; alternatively, the processing request may be self-triggering by the computing server, such as the computing server periodically triggering a processing request for a federated computing task. Of course, in other embodiments, other triggering manners may be used for the processing request, and the triggering manner of the processing request is not limited in the embodiments of the present disclosure.

In the embodiment of the disclosure, the service systems associated with the federation calculation task are at least two service systems to be subjected to federation calculation. Accordingly, the configuration information of the service systems associated with the federation calculation task is the configuration information of at least two service systems to be federally calculated.

In some embodiments, the configuration information may include task configuration information and storage configuration information. The task configuration information is used for indicating task content of the federation computing task. In particular, the task content is used to indicate information about at least two business systems associated with the federated computing task. In some embodiments, the task configuration information may include an environment variable (env) parameter, an alias (alias) parameter, a category (kine) parameter, a user authentication (user) parameter, a format (format) parameter, or a partition (partition) parameter, or the like. The storage configuration information is used to indicate metadata information of the storage server. In particular, the metadata information is used to describe the data structure of the storage server. In some embodiments, the storage configuration information may include a namespace (namespace) parameter, a database (database) parameter, a tag (table) parameter, a field (fields) parameter, a data Format (data Format) parameter, a compression Type (compression Type) parameter, or a separator (separator) parameter, and so forth.

In the above embodiment, by sending the information acquisition request to the control server to request to acquire the configuration information associated with the federal computing task, the task configuration information and the stored configuration information associated with the federal computing task can be quickly and efficiently acquired, and the efficiency of acquiring the configuration information is improved, so that the efficiency of federal computing is ensured.

S402, the control server receives an information acquisition request from the calculation server.

In the embodiment of the disclosure, the control server provides a control function for configuration information of different service systems. Therefore, the control server can be used for rapidly issuing the configuration information of the service system, so that the efficiency of acquiring the configuration information by the computing server is improved.

S403, the control server acquires configuration information of a service system associated with the federal computing task.

In some embodiments, the control server obtains configuration information of at least two business systems associated with the federal computing task from a set of configuration information according to business types of the at least two business systems.

The configuration information set comprises a plurality of service types and configuration information corresponding to the plurality of service types. The traffic type is used to indicate the type of traffic system, such as financial type, advertisement type, or other types. In some embodiments, the service type may be a service code, a service name or service identification number (Identity Document, ID), or the like. The embodiment of the disclosure does not limit the setting of the service type.

In some embodiments, the information obtaining request may carry service types of the at least two service systems, and further, after the control server receives the information obtaining request, the configuration information associated with the federal computing task is obtained according to the service types of the at least two service systems carried by the information obtaining request. Or in other embodiments, after the control server receives the information acquisition request, the control server determines the service types of the at least two service systems according to a preset field in the information acquisition request, and further obtains the configuration information associated with the federal computing task according to the determined service types of the at least two service systems. Of course, in other embodiments, the control server may also determine the service types of at least two service systems indicated by the federal computing task in other manners, and the process of determining the service types is not limited by the disclosed embodiments.

In the embodiment, the control server is used for maintaining the configuration information set, so that the configuration information can be rapidly issued when the computing server requests the configuration information of the federal computing task, and the efficiency of the computing server for acquiring the configuration information is improved.

Further, in some embodiments, the control server is also capable of updating the set of configuration information in real time. The corresponding procedure may be: the control server responds to the configuration information of any service system to change, and updates the configuration information of any service system in the configuration information set.

Thus, the control server updates the configuration information set in real time, so that flexible maintenance of the configuration information can be realized, and the data reliability of the configuration information set can be ensured.

S404, the control server sends the configuration information to the calculation server.

In some embodiments, the control Server is provided with a configuration service (Config Server) module and a metadata service (Meta Server) module. The configuration service module provides a function of managing and controlling task configuration information. The metadata service module is provided with a management and control function for storing configuration information.

Accordingly, S402 to S404 described above may be replaced with: the control server receives the information acquisition request, acquires task configuration information associated with the federal computing task through the configuration service module, transmits the task configuration information associated with the federal computing task to the computing server, acquires storage configuration information associated with the federal computing task through the metadata service module, and transmits the storage configuration information associated with the federal computing task to the computing server.

In the above embodiment, the control server issues the configuration information of the service system associated with the federation computing task to the computing server, so that cloud management and control of the configuration information of the service system are realized, dynamic management and control of the federation computing grid is realized, the whole data reading process is not perceived on the service program, that is, the computing server does not need to rely on the bottom program of each service system to acquire the configuration information associated with the federation computing task, so that the efficiency of acquiring the configuration information is improved, and the influence of data reading on the service system is avoided.

S405, the computing server receives the configuration information returned by the control server.

S406, the computing server analyzes the data format of the configuration information to convert the data format of the configuration information into a target data format, wherein the target data format is a data format supported by the computing server.

The data format of the configuration information may be a YAML data format or a part data format or other types of data formats. In the embodiment of the present disclosure, the parsing process of the data format refers to a process of converting the data format of the configuration information into the target data format.

In some embodiments, the process of data parsing described above may be: and the computing server calls a class function corresponding to the data format according to the data format of the configuration information to analyze the data format of the configuration information so as to convert the data format of the configuration information into a target data format. Wherein the class function is provided with a function of converting a data format.

Illustratively, taking the data format of the configuration information as part as an example, the computing server may call an HDFS File System class function and a part Reader class function to convert the configuration information into the data format of Avro.

In some embodiments, the computing server is provided with a data configuration module that provides functionality for data parsing. Accordingly, S405 to S406 described above may be replaced with: the computing server receives the configuration information returned by the control server through the data configuration module, and analyzes the data format of the configuration information so as to convert the data format of the configuration information into a target data format.

In this way, the data format of the configuration information is converted into the data format supported by the computing server by analyzing the data format of the configuration information, so that the federal computing process is performed by the computing server based on the configuration information.

S407, the computing server loads the data to be processed of the federal computing task from the storage server to the computing server according to the configuration information after the data format conversion.

In some embodiments, the computing server determines a data storage location of the storage server according to the storage configuration information, and loads the data to be processed corresponding to the task configuration information from the storage server to the computing server from the data storage location of the storage server.

In this way, since the task configuration information includes task content of the federation computing task and the storage configuration information includes metadata information of the storage server, the computing server can perceive data to be processed in the storage server according to the task configuration information and the storage configuration information, and further, through a data loading process, a subsequent federation computing process is performed.

In some embodiments, the computing server can determine, according to metadata information of the storage server indicated by the storage configuration information, a data storage location in the storage server that matches the metadata information, further determine, in the data storage location, data to be processed of at least two service systems according to related information of the at least two service systems indicated by the task configuration information, and load the data to be processed of the at least two service systems from the storage server to the computing server, so as to perform a process of subsequent federal computation.

In some embodiments, the computing server may perform the above-described data loading process in a Lazy Load (or Lazy Load) manner. Of course, in other embodiments, the computing server may also use other types of data loading methods to perform the above-described data loading process. The specific implementation of the data loading by the embodiments of the present disclosure is not limited.

In some embodiments, the computing server may load the data to be processed of the federated computing task from the storage server into the memory of the computing server, and then subsequently execute S408 in the memory of the computing server. Therefore, the efficiency of subsequent federation calculation can be further improved due to the high-efficiency reading speed of the memory.

In some embodiments, the data configuration module mentioned in S406 above is further provided with a data loading function. Accordingly, S407 described above may be replaced with: and the computing server loads the data to be processed of the federal computing task from the storage server to the computing server through the data configuration module according to the configuration information after the data format conversion.

In S406 to S407, the description is given of the procedure of the computing server performing the data analysis and then performing the data loading. In other embodiments, the computing server does not need to execute S406 after receiving the configuration information based on S405, but executes the process of loading data in S407 based on the received configuration information. Illustratively, if the data format of the configuration information is the data format supported by the computing server, S406 need not be performed.

S408, the computing server performs federation computation on the data to be processed of the federation computing task in the computing server to obtain a federation computing result.

In some embodiments, after loading the data to be processed of the federal computing task from the storage server into the memory of the computing server based on S407, the computing server performs federal computation on the data to be processed of the federal computing task in the memory of the computing server, to obtain a federal computing result.

In some embodiments, the computing server is provided with a computing execution module that provides the functionality of federal computing. Accordingly, S408 may be replaced with: and the computing server performs federation computation on the data to be processed of the federation computing task through a computing operation module in the computing server to obtain a federation computing result.

In some embodiments, the above-described federal computation process may be implemented based on operator orchestration results, and the corresponding process may be: and the computing server performs operator programming according to the task flow of the federation computing task to obtain an operator programming result of the federation computing task, and performs federation computing on data to be processed of the federation computing task according to the operator programming result to obtain the federation computing result.

The operator arrangement result is used for indicating the calculation flow of the federation calculation. Operator orchestration, i.e. ordering or combining different operators, forms an execution flow. In some embodiments, the operator orchestration result may be in the form of an operator tree, such as Transform Hierarchy operator tree. Illustratively, FIG. 5 is a schematic diagram of an operator orchestration result shown in an embodiment of the present disclosure. Referring to fig. 5, fig. 5 illustrates a schematic diagram of an operator tree with four-level nodes as an example, it may be found that, on the basis of a Root node (Root), a plurality of first-level operator nodes, such as a Text io.reader operator, a Select operator, a Group operator, a par do operator, and a part io.writer operator, may be connected. The Text IO.reader operator and the ParDo operator are original operators, and the Select operator, the Group operator and the Parque IO.writer operator are composite operators. Further, one or more secondary operator nodes can be connected on the basis of the primary operator nodes, as shown in fig. 5, a ParDo (DoFn) operator is also connected on the basis of the Select operator, and the ParDo (DoFn) operator is an original operator; on the basis of the Group operator, a Combine (DoFn) operator and a GroupBykey operator are also connected, wherein the Combine (DoFn) operator is a composite operator, and the GroupBykey operator is an original operator; and a ParDo (DoFn) operator is also connected on the basis of the Parque IO.Writer operator, and the ParDo (DoFn) operator is an original operator. Further, one or more three-level operator nodes can be connected on the basis of the two-level operator nodes, as shown in fig. 5, a ParDo (DoFn) operator is also connected on the basis of a Combine (DoFn) operator, and the ParDo (DoFn) operator is an original operator. In fig. 5, the original operator is represented by a solid line box, and the composite operator is represented by a dashed line box.

In some embodiments, the computing server is provided with an operator orchestration module that is provided with the functionality of operator orchestration. Accordingly, the operator orchestration process described above may be: and the computing server performs operator arrangement according to the task flow of the federation computing task through an operator arrangement module to obtain an operator arrangement result of the federation computing task.

In some embodiments, the task flow of the federated computing task may be determined based on the operations performed by the user on the user terminal. Accordingly, in some embodiments, the operator orchestration process described above is performed by the computing server and the user terminal in concert. For example, in a service program of a federation computing task presented by a user terminal, a user can flexibly call a computing function exposed in a computing server, and further encode a computing process of the federation computing task by using the called computing function, namely, the assignment of a task flow is completed, so that a process of arranging a subsequent operator is facilitated. Thus, flexible processing of the binding computing task can be achieved by calling rich computing functions from the computing server.

In the above embodiment, by performing operator arrangement on the task flow of the federation calculation task, an operator arrangement result for indicating the calculation flow of the federation calculation can be obtained, and further, the calculation server can smoothly execute the process of the federation calculation by using the operator arrangement result, thereby improving the accuracy of the federation calculation.

Further, in some embodiments, the computing server encapsulates multiple types of operators, and accordingly, the process of federation computation described above may be: and invoking a plurality of operators of the types corresponding to the operator programming result from the operators of the plurality of types packaged by the computing server, and performing federation computation on the data to be processed of the federation computing task according to the operator programming result and the operators to obtain the federation computing result.

An operator is a process of a series of instructions implemented to achieve a certain function or to achieve a certain goal. It should be understood that an operator is a computational unit that is capable of performing some function. Illustratively, join operators, group operators, filter operators, and so forth. The Join operator can connect a plurality of heterogeneous data sources based on preset conditions. The Group operator and the Filter operator are capable of performing a convolution operation and a filtering operation based on the data to be processed of the federated computing task. It should be noted that, various operators provided by the embodiments of the present disclosure mask internal complex implementation for a business program. Taking a Join operator as an example, the inside of the Join operator is realized based on a Sort Merge Join strategy, and the user does not need to code by himself.

In some embodiments, the computing server is provided with an operator management module for encapsulating multiple types of operators. Accordingly, the procedure of calling the operator can be as follows: the computing server calls a plurality of operators of the types corresponding to the operator arrangement result from a plurality of types of operators encapsulated by the operator management module.

In the embodiment, by encapsulating multiple types of operators, richer or more complex functions can be provided for users, the capacity of federal calculation is improved, the use cost of the users is simplified, and therefore the flexibility of federal calculation is improved. Compared with the federal computing method based on the HDFS in the related art, the program interface (Application Programming Interface, API) provided by the related art is lower in order, that is, the amount of information provided is smaller, more detailed information needs to be obtained through further data analysis, and the computing cost of federal computing is increased, so that the use cost and the operation and maintenance cost of a user are increased. In the embodiment of the disclosure, by packaging multiple types of operators, a higher-order program interface can be provided, so that richer information quantity is provided for a user, the safety and controllability of federal calculation can be ensured, and meanwhile, the use cost of the user is reduced, and the use of the user is easier.

In some embodiments, the computing server includes multiple types of computing engines. Accordingly, the process of federal computation described above may be: the computing server also determines a computing engine corresponding to the computing type of the federated computing task from a plurality of computing engines of the type included in the computing server according to the computing type of the federated computing task. Further, in the computing server, the computing engine corresponding to the computing type of the federation computing task is utilized to perform federation computing on the data to be processed of the federation computing task, and the federation computing result is obtained.

The computing engine is a computing tool for data processing, and is mainly responsible for computing the source of required data, the operation of the data and the management of the data so as to output a final computing result. It should be appreciated that computing engines can be of many different categories depending on the type of computation (or referred to as the purpose of computation) of the federated computing task. Illustratively, spark computing engines, flink computing engines or Click House computing engines, and the like.

In some embodiments, the compute server may maintain multiple types of compute engines in the compute run module. Accordingly, the process of determining the compute engine described above may be replaced with: and the computing server determines a computing engine corresponding to the computing type of the federal computing task from a plurality of computing engines included in the computing server according to the computing type of the federal computing task through a computing operation module, and further, in the computing server, performs federal computing on data to be processed of the federal computing task by utilizing the computing engine corresponding to the federal computing task to obtain a federal computing result.

In the above embodiment, by encapsulating multiple types of computing engines, the flexibility and scalability of multiple languages (Scala, python, java) and multiple languages (SQL and coding modes) can be realized, so as to improve the flexibility of federal computing. Compared with the federal computing method of the SQL-based execution plan in the related art, the related art is only applicable to a Data warehouse (Data Warehourse) scene comprising structured Data, and the applicable scene is single. In the embodiment of the disclosure, by packaging multiple types of calculation engines, the multi-language and multi-grammar flexible and extensible method can be realized, and can be suitable for semi-structured, unstructured and complex processing (such as machine learning or model training) Data Lake (Data Lake) scenes.

S409, the computing server sends the federal computing result to the storage server to be stored in the storage server.

In an embodiment of the present disclosure, a storage server is used to register or host a plurality of data warehouses or business systems. The writing and management of the data of the registered data source are controlled by an external system, and the writing of the data of the managed data source is controlled by a computing server and is provided with a management and control function of automatic data index cleaning (TTL).

Illustratively, FIG. 6 is a schematic diagram of the architecture of a federated computing system as illustrated by an embodiment of the present disclosure. Referring to fig. 6, the architecture of federal computing includes a three-tier architecture, a Control tier (Control plane), a framework tier (framework), and a storage tier (storage), corresponding to the Control server, the computing server, and the storage server, respectively, in embodiments of the present disclosure. Wherein a service program (Provider) layer is deployed in the framework layer. In the embodiment of the disclosure, after a user initiates a processing request for a federal computing task to a computing server through a service program layer, configuration information of a service system associated with the federal computing task is issued to the computing server through a control server, so that the configuration information of the service system associated with the federal computing task can be cloud-controllable without perception to a service program, that is, the control server only needs to interact with a framework layer of the computing server and does not need to interact with a service program layer inside the computing server. Further, after the computing server receives the configuration information issued by the control server, the computing server can perceive the data information of the storage server according to the configuration information, further record the data to be processed of the federal computing task into the computing server according to the configuration information, and then perform federal computing on the data to be processed of the federal computing task. In addition, after the federation calculation, the calculation server may also send the federation calculation result to the storage server to trigger the storage server to store the federation calculation result.

In the embodiment of the disclosure, a three-layer architecture of a control server, a computing server and a storage server is constructed, configuration information of a federal computing task can be rapidly and efficiently obtained through interactive communication between the control server and the computing server, and furthermore, the computing server utilizes the configuration information of the federal computing task to load data so as to facilitate subsequent federal computing, and federal computing of different service data can be realized. The control server, the computing server and the storage server are mutually communicated, namely, a data network technology is combined on the basis of an application framework of the computing server, so that a federal computing scheme with good expansibility, compatibility and low cost is realized, the data communication and the data processing process are not sensitive to service programs, the management and control risk of a data management platform is reduced, and the use cost of a user is also reduced.

In some embodiments, in addition to the data configuration module, the calculation operation module, the operator orchestration module, and the operator management module mentioned in the above embodiments, the calculation server is further provided with a flow triggering module, where the flow triggering module is provided with a control function for triggering a flow of federal calculation, and is capable of concatenating the full flows of information acquisition, data analysis, data loading, authority verification, operator orchestration, and start-up calculation. In the embodiment of the present disclosure, the computing server may trigger and execute the process of sending the information acquisition request to the control server in the above S401 through the flow triggering module; the process of analyzing the data format of the configuration information in S406 to convert the data format of the configuration information into the target data format; a process of loading the data to be processed of the federal computing task from the storage server to the computing server in S407; and (S408) performing federation calculation on the data to be processed of the federation calculation task to obtain a federation calculation result. In some embodiments, the data warehouse supported by the compute server is extended by the flow triggering module.

Illustratively, fig. 7 is a schematic diagram of an internal architecture of a computing server as shown in embodiments of the present disclosure. Referring to fig. 7, the computing server includes a data configuration (Config IO) module, a computing run (Runner) module, an operator orchestration (Pipeline) module, an operator management (Transform) module, and a flow trigger (Launcher) module. The data configuration module is provided with functions of data analysis and data loading. In the internal structure diagram shown in fig. 7, the data configuration module may implement data analysis on information with different data formats, such as Text information or part information. The computing operation module is provided with the function of federal computing. In the internal architecture diagram shown in fig. 7, the computing operation module further includes a plurality of types of computing engines, such as Spark Runner computing engines or Flink Runner computing engines. The operator orchestration module is provided with an operator orchestration function. In the internal architecture diagram shown in FIG. 7, the operator orchestration module performs operator orchestration based on federal computing tasks required by the user. The operator management module is used for packaging various types of operators. In the internal architecture diagram shown in FIG. 7, the operator management module may encapsulate operators such as ParDo operators, groupbykey operators, combine operators, flatten operators, join operators, select operators, filter operators, group operators, or RenameFields. The flow triggering module is provided with a control function for triggering the flow of the federal computation. In the internal structure diagram shown in fig. 7, the flow triggering module may also support multiple types of data warehouse, such as AFS, UDW, or Turing.

Illustratively, FIG. 8 is a schematic illustration of an interaction flow of federal computing as shown in an embodiment of the present disclosure. Referring to fig. 8, an interactive flow of the linkage calculation is described by taking a flow triggering (home) module, a calculation running (Runner) module, an operator orchestration (Pipeline) module, and a business program (Provider) layer set in a calculation Server, and a configuration service (Config Server) module and a metadata service (Meta Server) module in a control Server as examples. Firstly, a task information acquisition request is sent to a configuration service module through a flow triggering module to request to acquire task configuration information related to the federal computing task, then the configuration service module returns the task configuration information to the flow triggering module, meanwhile, a storage information acquisition request is also sent to a metadata service module through the flow triggering module to request to acquire storage configuration information related to the federal computing task, and then the metadata service module returns the storage configuration information to the flow triggering module. And then, after receiving the task configuration information and the storage configuration information, the flow triggering module sends an initialization instruction to the operator arranging module so as to instruct the operator arranging module to perform initialization operation, so that an operator arranging result of the federation computing task is generated later. And the flow triggering module acquires an operator arranging result of the federation computing task from the service program layer after the operator arranging module is initialized. And finally, the flow triggering module sends a calculation starting instruction to the calculation operation module so as to instruct the calculation operation module to start the process of performing federal calculation.

Fig. 9 is a block diagram of a federated computing device as applied to a computing server, as illustrated in an embodiment of the present disclosure. Referring to fig. 9, the apparatus includes a transmitting module 901, a loading module 902, and a calculating module 903. Wherein:

a sending module 901, configured to send an information acquisition request to a control server in response to a processing request of a federal computing task; the federation computing task is used for fusing data from different service systems; the control server provides a control function for configuration information of different service systems; the information acquisition request is used for requesting to acquire configuration information of a business system associated with the federal computing task;

A loading module 902, configured to receive the configuration information returned by the control server; according to the configuration information, loading the data to be processed of the federal computing task from a storage server to the computing server;

the computing module 903 is configured to perform federation computation on data to be processed of the federation computing task in the computing server, to obtain a federation computing result.

In some embodiments, the configuration information includes task configuration information and storage configuration information; the task configuration information is used for indicating task content of the federation computing task; the storage configuration information is used to indicate metadata information of the storage server.

In some embodiments, the loading module 902 is configured to:

determining a data storage position of the storage server according to the storage configuration information;

and loading the data to be processed corresponding to the task configuration information from the data storage position of the storage server to the calculation server.

In some embodiments, the computing module 903 includes:

the operator arranging sub-module is used for arranging operators according to the task flow of the federation calculation task to obtain an operator arranging result of the federation calculation task; the operator programming result is used for indicating the calculation flow of the federation calculation;

and the federation calculation sub-module is used for performing federation calculation on the data to be processed of the federation calculation task according to the operator arrangement result in the calculation server to obtain the federation calculation result.

In some embodiments, the computing server encapsulates multiple types of operators;

The federation computation sub-module is used for:

invoking a plurality of operators of the types corresponding to the operator arranging result from a plurality of types of operators encapsulated by the computing server;

and performing federation calculation on the data to be processed of the federation calculation task according to the operator arrangement result and the operators to obtain the federation calculation result.

In some embodiments, the computing server includes multiple types of computing engines;

the device also comprises a determining module, a calculating module and a processing module, wherein the determining module is used for determining a calculating engine corresponding to the calculating type of the federal calculating task from a plurality of calculating engines included in the calculating server according to the calculating type of the federal calculating task;

the computing module 903 is further configured to perform, in the computing server, federation computation on data to be processed of the federation computing task by using a computing engine corresponding to a computing type of the federation computing task, to obtain a federation computing result.

In some embodiments, the system further includes an parsing module, configured to parse the data format of the configuration information to convert the data format of the configuration information into a target data format; the target data format is a data format supported by the computing server;

The loading module 902 is further configured to load data to be processed of the federated computing task from the storage server to the computing server according to the configuration information after the data format is converted.

Fig. 10 is a block diagram of a federated computing device as applied to a control server, as illustrated in an embodiment of the present disclosure. Referring to fig. 10, the apparatus includes a receiving module 1001, an acquiring module 1002, and a transmitting module 1003. Wherein:

a receiving module 1001, configured to receive an information acquisition request from a computing server; triggering the information acquisition request based on a processing request of the federal computing task; the federation computing task is used for fusing data from different service systems; the information acquisition request is used for requesting to acquire configuration information of a business system associated with the federal computing task;

an obtaining module 1002, configured to obtain configuration information of a service system associated with the federal computing task;

a transmitting module 1003, configured to transmit the configuration information to the computing server; the configuration information is used for loading the data to be processed of the federal computing task from a storage server to the computing server.

In some embodiments, the obtaining module 1002 is configured to:

acquiring configuration information of at least two service systems from a configuration information set according to service types of the at least two service systems associated with the federal computing task; the configuration information set comprises a plurality of service types and configuration information corresponding to the plurality of service types.

In some embodiments, further comprising:

and the updating module is used for responding to the change of the configuration information of any service system, and updating the configuration information of any service system in the configuration information set.

According to an embodiment of the present disclosure, the present disclosure also provides an electronic device including at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the federal computing method provided by the present disclosure.

According to an embodiment of the present disclosure, the present disclosure also provides a non-transitory computer-readable storage medium storing computer instructions for causing an electronic device to perform the federal computing method provided by the present disclosure.

According to an embodiment of the present disclosure, the present disclosure also provides a computer program product comprising a computer program which, when executed by a processor, implements the federal computing method provided by the present disclosure.

In some embodiments, the electronic device may be the server illustrated in fig. 1 above. Fig. 11 illustrates a schematic block diagram of an example electronic device 1100 that can be used to implement embodiments of the present disclosure. The electronic device 1100 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device 1100 may also represent various forms of mobile apparatuses, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing apparatuses. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 11, the electronic device 1100 includes a computing unit 1101 that can execute various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 1102 or a computer program loaded from a storage unit 1108 into a random access Memory (Random Access Memory, RAM) 1103. In the RAM 1103, various programs and data required for the operation of the electronic device 1100 can also be stored. The computing unit 1101, ROM 1102, and RAM 1103 are connected to each other by a bus 1104. An input/output (I/O) interface 1105 is also connected to bus 1104.

A number of components in the electronic device 1100 are connected to the I/O interface 1105, including: an input unit 1106 such as a keyboard, a mouse, etc.; an output unit 1107 such as various types of displays, speakers, and the like; a storage unit 1108, such as a magnetic disk, optical disk, etc.; and a communication unit 1109 such as a network card, modem, wireless communication transceiver, or the like. The communication unit 1109 allows the electronic device 1100 to exchange information/data with other devices through a computer network such as the internet and/or various telecommunications networks.

The computing unit 1101 may be a variety of general purpose and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 1101 include, but are not limited to, a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU), various dedicated artificial intelligence (Artificial Intelligence, AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (Digital Signal Processing, DSP), and any suitable processors, controllers, microcontrollers, etc. The computing unit 1101 performs the various methods and processes described above, such as the federal computing method. For example, in some embodiments, the federal computing method can be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 1108. In some embodiments, some or all of the computer programs may be loaded and/or installed onto electronic device 1100 via ROM 1102 and/or communication unit 1109. When the computer program is loaded into the RAM 1103 and executed by the computing unit 1101, one or more steps of the federal computing method described above may be performed. Alternatively, in other embodiments, the computing unit 1101 may be configured to perform the federal computing method by any other suitable means (e.g., by means of firmware).

Various implementations of the systems and techniques described here above can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (Field Programmable Gate Array, FPGAs), application specific integrated circuits (Application Specific Integrated Circuit, ASICs), application specific standard products (Application Specific Standard Parts, ASSPs), systems On Chip (SOC), complex programmable logic devices (Complex Programmable Logic Device, CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access Memory, a read-Only Memory, an erasable programmable read-Only Memory (Erasable Programmable Read Only Memory, EPROM, or flash Memory), an optical fiber, a compact disc read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device for displaying information to a user, for example, a Cathode Ray Tube (CRT) or a liquid crystal display (Liquid Crystal Display, LCD) monitor; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local area network (Local Area Network, LAN), wide area network (Wide Area Network, WAN) and the internet.

The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server incorporating a blockchain.

It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel or sequentially or in a different order, provided that the desired results of the technical solutions of the present disclosure are achieved, and are not limited herein.

The above detailed description should not be taken as limiting the scope of the present disclosure. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present disclosure are intended to be included within the scope of the present disclosure.

Claims

1. A federal computing method, applied to a computing server, comprising:

Receiving the configuration information returned by the control server; loading data to be processed of the federal computing task from a storage server to the computing server according to the configuration information;

2. The method of claim 1, wherein the configuration information includes task configuration information and storage configuration information; the task configuration information is used for indicating task content of the federation computing task; the storage configuration information is used for indicating metadata information of the storage server.

3. The method of claim 2, wherein the loading the data to be processed of the federated computing task from a storage server to the computing server in accordance with the configuration information comprises:

and loading the data to be processed corresponding to the task configuration information from the data storage position of the storage server to the computing server.

4. The method according to claim 1, wherein the performing, in the computing server, the federation calculation on the data to be processed of the federation computing task to obtain a federation calculation result includes:

Operator programming is carried out according to the task flow of the federation computing task, and an operator programming result of the federation computing task is obtained; the operator arranging result is used for indicating the calculation flow of the federation calculation;

and in the calculation server, performing federation calculation on the data to be processed of the federation calculation task according to the operator programming result to obtain the federation calculation result.

5. The method of claim 4, wherein the computing server encapsulates multiple types of operators;

and in the computing server, performing federation computation on the data to be processed of the federation computation task according to the operator programming result to obtain the federation computation result, including:

invoking a plurality of operators of the types corresponding to the operator arranging result from the operators of the types encapsulated by the computing server;

6. The method of claim 1, wherein the computing server comprises multiple types of computing engines; the method further comprises the steps of:

Determining a computing engine corresponding to the computing type of the federation computing task from a plurality of computing engines included in the computing server according to the computing type of the federation computing task;

and in the computing server, performing federation computation on the data to be processed of the federation computing task to obtain a federation computing result, including:

and in the computing server, performing federation computation on the data to be processed of the federation computing task by utilizing a computing engine corresponding to the computing type of the federation computing task to obtain a federation computing result.

7. The method of claim 1, further comprising:

analyzing the data format of the configuration information to convert the data format of the configuration information into a target data format; the target data format is a data format supported by the computing server;

the loading the data to be processed of the federal computing task from a storage server to the computing server according to the configuration information includes:

and loading the data to be processed of the federal computing task from the storage server to the computing server according to the configuration information after the data format is converted.

8. The federal computing method is applied to a control server, and the control server provides a control function for configuration information of different service systems; the method comprises the following steps:

receiving an information acquisition request from a computing server; the information acquisition request is triggered based on a processing request of the federal computing task; the federation computing task is used for fusing data from different service systems; the information acquisition request is used for requesting to acquire configuration information of a business system associated with the federal computing task;

transmitting the configuration information to the computing server; the configuration information is used for loading the data to be processed of the federal computing task from a storage server to the computing server by the computing server.

9. The method of claim 8, wherein the obtaining configuration information for a business system associated with the federated computing task comprises:

10. The method of claim 9, further comprising:

and responding to the change of the configuration information of any service system, and updating the configuration information of any service system in the configuration information set.

11. A federal computing device for use in a computing server, comprising:

the loading module is used for receiving the configuration information returned by the control server; loading data to be processed of the federal computing task from a storage server to the computing server according to the configuration information;

and the calculation module is used for carrying out federation calculation on the data to be processed of the federation calculation task in the calculation server to obtain a federation calculation result.

12. The apparatus of claim 11, wherein the configuration information comprises task configuration information and storage configuration information; the task configuration information is used for indicating task content of the federation computing task; the storage configuration information is used for indicating metadata information of the storage server.

13. The apparatus of claim 12, wherein the loading module is to:

14. The apparatus of claim 11, wherein the computing module comprises:

the operator arranging sub-module is used for carrying out operator arranging according to the task flow of the federation computing task to obtain an operator arranging result of the federation computing task; the operator arranging result is used for indicating the calculation flow of the federation calculation;

and the federation calculation sub-module is used for performing federation calculation on the data to be processed of the federation calculation task in the calculation server according to the operator arrangement result to obtain the federation calculation result.

15. The apparatus of claim 14, wherein the computing server encapsulates multiple types of operators;

the federation computation sub-module is used for:

16. The apparatus of claim 11, wherein the computing server comprises a plurality of types of computing engines;

and the calculation module is also used for carrying out federation calculation on the data to be processed of the federation calculation task by using a calculation engine corresponding to the calculation type of the federation calculation task in the calculation server to obtain the federation calculation result.

17. A federal computing device, applied to a control server, the control server providing control functions for configuration information of different service systems; the device comprises:

the receiving module is used for receiving an information acquisition request from the computing server; the information acquisition request is triggered based on a processing request of the federal computing task; the federation computing task is used for fusing data from different service systems; the information acquisition request is used for requesting to acquire configuration information of a business system associated with the federal computing task;

a sending module, configured to send the configuration information to the computing server; the configuration information is used for loading the data to be processed of the federal computing task from a storage server to the computing server by the computing server.

18. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 7 or 8 to 10.

19. A non-transitory computer readable storage medium storing computer instructions for causing an electronic device to perform the method of any one of claims 1-7 or 8-10.

20. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 7 or 8 to 10.