CN117278554A

CN117278554A - Data processing method based on cloud management platform and cloud management platform

Info

Publication number: CN117278554A
Application number: CN202210667087.2A
Authority: CN
Inventors: 张洪; 张珊铭; 杨明雄; 裴名明
Original assignee: Huawei Cloud Computing Technologies Co Ltd
Current assignee: Huawei Cloud Computing Technologies Co Ltd
Priority date: 2022-06-14
Filing date: 2022-06-14
Publication date: 2023-12-22
Also published as: WO2023241616A1

Abstract

The application discloses a data processing method based on a cloud management platform and the cloud management platform, which can save a great deal of labor cost and physical resources and can meet various data processing requirements of users. The method comprises the following steps: the cloud management platform determines a second client matched with the data processing requirement of the first client; the cloud management platform provides a ranking interface for receiving rankings from the first client for at least one operator provided by the second client; the cloud management platform receives a data processing request for target data from a first client; and the cloud management platform calls at least one operator to process the target data according to the sequence, obtains processed target data, and sends the processed target data to the first client.

Description

Data processing method based on cloud management platform and cloud management platform

Technical Field

The embodiment of the application relates to the technical field of clouds, in particular to a data processing method based on a cloud management platform and the cloud management platform.

Background

With the development and popularization of cloud storage technology, more and more users (e.g., enterprise users and individual users, etc.) select object storage services (object storage service, OBS) provided by a cloud service system, i.e., users can store data in an object storage bucket deployed by the cloud service system.

As the number of users increases, the size of the object buckets in the cloud service system increases, and the number and types of data stored in the object buckets also increases. For cloud service systems, users have a need for data storage as well as data processing. Based on the above, a developer of the cloud service system can deploy cloud instances with data processing capability in the cloud service system in a self-research manner so as to realize the data processing requirements of users.

However, as the data processing requirements of users are increasing, developers often need to consume a great deal of manpower cost and resources to develop and provide data processing capability, and users often have various data processing requirements, so that the developers often have difficulty in comprehensively considering the requirements of different types of data processing requirements of the users, and thus cannot meet the requirements of the users.

Disclosure of Invention

The embodiment of the application provides a data processing method based on a cloud management platform and the cloud management platform, which can save a great deal of labor cost and physical resources and can meet various data processing requirements of users.

A first aspect of an embodiment of the present application provides a data processing method based on a cloud management platform, where the method includes:

when the user has data processing requirements, the cloud management platform can provide a management interface for a first client used by the user, so that the user can send the data processing requirements of the user to the management interface through the first client, and the cloud management platform receives the data processing requirements sent by the first client through the management interface, namely the cloud management platform receives the data processing requirements of the user through the management interface.

After the cloud management platform obtains the data processing requirement sent by the first client, that is, after the cloud management platform obtains the data processing requirement of the user, the cloud management platform can select an ISV matched with the data processing requirement of the user in an independent software developer (independent software vendors, ISV) resource pool. The second client used by the ISV is then the second client that matches the data processing requirements sent by the first client.

The ISV may register at least one operator on the cloud management platform using the second client, the at least one operator being developed by the ISV, each operator being capable of implementing a particular data processing in the at least one operator developed by the ISV, so that the cloud management platform completes deployment of the at least one operator (e.g., deploys the at least one operator in a virtual instance) for subsequent direct invocation of the at least one operator by the cloud management platform.

Based on this, the cloud management platform may provide a ranking interface to the first client used by the user, so the user may send the ranking (which may also be understood as an execution order) of the user for the at least one operator to the ranking interface through the first client, such that the cloud management platform receives the ranking for the at least one operator sent by the first client through the ranking interface.

After deriving the ordering for the at least one operator, the cloud management platform may record the ordering for the at least one operator sent by the first client. Then, when the cloud management platform receives a data processing request for the target data sent by the first client, the cloud management platform may send an instruction to the virtual instance where the at least one operator is deployed. Then, the virtual instance acquires target data from an object storage bucket corresponding to the first client based on the instruction, and sequentially runs the at least one operator according to the sequence of the at least one operator to process the target data, so that processed target data are obtained. Then, the virtual instance may send the processed target data to the cloud management platform, so that the cloud management platform returns the processed test data as a data processing response to the first client used by the user. Thus, the data processing requirement of the user is met.

From the above method, it can be seen that: upon determining a second client that matches the data processing requirements of the first client, the cloud management platform may receive a ranking from the first client via the ranking interface for at least one operator provided by the second client. Then, after receiving a data processing request for the target data from the first client, the cloud management platform may call at least one operator to process the target data according to the order for the at least one operator, obtain processed target data, and send the processed target data to the first client. In the foregoing process, the user of the first client may perform custom ordering on at least one operator developed by using the ISV of the second client according to the data processing requirement of the user. When the user has target data to be processed, the cloud management platform in the cloud service system can call the ready at least one operator to process the target data according to the sequence, and the obtained processed data can be used by the user. Therefore, a developer of the cloud service system does not need to self-grind operators with data processing capacity, a large amount of labor cost and physical resources can be saved, and specific operators can be provided for users in a targeted manner, so that various data processing requirements of the users are met.

In one possible implementation, after the cloud management platform determines the second client that matches the data processing requirement of the first client, before the cloud management platform provides the ordering interface, the method further includes: the cloud management platform provides a registration interface, and the registration interface is used for receiving information of at least one operator from the second client; the cloud management platform deploys at least one operator based on the information. In the foregoing implementation, after determining the ISV that matches the data processing requirement of the user, the cloud management platform may provide a registration interface to a second client used by the ISV, so the ISV may send information of at least one operator to the registration interface through the second client, so that the cloud management platform receives the information of the at least one operator through the registration interface, thereby registering the at least one operator on the cloud management platform. After registration is completed, the cloud management platform can complete deployment of the at least one operator in a certain virtual instance based on information of the at least one operator, so that the at least one operator can be directly called by the cloud management platform.

In one possible implementation, after the cloud management platform provides the registration interface, before the cloud management platform deploys the at least one operator based on the information, the method further includes: the cloud management platform provides an auditing interface, and the auditing interface is used for receiving an auditing passing result aiming at the information from the third client; the cloud management platform deploying at least one operator based on the information comprises: and the cloud management platform deploys at least one operator based on the auditing passing result and the information. In the foregoing implementation manner, after obtaining the information of the at least one operator, the cloud management platform may provide the information of the at least one operator to a third client used by the operation and maintenance personnel, so as to provide an audit interface for the operation and maintenance personnel. And then, the operation and maintenance personnel can audit the information displayed by the third client, and if the audit is passed, the operation and maintenance personnel can input an audit passing result aiming at the information of the at least one operator to the audit interface through the third client so that the cloud management platform can receive the audit passing result through the audit interface. After the auditing passing result is obtained, the cloud management platform can determine that the at least one operator is available, and the at least one operator is deployed on a certain virtual instance based on the information of the at least one operator, so that the at least one operator can be operated on the virtual instance subsequently to realize data processing. Therefore, operators provided by the ISV can be checked by operation and maintenance personnel of the cloud service system, so that the reliability of the operators is ensured.

In one possible implementation, after the cloud management platform deploys the at least one operator based on the audit passing result and the information, before the cloud management platform provides the ordering interface, the method further includes: the cloud management platform provides a test interface, and the test interface is used for receiving test data from the third client; and the cloud management platform calls at least one operator to process the test data, obtains processed test data, and sends the processed test data to the third client. In the foregoing implementation manner, after the at least one operator is deployed, the cloud management platform may provide a test interface to a third client used by the operation and maintenance personnel, so the operation and maintenance personnel may send test data to the test interface through the third client, so that the cloud management platform receives the test data through the test interface. Upon receiving the test data, the cloud management platform may send a test request to the virtual instance deployed by the at least one operator, the test request containing the test data. The virtual instance may then run the at least one operator to process the test data to obtain processed test data. Then, the virtual instance may send the processed test data to the cloud management platform, so that the cloud management platform returns the processed test data as a test response to a third client used by the operation and maintenance personnel. And the cloud management platform does not receive any request or any instruction of the third client used by the operation and maintenance personnel within a preset duration, and can determine that the functional test of the at least one operator is successful. It can be seen that operators provided by ISVs can be tested by operation and maintenance personnel of the cloud service system, so that reliability of the operators is further guaranteed.

In one possible implementation manner, the target data is stored in an object storage bucket corresponding to the first client, the cloud management platform calls at least one operator to process the target data according to the ordering, and after the processed target data is obtained, the method further includes: and the cloud management platform stores the processed target data in an object storage bucket. In the foregoing implementation manner, since the target data to be processed is stored in the object storage bucket corresponding to the first client, after the processed target data is obtained, the cloud management platform may store the processed target data in the object storage bucket corresponding to the first client, so as to ensure the data security of the user.

In one possible implementation, the data processing requirements sent by the first client include at least one of: the type of data required to be processed by the first client, namely the type of data required to be processed by the user; the type of processing required to be implemented by the first client on the data, namely the type of processing required to be implemented by the user on the data; the data volume required to be processed by the first client, namely the data volume required to be processed by the user; the service level agreement that the first client conforms to, i.e. the service level agreement that the user requires.

In one possible implementation, the information of the at least one operator includes at least one of: attribute information of the at least one operator including a name, a size, an icon, and a data processing that can be implemented, etc. of the at least one operator; code of the at least one operator operable to deploy and run the at least one operator; instructions for the at least one operator, which may be used to introduce a specific manner of use of the at least one operator, data processing capabilities, and the like; a service level agreement to which the at least one operator conforms; open source information for at least one operator indicating that the at least one operator is implemented based on an open source software library; a test report of at least one operator comprising a functional test, a performance test, a reliability test, etc. of the at least one operator for describing a functional stability and reliability, etc. of the at least one operator.

A second aspect of the embodiments of the present application provides a cloud management platform, including: the determining module is used for determining a second client matched with the data processing requirement of the first client; a first providing module for providing a ranking interface for receiving a ranking from a first client for at least one operator provided by a second client; the receiving module is used for receiving a data processing request for target data from the first client; the first processing module is used for calling at least one operator to process the target data according to the sequence, obtaining processed target data, and sending the processed target data to the first client.

From the cloud management platform, it can be seen that: upon determining a second client that matches the data processing requirements of the first client, the cloud management platform may receive a ranking from the first client via the ranking interface for at least one operator provided by the second client. Then, after receiving a data processing request for the target data from the first client, the cloud management platform may call at least one operator to process the target data according to the order for the at least one operator, obtain processed target data, and send the processed target data to the first client. In the foregoing process, the user of the first client may perform custom ordering on at least one operator developed by using the ISV of the second client according to the data processing requirement of the user. When the user has target data to be processed, the cloud management platform in the cloud service system can call the ready at least one operator to process the target data according to the sequence, and the obtained processed data can be used by the user. Therefore, a developer of the cloud service system does not need to self-grind operators with data processing capacity, a large amount of labor cost and physical resources can be saved, and specific operators can be provided for users in a targeted manner, so that various data processing requirements of the users are met.

In one possible implementation, the cloud management platform further includes: a second providing module for providing a registration interface for receiving information of at least one operator from a second client; and the deployment module is used for deploying at least one operator based on the information.

In one possible implementation, the cloud management platform further includes: the third providing module is used for providing an auditing interface, and the auditing interface is used for receiving an auditing passing result for the information from the third client; the deployment module is used for deploying at least one operator based on the auditing passing result and the information.

In one possible implementation, the cloud management platform further includes: the fourth providing module is used for providing a test interface, and the test interface is used for receiving test data from the third client; the second processing module is used for calling at least one operator to process the test data, obtaining processed test data and sending the processed test data to the third client.

In one possible implementation, the target data is stored in an object storage bucket corresponding to the first client, and the cloud management platform further includes: and the storage module is used for storing the processed target data in the object storage bucket.

In one possible implementation, the data processing requirements include at least one of: the type of data that the first client needs to process, the type of processing that the first client needs to implement on the data, the amount of data that the first client needs to process, and the service level agreement that the first client conforms to.

In one possible implementation, the information of the at least one operator includes at least one of: the system comprises attribute information of at least one operator, codes of at least one operator, a service level protocol which the at least one operator accords with, open source information of the at least one operator and a test report of the at least one operator.

A third aspect of the embodiments of the present application provides a cloud management platform, where the cloud management platform includes a memory and a processor; the memory stores code, the processor being configured to execute the code, the cloud management platform performing the method according to the first aspect or any one of the possible implementations of the first aspect when the code is executed.

A fourth aspect of embodiments of the present application provides a computer storage medium storing one or more instructions that, when executed by one or more computers, cause the one or more computers to implement the method as described in the first aspect or any one of the possible implementations of the first aspect.

A fifth aspect of embodiments of the present application provides a computer program product storing instructions that, when executed by a computer, cause the computer to carry out the method according to the first aspect or any one of the possible implementations of the first aspect.

In this embodiment, after determining the second client that matches the data processing requirement of the first client, the cloud management platform may receive, through the ordering interface, an ordering from the first client for at least one operator, where the at least one operator is provided by the second client. Then, after receiving a data processing request for the target data from the first client, the cloud management platform may call at least one operator to process the target data according to the order for the at least one operator, obtain processed target data, and send the processed target data to the first client. In the foregoing process, the user of the first client may perform custom ordering on at least one operator developed by using the ISV of the second client according to the data processing requirement of the user. When the user has target data to be processed, the cloud management platform in the cloud service system can call the ready at least one operator to process the target data according to the sequence, and the obtained processed data can be used by the user. Therefore, a developer of the cloud service system does not need to self-grind operators with data processing capacity, a large amount of labor cost and physical resources can be saved, and specific operators can be provided for users in a targeted manner, so that various data processing requirements of the users are met.

Drawings

Fig. 1 is a schematic structural diagram of a cloud service system according to an embodiment of the present application;

fig. 2 is a schematic flow chart of a data processing method based on a cloud management platform according to an embodiment of the present application;

FIG. 3 is a schematic diagram of a registration interface according to an embodiment of the present application;

FIG. 4 is a schematic diagram of an audit interface provided by an embodiment of the present application;

FIG. 5 is a schematic diagram of a test interface according to an embodiment of the present application;

FIG. 6 is a schematic diagram of a sorting interface provided by an embodiment of the present application;

fig. 7 is a schematic structural diagram of a cloud management platform according to an embodiment of the present application;

fig. 8 is another schematic structural diagram of the cloud management platform according to the embodiment of the present application.

Detailed Description

The terms first, second and the like in the description and in the claims of the present application and in the above-described figures, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances and are merely illustrative of the manner in which the embodiments of the application described herein have been described for objects of the same nature. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of elements is not necessarily limited to those elements, but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.

With the development and popularization of cloud storage technology, more and more users (e.g., enterprise users and individual users, etc.) select object storage services provided by a cloud service system, that is, users can store data in an object storage bucket (also referred to as an OBS bucket) deployed by the cloud service system.

As the number of users increases, the size of the object buckets in the cloud service system increases, and the number and types of data stored in the object buckets also increases. For cloud service systems, users have a need for data storage as well as data processing. Based on the above, a developer of the cloud service system can deploy cloud instances with data processing capability (function) in the cloud service system in a self-research manner so as to realize the data processing requirement of a user. For example, in order to meet the data storage requirement of a certain user, the cloud management platform of the cloud service system may deploy an object storage bucket for the user, so as to store the data of the user, and in order to meet the data processing requirement of the user, the cloud management platform may deploy a virtual machine (also may be a container or a physical machine, etc.) with data processing capability for the user. Then, after receiving the data processing request sent by the client used by the user, the cloud management platform may issue an instruction to the virtual machine, so that the virtual machine pulls data from the object storage bucket, processes the data, and returns the processed data to the cloud management platform. Finally, the cloud management platform may return the processed data to the user's client for use by the user.

However, as the data processing requirements of users are increasing, developers often need to consume a great deal of manpower cost to develop the data processing capability (and consume a great deal of physical resources to additionally deploy cloud instances with the data processing capability), and users often have various data processing requirements, which are often difficult for developers to comprehensively consider and consider, and cannot develop and provide various types of data processing capability, so that the requirements of different types of data processing requirements of users cannot be met.

Further, the data processing capability provided by the developer is often not the best and most specialized, and the data processing requirement of the user cannot be rapidly and high-quality completed, so that the data processing efficiency of the whole cloud service system is low, and further, the user experience is poor.

Further, the data processing capability provided by the developer is invisible to the user, and how to trust the data processing capability provided by the developer to be reliable and safe, so as to ensure the stable landing of the cloud service system is also a problem to be solved urgently.

In order to solve the above problems, the embodiments of the present application provide a data processing method, which may be implemented by a cloud service system. Fig. 1 is a schematic structural diagram of a cloud service system provided in an embodiment of the present application, as shown in fig. 1, where the cloud service system includes: the cloud management platform, the plurality of first physical nodes and the plurality of second physical nodes are described below as follows:

The cloud management platform can carry out overall management on a plurality of physical nodes in the whole cloud service system, can also be opened for users outside the system, independent software providers (independent software vendor, ISVs) and operation and maintenance personnel of the cloud service system, and can respond to the requests of the users and the independent software providers (independent software vendor, ISVs). For example, the cloud management platform may create an OBS bucket on a first physical node for storing data uploaded by a user's client. As another example, the cloud management platform may also create virtual instances on the second physical node for deploying ISV-provided operators (also referred to as functions), so these virtual instances may implement data processing. For another example, the cloud management platform may provide various interfaces (such as a login interface, a registration interface, an audit interface, a test interface, and a sequencing interface, etc.) for accessing the client of the user, the client of the ISV, and the client of the operation and maintenance personnel, where the cloud management platform may perform identity verification on the client of the user, the client of the ISV, and the client of the operation and maintenance personnel through the login interface, and may allow the client to log into the cloud management platform after the verification is passed. The cloud management platform may also allow clients of the ISV to register operators provided by the ISV on the cloud management platform through a registration interface. The cloud management platform can also audit and test operators provided by the ISV through the audit interface and the test interface so as to ensure the reliability, safety and credibility of the operators. The cloud management platform can also display operators provided by the ISV through the sequencing interface and allow the client of the user to sequence the operators selected by the user, so that the user-defined flow of the user for data processing is realized.

The first physical node is a physical server. The physical node is based on a virtualization technology, can be deployed with one or more OBS barrels, and under the allocation of a cloud management platform, the OBS barrels can be used for storing data uploaded by a client of a user, so that the data storage requirement of the user is met.

The second physical node is a physical server. The physical node may be deployed with one or more virtual instances (e.g., virtual Machines (VMs) or containers, etc.) based on virtualization technology, and under deployment of the cloud management platform, these virtual instances may be used to deploy operators provided by the ISV, and since these operators may implement data processing, they may meet the data processing requirements of the user.

To further understand the workflow of the cloud service system, the workflow will be described with reference to fig. 2, and for convenience of explanation, the client of the user will be referred to as a first client, the client of the ISV will be referred to as a second client, and the client of the operation staff of the cloud service system will be referred to as a third client. Fig. 2 is a schematic flow chart of a data processing method based on a cloud management platform according to an embodiment of the present application, as shown in fig. 2, the method includes:

201. The cloud management platform determines a second client that matches the data processing requirements of the first client.

In this embodiment, when a user has a data processing requirement, the cloud management platform may provide a management interface (e.g., a requirement input field of a user interface) to a first client used by the user (e.g., a browser used by the user, etc.), so the user may send the data processing requirement of the user to the management interface through the first client, so that the cloud management platform receives the data processing requirement sent by the first client through the management interface, that is, the cloud management platform receives the data processing requirement of the user through the management interface.

Specifically, the data processing requirements sent by the first client include one or more of the following information: (1) The type of data that the first client needs to process, i.e., the type of data that the user needs to process, e.g., the data that the user needs to process is image data, etc. (2) The type of processing that the first client needs to implement on the data, i.e., the type of processing that the user needs to implement on the data, e.g., the processing that the user needs to implement on the image is a zoom-in operation, etc. (3) The amount of data that the first client needs to process, i.e., the amount of data that the user needs to process, for example, the amount of image data that the user needs to enlarge is 1G or the like. (4) The service level agreement (service level agreement, SLA) the first client is compliant with, i.e. the service level agreement (service level agreement, SLA) the user requires.

After the cloud management platform obtains the data processing requirement sent by the first client, that is, after the cloud management platform obtains the data processing requirement of the user, the cloud management platform can select an ISV matching the data processing requirement of the user (that is, the ISV can meet the data processing requirement of the user) from an ISV resource pool (including a plurality of candidate ISVs). The second client used by the ISV is then the second client that matches the data processing requirements sent by the first client.

It should be appreciated that before the user sends the data processing requirements, the cloud management platform may provide a login interface (e.g., an account input field and a password input field of the login interface, etc.) to a first client used by the user, and the user may input a user account number and a user password (an account number and a password that have been registered with the cloud management platform) to the login interface through the first client, so the cloud management platform may perform identity verification on the information, and after the verification is passed, the first client may be allowed to log into the cloud management platform.

202. The cloud management platform receives information of at least one operator from the second client through the registration interface.

After determining an ISV that matches the user's data processing needs, the cloud management platform may provide a registration interface (e.g., an operator information input field of an operator registration interface, etc.) to a second client used by the ISV, so the ISV may send information of at least one operator (i.e., one or more operators) to the registration interface through the second client, such that the cloud management platform receives information of the at least one operator through the registration interface, thereby registering the at least one operator on the cloud management platform. It is noted that the at least one operator is developed by the ISV, in which each operator is capable of performing a certain data processing (e.g., image magnification, image recognition, etc.).

In particular, the information of the at least one operator comprises one or more of the following information: (1) Attribute information of the at least one operator including a name, a size, an icon, and data processing enabled by the at least one operator, and the like. (2) The code (which may also be referred to as an operator package) of the at least one operator is operable to deploy and run the at least one operator. (3) The instruction of the at least one operator can be used to introduce the specific usage of the at least one operator, the data processing capability, and so on. (4) The service level agreement to which the at least one operator conforms (satisfies). (5) Open source information (also referred to as open source declarations) for at least one operator indicating that the at least one operator is implemented based on a trusted open source software library. (6) A test report of at least one operator comprising a functional test, a performance test, a reliability test, etc. of the at least one operator for describing a functional stability and reliability, etc. of the at least one operator.

To further understand the registration interface provided by the cloud management platform, the registration interface is further described below in connection with a specific example. For example, as shown in fig. 3 (fig. 3 is a schematic diagram of a registration interface provided by an embodiment of the present application), the cloud management platform provides an operator registration interface for a client used by an ISV (e.g., a browser used by ISV1, etc.), and the operator registration interface may be displayed by the client for viewing and operation by the ISV. The operator registration interface comprises an attribute information input field, an operator package input field, a use instruction input field, a service level protocol input field, an open source declaration input field, a test report input field and other operator information input fields. When the ISV needs to register the operator 1, attribute information of the operator 1 can be input at an attribute information input field, codes of the operator 1 can be input at an operator package input field, a use instruction of the operator 1 can be input at a use instruction input field, a service level protocol satisfied by the operator 1 can be input at a service level protocol input field, an open source declaration of the operator 1 can be input at an open source declaration input field, a test report of the operator 1 can be input at a test report input field, and the like, so that information of the operator 1 can be uploaded to the cloud management platform. If the ISV also requires a registry operator 2, an operator 3, an operator N, operations as performed on operator 1 may also be performed, and are not described in detail herein.

It should be appreciated that before the ISV sends the information of the at least one operator, the cloud management platform may provide a login interface (e.g., an account input field and a password input field of the login interface, etc.) to a second client used by the ISV, and the ISV may input an ISV account number and an ISV password (the account number and the password that have been registered with the cloud management platform) to the login interface through the second client, so the cloud management platform may perform identity verification on the information, and after the verification passes, may allow the second client to log into the cloud management platform.

203. And the cloud management platform receives an audit passing result of information aiming at least one operator from the third client through an audit interface.

After obtaining the information of the at least one operator, the cloud management platform may provide the information of the at least one operator to a third client used by the operation and maintenance personnel (e.g., the cloud management platform may provide an operator audit interface, the operator audit interface may display the information of the operator, etc.) for the operation and maintenance personnel to audit, and provide an audit interface (e.g., an audit result input field in the operator audit interface, etc.) to the third client. And if the information displayed by the third client passes the audit, the operation and maintenance personnel can input the audit passing result of the information of the at least one operator to the audit interface through the third client so that the cloud management platform receives the audit passing result through the audit interface, and therefore, the cloud management platform can determine that the at least one operator has certain functional stability and reliability based on the audit passing result and is available.

In order to further understand the audit interface provided by the cloud management platform, the audit interface is further described below in connection with a specific example. Still as shown in fig. 4 (fig. 4 is a schematic diagram of an audit interface provided by an embodiment of the present application, and fig. 4 is drawn based on fig. 3), the cloud management platform provides an operator audit interface for a client (e.g., a browser used by an operator, etc.) used by the operator, where the operator audit interface displays information of the operator 1, information of the operator 2, information of the operator N, and the operator audit interface includes an audit result input field. The operator auditing interface can be displayed by the client side to be watched by operation and maintenance personnel, so that the operation and maintenance personnel can browse the operator auditing interface to audit the information of each operator provided by the ISV, if the auditing is passed, the operation and maintenance personnel can input an auditing passing result in an auditing result input field through the client side, and the cloud management platform can determine that each operator provided by the ISV is available based on the result.

It should be appreciated that before the operation and maintenance personnel send the audit passing result, the cloud management platform may provide a login interface (e.g., an account input field and a password input field of the login interface, etc.) to a third client used by the operation and maintenance personnel, and the operation and maintenance personnel may input an operation and maintenance personnel account number and an operation and maintenance personnel password (an account number and a password that have been registered in the cloud management platform) to the login interface through the third client, so the cloud management platform may perform identity verification on the information, and after the verification passes, the third client may be allowed to log into the cloud management platform.

204. The cloud management platform deploys at least one operator based on the audit passing result and information of the at least one operator.

Upon receipt of the audit pass result, the cloud management platform may determine that the at least one operator is available, then deploy the at least one operator on a virtual instance based on information of the at least one operator (e.g., code of the at least one operator, etc.), so that the at least one operator may be subsequently run on the virtual instance to enable data processing.

It should be understood that step 203 is optional, and the cloud management platform may also directly perform step 204 without performing step 203. In this case, no operation staff is required to audit the information of the at least one operator, so the cloud management platform does not need to pass the audit result for the at least one operator, and the at least one operator can be deployed directly based on the information of the at least one operator.

205. And the cloud management platform receives the test data from the third client through the test interface.

206. And the cloud management platform calls at least one operator to process the test data, obtains processed test data, and sends the processed test data to the third client.

After deploying the at least one operator, the cloud management platform may provide a test interface (e.g., a data input field of the operator test interface, etc.) to a third client used by the operation and maintenance personnel, so the operation and maintenance personnel may send test data to the test interface through the third client, so that the cloud management platform receives the test data through the test interface. Upon receiving the test data, the cloud management platform may send a test request to the virtual instance deployed by the at least one operator, the test request containing the test data. The virtual instance may then run the at least one operator to process the test data to obtain processed test data. Then, the virtual instance may send the processed test data to the cloud management platform, so that the cloud management platform returns the processed test data as a test response to a third client used by the operation and maintenance personnel.

It should be noted that, the third client may display the processed test data to the operation and maintenance personnel for viewing, the operation and maintenance personnel may determine whether the processed test data is qualified, if the processed test data is qualified, no operation needs to be performed any more, and the cloud management platform may determine that the functional test for the at least one operator is successful.

To further understand the test interface provided by the cloud management platform, the test interface is further described below in conjunction with a specific example. Still as an example, as shown in fig. 5 (fig. 5 is a schematic diagram of a test interface provided by an embodiment of the present application, and fig. 5 is drawn based on fig. 4), the cloud management platform provides an operator test interface for a client used by an operator (e.g., a browser used by the operator, etc.), where the operator test interface shows a test specification for an operator 1, an operator 2, an operator N, and the operator test interface includes a data input field. The operator test interface can be displayed by the client side for an operator to watch, so that the operator can input certain test data in a data input field through the client side, the cloud management platform calls an operator 1, an operator 2 and an operator N to process the test data, and the processed test data is displayed on a data output field of the operator test interface so that the operator can judge whether the functional test of the operators is successful or not.

It should be understood that step 205 and step 206 are optional, and the cloud management platform may also directly perform step 207 without performing step 205 and step 206. In this case, no operation staff is required to test the function of the at least one operator, so the cloud management platform may directly perform step 207.

207. The cloud management platform receives a ranking for at least one operator from a first client via a ranking interface.

Upon determining that the functional test for the at least one operator is successful, the cloud management platform may provide a ranking interface (e.g., a ranking input field of the operator ranking interface, etc.) to a first client used by the user, so the user may send a ranking (also known as an execution order) of the user for the at least one operator to the ranking interface through the first client, such that the cloud management platform receives the ranking of the at least one operator sent by the first client through the ranking interface.

To further understand the ranking interface provided by the cloud management platform, the ranking interface is further described below in connection with a specific example. Still as shown in fig. 6 (fig. 6 is a schematic diagram of a ranking interface provided by an embodiment of the present application, and fig. 6 is drawn based on fig. 5), the cloud management platform provides an operator ranking interface for a client used by a user (e.g., a browser used by the user, etc.), the operator ranking interface including an operator selection area including an operator 1, an operator 2,... The operator sequencing interface may be displayed to a user for viewing by the client, so that the user may select the icon of operator 1 in the operator selection area and pull the icon of operator 1 into the sequencing input field, then the user may continue to select the icon of operator 3 in the operator selection area and pull the icon of operator 3 into the sequencing input field (in the sequencing input field, the position of the icon of operator 3 is concatenated after the position of the icon of operator 1) through the client, until all of the N operator icons in the operator selection area are pulled into the sequencing input field. The user has thus successfully entered the order for the N operators, operator 1→operator 3→operator 2→operator N, into the order input field via the client.

208. The cloud management platform receives a data processing request for target data from a first client.

209. And the cloud management platform calls at least one operator to process the target data according to the sequence, obtains processed target data, and sends the processed target data to the first client.

After deriving the ordering for the at least one operator, the cloud management platform may record the ordering for the at least one operator sent by the first client. Then, when the cloud management platform receives the data processing request for the target data sent by the first client, the cloud management platform may send an instruction to the virtual instance deployed with the at least one operator, the instruction including the pull address of the target data and the ordering of the at least one operator. Then, the virtual instance obtains target data from an OBS bucket corresponding to the first client (i.e., the OBS bucket of the user, which stores a plurality of data of the user, the plurality of data including target data), and sequentially runs the at least one operator according to the order of the at least one operator to process the target data, thereby obtaining processed target data. Then, the virtual instance may send the processed target data to the cloud management platform, so that the cloud management platform returns the processed test data as a data processing response to the first client used by the user. Thus, the data processing requirement of the user is met.

Still as in the above example, after the cloud management platform determines that the target data to be processed needs to be specified by the user, the virtual machine deployed with the operator 1, the operator 2, the operator N may pull the target data from the OBS bucket of the user. Then, the order of the N operators is operator 1- & gt operator 3- & gt operator 2- & gt operator N, so that the virtual machine can process target data through the operator 1 to obtain target data after the 1 st process, process the target data after the 1 st process through the operator 3 to obtain target data after the 2 nd process, process the target data after the 2 nd process through the operator 2 to obtain target data after the 3 rd process, and process the target data after the N-1 th process through the operator N to obtain target data after the N process. Finally, the virtual machine can send the target data after the Nth processing to the cloud management platform, so that the cloud management platform returns the target data after the Nth processing to the user.

Further, the cloud management platform is further provided with a double-end fixing mechanism between the virtual private cloud (virtual private cloud, VPC) and the OBS barrel, and the target data to be processed is stored in the OBS barrel corresponding to the first client, so that after the processed target data is obtained, the cloud management platform stores the processed target data in the OBS barrel corresponding to the first client, and the data security of a user is ensured.

Further, in the embodiment of the present application, operators provided by ISVs often have high-quality and professional data processing capabilities, so that the cloud management platform can rapidly and high-quality complete the data processing requirements of users, thereby improving the data processing efficiency of the whole cloud service system and further improving the user experience.

Furthermore, in the embodiment of the present application, operators provided by the ISV may be checked and tested by operation and maintenance personnel of the cloud service system, and may also be tested personally by a user (the testing process is similar to the foregoing steps 208 and 209, and will not be repeated here), so as to ensure the reliability of these operators. Moreover, the cloud service system can ensure the security of user data through a double-ended fixing mechanism between the VPC and the OBS barrel. In this way, the trust of the user to the cloud service system can be obtained, and the cloud service system is facilitated to fall to the ground.

The foregoing is a specific description of the data processing method based on the cloud management platform provided by the embodiment of the present application, and the following describes the cloud management platform provided by the embodiment of the present application. Fig. 7 is a schematic structural diagram of a cloud management platform according to an embodiment of the present application, where as shown in fig. 7, the cloud management platform includes:

A determining module 701, configured to determine a second client that matches the data processing requirement of the first client;

a first providing module 702 for providing a ranking interface for receiving a ranking from a first client for at least one operator provided by a second client;

a receiving module 703, configured to receive a data processing request for target data from a first client;

the first processing module 704 is configured to call at least one operator to process the target data according to the ordering, obtain processed target data, and send the processed target data to the first client.

In one possible implementation, the information includes at least one of: the system comprises attribute information of at least one operator, codes of at least one operator, a service level protocol which the at least one operator accords with, open source information of the at least one operator and a test report of the at least one operator.

It should be noted that, because the content of information interaction and implementation process between the modules/units of the above-mentioned apparatus is based on the same concept as the method embodiment of the present application, the technical effects brought by the content are the same as the method embodiment of the present application, and specific content may refer to the description in the foregoing illustrated method embodiment of the present application, which is not repeated herein.

Fig. 8 is another schematic structural diagram of the cloud management platform according to the embodiment of the present application. As shown in fig. 8, one embodiment of a cloud management platform may include one or more central processors 801, memory 802, input output interfaces 803, wired or wireless network interfaces 804, power supplies 805.

Memory 802 may be transient or persistent. Still further, the central processor 801 may be configured to communicate with the memory 802 to execute a series of instruction operations in the memory 802 on the cloud management platform.

In this embodiment, the central processor 801 may execute the method steps executed by the cloud management platform in the embodiment shown in fig. 2, which is not described herein.

In this embodiment, the specific functional module division in the central processing unit 801 may be similar to the division manner of the determining module, the first providing module, the receiving module, the first processing module, the second providing module, the deployment module, the third providing module, the fourth providing module, the second processing module, and the storage module described in fig. 7, which is not repeated here.

The present embodiment also relates to a computer storage medium in which a program for performing signal processing is stored, which when run on a computer causes the computer to perform the steps in the embodiment shown in fig. 2.

Embodiments of the present application also relate to a computer program product storing instructions that, when executed by a computer, cause the computer to perform the steps of the embodiment shown in fig. 2.

It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, which are not repeated herein.

In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods may be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a storage medium, including several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a random access memory (random access memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

Claims

1. A data processing method based on a cloud management platform, the method comprising:

the cloud management platform determines a second client matched with the data processing requirement of the first client;

the cloud management platform provides a ranking interface for receiving rankings from the first client for at least one operator provided by the second client;

the cloud management platform receives a data processing request for target data from the first client;

and the cloud management platform calls the at least one operator to process the target data according to the ordering to obtain processed target data, and sends the processed target data to the first client.

2. The method of claim 1, wherein after the cloud management platform determines a second client that matches the data processing requirements of the first client, the method further comprises, before the cloud management platform provides the ordering interface:

the cloud management platform provides a registration interface for receiving information of at least one operator from the second client;

The cloud management platform deploys the at least one operator based on the information.

3. The method of claim 2, wherein after the cloud management platform provides the registration interface, the cloud management platform is further configured to:

the cloud management platform provides an auditing interface, and the auditing interface is used for receiving an auditing passing result for the information from a third client;

the cloud management platform deploying the at least one operator based on the information comprises:

and the cloud management platform deploys the at least one operator based on the auditing passing result and the information.

4. The method of claim 3, wherein the cloud management platform, based on the audit pass result and the information, after deploying the at least one operator, before the cloud management platform provides a ranking interface, the method further comprises:

the cloud management platform provides a test interface, and the test interface is used for receiving test data from a third client;

and the cloud management platform calls the at least one operator to process the test data to obtain processed test data, and sends the processed test data to the third client.

5. The method according to any one of claims 1 to 4, wherein the target data is stored in an object storage bucket corresponding to the first client, and the cloud management platform calls the at least one operator to process the target data according to the ordering, so as to obtain processed target data, and the method further includes:

and the cloud management platform stores the processed target data in the object storage bucket.

6. The method according to any one of claims 1 to 5, wherein the data processing requirements include at least one of: the method comprises the steps of determining the type of data required to be processed by the first client, the type of processing required to be realized by the first client on the data, the data amount required to be processed by the first client and the service level agreement which the first client accords with.

7. The method according to any one of claims 1 to 6, wherein the information comprises at least one of: attribute information of the at least one operator, code of the at least one operator, instructions for use of the at least one operator, service level agreements to which the at least one operator conforms, open source information of the at least one operator and test reports of the at least one operator.

8. A cloud management platform, the cloud management platform comprising:

the determining module is used for determining a second client matched with the data processing requirement of the first client;

a first providing module for providing a ranking interface for receiving a ranking for at least one operator from the first client, the at least one operator being provided by the second client;

the receiving module is used for receiving a data processing request for target data from the first client;

and the first processing module is used for calling the at least one operator to process the target data according to the ordering to obtain processed target data, and sending the processed target data to the first client.

9. The cloud management platform of claim 8, further comprising:

a second providing module for providing a registration interface for receiving information of at least one operator from the second client;

and the deployment module is used for deploying the at least one operator based on the information.

10. The cloud management platform of claim 9, further comprising:

The third providing module is used for providing an auditing interface, and the auditing interface is used for receiving an auditing passing result for the information from a third client;

the deployment module is used for deploying the at least one operator based on the auditing passing result and the information.

11. The cloud management platform of claim 10, further comprising:

a fourth providing module, configured to provide a test interface, where the test interface is configured to receive test data from a third client;

and the second processing module is used for calling the at least one operator to process the test data, obtaining processed test data and sending the processed test data to the third client.

12. The cloud management platform of any of claims 8 to 11, wherein the target data is stored in an object bucket corresponding to the first client, the cloud management platform further comprising:

and the storage module is used for storing the processed target data in the object storage bucket.

13. The cloud management platform of any of claims 8 to 12, wherein the data processing requirements comprise at least one of: the method comprises the steps of determining the type of data required to be processed by the first client, the type of processing required to be realized by the first client on the data, the data amount required to be processed by the first client and the service level agreement which the first client accords with.

14. The cloud management platform of any of claims 8 to 13, wherein the information comprises at least one of: attribute information of the at least one operator, code of the at least one operator, instructions for use of the at least one operator, service level agreements to which the at least one operator conforms, open source information of the at least one operator and test reports of the at least one operator.

15. A cloud management platform, wherein the cloud management platform comprises a memory and a processor; the memory stores code, the processor being configured to execute the code, the cloud management platform performing the method of any of claims 1 to 7 when the code is executed.

16. A computer storage medium storing one or more instructions which, when executed by one or more computers, cause the one or more computers to carry out the method of any one of claims 1 to 7.

17. A computer program product, characterized in that it stores instructions that, when executed by a computer, cause the computer to implement the method of any one of claims 1 to 7.