CN110908774A

CN110908774A - Resource scheduling method, device, system and storage medium

Info

Publication number: CN110908774A
Application number: CN201811075246.XA
Authority: CN
Inventors: 刘敬良
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2018-09-14
Filing date: 2018-09-14
Publication date: 2020-03-24
Anticipated expiration: 2038-09-14
Also published as: CN110908774B

Abstract

The embodiment of the application provides a resource scheduling method, equipment, a system and a storage medium, wherein the system comprises: a control platform and a container scheduling center; the management and control platform is used for receiving a service request of a user, generating a resource scheduling command for the service request and sending the resource scheduling command to the container scheduling center, wherein the resource scheduling command comprises a resource requirement corresponding to the service request; the container scheduling center is configured to receive the resource scheduling command, and dynamically acquire a target container meeting the resource demand for the service request based on a currently allocable resource, so as to provide a data service for the service request by using a service engine deployed in the target container. According to the embodiment of the application, the resources for providing the data service can be dynamically allocated according to the change of the resource requirement of the service request, and the monopolization of the resources is avoided.

Description

Resource scheduling method, device, system and storage medium

Technical Field

The present application relates to the field of data processing technologies, and in particular, to a resource scheduling method, device, system, and storage medium.

Background

In some business scenarios, private data of a user may be involved, and it is necessary to ensure the security of the user data. Based on the privacy of the user data, the providers of relevant data services need to provide independent resources for the user's data service requests, which results in the data services for each user requiring resource monopolization, while the data services typically require a large loss of resources, which affects the resource utilization of the data service providers.

Disclosure of Invention

Aspects of the present application provide a resource scheduling method, device, system, and storage medium, to solve the technical problem that an existing data service requires exclusive resource.

An embodiment of the present application provides a resource scheduling system, including: a control platform and a container scheduling center;

the management and control platform is used for receiving a service request of a user, generating a resource scheduling command for the service request and sending the resource scheduling command to the container scheduling center, wherein the resource scheduling command comprises a resource requirement corresponding to the service request;

the container scheduling center is configured to receive the resource scheduling command, and dynamically acquire a target container meeting the resource demand for the service request based on a currently allocable resource, so as to provide a data service for the service request by using a service engine deployed in the target container.

The embodiment of the present application further provides a resource scheduling method, including:

receiving a service request of a user, and generating a resource scheduling command for the service request;

sending the resource scheduling command to a container scheduling center, so that the container scheduling center dynamically acquires a target container meeting the resource demand for the service request based on the currently allocable resource, and provides data service for the service request by using a service engine deployed in the target container;

and the resource scheduling command comprises a resource requirement corresponding to the service request.

receiving a resource scheduling command sent by a control platform, wherein the resource scheduling command comprises a resource requirement corresponding to a service request of a user;

and dynamically acquiring a target container meeting the resource requirement for the service request based on the currently allocable resource so as to provide data service for the service request by utilizing a service engine deployed in the target container.

recording the specification information of a target container of a service request distributed to a user by a container scheduling center;

and monitoring the use condition of resources, and when the allocable resources meet preset requirements, acquiring a target container meeting the resource requirements for the service request again according to the recorded specification information of the target container allocated to the service request, so as to restart the data service corresponding to the service request by using a service engine deployed in the target container.

The embodiment of the application also provides a management and control device, which comprises a memory, a processor and a communication component;

the memory to store one or more computer instructions;

the processor, coupled with the memory and the communication component, to execute one or more computer instructions to:

receiving a service request of a user through the communication assembly, and generating a resource scheduling command for the service request;

sending the resource scheduling command to a container scheduling center through the communication component, so that the container scheduling center dynamically acquires a target container meeting the resource requirement for the service request based on the currently allocable resource, and providing data service for the service request by using a service engine deployed in the target container;

The embodiment of the application also provides container scheduling equipment, which comprises a memory, a processor and a communication component;

the memory to store one or more computer instructions;

receiving a resource scheduling command sent by a management and control platform through the communication assembly, wherein the resource scheduling command comprises a resource requirement corresponding to a service request of a user;

The embodiment of the application also provides a container management server, which comprises a memory, a processor and a communication component;

the memory to store one or more computer instructions;

Embodiments of the present application also provide a computer-readable storage medium storing computer instructions, which, when executed by one or more processors, cause the one or more processors to execute the aforementioned resource scheduling method executed by the management and control device.

Embodiments of the present application also provide a computer-readable storage medium storing computer instructions, which, when executed by one or more processors, cause the one or more processors to execute the resource scheduling method performed by the foregoing container scheduling apparatus.

Embodiments of the present application also provide a computer-readable storage medium storing computer instructions, which, when executed by one or more processors, cause the one or more processors to execute the resource scheduling method performed by the foregoing container management server.

In the embodiment of the present application, resources for providing data services are managed in a container manner. Aiming at a service request of a user, a container scheduling center can dynamically acquire a target container meeting the resource requirement for the service request based on the currently allocable resource according to the resource requirement corresponding to the service request, so that the resource for providing data service can be dynamically allocated according to the change of the resource requirement of the service request, and the resource monopolization is avoided.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:

fig. 1 is a schematic structural diagram of a resource scheduling system according to an embodiment of the present application;

fig. 2 is a schematic structural diagram of another resource scheduling system according to an embodiment of the present application;

fig. 3 is a flowchart illustrating a resource scheduling method according to another embodiment of the present application;

fig. 4 is a flowchart illustrating a resource scheduling method according to another embodiment of the present application;

fig. 5 is a flowchart illustrating a resource scheduling method according to another embodiment of the present application;

fig. 6 is a schematic structural diagram of a management device according to another embodiment of the present application;

fig. 7 is a schematic structural diagram of a container scheduling apparatus according to another embodiment of the present application;

fig. 8 is a schematic structural diagram of a container management server according to yet another embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be described in detail and completely with reference to the following specific embodiments of the present application and the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

In the prior art, based on the privacy of user data, providers of related data services need to provide independent resources for data service requests of users, which results in that data services for each user need resource monopolization, and data services usually need to consume a large amount of resources, which affects the resource utilization rate of data service providers. To address the problems with the prior art, in some embodiments of the present application: resources for providing data services are managed in a container manner. Aiming at a service request of a user, a container scheduling center can dynamically acquire a target container meeting the resource requirement for the service request based on the currently allocable resource according to the resource requirement corresponding to the service request, so that the resource for providing data service can be dynamically allocated according to the change of the resource requirement of the service request, and the resource monopolization is avoided.

The technical solutions provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings.

Fig. 1 is a schematic structural diagram of a resource scheduling system according to an exemplary embodiment of the present application. As shown in fig. 1, the resource scheduling system includes: a regulatory platform 10 and a container dispatch center 20.

The management and control platform 10 is configured to receive a service request of a user, generate a resource scheduling command for the service request, and send the resource scheduling command to the container scheduling center 20, where the resource scheduling command includes a resource requirement corresponding to the service request;

and the container scheduling center 20 is configured to receive the resource scheduling command, and dynamically acquire a target container 60 meeting the resource requirement for the service request based on the currently allocable resource, so as to provide a data service for the service request by using the service engine 50 deployed in the target container 60.

The resource scheduling system provided in this embodiment is suitable for various data service scenarios, for example, a data discovery scenario, a data consistency detection scenario, and the like, which is not limited in this embodiment. The embodiment is particularly suitable for data service scenes with large resource usage and/or long resource monopolizing time, can effectively solve the problem of resource monopolizing, and improves the resource utilization rate.

The resource scheduling system provided by this embodiment may be applied to various service architectures, such as an OSS platform, an ODPS platform, or an RDS platform. The user data of the user can be stored in the OSS platform, the ODPS platform, or the RDS platform, and in this embodiment, data services can be provided for the user data stored in various platforms through a service engine in a dynamically acquired target container.

The user in this embodiment refers to a user of the data service, that is, a company or an individual who requests the data service from a provider of the data service. In this embodiment, the user may submit a data service provisioning application to the data service provider, and after the data service provider configures the data service usage right for the user, the user may send a service request to the management and control platform 10. The service request of the user may carry information such as the identity of the user, the type of the requested service, and the like, so that the management and control platform 10 can recognize the request intention of the user.

After receiving a service request of a user, the management and control platform 10 may determine a resource requirement corresponding to the service request, and generate a resource scheduling command for the service request according to the resource requirement. In this embodiment, the resource refers to a resource supporting a data service, such as a CPU resource, a network resource, and the like, and correspondingly, the resource requirement refers to a resource specification, such as a CPU usage amount, a network bandwidth amount, and the like, required for processing the service request. After generating the resource scheduling command, the management and control platform 10 may issue the resource scheduling command to the container scheduling center 20.

For the container scheduling center 20, when receiving the resource scheduling command, the resource requirement carried in the resource scheduling command to the service request can be analyzed. The container scheduling center 20 manages the resources in a container manner, and based on this, the container scheduling center 20 can dynamically obtain the target container 60 meeting the resource demand for the service request based on the currently allocable resources according to the resource demand. The target container 60 may be one or more containers according to the resource requirement and the container specification. For example, when the data amount of the user data of the user is larger, the corresponding resource requirement is also larger, and the container scheduling center 20 may allocate a plurality of target containers 60 for the service request of the user to meet the resource requirement of the service request.

In this embodiment, when the resource requirement of the user changes, the container scheduling center 20 can dynamically allocate the resource managed by the user. For example, when the user data of the user decreases, the container scheduling center 20 may dynamically decrease the number of containers corresponding to the service request of the user, and the resource corresponding to the vacated container will not process the service request of the user any more, that is, will not be monopolized by the user any more. For another example, when a service request of a user is suspended, the container scheduling center 20 may release all containers 60 corresponding to the service request, and resources corresponding to the containers may be used for other data services as allocable resources.

The currently allocable resource refers to a resource that is not occupied currently, and the currently allocable resource may be a resource that is not defined in any container, or a resource that is defined in a container but the container where the resource is located is not occupied, which is not limited in this embodiment.

In this embodiment, a service engine 50 is deployed in each container 60, and the service engine 50 is used to provide data services. When the container scheduling center 20 determines that the service request of the user corresponds to the target container 60, the service engine 50 in the target container 60 will provide data service for the service request.

In this embodiment, to ensure the security of the user data, when the container scheduling center 20 performs container scheduling, the service request of a single user is scheduled to an integer number of containers 60, that is, the service requests of different users are not processed by the same container 60, which realizes user data isolation between different users and can effectively ensure the security and privacy of the user data.

In the present embodiment, resources for providing data services are managed in a container manner. Aiming at the service request of the user, the container scheduling center can dynamically acquire the target container meeting the resource requirement for the service request based on the currently allocable resource according to the resource requirement corresponding to the service request, so that the resource for providing the data service can be dynamically allocated according to the change of the resource requirement of the service request, and the resource monopolization is avoided.

In the above or below embodiments, the container scheduling center 20 may perform container creation, container release, container deployment, etc. operations on the resources it manages. The container scheduling center 20 can implement the dynamic acquisition of the target container 60 in at least the following two implementations.

In one implementation, the container scheduling center 20 may select a target container 60 for the service request that satisfies the resource requirements from among the containers 60 that have been created on the currently allocable resources.

In this implementation, the container scheduling center 20 may create a plurality of containers 60 in advance based on the resources it manages, and upon receiving the resource scheduling command, the container scheduling center 20 may select a target container 60 that meets the resource requirement from a type of container 60 that is not currently occupied from the plurality of containers 60. For example, the container scheduling center 20 may create 100 containers in advance based on the resources it manages, and when a resource scheduling command is received, 30 containers are currently occupied, and the container scheduling center 20 may select a target container 60 for the service request of the current user from the remaining 70 containers. According to the resource requirement carried in the resource scheduling command, the container scheduling center 20 may select 3 target containers 60 from the remaining 70 containers, and the service engine 50 deployed in the 3 target containers 60 may provide data service for the service request.

In another implementation, the container scheduling center 20 may create a target container 60 that satisfies the resource requirements for the service request on the currently allocable resources and deploy the service engine 50 in the created target container 60.

In this implementation, the container scheduling center 20 may create at least one target container 60 according to the resource requirement carried in the resource scheduling command when receiving the resource scheduling command, and deploy the service engine 50 in each target container 60 respectively. The container scheduling center 20 may determine the specification information of the target container 60 required by the current service request according to the resource requirement corresponding to the service request of the current user, and create the target container 60 according to the specification information of the target container 60. For example, the amount of user data requested to be processed by the service request of the current user is 2T, the container scheduling center 20 may determine, according to the resource requirements of the user data, that the number of target containers 60 required by the current service request is 2, and each target container 60 needs to have a specification capable of processing 1T data amount, and accordingly, the container scheduling center 20 may create 2 target containers 60 capable of respectively processing 1T data amount for the current service request.

In this embodiment, the specifications of the containers 60 managed by the container scheduling center 20 may not be identical, that is, the specifications of the containers 60 may be flexibly configured. Accordingly, when there are a plurality of target containers 60 corresponding to the service request of the user, the specifications of the plurality of target containers 60 may not be completely the same. This effectively improves the flexibility of container management, so that the container scheduling center 20 can allocate resources according to the actual resource requirement of the service request, and avoid resource idleness.

Fig. 2 is a schematic structural diagram of another resource scheduling system according to an embodiment of the present application. As shown in fig. 2, the number of the container scheduling centers 20 in the resource scheduling system may be multiple, and each container scheduling center 20 manages resources in a geographic area in a container 60 manner. For example, the leftmost container scheduling center 20 in fig. 2 is used to manage the resources in the a city as a whole, and the container scheduling center 20 may perform operations such as creating, releasing, scheduling, etc. of the container 60 for the resources in the a city.

In view of the large number of geographical areas in which user data are distributed, in this embodiment, an optimization scheme for providing resources for user data in the near vicinity according to the geographical areas to implement data services is proposed. The management and control platform 10 may determine a distribution geographic area of the user data according to the service request, and send the resource scheduling commands to the container scheduling centers 20 corresponding to the distribution geographic area of the user data, respectively.

There may be multiple geographical areas where the user data may be distributed, for example, a portion of the user data of user a is stored in the OSS storage space of city a, a portion of the user data is stored in the OSS storage space of city B, and a portion of the user data is stored in the OSS storage space of the guangzhou region. In this way, the management and control platform 10 may issue the resource scheduling command to the nearby container scheduling center 20 for the user data in different distributed geographic areas. For example, for the user data of the user stored in city a, the management and control platform 10 may issue a resource scheduling command to the container scheduling center 20 for managing the resources of city a. Accordingly, the container scheduling center 20 for managing resources of city a can schedule the resources of city a to the service request of the user in the near so as to provide data service for the user data stored in city a by the user using the resources of city a, so as to implement the service request of the user in the near so processing.

In this embodiment, in order to obtain a distribution geographic area of user data, the management and control platform 10 may send an access request for the user data to the data authorization center according to a service request of a user; after the authorization key is obtained, reading attribute information corresponding to the user data; and acquiring the distribution geographical area of the user data from the attribute information corresponding to the user data.

In consideration of the security of the user data, the management and control platform 10 needs to obtain access authorization for the user data and then perform access to the user data. In a practical application, in order to avoid leakage of user identity information of a user, the management and control platform 10 may apply a temporary access right to the data authorization center, if the application is successful, the management and control platform 10 may obtain information, such as a temporary ID, a temporary password, and a security token, used for accessing user data as an authorization key, the management and control platform 10 may temporarily access the user data through the authorization key, and the management and control platform 10 may read attribute information corresponding to the user data and obtain a distribution geographical location of the user data therefrom. The attribute information corresponding to the user data is information for describing an attribute of the user data, and is usually stored in a storage space where the user data is located. For example, the attribute may include a data amount of the user data, a distribution geographical area, a name of a storage space where the user data is located, and the like, which is not limited by the embodiment.

In this embodiment, the resource scheduling command is allocated to the nearby container scheduling center 20 according to the geographic area in which the user data is distributed, and in this way, resource allocation is performed from the dimension of the geographic area to provide nearby resources for the service request of the user, so that the speed of data service can be effectively increased.

In the above or below embodiments, as shown in fig. 2, the resource allocation system may further include container management servers 30, each container management server 30 is deployed in a geographic area and is used for managing several resource servers 40 in its corresponding geographic area, and several resource servers 40 are used for providing resources for data services.

In the embodiment, considering that the execution frequency of the data service may not be full time domain, an optimization scheme for providing the data service in a time sharing manner and automatically recovering resources during the suspension period of the data service is provided. For example, for a data discovery service, since the frequency of change of user data is small, a next data discovery service may be executed after a preset time interval after one data discovery service is completed, and resources occupied by the data discovery service are idle during the interval between two data discovery services. In the embodiment, the idle resources can be repeatedly utilized by time-sharing use of the resources.

The management and control platform 10 may set a time-sharing interval for providing data service according to service requirements in different scenarios. The management and control platform 10 may restart the data service at regular time according to a preset time-sharing interval. The management and control platform 10 may take the restart data service as a new service request to re-execute the resource scheduling operation. The management and control platform 10 may generate a resource scheduling command for the new service request, and issue the resource scheduling command to the container scheduling center 20, and the container scheduling center 20 may dynamically acquire the target container 60 again for the service request according to the resource scheduling command.

The resource demand of the service request of the user is changed due to the repeated restarting of the data service, and the resource demand of the service request is changed to 'none' during the suspension period of the service request, so that the container scheduling center 20 can release the occupation of the service request on the target container 60 during the suspension period of the service request, and uses the resource corresponding to the target container 60 for other purposes, thereby avoiding the resource idling and effectively improving the resource utilization rate.

Because of the persistence of data services, service requests may be restarted a relatively large number of times, which may increase the operating pressure of the governing platform 10 and the container scheduling center 20. In view of this, in the present embodiment, the specification information of the target container allocated to the service request by the container scheduling center may be recorded by the container management server 30; and monitoring the resource use condition in the local geographic area, and when the allocable resource in the local geographic area meets the preset requirement, acquiring a target container meeting the resource requirement for the service request again according to the recorded specification information of the target container allocated to the service request, so as to restart the data service corresponding to the service request by using a service engine deployed in the target container.

The specification information of the target container allocated to the service request by the container scheduling center 20 is determined according to the resource requirement thereof, and therefore, the container management server 30 can re-acquire the target container 60 satisfying the resource requirement thereof for the service request according to the history of the specification information of the target container required for the service request. The container management server 30 may obtain specification information of each container 60 in its local geographic area, restart the data service for the service request when there are more allocable resources in its local geographic area, and obtain the target container 60 corresponding to the service request again.

The container management server 30 may use at least the following two implementations to obtain the target container 60 satisfying the resource requirement for the service request again.

In one implementation, the container management server 30 may reselect a container 60 of a corresponding size from among existing containers 60 of a geographic area according to the recorded size information of the target container 60.

In this implementation, a plurality of containers 60 are created in advance in the geographic area where the container management server 30 is located, the containers 60 may be created in advance by the container scheduling center 20, and the specifications of the containers 60 may be the same or may not be completely the same. When the specifications of the containers 60 are the same, the container management server 30 may record the number of target containers 60 required by the service request in the geographic area where the container management server is located, and when the service request is restarted, reselect a corresponding number of target containers 60 to perform the data service according to the recorded number of target containers 60 required by the service request. When the sizes of the containers 60 are not completely the same, the container management server 30 may record the total size of the target container 60 required for the service request in the geographic area where the container management server is located and the respective sizes of the managed containers 60, and accordingly select one or more containers 60 that meet the total size requirement of the target container 60 of the restarted service request to perform the data service. Of course, the container management server 30 may also select the target container 60 in other manners, for example, randomly or in order of processing capacity, which is not limited in this embodiment.

In another implementation, the container management server 30 may recreate the container 60 of the corresponding specification from the recorded specification information of the target container 60 in the allocable resources in the geographic area.

In this implementation, the container management server 30 may create at least one target container 60 according to the specification information of the target container 60 required by each service request in its local management area, and deploy the service engine 50 in each target container 60. For example, the container management server 30 records that the specification information of the target container 60 corresponding to the currently restarted service request is a specification that needs to be provided with a specification capable of processing 2T data volume, and accordingly, the container management server 30 may create 2 target containers 60 capable of processing 1T data volume for the currently restarted service request, and of course, the container management server 30 may create other numbers of target containers 60 with other specifications, which is not limited in this embodiment.

In this embodiment, after the resource allocation of the geographic area dimension is completed through the management and control platform 10 and the resource allocation of the container 60 dimension in the geographic area is completed through the container scheduling center 20, the container management server 30 may store the configuration record of the resource allocation. When the data service is restarted, the container management server 30 may perform flexible control of resources based on the containers 60 in the managed geographic area according to the history of resource allocation without repeating the scheduling process executed by the management and control platform 10 and the container scheduling center 20. The working pressure of the management and control platform 10 and the container management center is effectively relieved.

In the above or following embodiments, when performing resource allocation in the container 60 dimension, if there are a plurality of target containers 60, the container scheduling center 20 may determine the maximum amount of data processed by each target container 60 according to the specification of each target container 60; and determines at least one data unit served by each target container 60 according to the maximum data amount processed by each target container 60, the number of data units included in the user data, and the data amount of each data unit.

In this embodiment, after obtaining the target containers 60 according to the resource requirements carried in the resource allocation command, if the number of the target containers 60 is multiple, the container scheduling center 20 may determine the user data condition processed by each target container 60. In this embodiment, the user data may include at least one data unit, and the unit of the data unit may be different according to the storage form of the user data in different types of storage spaces. For example, in the OSS storage space, the user data is stored in units of buckets (buckets), and the user data may include at least one bucket, and the data amount of each bucket may not be completely the same.

The container scheduling center 20 may determine the maximum amount of data that the target container 60 can handle based on the specification of the target container 60. For example, the maximum data amount that can be processed by the target container 60A can be determined to be 2G according to the specifications of CPU resources, network resources, and the like in the target container 60A. The container scheduling center 20 may further determine at least one data unit served by the target container 60 according to the maximum data amount processed by the target container 60, the number of data units included in the user data, and the data amount of each data unit. Bearing the above example, when the total amount of user data of the user is 2T, wherein 8G of user data is stored in the OSS storage space of the city a, the 8G data includes three buckets, the data volume of the first bucket is 4G, the data volume of the second bucket is 3G, and the data volume of the third bucket is 1G, the container scheduling center 20 for managing resources of the city a allocates two containers 60 capable of processing 4G data volumes for the 8G data of the user. Moreover, the pod dispatching center 20 may dispatch a first pod into one pod 60 and a second and third pod into another pod 60.

Of course, this is only exemplary, and the container scheduling center 20 may also allocate two containers 60 capable of handling 7G data volume and 1G data volume for user data, allocate the first and second buckets into the container 60 capable of handling 7G data volume, and allocate the third bucket into the container 60 capable of handling 1G data volume. In the present embodiment, the specification of the target container 60 allocated by the container scheduling center 20, the pairing manner between the target container 60 and the data unit, and the like are not limited.

It should be noted that, in order to ensure the continuity of data, in the present embodiment, when the container scheduling center 20 allocates the service request, the number of data units corresponding to the service request processed by each target container 60 is an integer.

In this embodiment, based on the data amount of each data unit included in the user data, the container scheduling center 20 may allocate the service request for each data unit included in the user data to each target container 60, so as to improve the full load rate of the target container 60, and further reduce the resources occupied by each user on the premise of ensuring that the resource demand of the data service can be satisfied, and therefore, the resource utilization rate may be further improved.

Based on the at least one data unit serviced by the target container 60, the service engine 50 in the target container 60 may create at least one thread and configure the amount of data processed by each thread; at least one data unit serviced by the target container 60 is assigned to each thread according to the amount of data handled by each thread to service the at least one data unit by each thread.

For example, the target container 60 is used to process a bucket of user a, where the bucket includes 3000 data segments, and the service engine 50 may configure the number of data segments processed by each thread in a main thread, e.g., each thread processes 1000 data segments, the service engine 50 may create 3 threads, each thread is used to process 1000 data segments in the bucket.

In this embodiment, inside the container 60, the service engine 50 performs parallel deployment on the threads running in the container 60, so that resource deployment of the thread dimension in the container 60 is realized, and the resource utilization efficiency in the container 60 is further optimized.

In the above or below embodiments, in order to ensure the security of the user data, after completing resource deployment, the service engine 50 in the target container 60 may send an access request for the user data of the user to the management and control platform 10 based on the user identifier of the user; and after receiving the authorization key returned by the management and control platform 10, accessing the user data according to the authorization key.

In this embodiment, when receiving an access request for user data of a user sent by the service engine 50 in the target container 60, the management and control platform 10 may request an authorization key of the user from the authorization center, and return the authorization key to the service engine 50.

In an actual application, the service engine 50 may apply for temporary access to the user data from the management and control platform 10 according to a service request of a user, the management and control platform 10 may apply for a temporary access right from the data authorization center, and if the application is successful, the management and control platform 10 may obtain information, such as a temporary ID, a temporary password, and a security token, for accessing the user data as an authorization key, and return the authorization key to the service engine 50. The service engine 50 may initiate data access to the storage space of the user data using the information such as the temporary ID, the temporary password, and the security token as an authorization key, thereby successfully executing the data service.

In this embodiment, the temporary identity is allocated to the service engine 50 in a temporary authorization manner, so that leakage of real identity information of the user is avoided, and the service engine 50 can successfully access the user data after obtaining the temporary authorization, thereby ensuring security of the user data.

In the above or below embodiments, the container scheduling center 20 may release the target container 60 after the data service on the target container 60 is finished, so as to recreate a new container 60 on the resources released by the target container 60.

In this embodiment, after the data service on the target container 60 is finished, the corresponding resources of the target container 60 will be in an idle state, the container scheduling center 20 may release the target container 60 to recover the resources released by the target container 60, and the resources released by the target container 60 will be used as allocable resources for subsequent resource allocation by the container management center, where the resources released by the target container 60 may be used to create a new container 60, so that the resources released by the target container 60 may be recovered and may be used by other service requests.

Therefore, the container management center can automatically recycle the serviceable resources, effectively avoid the idle resources, save the occupied resources and improve the utilization rate of the resources.

In the above or following embodiments, the stability problems of active/standby, disaster recovery, etc. can also be solved by the container scheduling center 20. The container scheduling center 20 may forward the resource scheduling command to the container scheduling centers 20 corresponding to other geographic areas when the allocable resources of the geographic area corresponding to the container scheduling center cannot meet the resource requirements of the user data distributed in the geographic area corresponding to the container scheduling center.

In this embodiment, after receiving the resource allocation command, the container scheduling center 20 may check the allocable condition of the resource in the geographic area managed by the container scheduling center 20, and when the allocable resource managed by the container scheduling center 20 cannot meet the resource requirement of the service request, the container scheduling center 20 may forward the resource scheduling command for the service request to the container scheduling centers 20 corresponding to other geographic areas. For example, a part of user data of user a is stored in the OSS storage space of city a, and due to a disaster tolerance problem, the allocable resource of city a cannot meet the service request of user a, and the container scheduling center 20 managing the resource of city a may forward the resource scheduling command to the container scheduling center 20 managing the resource of city B, and the container scheduling center 20 allocates the resource for the service request of user a from the allocable resource of city B.

Therefore, in this embodiment, the stability problems of the active/standby system, the disaster recovery system, and the like can be solved by forwarding the resource allocation command.

In the above or following embodiments, the container 60 allocation center may also monitor the operation status of the target container 60, and when an abnormal container 60 occurs, release the abnormal container 60; a new container 60 is created to replace the anomalous container 60.

In this embodiment, the container 60 deployment center may also continuously monitor the operation state of the target container 60 after acquiring the target container 60 for the service request of the user, and when the target container 60 has abnormal situations such as disaster tolerance, the container scheduling center 20 may release the abnormal container 60, and the resources released by the abnormal container 60 may be used for other purposes. Container dispatch center 20 may also create a new container 60 to replace the exception container 60 to ensure proper execution of the data service.

In this embodiment, the sequence of the above processes of releasing the container 60 with the abnormal condition and creating a new container 60 is not limited in this embodiment, and in practical application, the two processes are fast, and the difference caused by the execution sequence is negligible.

In this embodiment, the container 60 deployment center can find the abnormal operation state of the target container 60 in time, and can create a new container 60 in time to replace the abnormal container 60, thereby ensuring the stability of the data service.

Fig. 3 is a flowchart illustrating a resource scheduling method according to another embodiment of the present application. As shown in fig. 3, the method includes:

300. receiving a service request of a user, and generating a resource scheduling command for the service request;

301. sending the resource scheduling command to a container scheduling center, so that the container scheduling center dynamically acquires a target container meeting the resource requirement for a service request based on the currently allocable resource, and providing data service for the service request by using a service engine deployed in the target container;

The resource scheduling method provided by the embodiment is suitable for a control platform in a resource scheduling system.

In an alternative embodiment, step 301 further comprises:

and determining the distribution geographical area of the user data according to the service request, and respectively sending the resource scheduling command to a container scheduling center corresponding to the distribution geographical area of the user data.

In an alternative embodiment, the step of determining the geographical area of distribution of the user data based on the service request comprises:

according to the service request, sending an access request aiming at user data to a data authorization center;

after the authorization key is obtained, reading attribute information corresponding to the user data;

and acquiring the distribution geographical area of the user data from the attribute information corresponding to the user data.

In an optional embodiment, the method further comprises:

receiving an access request for user data of a user sent by a service engine in a target container;

requesting the authorization key of the user from the authorization center and returning the authorization key to the service engine.

Fig. 4 is a flowchart illustrating a resource allocation method according to another embodiment of the present application. As shown in fig. 4, the method includes:

400. receiving a resource scheduling command sent by a control platform, wherein the resource scheduling command comprises a resource requirement corresponding to a service request of a user;

401. and dynamically acquiring a target container meeting the resource requirement for the service request based on the currently allocable resource so as to provide data service for the service request by utilizing a service engine deployed in the target container.

The resource scheduling method provided by the embodiment is suitable for a container scheduling center in a resource scheduling system.

In an optional embodiment, the number of the container scheduling centers is multiple, and each container scheduling center manages resources in one geographic area in a container mode.

In an alternative embodiment, step 401 comprises:

selecting a target container meeting the resource requirement for the service request from containers established on the currently allocable resources; or

A target container satisfying the resource requirements is created for the service request on the currently allocable resources, and a service engine is deployed in the created target container.

In an optional embodiment, the method further comprises:

and after the data service is finished, releasing the target container so as to recreate a new container on the resources released by the target container.

In an optional embodiment, the method further comprises:

and when the allocable resources in the current geographic area cannot meet the resource requirement of the distributed user data, forwarding the resource scheduling command to container scheduling centers corresponding to other geographic areas.

In an optional embodiment, the method further comprises:

when the target container is multiple, determining the maximum data volume processed by each target container according to the specification of each target container;

and determining at least one data unit served by each target container according to the maximum data volume processed by each target container, the number of data units contained in the user data and the data volume of each data unit.

In an optional embodiment, the method further comprises:

monitoring the running state of the target container, and releasing the abnormal container when the abnormal container appears;

a new container is created to replace the container that is anomalous.

Fig. 5 is a flowchart illustrating a resource scheduling method according to another embodiment of the present application. As shown in fig. 5, the method includes:

500. recording the specification information of a target container of a service request distributed to a user by a container scheduling center;

501. and monitoring the use condition of the resources, and when the allocable resources meet the preset requirement, acquiring the target container meeting the resource requirement for the service request again according to the recorded specification information of the target container allocated to the service request so as to restart the data service corresponding to the service request by using the service engine deployed in the target container.

The resource scheduling method provided by the embodiment is suitable for a container management server in a resource scheduling system.

In an alternative embodiment, step 501 comprises:

in the existing container, reselecting a container with a corresponding specification according to the recorded specification information of the target container; or

In the allocable resource, the container with the corresponding specification is recreated according to the recorded specification information of the target container.

Fig. 6 is a schematic structural diagram of a management device according to another embodiment of the present application. As shown in fig. 6, the management device includes a memory 60, a processor 61, and a communication component 62.

A processor 61, coupled to the memory 60, for executing computer programs in the memory 60 for:

receiving a service request from a user via the communication component 62, and generating a resource scheduling command for the service request;

sending the resource scheduling command to the container scheduling center through the communication component 62, so that the container scheduling center dynamically obtains a target container meeting the resource requirement for the service request based on the currently allocable resource, and provides data service for the service request by using a service engine deployed in the target container;

In an alternative embodiment, the processor 61, when sending the resource scheduling command to the container scheduling center through the communication component 62, is configured to:

In an alternative embodiment, the processor 61 is configured to determine the geographical area of distribution of the user data according to the service request, and is configured to:

In an alternative embodiment, the processor 61 is further configured to:

Further, as shown in fig. 6, the data processing apparatus may further include: display 63, power components 64, audio components 65, and the like. Only some of the components are schematically shown in fig. 6, and it is not intended that the regulating apparatus includes only the components shown in fig. 6.

Accordingly, embodiments of the present application also provide a computer-readable storage medium storing computer instructions, which, when executed by one or more processors, cause the one or more processors to execute the aforementioned resource scheduling method performed by the management and control device.

Fig. 7 is a schematic structural diagram of a container scheduling apparatus according to another embodiment of the present application. As shown in fig. 7, the container scheduling apparatus includes: a memory 70, a processor 71 and a communication component 72;

a memory 70 for storing one or more computer instructions;

a processor 71, coupled to the memory 70 and the communication component 72, for executing one or more computer instructions for:

receiving a resource scheduling command sent by the management and control platform through the communication component 72, wherein the resource scheduling command includes a resource requirement corresponding to a service request of a user;

In an alternative embodiment, the processor 71, when dynamically acquiring for a service request a target container satisfying a resource requirement based on currently allocable resources, is configured to:

In an alternative embodiment, processor 71 is further configured to:

a new container is created to replace the container that is anomalous.

Further, as shown in fig. 7, the data processing apparatus may further include: a display 73, power components 74, audio components 75, and the like. Only some of the components are schematically shown in fig. 7, and it is not meant that the container scheduling apparatus includes only the components shown in fig. 7.

Accordingly, embodiments of the present application also provide a computer-readable storage medium storing computer instructions, which, when executed by one or more processors, cause the one or more processors to execute the aforementioned resource scheduling method performed by the container scheduling apparatus.

Fig. 8 is a schematic structural diagram of a container management server according to yet another embodiment of the present application. As shown in fig. 8, the container management server includes: memory 80, processor 81, and communications component 82;

a memory 80 for storing one or more computer instructions;

a processor 81, coupled to the memory 80 and the communication component 82, for executing one or more computer instructions for:

and monitoring the use condition of the resources, and when the allocable resources meet the preset requirement, acquiring the target container meeting the resource requirement for the service request again according to the recorded specification information of the target container allocated to the service request so as to restart the data service corresponding to the service request by using the service engine deployed in the target container.

In an alternative embodiment, when the processor 81 acquires a target container satisfying the specification information for the service request again according to the specification information of the target container corresponding to the recorded service request, the processor is configured to:

Further, as shown in fig. 8, the data processing apparatus may further include: power supply components 83, and the like. Only some of the components are schematically shown in the figure, and it is not meant that the container management server includes only the components shown in fig. 8.

Accordingly, embodiments of the present application also provide a computer-readable storage medium storing computer instructions, which, when executed by one or more processors, cause the one or more processors to perform the aforementioned resource scheduling method performed by the container management server.

The memories of fig. 6, 7 and 8 are used to store computer programs and may be configured to store various other data to support the operations on the master node. Examples of such data include instructions for any application or method operating on the master node, contact data, phonebook data, messages, pictures, videos, and the like.

The memory may be implemented by any type or combination of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

The communication components of fig. 6, 7, and 8 may be configured to facilitate wired or wireless communication between the device in which the communication component is located and other devices. The device in which the communication component is located may access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

Among them, the displays in fig. 6 and 7 include screens, which may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

The power supply components of fig. 6, 7, and 8 provide power to various components of the device in which the power supply components are located. The power components may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device in which the power component is located.

The audio components in fig. 6 and 7 may be configured to output and/or input audio signals. For example, the audio component includes a Microphone (MIC) configured to receive an external audio signal when the device in which the audio component is located is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal may further be stored in a memory or transmitted via a communication component. In some embodiments, the audio assembly further comprises a speaker for outputting audio signals.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A resource scheduling system, comprising: a control platform and a container scheduling center;

2. The system of claim 1, wherein the number of the container scheduling centers is plural, and each container scheduling center manages resources in a geographic area in a container manner.

3. The system of claim 2, wherein the management and control platform is specifically configured to: and determining the distribution geographical area of the user data according to the service request, and respectively sending the resource scheduling command to a container scheduling center corresponding to the distribution geographical area of the user data.

4. The system of claim 3, wherein the management and control platform is specifically configured to:

according to the service request, sending an access request aiming at the user data to a data authorization center;

after obtaining the authorization key, reading attribute information corresponding to the user data;

5. The system of claim 2, wherein a container management server is further deployed in each geographic region; the container management server is configured to:

recording specification information of a target container allocated to the service request by the container scheduling center;

and monitoring the resource use condition in the local geographic area, and when the allocable resource in the local geographic area meets the preset requirement, acquiring a target container meeting the resource requirement for the service request again according to the recorded specification information of the target container allocated to the service request, so as to restart the data service corresponding to the service request by using a service engine deployed in the target container.

6. The system of claim 5, wherein the container management server, when retrieving the target container satisfying the resource requirement for the service request, is configured to:

in the existing containers in the geographic area, reselecting containers with corresponding specifications according to the recorded specification information of the target containers; or

And in the allocable resources in the geographic area, recreating containers with corresponding specifications according to the recorded specification information of the target containers.

7. The system of claim 1, wherein the container scheduling center is specifically configured to: selecting a target container meeting the resource requirement for the service request from containers which are created on the currently allocable resources; or

And creating a target container meeting the resource requirement for the service request on the currently allocable resource, and deploying a service engine in the created target container.

8. The system of claim 1, wherein if there are multiple target containers, the specifications of the multiple target containers are not identical.

9. The system of claim 1, wherein the container scheduling center is further configured to:

10. The system of claim 3, wherein the container scheduling center is specifically configured to:

and when the allocable resources of the corresponding geographic area can not meet the resource requirements of the user data distributed in the corresponding geographic area, forwarding the resource scheduling command to container scheduling centers corresponding to other geographic areas.

11. The system of claim 1, wherein the container scheduling center is further configured to:

12. The system of claim 11, wherein the server engine in the target container is further configured to:

creating at least one thread and configuring the data volume processed by each thread;

and distributing at least one data unit served by the target container to each thread according to the data volume processed by each thread, so as to serve the at least one data unit through each thread.

13. The system of claim 1, wherein before the service engine in the target container provides data services for the service request, the system is further configured to:

sending an access request aiming at user data of the user to the management and control platform based on the user identification of the user;

after receiving an authorization key returned by the management and control platform, accessing the user data according to the authorization key;

the management and control platform is also used for;

receiving an access request sent by a service engine in the target container for user data of the user;

requesting an authorization key for the user from an authorization center and returning the authorization key to the service engine.

14. The system of claim 1, wherein the container scheduling center is further configured to:

monitoring the running state of a target container, and releasing an abnormal container when the abnormal container occurs;

a new container is created to replace the container of the exception.

15. A method for scheduling resources, comprising:

and sending the resource scheduling command to a container scheduling center, wherein the resource scheduling command comprises a resource requirement corresponding to the service request, so that the container scheduling center dynamically acquires a target container meeting the resource requirement for the service request based on the currently allocable resource, and provides data service for the service request by using a service engine deployed in the target container.

16. The method of claim 15, wherein the sending the resource scheduling command to a container scheduling center comprises:

17. A method for scheduling resources, comprising:

18. The method of claim 17, wherein the number of the container scheduling centers is multiple, and each container scheduling center manages resources in a geographic area in a container manner.

19. The method of claim 17, wherein dynamically acquiring a target container for the service request based on the currently allocable resource to meet the resource requirement comprises:

20. The method of claim 17, further comprising:

21. The method of claim 18, further comprising:

22. The method of claim 17, further comprising:

when the target containers are multiple, determining the maximum data volume processed by each target container according to the specification of each target container;

23. The method of claim 17, further comprising:

a new container is created to replace the container of the exception.

24. A method for scheduling resources, comprising:

25. The method of claim 24, wherein said retrieving the target container satisfying the resource requirement for the service request comprises:

selecting a target container meeting the resource requirement for the service request from containers which are created on the currently allocable resources; or

26. A management device comprising a memory, a processor, and a communication component;

the memory to store one or more computer instructions;

27. A container scheduling apparatus comprising a memory, a processor and a communication component;

the memory to store one or more computer instructions;

28. A container management server comprising a memory, a processor, and a communication component;

the memory to store one or more computer instructions;

29. A computer-readable storage medium storing computer instructions, which when executed by one or more processors, cause the one or more processors to perform the resource scheduling method of claim 15 or 16.

30. A computer-readable storage medium storing computer instructions, which when executed by one or more processors, cause the one or more processors to perform the method of resource scheduling of any one of claims 17 to 23.

31. A computer-readable storage medium storing computer instructions, which when executed by one or more processors, cause the one or more processors to perform the resource scheduling method of claim 24 or 25.