WO2024004179A1 - Resource sharing within cloud group formed from plurality of clouds - Google Patents

Abstract

In the present invention, a control device is provided with one or more processors. When creation of a new application is entrusted to a primary cloud, the one or more processors select, from among existing applications that are deployed by the primary cloud within a cloud group formed from a plurality of clouds, a subject application that is to be deployed by a secondary cloud within the cloud group, with tightening of a resource of the primary cloud being used as a trigger. The one or more processors select, from among containerized network functions (CNFs) that constitute the subject application, at least one subject CNF that is to be deployed by the secondary cloud within the cloud group. The one or more processors set the same IP addresses as virtual IP addresses for a port of the subject CNF that is deployed by the primary cloud and a port of the subject CNF that is deployed by the secondary cloud.

Description

Resource sharing within a cloud group formed by multiple clouds

The present disclosure relates to resource sharing within a cloud group formed by multiple clouds.

With the increase in demand for IoT (Internet of Things), VR (Virtual Reality), AR (Augmented Reality), video distribution applications, etc., edge computing, which is a distributed architecture, has been attracting attention in recent years. Edge computing is a distributed computing system that performs at least some of its processing in an edge cloud built on edge servers installed near user terminals (UEs), instead of a central cloud built on a central data center. It is ing. By using the edge cloud, the processing for providing services to user terminals, which was concentrated in the center cloud, is distributed to the edge cloud, thereby distributing the processing load.

Japanese Patent Application Publication No. 2017-143365

However, due to constraints such as where the edge server is installed, the computing resources allocated to the edge cloud may not be sufficient, and it may be difficult to stably provide services to user terminals.

A control device according to one aspect of the present disclosure includes one or more processors.
The one or more processors are deployed to the primary cloud when a request to create a new application is made to the primary cloud in a cloud group formed by multiple clouds, triggered by the fact that the resources of the primary cloud are tight. Select a target application to be deployed to the secondary cloud in the cloud group from existing applications in the cloud group.
Furthermore, the one or more processors select at least one target CNF to be deployed to a secondary cloud in the cloud group from among CNFs (Containerized Network Functions) that construct the target application.
The one or more processors also deploy the target CNF to the secondary cloud.
Furthermore, the one or more processors set the same IP address as a virtual IP address for a port of the target CNF deployed in the primary cloud and a port of the target CNF deployed in the secondary cloud. do.

A control method according to an aspect of the present disclosure is such that when a request is made to create a new application to a primary cloud in a cloud group formed by a plurality of clouds, the primary cloud is triggered by the fact that the resources of the primary cloud are tight. The method includes selecting, using a processor, a target application to be deployed to a secondary cloud in the cloud group from existing applications deployed to the cloud.
The control method also includes selecting, using a processor, at least one target CNF to be deployed to a secondary cloud in the cloud group from among CNFs (Containerized Network Functions) that construct the target application. .
Further, the control method includes deploying the target CNF to the secondary cloud using a processor.
The control method also includes using a processor to assign the same IP address to a port of the target CNF deployed in the primary cloud and a port of the target CNF deployed in the secondary cloud. This includes setting it as an IP address.

A non-transitory computer-readable medium according to one aspect of the present disclosure records a program.
When the program is read by a computer, one or more processors of the computer execute a request to create a new application to a primary cloud in a cloud group formed by a plurality of clouds, when the primary cloud's resources are tight. Using this as a trigger, a target application to be deployed to the secondary cloud in the cloud group is selected from the existing applications deployed to the primary cloud.
Furthermore, the one or more processors select at least one target CNF to be deployed to a secondary cloud in the cloud group from among CNFs (Containerized Network Functions) that construct the target application.
The one or more processors also deploy the target CNF to the secondary cloud.
Furthermore, the one or more processors set the same IP address as a virtual IP address for a port of the target CNF deployed in the primary cloud and a port of the target CNF deployed in the secondary cloud. do.

FIG. 1 is a conceptual diagram showing a cloud group to which a control device according to the present disclosure is applied. FIG. 2 is a diagram showing the relationship between services and applications in a microservice architecture. FIG. 3 is a functional block diagram illustrating a configuration example of a control device according to the present disclosure. FIG. 4A is a diagram for explaining an application example of the control device according to the present disclosure. FIG. 4B is a diagram for explaining an application example of the control device according to the present disclosure. FIG. 5A is a diagram illustrating an example of the correspondence between CNF, Port, and IP address. FIG. 5B is a diagram illustrating an example of the correspondence between CNF, Port, and IP address. FIG. 6 is a flowchart illustrating an example of the operation of the control device according to the present disclosure. FIG. 7 is a diagram illustrating an example of a processing sequence in the control device according to the present disclosure. FIG. 8 is a block diagram showing a configuration example of the control device shown in FIG. 3.

Embodiments of the present disclosure will be described in detail below with reference to the drawings.
A control device according to the present disclosure controls resource sharing within a cloud group formed by a plurality of clouds. As discussed below, the controller may be an orchestrator.

(One embodiment)
FIG. 1 is a conceptual diagram showing a cloud group to which a control device 100 according to the present disclosure is applied. The cloud group includes a plurality of (L pieces, L is an integer) edge clouds.
FIG. 1 shows an example in which a cloud group includes four (L=4) edge clouds.
Applications for providing various services to user terminals (not shown) within a coverage area formed by one or more Radio Units (RUs) may be deployed in each edge cloud. The user terminal is an ICT (Information and Communication Technology) device that can execute an application for receiving services and can communicate. Examples of user terminals include portable PCs (Personal Computers) such as smartphones and tablet terminals. The user terminal may be a non-portable device such as a desktop PC.

A cloud group is formed from multiple edge clouds. A cloud group is formed, for example, by edge clouds built on edge servers that are geographically close to each other. Note that a cloud group does not necessarily have to be formed only by edge clouds that are geographically close to each other. For example, a cloud group may be formed by edge clouds built on edge servers installed in logically close locations.
Further, the cloud group may include a center cloud built on the central data center.
Further, the cloud group may include a cloud built on a regional data center.
Hereinafter, the edge cloud, center cloud, etc. included in a cloud group will be simply referred to as "cloud".

Before describing the control device 100 according to the present disclosure, the relationship between services provided to user terminals and applications deployed in the cloud will be described using FIG. 2. The application is an application for providing a service to a user terminal.
In the present disclosure, services provided to user terminals are constructed using a microservice architecture.
The control device 100 according to the present disclosure may be installed in a data center where these clouds are constructed, or may be installed in a data center outside the cloud group.

FIG. 2 is a diagram showing the relationship between services and applications (hereinafter also referred to as "APP") in a microservice architecture.
As shown in FIG. 2, a service is constructed from one or more applications. The number of applications that construct one service varies depending on the service.
Each application is built from one or more Containerized Network Functions (CNFs).
CNF operates on "Kubernetes" (registered trademark), for example. Kubernetes is open source software (OSS) that automatically configures and manages containerized services.
The number of CNFs to build one application may vary depending on the application and design. FIG. 2 shows how the application APP#1 is constructed from n (n is an integer) CNF#11, CNF#12, . . . , CNF#1n.
CNF running on Kubernetes is constructed from one or more Pods. A Pod is the smallest unit of application that can run on Kubernetes. Multiple Pods are operationally managed by Kubernetes. Pods that operate on CNF have a self-repair function; for example, when a certain Pod stops operating, another Pod is activated and the CNF function self-repairs.
Each Pod is constructed from one or more containers.

The use of microservice architecture is also being considered in 5G (fifth generation mobile communication system).
Examples of the above-mentioned services in 5G include a radio access network (RAN) and a core network (CN).
When the service is RAN, the Central Unit (CU) and Distributed Unit (DU) correspond to the application.
Furthermore, if the service is a CN, a Service Network Function (SNF) that constructs the CN, such as an AMF (Access and Mobility Management Function) or a UPF (User Plane Function), corresponds to the application.

When the application indicated by APP #1 is a CU, the CNF indicated by CNF #11 can be set as a CU-CP (Control Plane), and the CNF indicated by CNF #12 can be set as a CU-UP (User Plane).
CU-CP includes radio resource control (RRC). CU-UP includes SDAP (Service Data Adaptation Protocol) and PDCP (Packet Data Convergence Protocol) related to the user plane.
The CU-CP and CU-UP are capable of functionally linked communication via the E1 interface. Both CU-CP and CU-UP can be connected to DU with F1 interface.

Furthermore, examples of services other than the above-mentioned RAN and CN include a web distribution service, a game provision service, a video distribution service, a music distribution service, a monitoring service, a navigation service, an automatic driving service, an email provision service, and a sensor service.
Examples of applications that provide these services include a web distribution application, a game provision application, a video distribution application, a music distribution application, a monitoring execution application, a navigation application, an automatic driving application, a mail provision application, and a sensor execution application.

FIG. 3 is a functional block diagram showing a configuration example of the control device 100 according to the present disclosure.
The control device 100 includes a resource monitoring section 110, a resource determination section 120, an application selection section 130, a secondary cloud determination section 140, a CNF selection section 150, a virtual IP address setting section 160, and a deployment section 170.

The resource monitoring unit 110 monitors resource usage of clouds within a cloud group. Monitoring may be performed periodically, or may be performed at the timing when a request to create a new application, which will be described later, is notified to one of the clouds.
Note that applications already deployed in the cloud and new applications are built using microservice architecture. That is, these applications are constructed from one or more CNFs.
The resource usage status includes, for example, information on the total number of CNFs that can be placed (deployed) in the cloud and the total number of CNFs of all applications actually deployed on the cloud. The total number of CNFs that can be placed in the cloud may be determined by the cloud's computing resources. The total number of CNFs for all applications actually deployed in the cloud is the total number of CNFs already deployed in the cloud.

The resource determination unit 120 determines whether the cloud has the resources to deploy all the CNFs that construct the new application when requesting the creation of a new application to the cloud in the cloud group. In other words, the resource determination unit 120 determines whether or not the resources of the cloud that has received the request to create a new application are tight. Hereinafter, the cloud that received the request to create a new application will be referred to as the "primary cloud." A method for determining whether or not resources are tight in the resource determination unit 120 will be described later.

The resource determination unit 120 receives, for example, directly from the primary cloud that the primary cloud has been requested to create a new application. The resource determination unit 120 may receive a request to create a new application in the primary cloud from a management device that manages registration of new applications.
The new application creation request includes information about the new application (hereinafter also referred to as "application information"). The application information includes information on the total number of CNFs for constructing new applications.

For example, the resource determination unit 120 can determine whether the primary cloud has resources for deploying all CNFs for building a new application, using the following information (1) and (2). can.
(1) Maximum number of CNFs that can be placed in the primary cloud: f _max
(2) Total number of CNFs requested to be placed in the primary cloud: f _sum
f _max and f _sum are counted in units of one CNF. That is, f _max and f _sum represent how many CNFs they correspond to.

f _max is the maximum number of CNFs that can be placed in the primary cloud. In other words, f _max is the sum of the number of CNFs already placed in the primary cloud and the number of CNFs that can be further placed in the primary cloud. The maximum number of CNFs that can be placed in the primary cloud may be determined by the primary cloud's computing resources.
Furthermore, f _sum is the sum of the number of CNFs that have already been placed in the primary cloud plus the number of CNFs that will construct a new application that has been requested to be placed in the primary cloud. In other words, the total number of CNFs f _sum is the total number of CNFs scheduled to be placed in the primary cloud when all CNFs for building a new application are placed (deployed) in the primary cloud.

For example, the resource determination unit 120 determines whether the primary cloud has resources for deploying all CNFs that construct a new application, based on the resource usage ratio R defined by the ratio f _sum /f _max . judge.

If the resource usage ratio R is less than 1, that is,

In this case, it is determined that the primary cloud has the resources to deploy all the CNFs that build the new application.

On the other hand, if the resource usage ratio R is greater than 1, that is,

In this case, the resource determination unit 120 determines that the primary cloud does not have the resources to deploy all the CNFs that construct the new application.
If the resource usage ratio R>1, the resource determination unit 120 notifies the application selection unit 130 of a request to select an application to be deployed in a cloud different from the primary cloud (hereinafter referred to as “secondary cloud”). In the following explanation, the application that will be deployed to the secondary cloud is also referred to as the "target application."

Note that even if the primary cloud has the resources to deploy all the CNFs for building a new application, some CNFs may be deployed to the secondary cloud. In other words, even if the resources of the primary cloud are not actually in short supply, the resource determining unit 120 may make a hypothetical determination that the resources are in short supply only in a preset situation. As a result, it is possible to deploy a part of the CNF to the secondary cloud in advance before a resource shortage occurs, thereby delaying the timing at which the primary cloud's resources become scarce. For example, the resource determination unit 120 hypothetically determines that resources are tight in a situation where it is expected that new application creation requests will continue. The situations in which it is determined that resources are tight are not limited to this.
As described above, the resource determination unit 120 determines whether or not the resources of the primary cloud are tight, and if the resources are tight, the resource determination unit 120 requests the application selection unit 130 to select an application to be deployed to the secondary cloud. Notice. Note that the method used by the resource determination unit 120 to determine whether or not resources are tight is not particularly limited.

When the application selection unit 130 receives a request to select an application (target application) to be deployed to the secondary cloud from the resource determination unit 120, the application selection unit 130 selects the target application from existing applications deployed in the primary cloud. The selection method is not particularly limited. For example, the application selection unit 130 may select a target application based on the operating status of existing applications.
The operating status includes at least one of the operating time of the application, the number of user terminals connected to the application, the amount of data of the application, and the throughput of the application. The application selection unit 130 can obtain information about the operating time by, for example, inquiring a management device that manages the cloud. The application selection unit 130 can obtain information about the operating time by, for example, inquiring a management device that manages the cloud. Further, the application selection unit 130 can obtain information about the number of user terminals connected to the application, the amount of data of the application, and the throughput of the application by, for example, inquiring the UPF.

When using the operating time of an application as the operating status, the application selection unit 130 selects the application with the shortest operating time as the target application. When using the number of user terminals connected to an application as the operating status, the application selection unit 130 selects the application with the least number of user terminals as the target application. When using the amount of data of an application as the operating status, the application selection unit 130 selects the application with the least amount of data as the target application. When using the throughput of an application as the operating status, the application selection unit 130 selects the application with the lowest throughput as the target application.

As a result, applications with low uptime are relocated to the secondary cloud, while applications with long uptime are maintained in the primary cloud. Further, applications with a small number of connected user terminals are relocated to the secondary cloud, and applications with a large number of connected user terminals are maintained in the primary cloud. Also, applications with a small amount of data are relocated to the secondary cloud, while applications with a large amount of data are kept in the primary cloud. Also, applications with low throughput are relocated to the secondary cloud, while applications with high throughput are kept in the primary cloud. In this way, by determining the target application according to the operating status of the application in the primary cloud, an application that will have a low impact on usability when relocated to the secondary cloud is determined as the target application.

Note that the application selection unit 130 may use the instantaneous operating status in a situation where primary cloud resources are tight, or may use the average operating status within a time span such as one day, several days, or one week. May be used. When the operating status in a situation where primary cloud resources are tight is used, it is possible to suppress the impact on user terminals immediately after existing applications are relocated to the secondary cloud. When the average operating status within a time span such as one day, several days, or one week is used, the influence on user terminals within the time span can be suppressed.

In addition to the existing applications deployed in the primary cloud, the application selection unit 130 may also use the operating status of existing applications deployed in the clouds in the cloud group. In this case, the application is selected as the target application because the application's operating time, the number of user terminals connected to the application, the amount of data of the application, or the throughput of the application are temporarily low in the primary cloud. This can be prevented.

Further, the application selection unit 130 may select target applications based on the priority order of the applications.
Specifically, the application selection unit 130 selects a target application from among existing applications placed in the primary cloud based on the priority order of the applications. The application selection unit 130 can acquire information about existing applications placed in the primary cloud, for example, from a management device that manages application registration. Alternatively, the control device 100 may hold information on existing applications placed in each cloud in the cloud group in a storage, and the application selection unit 130 may refer to this storage.

Application priorities may be set in advance. Examples of the applications include a web distribution application, a game provision application, a video distribution application, a music distribution application, a monitoring execution application, a navigation application, an automatic driving application, a mail provision application, and a sensor execution application.
These applications require different quality of service (QoS) for each service provided by these applications. Therefore, application priorities may be set according to QoS. Examples of QoS include high speed best effort, high speed guaranteed bandwidth, low speed best effort, and low speed guaranteed bandwidth.
For example, in autonomous driving applications, ultra-low delay and ultra-high reliability are required, so high-speed bandwidth guarantee is required as QoS. On the other hand, in mail providing applications, since real-time performance is less important, low-band best effort QoS may be sufficient. Therefore, in a case where the priority order is set according to QoS, the priority order of the automatic driving application may be set higher than that of the mail providing application.
The application selection unit 130 may select target applications in the order of the earliest deployment timing to the primary cloud. In this case, the target application is selected simply by checking the deployment timing, so the time required to select the target application can be kept short.

After determining the target application, the application selection unit 130 notifies the secondary cloud determination unit 140 of information on the total number of CNFs (f _total ) for constructing the target application. Further, the application selection unit 130 notifies the virtual IP address setting unit 160 of information about the target application.

The secondary cloud determining unit 140 determines a secondary cloud within the cloud group in which at least a portion of the CNF for constructing the target application will be placed. An example of determining the secondary cloud will be described later. Furthermore, the secondary cloud determining unit 140 sets the number (f) of CNFs to be placed in the secondary cloud among all CNFs that construct the target application. For example, the secondary cloud determining unit 140 sets the number of surplus CNFs (f _d ) that cannot be deployed to the primary cloud to the number of CNFs (f) to be placed in the secondary cloud.

The secondary cloud determining unit 140 calculates the number of surplus CNFs (f _d ) using, for example, the following formula.

The number of surplus CNFs (f _d ) calculated using equation (3) is the number of CNFs that cannot be deployed to the primary cloud when attempting to deploy as many CNFs as possible to the primary cloud among the CNFs that construct the target application. The number of pieces.

Note that the method for calculating the number of surplus CNFs (f _d ) is not limited to the above. For example, the number of CNFs that can be placed in the primary cloud is set in advance as a threshold value (f _threshold , f _threshold < f _max ), and the number (f _sum - f _threshold ) exceeding this threshold value (f _threshold ) is set as surplus. It may also be the number of CNFs (f _d ).

In this case, (f _max - f _threshold ) free resources are secured in the primary cloud for each CNF. As a result, when there is an urgent request to create a new application in the primary cloud, there is a high possibility that the primary cloud will be able to respond to the urgent request immediately.
The secondary cloud determining unit 140 sets the thus calculated number of surplus CNFs (f _d pieces) that cannot be deployed to the primary cloud as the number (f) of CNFs to be placed in the secondary cloud. Note that the secondary cloud determining unit 140 may set the number of all CNFs (f _total ) that constructs the target application to the number of CNFs (f). In this case, all CNFs that construct the target application will be deployed to the secondary cloud.

The secondary cloud determining unit 140 determines the secondary cloud based on, for example, the size of free resources when a cloud other than the primary cloud in the cloud group deploys f CNFs. The size of empty resources is counted in units of one CNF. The secondary cloud determining unit 140 can obtain information about the resource usage status of clouds in the cloud group from the resource determining unit 120.

Note that the secondary cloud determining unit 140 may determine the secondary cloud based not only on the size of free resources but also on the geographical (or logical) distance between the primary cloud and a cloud other than the primary cloud. .
For example, if there are multiple clouds with the same size of free resources, the secondary cloud determining unit 140 determines the cloud with the smallest distance from the primary cloud as the secondary cloud among the multiple clouds with the same size of free resources. It's okay.

Additionally, the secondary cloud determining unit 140 may determine the secondary cloud based on the geographical (or logical) distance between the primary cloud and a cloud other than the primary cloud. For example, the secondary cloud determining unit 140 may determine the cloud with the smallest distance from the primary cloud as the secondary cloud, regardless of the size of free resources. When determining a secondary cloud based on distance, it is possible to reduce delay time caused by port forwarding, which will be described later.

Additionally, the secondary cloud determining unit 140 may determine the secondary cloud based on a delay time due to port forwarding, which will be described later. For example, the secondary cloud determining unit 140 can estimate the delay time caused by port forwarding based on the geographical location of the server where each cloud is installed, the specifications of the computing resources of each cloud, etc. . The secondary cloud determining unit 140 may determine the cloud with the minimum delay time as the secondary cloud. This allows the impact of port forwarding to be minimized.

Further, the secondary cloud determining unit 140 may determine the secondary cloud based on the current free resource size of clouds other than the primary cloud in the cloud group. In this case, the secondary cloud determining unit 140 may determine the secondary cloud and then set the number (f) of CNFs to be placed in the secondary cloud. This also allows the maximum number of CNFs to be placed in the secondary cloud.
Further, the secondary cloud determining unit 140 may always determine a specific cloud within the cloud group as the secondary cloud. In this case, the time it takes for a new application to be deployed and service provision to start can be shortened.

The secondary cloud determination unit 140 notifies the deployment unit 170 of information on the determined secondary cloud. Further, the secondary cloud determining unit 140 notifies the CNF selecting unit 150 of information about the number (f) of CNFs to be placed in the secondary cloud.

The CNF selection unit 150 acquires information about the characteristics of the CNF that constructs the target application from, for example, a management device that manages application registration. The characteristics of the CNF are information representing the role played by the CNF when a service is provided, such as, for example, the CNF is related to user data or control data.
CNF related to user data is accessed directly from the user terminal. CNFs related to user data are directly connected to user terminals. CNFs associated with control data are typically not directly accessed by user terminals, but may be connected to CNFs associated with user data.

The CNF selection unit 150 also acquires information on the number (f) of CNFs placed in the secondary cloud from the secondary cloud determination unit 140.
The CNF selection unit 150 selects CNFs to be placed in the secondary cloud from among the CNFs that construct the target application, based on the priority order of the CNFs, in descending order of the priority order of the CNFs. In the following, the CNF placed in the secondary cloud will also be referred to as "target CNF."

The CNF priorities may be set in advance. For example, CNF priorities may be set based on CNF characteristics. The characteristics of the CNF are information representing the role played by the CNF when a service is provided, such as whether the CNF is related to user data or control data.
When priorities are set focusing on functions related to user data and functions related to control data, the priority of CNFs related to user data is higher than the priority of CNFs related to control data. Set.
Based on the CNF priorities set in this way, when the CNF selection unit 150 selects CNFs to be placed in the secondary cloud, CNFs that can be directly accessed from user terminals are preferentially placed in the primary cloud. It becomes like this. Therefore, the influence of communication delay when a user terminal receives service provision can be suppressed.

Additionally, the priority of a CNF that is accessed (connected) only by the CNF that builds the target application and is not accessed by applications other than the target application is set lower than the priority of the CNF that is also accessed by applications other than the target application. You may also do so.
When a CNF that is not accessed from outside the target application is placed in a secondary cloud, it is not necessary to notify applications other than the target application of virtual IP address information, which will be described later.

Upon selecting a target CNF, the CNF selection unit 150 notifies the deployment unit 170 of information on the target CNF.
Note that if the number (f) of CNFs arranged in the secondary cloud, which is notified from the secondary cloud determining unit 140, is equal to the total number of CNFs that construct the target application, the CNF selecting unit 150 selects all CNFs of the target application. Select.

The virtual IP address setting unit 160 sets the same virtual IP address for the port of the target CNF deployed in the primary cloud and the port of the target CNF deployed in the secondary cloud. In the following, the port of the target CNF deployed in the primary cloud will also be referred to as a "primary port." Furthermore, the port of the target CNF deployed in the secondary cloud is also referred to as a "secondary port."

The virtual IP address setting unit 160 sets a specific IP address as a virtual IP address for the primary port and secondary port, for example. The specific IP address set as the virtual IP address may be an IP address reserved in advance for the virtual IP address, other than the IP addresses currently assigned to the primary port and the secondary port. Additionally, the virtual IP address may be the IP address currently assigned to the primary port. Alternatively, it may be the IP address currently assigned to the secondary port.

The primary port and the secondary port may be connected by L2 (layer 2) extension. As a result, L2 data destined for the primary port is port forwarded to the secondary port. Further, the primary port and the secondary port serve as connection windows for other CNFs different from the target CNF. In this way, by assigning the same IP address to the primary port and the secondary port as a virtual IP address, the target CNF can be used in the secondary cloud instead of the primary cloud while maintaining the connection between the target CNF and other CNFs. will be put into operation. This allows the processing load to be distributed within the cloud group. Furthermore, since the primary port and the secondary port are L2-extended, it is possible to provide the user terminal with a service by the target application including the target CNF.
The virtual IP address setting unit 160 notifies the deployment unit 170 of virtual IP address information.

The deploying unit 170 deploys the f target CNFs selected by the CNF selecting unit 150 to the secondary cloud. When the deployment of the target CNF to the secondary cloud is completed, the deployment unit 170 checks whether a traffic transmission path has been formed.
Specifically, the deployment unit 170 transmits traffic via a virtual IP address by a CNF that builds a target application other than the target CNF deployed in the primary cloud and a target CNF deployed in the secondary cloud. Check whether the route is created. In response to confirmation that the traffic transmission route has been formed, the deployment unit 170 executes processing for deleting the target CNF deployed in the primary cloud.

Next, resource sharing within a cloud group by the control device 100 according to the present disclosure will be described using FIGS. 4A and 4B. FIG. 4A shows the situation before resource sharing. FIG. 4B shows the situation after resource sharing.
FIGS. 4A and 4B show how a cloud group is formed by three edge clouds #1, #2, and #3. The number of edge clouds included in a cloud group is not limited to three. FIG. 4A is an example of the arrangement of CNFs (before arrangement change) before applying resource sharing by the control device 100 according to the present disclosure. FIG. 4B is an example of the arrangement of CNFs (after arrangement change) after applying resource sharing by the control device 100 according to the present disclosure.

FIG. 4A shows a simplified view of how CNFs that build applications are arranged on the virtualization infrastructure in each edge cloud. CNFs #11, #12, #13, and #14 that construct application #1 are arranged in edge cloud #1. CNF#11, CNF#12, CNF#13, and CNF#14 are associated with Port11, Port12, Port13, and Port14 of edge cloud #1. CNF #21 and #22 that construct application #2 are arranged in edge cloud #2. CNF#21 and CNF#22 are associated with Port21 and Port22 of edge cloud #2. Note that Port23 and Port24 of edge cloud #2 have no associated CNF and are empty ports. A large number of CNFs that build applications are arranged in edge cloud #3.
The virtualization infrastructure is provided with a vSwitch (virtual switch). The vSwitch is used to manage multiple ports and provide L2 (layer 2) connections between multiple CNFs.

Each port is assigned an IP address. FIG. 5A is a diagram showing the correspondence between CNFs, ports, and IP addresses when the arrangement shown in FIG. 4A is adopted.
In the example shown in FIG. 5A, IP address 199.0.2.11, IP address 199.0.2.12, and IP address 199.0.2 are assigned to Port11, Port12, Port13, and Port14 of edge cloud #1. .13, and the IP address 199.0.2.14 is assigned. Further, CNF#11, CNF#12, CNF#13, and CNF#14 are associated with Port11, Port12, Port13, and Port14.
Also, for Port21, Port22, Port23, and Port24 of Edge Cloud #2, IP address 199.0.2.21, IP address 199.0.2.22, IP address 199.0.2.23, IP address 199.0.2.24 is assigned. Furthermore, CNF#21 and CNF#22 are associated with Port21 and Port22. CNF is not associated with Port23 and Port24 (“No deployed”) and they are empty ports.

In this case, it is assumed that the application selection unit 130 selects the application #1 as the target application, and the secondary cloud determination unit 140 determines the edge cloud #2 as the secondary cloud. Further, assume that the CNF selection unit 150 selects CNF #14 as the target CNF. In this case, the deployment unit 170 deploys CNF #14 to edge cloud #2, as shown in FIG. 4B. Note that the example shown in FIG. 4B shows how CNF #14 is associated with Port24, which was an empty port in edge cloud #2.
In this example, Port14 of edge cloud #1 is the primary port, and Port24 of edge cloud #2 is the secondary port.

The virtual IP address setting unit 160 sets a virtual IP address for Port14, which is the primary port. FIG. 5B is a diagram showing the correspondence between CNFs, ports, and IP addresses when the arrangement shown in FIG. 4B is adopted. As shown in FIG. 5B, the virtual IP address setting unit 160 sets, for example, the IP address "199.0.2.255", which is secured as a virtual IP address, as the IP address of Port14, which is the primary port. Furthermore, the virtual IP address setting unit 160 sets the same IP address "199.0.2.255" as the IP address of Port24, which is the secondary port. As a result, the same IP address "199.0.2.255" is set as a virtual IP address for the primary port and the secondary port.

Port14 and Port24 may be connected by L2 (layer 2) extension. Port14 serves as a connection window for CNFs other than CNF#14, and L2 data to Port14 is port forwarded to Port24. By assigning the same IP address to Port14 and Port24 as a virtual IP address, processing of the target CNF is executed in edge cloud #2 instead of edge cloud #1. This allows the processing load to be distributed within the cloud group. Further, since L2 data to Port14 is port forwarded to Port24 by L2 extension, other CNFs accessing the target CNF do not need to know in which cloud the target CNF is deployed. In this way, even when dynamically changing the placement of CNFs that build applications, it is possible to stably provide services to user terminals while suppressing the effects of the placement changes.

FIG. 6 is a flowchart illustrating an example of the operation of the control device 100 according to the present disclosure.
The resource monitoring unit 110 monitors the resource usage status of the cloud in the cloud group (S11).
When requesting the primary cloud to create a new application, the resource determining unit 120 determines whether or not the resources of the primary cloud are tight (S12). Specifically, the resource determination unit 120 determines that resources are tight if the primary cloud does not have the resources to deploy all CNFs that construct a new application.
The application selection unit 130 selects an application (target application) to be deployed to the secondary cloud from existing applications deployed to the primary cloud (S13). In S13, the application selection unit 130 may select an application selection unit 130 based on the priority order or operating status of the application.

The secondary cloud determining unit 140 sets the number (f) of CNFs (target CNFs) to be placed in the secondary cloud among all CNFs that construct the target application (S14). The secondary cloud determining unit 140 determines a secondary cloud (S15). Note that, as described above, the number (f) of CNFs (target CNFs) may be set after determining the secondary cloud.
The CNF selection unit 150 selects CNFs (target CNFs) to be placed in f secondary clouds from the CNFs that construct the target application (S16). In S16, the CNF selection unit 150 may select the target CNF based on a preset CNF priority order.
The deployment unit 170 deploys the selected target CNF to the secondary cloud (S17).
The virtual IP address setting unit 160 sets the same IP address as a virtual IP address for the primary port and the secondary port (S18). The primary port is the port of the target CNF deployed in the primary cloud. The secondary port is a port of the target CNF deployed in the secondary cloud.
Note that in S12, if it is determined that the resources of the primary cloud for which the new application creation request was made are not under strain, the deployment unit 170 creates the new application in the primary cloud. (S19).

Next, the operation of the control device 100 according to the present disclosure will be described using FIG. 7. FIG. 7 is a diagram illustrating an example of a processing sequence in the control device 100 according to the present disclosure.
The control device 100 monitors the resource usage status of the cloud in the cloud group (“resource information management”, T11).

FIG. 7 shows how the resource usage status of the i-th cloud (also referred to as cloud (i)) other than the primary cloud in the cloud group is monitored. Here, i=1 to N-1, and N is the total number of clouds in the cloud group.
Further, the resource usage status may be monitored periodically or triggered by a request to create a new application.

The control device 100 triggers a request to create a new application to the primary cloud in the cloud group (“new application creation request”, T12), and the control device 100 determines that the resources for deploying all CNFs for building the new application are available in the primary cloud. ("resource determination", T13). For example, if the resource usage ratio R defined by the ratio f _sum /f _max is 1 or less, the control device 100 determines that the primary cloud has resources for deploying all CNFs that construct a new application ("resources are tight"). "No"). On the other hand, if the resource usage ratio R is greater than 1, the control device 100 determines that the primary cloud does not have the resources to deploy all the CNFs that construct the new application (“resources are tight”).

If it is determined in the "resource determination" (T13) that "resources are tight", the control device 100 selects an application (target application) to be deployed to the secondary cloud ("application selection", T14). The target application may be selected from existing applications deployed in the primary cloud based on the priority or operating status of the application.
When the target application is selected, the control device 100 determines a secondary cloud in which to place at least a portion of the CNF that constructs the target application (“determine secondary cloud”, T15). Furthermore, in "Secondary Cloud Determination" (T15), the control device 100 sets the number of CNFs (target CNFs) to be placed in the secondary cloud.
Further, the control device 100 selects a CNF (target CNF) to be placed in the secondary cloud, for example, based on the priority order of the CNF (“CNF selection”, T16).

The control device 100 deploys the selected target CNF to the secondary cloud (T17).
Note that in the "resource determination" (T13), if it is determined that "there is no resource strain", the control device 100 may decide to deploy all CNFs for constructing a new application in the primary cloud (T14, T15). , T16). In this case, the control device 100 deploys all CNFs for constructing a new application to the primary cloud (T17A).

The control device 100 sets the same IP address as a virtual IP address for the primary port and the secondary port (“virtual IP address setting”, T18). The control device 100 checks whether a traffic transmission path is formed by a CNF other than the target CNF deployed in the primary cloud and a target CNF deployed in the secondary cloud via the virtual IP address. . In response to confirmation that the traffic transmission path has been formed, the control device 100 executes processing for deleting the target CNF deployed in the primary cloud (T19).

As described above, the control device 100 according to the present disclosure includes at least the application selection section 130, the CNF selection section 150, the virtual IP address setting section 160, and the deployment section 170. The application selection unit 130 selects a target application to be deployed to a secondary cloud in a cloud group from existing applications deployed to a primary cloud in a cloud group formed by a plurality of clouds. For example, when requesting the creation of a new application in the primary cloud, the target application is selected using the fact that the primary cloud's resources are tight as a trigger. The CNF selection unit 150 selects at least one target CNF to be deployed to the secondary cloud in the cloud group from among the CNFs (Containerized Network Functions) that construct the target application. The deployment unit 170 deploys the target CNF to the secondary cloud. The virtual IP address setting unit 160 sets the same IP address as a virtual IP address for the port of the target CNF deployed in the primary cloud and the port of the target CNF deployed in the secondary cloud.

By adopting the configuration described above, the control device 100 can stably provide services to user terminals while suppressing the influence of the placement change even when dynamically changing the placement of the CNF that builds the application. be able to.
Furthermore, since CNFs that build applications can be distributed and placed in different clouds, it is possible to level out the consumption of computing resources for the cloud group as a whole.

Further, the application selection unit 130 may select a target application based on the priority order or operating status of existing applications deployed in the primary cloud. As a result, applications with high priority or operating status will be placed in the primary cloud, and applications with low priority or operating status will be placed in the secondary cloud, so that the PDU session may be temporarily disconnected, etc. The effects of changes in layout can be suppressed.
Note that, before selecting a target application from existing applications deployed in the primary cloud, the application selection unit 130 may compare the priorities of the existing application and the new application. If the priority of the new application is higher than the existing application, the application selection unit 130 may select the target application from the existing applications. If the new application has a lower priority than the existing application, the application selection unit 130 may select the new application as the target application to be deployed to the secondary cloud. The application selection unit 130 may instruct the secondary cloud determination unit 140 and the CNF selection unit 150 to deploy the new application to the secondary cloud. In this case, no instruction to set a virtual IP address will be output to the virtual IP address setting unit 160.

Note that in the above explanation, an example has been explained in which an application for providing services provided to a user terminal has a configuration as shown in FIG. 2, but the present invention is not limited to this. . Even if an application is built from role units that are divided into roles when providing a service, and these role units are deployed to different clouds, an architecture is adopted that allows functional coordination communication between role units. It would be fine if it was.

FIG. 8 is a diagram showing a computer 200 implementing the control device 100 according to the present disclosure. Computer 200 includes a processor 210, a memory 220, and a communication unit 230. The number of processors 210, memories 220, and communication units 230 is not limited, and may be one or more. Further, the processor 210, the memory 220, and the communication unit 230 may be deployed together at each location where each unit that constitutes the control device 100 is arranged. The processor 210 is a program-controlled device such as a microprocessor that operates according to a program installed in the control device 100. The memory 220 is a storage device such as a storage element such as ROM or RAM, a solid state drive (SSD), or a hard disk drive (HDD). The memory 220 stores programs and the like executed by the processor 210. The communication unit 230 is, for example, a communication interface such as a NIC or a wireless LAN module. Note that SDN (Software-Defined Networking) may be implemented in the communication unit 230.
Computer 200 may further include storage 240.
Examples of the application information stored in the storage 240 include information on the total number of CNFs for constructing a new application, information on CNF characteristics, and information on application priorities.
Computer 200 may further include other components not shown.

The control device 100 according to the present disclosure includes one or more processors. When one or more processors requests the creation of a new application to the primary cloud in a cloud group formed by multiple clouds, the primary cloud's resources are tight, which triggers the deployment to the primary cloud in the cloud group. Select the target application to be deployed to the secondary cloud in the cloud group from among the existing applications that are currently available. One or more processors select at least one target CNF to be deployed to a secondary cloud in a cloud group from among CNFs (Containerized Network Functions) that construct a target application. One or more processors deploy the target CNF to a secondary cloud. One or more processors set the same IP address as a virtual IP address for a port of a target CNF deployed in a primary cloud and a port of a target CNF deployed in a secondary cloud.

Note that when container-type virtualization technology is applied to the construction of an application, the control device 100 according to the present disclosure may realize resource sharing between clouds using the function of an orchestrator.

The present disclosure is not limited to the above configuration, and the present disclosure also includes a control program. That is, the present disclosure also includes a program that, when read by a computer, causes one or more processors of the computer to execute each part of the control device 100.
The above-mentioned program may be provided recorded on a computer-readable non-transitory storage medium.

The present disclosure includes the following aspects.

[1] Comprising one or more processors, the one or more processors:
When requesting the creation of a new application on a primary cloud in a cloud group formed by multiple clouds, the resource shortage of the primary cloud will trigger the request to create a new application from the existing application deployed on the primary cloud. Selecting target applications to be deployed to secondary clouds within the cloud group;
Selecting at least one target CNF to be deployed to a secondary cloud in the cloud group from among CNFs (Containerized Network Functions) that construct the target application;
deploying the target CNF on the secondary cloud;
Setting the same IP address as a virtual IP address for a port of the target CNF deployed in the primary cloud and a port of the target CNF deployed in the secondary cloud;
A control device that executes.

[2] The one or more processors:
A traffic transmission path is formed by a CNF that builds the target application other than the target CNF deployed in the primary cloud and the target CNF deployed in the secondary cloud via the virtual IP address. Check whether the
Deleting the target CNF deployed in the primary cloud in response to the formation of the traffic transmission route;
The control device according to [1], which further executes.

[3] The one or more processors:
Monitoring resource usage of the primary cloud;
The control device according to [1] or [2], which further executes the following.

[4] The one or more processors are:
The control device according to any one of [1] to [3], which selects the target application based on the operating status of the existing application.

[5] The operating status includes at least one of the operating time of the application, the number of user terminals connected to the application, the amount of data of the application, and the throughput of the application, as described in [4] control device.

[6] The one or more processors are:
The control device according to any one of [1] to [3], which selects the target application based on a priority order of the existing applications.

[7] The control device according to [6], wherein the priority order of the applications is set according to QoS (Quality of service).

[8] The one or more processors,
Selecting the at least some CNFs based on a preset CNF priority order;
The control device according to any one of [1] to [7], which further executes the following.

[9] The control device according to [8], wherein in the preset CNF priorities, a CNF related to user data has a higher priority than a CNF related to control data.

[10] The one or more processors include:
determining a cloud with the largest amount of free resources as the secondary cloud;
The control device according to any one of [1] to [9], which further executes.

[11] The one or more processors include:
If there are multiple clouds with the same free resource size, the cloud with the smallest distance from the primary cloud is determined as the secondary cloud among the multiple clouds with the same free resource size. Control device as described.

[12] The one or more processors include:
determining the secondary cloud based on a distance between the primary cloud and a cloud other than the primary cloud;
The control device according to any one of [1] to [9], which further executes.

[13] The one or more processors include:
The control device according to [12], wherein a cloud having a minimum distance from the primary cloud is determined as the secondary cloud.

[14] The one or more processors include:
determining the secondary cloud based on a delay time caused by port forwarding from a port of the target CNF deployed in the primary cloud to a port of the target CNF deployed in the secondary cloud;
The control device according to any one of [1] to [9], which further executes.

[15] The one or more processors include:
The control device according to [14], which determines the cloud with the minimum delay time as the secondary cloud.

[16] The control device according to any one of [1] to [15], wherein the target application is constructed using a microservice architecture.

[17] When requesting the creation of a new application to a primary cloud in a cloud group formed by multiple clouds, the existing application deployed to the primary cloud is triggered by the fact that the primary cloud's resources are tight. Selecting, using a processor, a target application to be deployed to a secondary cloud in the cloud group from
Using a processor, selecting at least one target CNF to be deployed to a secondary cloud in the cloud group from among CNFs (Containerized Network Functions) that construct the target application;
Deploying the target CNF in the secondary cloud using a processor;
The same IP address is set as a virtual IP address using a processor for a port of the target CNF deployed in the primary cloud and a port of the target CNF deployed in the secondary cloud. ,
including control methods.

[18] When read by a computer, one or more processors of said computer:
When requesting the creation of a new application on a primary cloud in a cloud group formed by multiple clouds, the resource shortage of the primary cloud will trigger the request to create a new application from the existing application deployed on the primary cloud. Selecting target applications to be deployed to secondary clouds within the cloud group;
Selecting at least one target CNF to be deployed to a secondary cloud in the cloud group from among CNFs (Containerized Network Functions) that construct the target application;
deploying the target CNF on the secondary cloud;
Setting the same IP address as a virtual IP address for a port of the target CNF deployed in the primary cloud and a port of the target CNF deployed in the secondary cloud;
A non-transitory computer-readable medium on which a program for executing is recorded.

Note that the present disclosure is not limited to the above-described embodiments, but also includes various modifications in which components are added, deleted, or converted to the above-described configuration.

100 Control device 110 Resource monitoring unit 120 Resource determination unit 130 Application selection unit 140 Secondary cloud determination unit 150 CNF selection unit 160 Virtual IP address setting unit 170 Deployment unit 200 Computer 210 Processor 220 Memory 230 Communication unit 240 Storage

Claims (18)

1. comprising one or more processors, the one or more processors comprising:
   When requesting the creation of a new application on a primary cloud in a cloud group formed by multiple clouds, the resource shortage of the primary cloud will trigger the request to create a new application from the existing application deployed on the primary cloud. Selecting target applications to be deployed to secondary clouds within the cloud group;
   Selecting at least one target CNF to be deployed to a secondary cloud in the cloud group from among CNFs (Containerized Network Functions) that construct the target application;
   deploying the target CNF on the secondary cloud;
   Setting the same IP address as a virtual IP address for a port of the target CNF deployed in the primary cloud and a port of the target CNF deployed in the secondary cloud;
   A control device that executes.
2. the one or more processors,
   A traffic transmission path is formed by a CNF that builds the target application other than the target CNF deployed in the primary cloud and the target CNF deployed in the secondary cloud via the virtual IP address. Check whether the
   Deleting the target CNF deployed in the primary cloud in response to the formation of the traffic transmission route;
   The control device according to claim 1, further performing:
3. the one or more processors,
   Monitoring resource usage of the primary cloud;
   The control device according to claim 1 or 2, further performing the following.
4. The one or more processors are:
   The control device according to claim 1, wherein the target application is selected based on the operating status of the existing application.
5. The control device according to claim 4, wherein the operating status includes at least one of the operating time of the application, the number of user terminals connected to the application, the amount of data of the application, and the throughput of the application. .
6. The one or more processors are:
   The control device according to claim 1, wherein the target application is selected based on a priority order of the existing applications.
7. The control device according to claim 6, wherein the priority of the applications is set according to QoS (Quality of service).
8. the one or more processors,
   Selecting the at least some CNFs based on a preset CNF priority order;
   The control device according to claim 1, further performing:
9. The control device according to claim 8, wherein, in the preset CNF priorities, a CNF related to user data has a higher priority than a CNF related to control data.
10. The one or more processors are:
    determining a cloud with the largest amount of free resources as the secondary cloud;
    The control device according to claim 1, further performing:
11. The one or more processors are:
    11. If there are a plurality of clouds with the same size of free resources, a cloud with a minimum distance from the primary cloud is determined as the secondary cloud among the plurality of clouds with the same size of free resources. Control device as described.
12. The one or more processors are:
    determining the secondary cloud based on a distance between the primary cloud and a cloud other than the primary cloud;
    The control device according to claim 1, further performing:
13. The one or more processors are:
    The control device according to claim 12, wherein a cloud having a minimum distance from the primary cloud is determined as the secondary cloud.
14. The one or more processors are:
    determining the secondary cloud based on a delay time caused by port forwarding from a port of the target CNF deployed in the primary cloud to a port of the target CNF deployed in the secondary cloud;
    The control device according to claim 1, further performing:
15. The one or more processors are:
    The control device according to claim 14, wherein a cloud with the minimum delay time is determined as the secondary cloud.
16. The control device according to claim 1, wherein the target application is constructed using a microservice architecture.
17. When requesting the creation of a new application on a primary cloud in a cloud group formed by multiple clouds, the resource shortage of the primary cloud will trigger the request to create a new application from the existing application deployed on the primary cloud. Selecting, using a processor, a target application to be deployed to a secondary cloud in the cloud group;
    Using a processor, selecting at least one target CNF to be deployed to a secondary cloud in the cloud group from among CNFs (Containerized Network Functions) that construct the target application;
    Deploying the target CNF in the secondary cloud using a processor;
    Using a processor, the same IP address is set as a virtual IP address for a port of the target CNF deployed in the primary cloud and a port of the target CNF deployed in the secondary cloud. ,
    including control methods.
18. When read by a computer, one or more processors of said computer:
    When requesting the creation of a new application on a primary cloud in a cloud group formed by multiple clouds, the resource shortage of the primary cloud will trigger the request to create a new application from the existing application deployed on the primary cloud. Selecting target applications to be deployed to secondary clouds within the cloud group;
    Selecting at least one target CNF to be deployed to a secondary cloud in the cloud group from among CNFs (Containerized Network Functions) that construct the target application;
    deploying the target CNF on the secondary cloud;
    Setting the same IP address as a virtual IP address for a port of the target CNF deployed in the primary cloud and a port of the target CNF deployed in the secondary cloud;
    A non-transitory computer-readable medium on which a program for executing is recorded.
    
    

Images