WO2024095745A1 - Information processing device, and information processing method - Google Patents

Information processing device, and information processing method Download PDF

Info

Publication number
WO2024095745A1
WO2024095745A1 PCT/JP2023/037327 JP2023037327W WO2024095745A1 WO 2024095745 A1 WO2024095745 A1 WO 2024095745A1 JP 2023037327 W JP2023037327 W JP 2023037327W WO 2024095745 A1 WO2024095745 A1 WO 2024095745A1
Authority
WO
WIPO (PCT)
Prior art keywords
node
application container
movement
priority
pod
Prior art date
Application number
PCT/JP2023/037327
Other languages
French (fr)
Japanese (ja)
Inventor
歩 小林
Original Assignee
ソニーグループ株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ソニーグループ株式会社 filed Critical ソニーグループ株式会社
Publication of WO2024095745A1 publication Critical patent/WO2024095745A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04WWIRELESS COMMUNICATION NETWORKS
    • H04W28/00Network traffic management; Network resource management
    • H04W28/02Traffic management, e.g. flow control or congestion control
    • H04W28/08Load balancing or load distribution
    • H04W28/084Load balancing or load distribution among network function virtualisation [NFV] entities; among edge computing entities, e.g. multi-access edge computing

Definitions

  • This disclosure relates to an information processing device and an information processing method, and in particular to an information processing device and an information processing method that enable offloading of more appropriate targets.
  • MEC Multi-access Edge Computing
  • Patent Document 1 discloses a technology in which a management system predicts the number of UEs using edge applications for the finite resources of an MEC host, and based on the results of the prediction, predicts the degree of congestion of computing resources of an edge server, and switches the connection destination of some of the UEs to a cloud application based on the predicted result of the congestion.
  • Patent Document 1 treats application programs and UEs uniformly based on the correlation between the number of UEs and computing, but does not take into account the impact that application programs that impose specific loads have on the computing resources of the MEC. As a result, there is a possibility that the targets for offloading are not appropriately determined.
  • An information processing device includes a management unit that manages a first node, which is a computer at an edge site connected to a mobile communication network, and a second node, which is a computer at a global site connected to the Internet; a determination unit that, when a first resource value indicating the computing resources of the first node exceeds a threshold, determines a priority for moving the application container based on detailed information on the details of an application container consisting of multiple containers operating on the source node and a second resource value indicating the computing resources of the application container, with the first node whose first resource value exceeds the threshold as the source node, and a control unit that controls the movement of the application container from the source node to a destination node, which is the second node to which the application container is to be moved, according to the priority.
  • an information processing device manages a first node, which is a computer at an edge site connected to a mobile communication network, and a second node, which is a computer at a global site connected to the Internet, and when a first resource value indicating the computing resources of the first node exceeds a threshold, the information processing device determines a priority for moving the application container based on detailed information on the details of an application container consisting of multiple containers running on the source node and a second resource value indicating the computing resources of the application container, and controls the movement of the application container from the source node to a destination node, which is the second node to which the application container is to be moved, according to the priority.
  • a first node which is a computer at an edge site connected to a mobile communications network
  • a second node which is a computer at a global site connected to the Internet
  • a first resource value indicating the computing resources of the first node exceeds a threshold
  • the first node whose first resource value exceeds the threshold is set as a source node
  • a priority for moving the application container is determined based on detailed information regarding the details of an application container consisting of multiple containers operating on the source node and a second resource value indicating the computing resources of the application container, and control is performed to move the application container from the source node to a destination node, which is the second node to which the application container is to be moved, according to the priority.
  • the information processing device of one aspect of the present disclosure may be an independent device or an internal block constituting a single device.
  • FIG. 1 is a diagram illustrating an example of the configuration of a system to which the present disclosure is applied.
  • 2 is a block diagram showing a configuration example of the MEC management system of FIG. 1.
  • 3 is a block diagram showing an example of a functional configuration of an MEC control unit in FIG. 2.
  • FIG. 13 is a diagram illustrating an example of a node table.
  • FIG. 13 illustrates an example of a pod movement table.
  • FIG. 13 is a diagram illustrating an example of a UE mobility table.
  • FIG. 13 is a diagram illustrating an example of a priority table.
  • FIG. 11 is a diagram showing a first other example of the priority table.
  • FIG. 11 is a diagram showing a second example of a priority table;
  • FIG. 11 is a diagram showing another third example of the priority table.
  • FIG. 11 is a diagram illustrating an example of a sequence when a migration setting process is executed.
  • FIG. 11 is a diagram illustrating an example of a sequence when a movement control process is executed.
  • 13 is a flowchart illustrating the flow of a movement setting process. 13 is a flowchart illustrating the flow of a movement control process.
  • FIG. 1 is a block diagram illustrating an example of the configuration of a computer.
  • Fig. 1 is a diagram showing an example of the configuration of a system to which the present disclosure is applied.
  • the system to which the present disclosure is applied is composed of a MEC (Multi-access Edge Computing) management system 11, an MEC 12, a carrier network 13, a cloud data center 14, a name server 15, and a wireless terminal 16.
  • MEC Multi-access Edge Computing
  • the MEC management system 11 includes an information processing device that has a function of controlling the devices that make up the system in FIG. 1.
  • the information processing device is composed of a computer such as a server.
  • the MEC management system 11 may be provided as an on-premise type or as a cloud type.
  • the MEC management system 11 manages the nodes placed in the MEC 12 and the cloud data center 14, the pods that operate (run) on the nodes, and the exit of the carrier network 13 when connecting to the nodes using DNAI (DN Access Identifier) defined by 3GPP (Third Generation Partnership Project) (registered trademark).
  • DNAI DN Access Identifier
  • 3GPP Third Generation Partnership Project
  • the MEC management system 11 performs management by exchanging control signals (C-plain) between nodes located at at least one edge site and nodes located at at least one global site.
  • C-plain control signals
  • a VPN Virtual Private Network
  • the nodes at the edge sites are nodes located in the MEC 12, and the nodes at the global sites are nodes located in the cloud data center 14.
  • the detailed configuration of the MEC management system 11 is explained in Figure 2. Details of the pods will also be explained later.
  • MEC12 is configured with devices that distribute edge servers near wireless terminals 16 and perform as much processing as possible at the edge servers.
  • Edge site node 21A is placed in MEC12.
  • Node 21A is configured with computers such as servers (edge servers).
  • the carrier network 13 is a mobile communication network constructed by a mobile communication operator (communication carrier) and is composed of a core network, etc.
  • the carrier network 13 includes a 4G network corresponding to the fourth generation mobile communication system (4G) and a 5G network corresponding to the fifth generation mobile communication system (5G).
  • the group of servers that make up the core network of the carrier network 13 includes a NEF (Network Exposure Function) server. Note that while FIG. 1 shows an example in which the MEC 12 is placed outside the carrier network 13, the MEC 12 only needs to be connected to the carrier network 13 and may be placed inside the carrier network 13.
  • the carrier network 13 can be connected to the Internet and is connected to the cloud data center 14 via the Internet.
  • Cloud data center 14 is a cloud that provides a network environment for service providers to prepare and use servers and equipment, or a data center that provides space and facilities for collecting and managing servers in one place. Cloud data center 14 is connected to carrier network 13 via the Internet.
  • Node 21B of the global site is located in cloud data center 14.
  • Node 21B is composed of computers such as servers (cloud servers).
  • node 21 when there is no need to distinguish between node 21A and node 21B, they will also be referred to as node 21.
  • the name server 15 is a server that has a DNS (Domain Name System) function that converts domain names and IP addresses to perform name resolution.
  • DNS Domain Name System
  • the name server 15 may be provided in any communication network as long as it can be connected to the MEC management system 11 or the wireless terminal 16, and may be provided as a cloud, for example.
  • the wireless terminal 16 is a mobile communication terminal such as a smartphone or tablet terminal.
  • the wireless terminal 16 communicates via the carrier network 13.
  • the wireless terminal 16 can also connect to the name server 15 to perform name resolution.
  • the wireless terminal 16 will also be referred to as UE (User Equipment) as appropriate.
  • FIG. 1 shows a case where there is only one MEC 12 and one cloud data center 14, there may be multiple of each.
  • cloud data centers 14 this is the case when there are multiple instances on public clouds, private clouds, or data centers, or when instances (nodes) are located on multiple different public clouds, private clouds, or data centers.
  • nodes instances
  • edge sites when there are multiple MECs 12, this is the case when there are different edge sites.
  • FIG. 1 a typical configuration would be one in which there are multiple wireless terminals 16 such as smartphones, but to simplify the explanation, a case in which there is one wireless terminal 16 is shown as an example.
  • the name server 15 is not a required component and is provided as needed.
  • FIG. 2 is a block diagram showing an example of the configuration of the MEC management system 11 in FIG. 1.
  • the MEC management system 11 is composed of an MEC control unit 111, an API (Application Programming Interface) unit 112, an AF (Application Function) unit 113, and a database 114.
  • API Application Programming Interface
  • AF Application Function
  • the MEC control unit 111 cooperates with the API unit 112 and the AF unit 113 to perform processes related to the node 21 and the pods 31 operating (running) on the node 21, as well as route control for the wireless terminal 16 (UE).
  • the node 21 is placed in the MEC 12 or the cloud data center 14.
  • the node 21 is configured as hardware, and is capable of operating multiple pods 31 executed as software.
  • the pod 31 is an application program (hereinafter also referred to as an application container) that is composed of multiple containers.
  • the pod 31 can be said to be a collection of multiple containers.
  • a container is a unit in which software is packaged, and by using container-based virtualization technology, virtualization is achieved by constructing an application environment in an independent space called a container.
  • a container contains things necessary for software execution, such as libraries, system tools, code, and runtime.
  • the MEC control unit 111 also works in conjunction with the database 114 to refer to and update tables, perform movement setting processing, movement control processing, and other processing.
  • the MEC control unit 111 has a function of replacing the IP address of the DNS record when the pod 31 moves by controlling a communication I/F (not shown) and connecting to the name server 15. However, this IP address replacement function is not a required function.
  • Route53 from AWS (Amazon Web Services) (trademark) is used as the name server 15
  • the aws route53 change-resource-record-sets command can be rewritten using JSON (JavaScript Object Notation) that describes the contents of the record.
  • AWS is a cloud computing service provided by Amazon Web Services, Inc.
  • JSON is a data description language and a format for text-based data conversion.
  • the API unit 112 is configured as an API server that uses the API to communicate with the nodes 21 and the pods 31 on the control plane.
  • the API can use the Kubernetes API.
  • Kubernetes is an open source platform for orchestrating containers. Kubernetes defines the smallest deployable unit of computing that can be created and managed as a pod. The pod 31 corresponds to the pod defined in Kubernetes.
  • the API unit 112 can use the kubectl command in conjunction with the Kubernetes API to, for example, obtain computing resources of the node 21 and the pod 31, obtain detailed information (qosClass) about the pod 31, and specify a manifest file to deploy to or delete from the node 21.
  • the manifest file is a text file that describes resource information about the pod 31.
  • Requests represents the minimum resource capacity (CPU and memory capacity) required by pod 31.
  • Limits represents the maximum resource capacity (maximum CPU and memory capacity) that can be used when the load on pod 31 increases.
  • Burstable and BestEffort pods 31 are targets for movement (targets for offloading).
  • the AF unit 113 performs processing in cooperation with the NEF server 22 using the 3GPP API.
  • the 3GPP API is a function described in 3GPP TS 29.517 that cooperates with an API when a communications carrier provides a core node function of the carrier network 13 to a third party such as a service provider as an API.
  • the AF unit 113 cooperates with the Traffic Influence API described in 3GPP TS 29.522 5.4 to control the route of a UE accessing a node that has been mapped in advance to a DNAI.
  • the NEF server 22 exposes the functions of the carrier network 13 (NF: Network Function) so that they can be used by external application programs, allowing applications to grasp information from the network, and also provides an API that allows application programs to control the NF.
  • NF Network Function
  • the database 114 is recorded in a storage device such as a hard disk drive (HDD).
  • the database 114 may be provided by a database (DB) server or the like.
  • the database 114 manages various tables used in processing by the MEC control unit 111.
  • FIG. 3 is a block diagram showing an example of the functional configuration of the MEC control unit 111 in FIG. 2.
  • the MEC control unit 111 has a management unit 121, a setting unit 122, and a control unit 123.
  • the management unit 121 manages the nodes 21 and pods 31 using various tables managed in the database 114.
  • the management unit 121 also manages the exit of the carrier network 13 when connecting to the node 21 by mapping the DNAI and the node 21 in advance using the tables managed in the database 114.
  • the setting unit 122 performs a movement setting process for setting the movement of pods and UEs.
  • the priority (movement priority) of the pod movement is determined, and the pods and UEs that are candidates for movement are set with the determined priority.
  • a pod that is a candidate for movement is also referred to as a candidate pod for movement
  • a UE that is a candidate for movement is also referred to as a candidate UE for movement.
  • the setting unit 122 has a movement setting unit 131 and a priority determination unit 132.
  • the movement setting unit 131 performs settings related to the movement of candidate pods to be moved and candidate UEs to be moved.
  • the priority determination unit 132 determines the priority of the movement of pods.
  • the control unit 123 performs a movement control process for controlling the movement of pods and UEs.
  • a pod to be moved and a UE to be moved are determined from among the candidate pods to be moved and the candidate UEs to be moved according to the movement priority (movement priority), and the determined pod to be moved and UE to be moved are moved.
  • the pod that is actually selected as the movement target among the candidate pods to be moved is also referred to as the movement target pod.
  • the UE that is actually selected as the movement target among the candidate UEs to be moved is also referred to as the movement target UE.
  • the control unit 123 has a movement control unit 141 and a route control unit 142.
  • the movement control unit 141 controls the API unit 112 to control the movement of the pod to be moved.
  • the route control unit 142 controls the AF unit 113 to control the route of the UE to be moved.
  • the database 114 manages a node table 151, a pod movement table 152, a UE movement table 153, and a priority table 154.
  • the node table 151 maps and manages the DNAI defined by 3GPP and the nodes 21 on Kubernetes. In other words, the mapping is performed so that the carrier network 13 can identify the nodes 21 managed by Kubernetes as nodes (DN: DataNet) to which pods 31 can be deployed.
  • the node table 151 manages information such as site information, identification information as to whether the site is edge or global, latitude and longitude indicating the location, and IP address for each node 21.
  • the pod movement table 152 manages information on pods that are candidates for movement.
  • the pod movement table 152 manages the movement priority of the candidate pods for movement based on the source node, which is node 21A of the edge site from which the movement originates, the destination node, which is node 21B of the global site to which the movement originates, and the priority.
  • the pod movement table 152 also manages information such as the scheduled movement time, which is determined from the communication volume of the source node, for each candidate pod for movement.
  • UE movement table 153 manages information on UEs (candidate UEs) that are only accessing candidate pods to be moved.
  • UE movement table 153 manages the movement priority of candidate UEs to be moved based on the source DNAI, which is the DNAI from which the movement will be made, the destination DNAI, which is the DNAI to which the movement will be made, and the priority.
  • UE movement table 153 also manages information such as the scheduled movement time for each candidate UE to be moved.
  • the life cycle of UE movement table 153 is the same as that of pod movement table 152.
  • movement tables when there is no need to distinguish between pod movement table 152 and UE movement table 153, they will also be referred to as movement tables.
  • the priority table 154 manages detailed information (qosClass) of the pod 31 and a priority according to the utilization rate of the computing resources of the pod 31 as information for determining the priority of candidate pods to be moved. For example, in the priority table 154, the more unrestricted a Besteffort type pod is, the higher its priority as a pod to be moved, and among those, the higher its priority is set for pods with utilization rates of computing resources equal to or above the median. This makes it possible for a pod to be moved with a higher priority as a candidate pod to be moved (a target for offloading).
  • qosClass detailed information
  • a priority is set for a candidate UE to be moved that corresponds to the candidate pod to be moved that it is accessing, it can be said that by determining the priority of the candidate pod to be moved, the priority of the candidate UE to be moved that is accessing the UE to be moved is also determined.
  • FIG. 4 is a diagram showing an example of the node table 151 in FIG. 3.
  • the node table 151 uses the node ID that identifies the node as a key to manage information about the node's site, the DNAI mapped to the node, the latitude and longitude that indicate the node's location, and the node's IP address in association with each other.
  • a node assigned node ID 1 exists at an edge site, the Sendai Edge Site, is mapped to DNAI1, has latitude and longitude AA,BB, and an IP address aa.aa.aa.aa.
  • a node assigned node ID 5 exists at a global site, the Tokyo Global Site, is mapped to DNAI5, has latitude and longitude II,JJ, and an IP address ee.ee.e.e.
  • records with node IDs 1 and 5 are shown as examples, but other records are managed in the same way.
  • FIG. 5 is a diagram showing an example of the pod movement table 152 in FIG. 3.
  • the pod movement table 152 uses a pod ID that identifies a candidate pod to be moved as a key, and manages the source site (source node), destination site (destination node), scheduled movement time indicating the time of movement, name indicating the domain name, and priority indicating the priority of the movement in association with each other.
  • a pod assigned pod ID 1 is scheduled to move from a node in the Sendai Edge Site to a node in the Tokyo Global Site at 1:00 on 1 November 2021, and has the highest priority for movement because it has priority 1.
  • a pod assigned pod ID 3 is scheduled to move from a node in the Fukuoka Edge Site to a node in the Osaka Global Site at 2:00 on 1 November 2021, and has the highest priority for movement because it has priority 1.
  • records with pod IDs 1 and 3 are shown as examples, but other records are managed in the same way.
  • FIG. 6 is a diagram showing an example of the UE movement table 153 in FIG. 3.
  • the UE movement table 153 uses the UEID that identifies the candidate UE to be moved as a key, and manages the source DNAI, destination DNAI, scheduled movement time indicating the time of movement, IP address, and priority indicating the priority of the movement in association with each other.
  • SUPI Subscription Permanent Identifier
  • a UE assigned UEID 1 is scheduled to move from DNAI1 to DNAI5 at 1:00 on 1 November 2021, and has the highest priority for movement because its IP address is aa.aa.aa.1 and its priority is 1.
  • a UE assigned UEID 5 is scheduled to move from DNAI4 to DNAI6 at 2:00 on 1 November 2021, and has the highest priority for movement because its IP address is bb.bb.bb.1 and its priority is 1.
  • records for UEID 1 and 5 are shown as examples, but other records are managed in the same way.
  • FIG. 7 is a diagram showing an example of the priority table 154 in FIG. 3.
  • the priority table 154 is a table used to determine the priority of the movement of pods and UEs that are candidates for movement, and manages the priority according to the detailed information of the pod (qosClass) and the utilization rate of the pod's computing resources (pod resource utilization rate).
  • the lower the priority number the higher the priority.
  • the pod resource utilization rate includes the CPU utilization rate and memory utilization rate of the pod.
  • Burstable and BestEffort type pods are targeted for movement, and the more unlimited the Besteffort type pod is, the higher the priority for being targeted for movement.
  • pods with pod resource utilization rates equal to or greater than the median are given a higher priority, so that BestEffort type pods with pod resource utilization rates equal to or greater than the median are given priority 1, making them the highest priority for movement.
  • BestEffort type pods with pod resource utilization rates less than the median are given priority 2, making them the second highest priority for movement after priority 1.
  • burstable pods with pod resource utilization rates equal to or greater than the median they are assigned a priority of 3, making them the second highest priority for migration after priority 2.
  • burstable pods with pod resource utilization rates below the median they are assigned a priority of 4, making them the lowest priority for migration. Note that here, if multiple pods are running on the target node, the median pod resource utilization rates of those pods can be used as the standard.
  • the priority of the priority table 154 can be determined based on two axes: detailed information about the pod (qosClass) and the pod resource usage rate. Furthermore, it is not limited to these two axes, and more axes can be added to achieve a more suitable priority determination for the situation.
  • Figures 8 to 10 show other examples of priority tables.
  • the priority is determined based on whether the pod resource utilization rate is high or low, based on the median value of the pod resource utilization rate, but cases in which the pod resource utilization rates are concentrated near the median value are also assumed. Therefore, when the variance value of the pod resource utilization rate is less than the threshold value, the priority of a pod with a low number of accesses from UEs, etc., based on the access log for a specified period of time is increased.
  • Figure 8 shows a priority table 154A that manages the priority according to the detailed information (qosClass) of a pod and the number of accesses to the pod.
  • qosClass the detailed information
  • the priority table 154A of the Burstable and BestEffort type pods to be moved, BestEffort type pods with an access count below the median are assigned priority 1, making them the highest priority for movement. Also, BestEffort type pods with an access count equal to or greater than the median are assigned priority 2. Similarly, Burstable type pods with an access count below the median are assigned priority 3, and those with an access count equal to or greater than the median are assigned priority 4.
  • the priority table 154A is used to increase the priority of pods with high resource utilization rates despite a low number of accesses from UEs, etc., and to move them preferentially, thereby reducing the impact of communication interruptions, etc. that occur during the move.
  • FIG 9 shows priority table 154B, which manages detailed pod information (qosClass) and priority according to the 5G GBR/Non-GBR allocation in the communications used by the pod (UE PDU session).
  • priority table 154B among the Burstable and BestEffort pods to be moved, BestEffort pods that are both Non-GBR are assigned priority 1, making them the highest priority for movement. Also, BestEffort pods that are GBR are assigned priority 2. Similarly, Burstable pods that are Non-GBR are assigned priority 3, and GBR pods are assigned priority 4.
  • This axis for determining priorities is intended to be placed after the axis of pod resource utilization and the axis of number of accesses mentioned above.
  • GPU Graphics Processing Unit
  • FIG. 10 shows a priority table 154C that manages priorities according to the utilization rate of computing resources (pod resource utilization rate) of non-limited pods when targeting pods that use GPUs.
  • the pod resource utilization rate here is the GPU utilization rate.
  • pods with GPU utilization rates equal to or greater than the median are assigned priority 1, making them the highest priority for migration. Additionally, pods with GPU utilization rates less than the median are assigned priority 2.
  • pod resource utilization is not limited to CPU utilization and memory utilization, and resource utilization such as GPU utilization may be used depending on the resources used by the pod. For example, when targeting pods that use GPUs, a similar priority can be determined for non-limited pods based on whether their GPU utilization is above or below the median value.
  • Fig. 11 is a diagram showing an example of a sequence when a migration setting process is executed in the MEC management system 11.
  • the exchange of data (signals) between devices is indicated by bidirectional arrows.
  • the MEC management system 11 exchanges data between a node 21A at an edge site arranged in the MEC 12 and a node 21B at a global site arranged in the cloud data center 14, which is a migration source node.
  • the API unit 112 executes the kubectl top node command to obtain the computing resources (node resource usage) of node 21A (S11). Then, the MEC control unit 111 performs a threshold check to compare the obtained node resource usage with a threshold (S12).
  • the node resource usage includes the CPU usage and memory usage of node 21A, etc.
  • the MEC management system 11 determines in the threshold check (S12) that the node resource usage rate exceeds the threshold, the following process is carried out and the migration priority is determined.
  • the API unit 112 executes the kubectl describe pod command to obtain detailed information (qosClass) for each pod 31 running on node 21A (S13), and executes the kubectl top pod command to obtain the computing resources (pod resource utilization) of the pod 31 (S14).
  • the pod resource utilization includes the CPU utilization and memory utilization of the pod 31.
  • the MEC control unit 111 uses the obtained detailed information (qosClass) and pod resource utilization to determine the migration priority (S15). This creates the priority table 154.
  • the pods and UEs that are candidates for movement are set in the pod movement table 152 and the UE movement table 153, respectively, along with their priorities.
  • the MEC control unit 111 refers to the node table 151, performs a predetermined distance calculation using latitude and longitude information, and determines the destination node 21B (node at the global site) (S16).
  • the MEC control unit 111 also obtains the communication volume for a predetermined period from the communication log of the source node 21A (S17), and determines the movement time (scheduled movement time) based on the communication volume (S18).
  • the MEC control unit 111 sets and updates the pod movement table 152 with information about the candidate pod to be moved, such as the determined priority, destination node, and movement time (S19).
  • the MEC control unit 111 also extracts the IP address (UEIP) of the UE that is accessing only the candidate pod to be moved from the access log in the source node 21A, and sets the UE with the extracted UEIP as a candidate UE to be moved (S20). The MEC control unit 111 then sets and updates the UE movement table 153 with information about the candidate UE to be moved, such as the source DNAI, destination DNAI, movement time, and priority (S21).
  • UEIP IP address
  • FIG. 12 is a diagram showing an example of the sequence when the mobility control process is executed in the MEC management system 11.
  • the MEC management system 11 also exchanges data between the node 21B placed in the cloud data center 14, the name server 15, and the NEF server 22.
  • the MEC control unit 111 checks the time (S31), and when the target time arrives, the subsequent processing is performed. That is, the MEC control unit 111 identifies the record with the highest priority from the pod movement table 152 updated in step S19 of FIG. 11, and determines the candidate pod to be moved of the identified record as the pod to be moved (S32). The MEC control unit 111 also identifies the record with the highest priority from the UE movement table 153 updated in step S21 of FIG. 11, and determines the candidate UE to be moved of the identified record as the UE to be moved (S33).
  • the API unit 112 under control of the MEC control unit 111, deploys the pod to be moved that was running on the source node 21A to the destination node 21B (S34).
  • the deployment is performed by specifying a manifest file with the kubectl command.
  • the AF unit 113 uses the TrafficInfluence API to switch the route of the UE to be moved (S36).
  • the AF unit 113 under control of the MEC control unit 111, specifies the UEIP and destination DNAI of the UE to be moved, and performs route control of the UE in cooperation with the NEF server 22. In this way, the movement of the pod to be moved and the UE to be moved according to priority is realized.
  • the MEC control unit 111 updates the DNS record for the name of the pod to be moved in the pod movement table 152 (S37), and notifies the name server 15 of the IP address of the destination node 21B as the new IP address (S38).
  • the name of the pod to be moved is registered in the name server 15, it can update the IP address by replacing it with the IP address of the destination node 21B.
  • the API unit 112 under control of the MEC control unit 111, deletes the pod to be moved from the source node 21A (S39). The deletion here is performed by specifying the manifest file with the kubectl command. Finally, the MEC control unit 111 deletes the record of the pod to be moved from the pod movement table 152 (S40). In addition, the MEC control unit 111 deletes the record of the UE to be moved from the UE movement table 153 (S40).
  • Fig. 13 illustrates an example in which the utilization rate of computing resources (node resource utilization rate) of the node 21A, which is located at the edge sites Sendai Edge Site and Fukuoka Edge Site among the nodes managed in the node table 151, exceeds a threshold value.
  • the movement setting unit 131 controls the API unit 112 to acquire the node resource utilization rate of the node 21A placed at the edge site (S111), and determines whether the acquired node resource utilization rate exceeds a threshold value (S112).
  • the API unit 112 executes the kubectl top node command to acquire the CPU utilization rate and memory utilization rate of the node 21A as the node resource utilization rate, which are compared with the threshold value.
  • the node resource utilization rate is not limited to the utilization rate at a specific time (time period), and for example, the load average may be used.
  • the migration setting unit 131 checks whether or not there is a migration table in the database 114 (S113). For example, if the node resource usage exceeds the threshold for the first time, neither the pod migration table 152 nor the UE migration table 153 exists (No in S113), and the migration setting unit 131 controls the API unit 112 to acquire detailed information about each pod running on the node 21A (S114). In this example, the API unit 112 executes the kubectl describe pod command to acquire detailed information (qosClass) about each pod running on the node 21A of the Sendai Edge Site and the node 21A of the Fukuoka Edge Site. The migration setting unit 131 then checks the type of qosClass acquired, and performs the processes of steps S116 to S122 for pods determined to be of the BestEffort type or Burstable type (Yes in S115).
  • qosClass detailed information
  • the migration setting unit 131 controls the API unit 112 to obtain the utilization rate of the pod's computing resources (pod resource utilization rate) (S116).
  • the API unit 112 executes the kubectl top pod command to obtain the pod's CPU utilization rate and memory utilization rate as the pod resource utilization rate.
  • the pod resource utilization rate is not limited to the utilization rate at a specific time (time period), and for example, the load average may be used.
  • the priority determination unit 132 determines the priority of the pod movement based on the qosClass and the pod resource usage rate (S117). This creates a priority table 154. For example, in the priority table 154 of FIG. 7, the priority of unrestricted BestEffort type movement is higher, and among them, pods with pod resource usage rates equal to or higher than the median are assigned the highest movement priority of 1. Note that here, a priority table 154A (FIG. 8) using the number of accesses or a priority table 154B (FIG. 9) according to the GBR/Non-GBR allocation may also be created.
  • the movement setting unit 131 refers to the node table 151 and determines, from among the nodes of global sites stored in the node table 151, the node with the closest distance from the source node calculated using latitude and longitude as the destination node (S118). For example, in the node table 151 of FIG. 4, the node of the Tokyo Global Site, which is the global site with the closest distance to the source node of the Sendai Edge Site, is determined as the destination. Also, the node of the Osaka Global Site, which is the global site with the closest distance to the source node of the Fukuoka Edge Site, is determined as the destination.
  • the movement setting unit 131 determines the time period with the least amount of communication on the source node (S119). For example, by referring to the communication log for a specified period (e.g., one week) on the source node, the communication volume for that period is averaged for each day to calculate the time period with the least amount of communication. In this example, 1:00 is calculated as the time period with the least amount of communication on the node at the Sendai Edge Site, and 2:00 is calculated as the time period with the least amount of communication on the node at the Fukuoka Edge Site. Note that the time period to be determined is basically assumed to be at night, but if the global site extends overseas, for example, it may be daytime due to the time difference. The date and time corresponding to the time period determined here becomes the time of movement of the pod (scheduled time of movement).
  • a specified period e.g., one week
  • the movement setting unit 131 sets and updates the information, such as the priority, destination node, and movement time determined in steps S117 to S119, in the pod movement table 152 as information about the pod candidate to be moved (S120). For example, in the pod movement table 152 of FIG. 5, information about the pod to be moved from the Sendai Edge Site to the Tokyo Global Site at the scheduled movement time of 2021/11/1 1:00 is set as records for pod IDs 1 and 2, and information about the pod to be moved from the Fukuoka Edge Site to the Osaka Global Site at the scheduled movement time of 2021/11/1 2:00 is set as records for pod IDs 3, 4, 5, 6, and 7.
  • a priority is set for each pod, and for example, the pods candidate to be moved with pod IDs 1, 3, and 4 have a priority of 1, and therefore become the pods to be moved first at the corresponding scheduled movement times.
  • the destination node which is a node in the global site, is mapped to DNAI in node table 151.
  • the mobility setting unit 131 refers to the access log for a predetermined period (e.g., one week) and extracts the IP addresses (UEIP) of UEs that are accessing only the candidate pods to be moved for each priority (S121).
  • the mobility setting unit 131 sets and updates the UE mobility table 153 with information such as the source DNAI, destination DNAI, mobility time, and priority as the candidate UEs to be moved (S122).
  • the movement time and priority here correspond to the movement time and priority of the candidate pod to be moved that the candidate UE to be moved is accessing.
  • the source DNAI and destination DNAI can be set from the DNAI mapped to the node in the node table 151. For example, in the UE movement table 153 of FIG.
  • UEs information on UEs (UEs that only access pods at the Sendai Edge Site) whose routes will be switched from DNAI1 to DNAI5 at the scheduled movement time of 1:00 on November 1, 2021 is set as records for UEIDs 1, 2, 3, and 4, and information on UEs (UEs that only access pods at the Fukuoka Edge Site) whose routes will be switched from DNAI4 to DNAI6 at the scheduled movement time of 2:00 on November 1, 2021 is set as records for UEIDs 5, 6, 7, 8, and 9.
  • a priority is set for each UE according to the priority of the pod it accesses. For example, candidate UEs for movement with UEIDs 1, 2, 3, 5, and 6 will have a priority of 1, and will be the first UEs to be switched to at the scheduled movement time.
  • step S122 ends, the series of processes ends. If it is determined that the node resource utilization rate is below the threshold (No in S112), and there is a migration table (Yes in S123), the migration table is reset (S124) and the process ends. On the other hand, if there is no migration table (No in S123), the process ends as it is. Also, even if it is determined that the node resource utilization rate exceeds the threshold (Yes in S112), if there is a migration table (Yes in S113), the process ends.
  • the above-mentioned movement setting process updates the pod movement table 152 and the UE movement table 153, and sets the information of the movement target candidate pods and the movement target candidate UEs together with their priorities. Then, based on the information set in the movement setting process, a movement control process is performed to move the movement target pods and the movement target UEs according to the priorities.
  • the flow of the movement control process executed by the MEC control unit 111 of the MEC management system 11 will be described with reference to the flowchart in FIG. 14.
  • the control unit 123 refers to the pod movement table 152 (S131) and determines whether the current time has reached the target time indicated by the scheduled movement time (S132). If the control unit 123 determines that the target time has reached (Yes in S132), it performs the processes of steps S133 to S139.
  • control unit 123 refers to the pod movement table 152 and sets the candidate pod to be moved of the record with the highest priority as the pod to be moved (S133).
  • the control unit 123 also refers to the UE movement table 153 and sets the candidate UE to be moved of the record with the highest priority as the UE to be moved (S134).
  • the pod with pod ID 1 which has a scheduled movement time of 1:00 on November 1, 2021 and is set to priority 1 in the pod movement table 152 of FIG. 5, becomes the pod to be moved.
  • the UEs with UE IDs 1, 2, and 3, which have a scheduled movement time of 1:00 on November 1, 2021 and is set to priority 1 become the UEs to be moved.
  • the movement control unit 141 controls the API unit 112 to deploy the pod to be moved to the destination node (S135).
  • the destination of the pod to be moved with pod ID 1 is a node in the Tokyo Global Site.
  • the node in the Tokyo Global Site, which is the destination node, is mapped to DNAI, which is DNAI5, in the node table 151 in Figure 4.
  • the route control unit 142 controls the AF unit 113 to perform route control using the TrafficInfluence API and switch the route of the UE to be moved (S136).
  • route control is performed in cooperation with the NEF server 22 by specifying the UEIP and destination DNAI of the UE to be moved.
  • the routes of the UEs with UEIDs 1, 2, and 3 are switched by specifying the IP addresses aa.aa.aa.1, aa.aa.a.2, and aa.aa.aa.3 corresponding to UEIDs 1, 2, and 3 corresponding to priority 1, and the destination DNAI of DNAI 5.
  • the control unit 123 updates the IP address of the name of the pod to be moved, which is managed by the name server 15, to the IP address of the destination node (S137). For example, if aaa.co.jp, which is the name of the pod to be moved with pod ID 1 in the pod movement table 152 of Figure 5, is registered in the name server 15, the control unit 123 can replace the IP address of the name of the pod to be moved with the IP address of the destination node by notifying the name server 15 of ee.ee.e.ee, which is the IP address of the destination node, which is the Tokyo Global Site.
  • the movement control unit 141 controls the API unit 112 to delete the pod to be moved from the source node (S138).
  • the pod to be moved with pod ID 1 is deleted from the node of the Sendai Edge Site.
  • the control unit 123 deletes the records to be moved from the pod movement table 152 and the UE movement table 153 (S139). For example, in the pod movement table 152 of FIG. 5, the record of pod ID 1, which is the pod to be moved, is deleted. Also, in the UE movement table 153 of FIG. 6, the records of UE IDs 1, 2, and 3, which are the UEs to be moved, are deleted.
  • step S139 ends, the series of processes ends. Note that if it is determined that the target time has not yet arrived (No in S132), steps S133 to S139 are skipped and the process ends.
  • the following processing is performed after the movement of priority 1. That is, the movement control processing of priority 1 is performed, and the priority 1 records are deleted from both the pod movement table 152 and the UE movement table 153. Then, the movement setting processing of FIG. 13 is executed again, and the node resource utilization threshold is checked (S111, S112). In this example, if it is determined that the node resource utilization rates of the Sendai Edge Site and the Fukuoka Edge Site are below the threshold (No in S112), the pod movement table 152 and the UE movement table 153 become unnecessary, and these movement tables are reset (S124). That is, since the records of priorities 2 to 4 are also deleted, the movement control processing for the remaining candidate pods and candidate UEs to be moved is not performed and ends.
  • the same process is performed on the records with priority 3 and priority 4 until it is determined that the node resource utilization rate is less than the threshold (No in S112).
  • the processing will be completed in as little as one day (for example, 2021/11/1) and at most four days (for example, 2021/11/1 - 11/4).
  • the priority of pod movement can be determined from detailed information about each pod operating on an edge site node where node resource utilization has exceeded a threshold and the pod resource utilization, and the pod can be moved from the edge site node to a global site node according to the determined priority.
  • the route of a UE that is only accessing the pod to be moved can be switched according to the determined priority.
  • a pod (application container) with a specific load is preferentially moved to a global site node, and the route of a UE that is only accessing that pod is switched, so that the pod (application container) or UE to be offloaded can be appropriately determined, and more appropriate targets can be offloaded.
  • pods whose computing resources can be controlled tend to gather at MEC12 (edge site node) while minimizing the degradation of the quality of experience that accompanies route changes. This makes it possible to prevent a particular pod running on an edge site node from occupying too many of the node's computing resources, causing the node to shut down. In other words, it makes it possible to prevent a situation in which all application programs (pods, etc.) running on an edge site node shut down completely.
  • a system refers to a logical collection of multiple devices, but the MEC management system 11 may be composed of multiple devices (e.g., servers, etc.).
  • a device having the MEC control unit 111, a device having the API unit 112, a device having the AF unit 113, and a device having the database 114 may be composed of separate devices.
  • the MEC management system 11 may be regarded as an MEC management device.
  • This MEC management device is a device having the MEC control unit 111, the API unit 112, the AF unit 113, and the database 114. Note that the MEC management device only needs to have at least the MEC control unit 111, and the API unit 112, the AF unit 113, and the database 114 may be included in external devices.
  • priority level is not limited to four levels and may be more than four, as long as it is two or more.
  • Burstable and BestEffort pods are targeted for movement, but other classifications (Guaranteed, etc.) may be targeted for movement, or the pod resource utilization may be further divided into smaller categories using a threshold or the like rather than being above or below the median, thereby increasing the number of priority levels.
  • the median is used for the pod resource utilization, but other indices may be used instead of being limited to the median.
  • resource utilization rates such as node resource utilization rate and pod resource utilization rate are examples of resource values, and other indices may be used. Furthermore, when multiple resource utilization rates such as CPU utilization rate and memory utilization rate are obtained as node resource utilization rate and pod resource utilization rate, it is sufficient to use at least one of the resource utilization rates, as well as all of the resource utilization rates. When comparing resource utilization rate with a threshold value, median value, etc., the obtained resource utilization rate may be used as is, or a predetermined calculation may be performed before comparing with the threshold value, median value, etc.
  • latitude and longitude are used as location information regarding the positions of edge site nodes and global site nodes in node table 151, but latitude and longitude are only one example of location information, and other location information may be used as long as it is information that can identify the node position.
  • node 21A in FIG. 1 consisting of a server etc. in an MEC (MEC 12 in FIG. 1) is also called a worker node in Kubernetes and a data net (DN: DataNet) in 3GPP, but is called a node in this disclosure.
  • MEC 12 in FIG. 1 MEC 12 in FIG. 1
  • DN DataNet
  • FIG. 15 is a block diagram showing an example of the hardware configuration of a computer that executes the above-mentioned series of processes by a program.
  • a CPU 1001 In a computer, a CPU 1001, a ROM (Read Only Memory) 1002, and a RAM (Random Access Memory) 1003 are interconnected by a bus 1004.
  • An input/output interface 1005 is further connected to the bus 1004.
  • An input unit 1006, an output unit 1007, a memory unit 1008, a communication unit 1009, and a drive 1010 are connected to the input/output interface 1005.
  • the input unit 1006 includes a keyboard, a mouse, a microphone, etc.
  • the output unit 1007 includes a display, a speaker, etc.
  • the storage unit 1008 includes a hard disk, a non-volatile memory, etc.
  • the communication unit 1009 includes a network interface, etc.
  • the drive 1010 drives a removable recording medium 1011 such as a semiconductor memory, a magnetic disk, an optical disk, or a magneto-optical disk.
  • the CPU 1001 loads the programs stored in the ROM 1002 or memory unit 1008 into the RAM 1003 via the input/output interface 1005 and bus 1004, and executes them, thereby carrying out the above-mentioned series of processes.
  • the program executed by the computer (CPU 1001) can be provided by being recorded on a removable recording medium 1011, such as a package medium, for example.
  • the program can also be provided via a wired or wireless transmission medium, such as a local area network, the Internet, or digital satellite broadcasting.
  • a program can be installed in the storage unit 1008 via the input/output interface 1005 by inserting the removable recording medium 1011 into the drive 1010.
  • the program can also be received by the communication unit 1009 via a wired or wireless transmission medium and installed in the storage unit 1008.
  • the program can be pre-installed in the ROM 1002 or storage unit 1008.
  • the processing performed by a computer according to a program does not necessarily have to be performed chronologically in the order described in the flowchart.
  • the processing performed by a computer according to a program also includes processing executed in parallel or individually (for example, parallel processing or processing by objects).
  • a program may be processed by one computer (processor), or may be processed in a distributed manner by multiple computers.
  • a management unit that manages a first node, which is a computer at an edge site connected to a mobile communication network, and a second node, which is a computer at a global site connected to the Internet; a determination unit that, when a first resource value indicating a computing resource of the first node exceeds a threshold, determines a priority of migration of the application container based on detailed information on details of an application container consisting of a plurality of containers operating on the source node and a second resource value indicating a computing resource of the application container, with the first node whose first resource value exceeds the threshold as a source node; a control unit that performs control to move the application container from the source node to a destination node, which is the second node that is a destination of the application container, according to the priority.
  • the first node and the second node are pre-mapped with DNAI defined in 3GPP, The information processing device according to (1), wherein the control unit moves the application container to the second node mapped to the DNAI as the destination node.
  • the determination unit determines, based on a log related to communication in the source node, a time corresponding to a time period during which communication volume in the source node is the lowest, as a movement time for moving the application container;
  • the information processing device according to (2) wherein the control unit moves the application container in accordance with the priority when the current time becomes the movement time.
  • the information processing device described in (3) wherein the determination unit determines the second node located closest to the source node as the destination node based on location information regarding the locations of the first node and the second node.
  • the determination unit determines, as a movement target, a mobile communication terminal that is accessing only the application container to be moved, among the mobile communication terminals that communicate via the mobile communication network, based on a log related to access to the application container;
  • the second resource value is a resource usage rate including a CPU usage rate and a memory usage rate by the application container.
  • the detailed information includes a qosClass defined in kubernetes,
  • the determination unit is The application container that is BestEffort or Burstable is targeted for migration, Giving a higher priority to the movement of the BestEffort application container than the Burstable application container;
  • the information processing device according to (7), wherein a higher priority is given to the movement of the application container whose resource usage rate is equal to or greater than the median than to the application container whose resource usage rate is less than the median.
  • the decision unit assigns a higher priority to the movement of the application container whose access count is less than the median than to the application container whose access count is equal to or greater than the median.
  • the decision unit gives a higher priority to the movement of the application container of Non-GBR than the application container of GBR defined in 3GPP.
  • the second resource value is a resource usage rate including a GPU usage rate by the application container;
  • the decision unit assigns a higher priority to the movement of application containers with resource usage rates equal to or greater than a median among the application containers with no resource restrictions than to application containers with resource usage rates less than the median.
  • the application container corresponds to a pod defined in Kubernetes.
  • An information processing device Manage a first node, which is a computer at an edge site connected to a mobile communication network, and a second node, which is a computer at a global site connected to the Internet;
  • a first resource value indicating a computing resource of the first node exceeds a threshold
  • the first node whose first resource value exceeds the threshold is set as a source node
  • a priority for migration of the application container is determined based on detailed information on details of an application container consisting of a plurality of containers operating on the source node and a second resource value indicating a computing resource of the application container
  • an information processing method for controlling the movement of the application container from the source node to a destination node, which is the second node serving as a destination, according to the priority.
  • 11 MEC management system 12 MEC, 13 Carrier network, 14 Cloud data center, 15 Name server, 16 Wireless terminal, 21, 21A, 21B Node, 22 NEF server, 111 MEC control unit, 112 API unit, 113 AF unit, 114 Database, 121 Management unit, 122 Setting unit, 123 Control unit, 131 Mobility setting unit, 132 Priority determination unit, 141 Mobility control unit, 142 Route control unit, 151 Node table, 152 Pod mobility table, 153 UE mobility table, 154, 154A, 154B, 154C Priority table, 1001 CPU

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)
  • Mobile Radio Communication Systems (AREA)

Abstract

The present disclosure relates to an information processing device and an information processing method capable of offloading a more appropriate target. Provided is an information processing device comprising: a management unit for managing a first node, which is a computer at an edge site connected to a mobile communication network, and a second node, which is a computer at a global site connected to the Internet; a determining unit which, if a first resource value indicating a computing resource at the first node exceeds a threshold, determines a movement priority level, with the first node as a movement source node, on the basis of detailed information relating to details of an application container operating at the movement source node, and a second resource value indicating the computing resource; and a control unit for performing control to move the application container, in accordance with the priority level, from the movement source node to a movement destination node, which is the second node serving as a movement destination. The present disclosure is applicable to equipment constituting an MEC management system, for example.

Description

情報処理装置、及び情報処理方法Information processing device and information processing method
 本開示は、情報処理装置、及び情報処理方法に関し、特に、より適切な対象をオフロードすることができるようにした情報処理装置、及び情報処理方法に関する。 This disclosure relates to an information processing device and an information processing method, and in particular to an information processing device and an information processing method that enable offloading of more appropriate targets.
 近年、マルチアクセスエッジコンピューティング(MEC:Multi-access Edge Computing)に関する技術の研究開発が進められている。MECは、無線端末(UE:User Equipment)の近くにエッジ(エッジサーバ)を分散配置して、エッジサーバで可能な限り処理を実施する技術である。 In recent years, research and development into technology related to Multi-access Edge Computing (MEC) has been progressing. MEC is a technology in which edges (edge servers) are distributed close to wireless terminals (UE: User Equipment) and as much processing as possible is carried out on the edge servers.
 MECに関する技術としては、例えば、特許文献1に開示されている技術がある。特許文献1には、有限なMECホストのリソースに対して、管理システムが、エッジアプリケーションを利用するUE数を予測した予測結果に基づき、エッジサーバのコンピューティングリソースの逼迫度を予測し、その逼迫度の予測結果に基づき、一部のUEの接続先をクラウドアプリケーションに切り替える技術が開示されている。 An example of technology related to MEC is the technology disclosed in Patent Document 1. Patent Document 1 discloses a technology in which a management system predicts the number of UEs using edge applications for the finite resources of an MEC host, and based on the results of the prediction, predicts the degree of congestion of computing resources of an edge server, and switches the connection destination of some of the UEs to a cloud application based on the predicted result of the congestion.
特開2019-62510号公報JP 2019-62510 A
 しかしながら、特許文献1に開示されている技術では、UE数とコンピューティングとの相関性に基づき、アプリケーションプログラムとUEを一様に捉えているが、特定の負荷のかかるアプリケーションプログラムがMECのコンピューティングリソースに影響を与えることが考慮されていない。そのため、オフロードの対象が適切に決定されていない可能性がある。 However, the technology disclosed in Patent Document 1 treats application programs and UEs uniformly based on the correlation between the number of UEs and computing, but does not take into account the impact that application programs that impose specific loads have on the computing resources of the MEC. As a result, there is a possibility that the targets for offloading are not appropriately determined.
 本開示はこのような状況に鑑みてなされたものであり、より適切な対象をオフロードすることができるようにするものである。 This disclosure was made in light of these circumstances, and makes it possible to offload more appropriate targets.
 本開示の一側面の情報処理装置は、移動通信ネットワークに接続されたエッジサイトのコンピュータである第1のノード、及びインターネットに接続されたグローバルサイトのコンピュータである第2のノードを管理する管理部と、前記第1のノードのコンピューティングリソースを示す第1のリソース値が閾値を超えた場合、前記第1のリソース値が閾値を超えた前記第1のノードを移動元ノードとして、前記移動元ノードで動作している複数のコンテナから構成されるアプリケーションコンテナの詳細に関する詳細情報、及び前記アプリケーションコンテナのコンピューティングリソースを示す第2のリソース値に基づいて、前記アプリケーションコンテナの移動の優先度を決定する決定部と、前記移動元ノードから、移動先となる前記第2のノードである移動先ノードに対し、前記優先度に応じて前記アプリケーションコンテナを移動させる制御を行う制御部とを備える情報処理装置である。 An information processing device according to one aspect of the present disclosure includes a management unit that manages a first node, which is a computer at an edge site connected to a mobile communication network, and a second node, which is a computer at a global site connected to the Internet; a determination unit that, when a first resource value indicating the computing resources of the first node exceeds a threshold, determines a priority for moving the application container based on detailed information on the details of an application container consisting of multiple containers operating on the source node and a second resource value indicating the computing resources of the application container, with the first node whose first resource value exceeds the threshold as the source node, and a control unit that controls the movement of the application container from the source node to a destination node, which is the second node to which the application container is to be moved, according to the priority.
 本開示の一側面の情報処理方法は、情報処理装置が、移動通信ネットワークに接続されたエッジサイトのコンピュータである第1のノード、及びインターネットに接続されたグローバルサイトのコンピュータである第2のノードを管理し、前記第1のノードのコンピューティングリソースを示す第1のリソース値が閾値を超えた場合、前記第1のリソース値が閾値を超えた前記第1のノードを移動元ノードとして、前記移動元ノードで動作している複数のコンテナから構成されるアプリケーションコンテナの詳細に関する詳細情報、及び前記アプリケーションコンテナのコンピューティングリソースを示す第2のリソース値に基づいて、前記アプリケーションコンテナの移動の優先度を決定し、前記移動元ノードから、移動先となる前記第2のノードである移動先ノードに対し、前記優先度に応じて前記アプリケーションコンテナを移動させる制御を行う情報処理方法である。 In one aspect of the information processing method of the present disclosure, an information processing device manages a first node, which is a computer at an edge site connected to a mobile communication network, and a second node, which is a computer at a global site connected to the Internet, and when a first resource value indicating the computing resources of the first node exceeds a threshold, the information processing device determines a priority for moving the application container based on detailed information on the details of an application container consisting of multiple containers running on the source node and a second resource value indicating the computing resources of the application container, and controls the movement of the application container from the source node to a destination node, which is the second node to which the application container is to be moved, according to the priority.
 本開示の一側面の情報処理装置、及び情報処理方法においては、移動通信ネットワークに接続されたエッジサイトのコンピュータである第1のノード、及びインターネットに接続されたグローバルサイトのコンピュータである第2のノードが管理され、前記第1のノードのコンピューティングリソースを示す第1のリソース値が閾値を超えた場合、前記第1のリソース値が閾値を超えた前記第1のノードを移動元ノードとして、前記移動元ノードで動作している複数のコンテナから構成されるアプリケーションコンテナの詳細に関する詳細情報、及び前記アプリケーションコンテナのコンピューティングリソースを示す第2のリソース値に基づいて、前記アプリケーションコンテナの移動の優先度が決定され、前記移動元ノードから、移動先となる前記第2のノードである移動先ノードに対し、前記優先度に応じて前記アプリケーションコンテナを移動させる制御が行われる。 In one aspect of the information processing device and information processing method of the present disclosure, a first node, which is a computer at an edge site connected to a mobile communications network, and a second node, which is a computer at a global site connected to the Internet, are managed, and when a first resource value indicating the computing resources of the first node exceeds a threshold, the first node whose first resource value exceeds the threshold is set as a source node, and a priority for moving the application container is determined based on detailed information regarding the details of an application container consisting of multiple containers operating on the source node and a second resource value indicating the computing resources of the application container, and control is performed to move the application container from the source node to a destination node, which is the second node to which the application container is to be moved, according to the priority.
 なお、本開示の一側面の情報処理装置は、独立した装置であってもよいし、1つの装置を構成している内部ブロックであってもよい。 Note that the information processing device of one aspect of the present disclosure may be an independent device or an internal block constituting a single device.
本開示を適用したシステムの構成例を示す図である。FIG. 1 is a diagram illustrating an example of the configuration of a system to which the present disclosure is applied. 図1のMEC管理システムの構成例を示すブロック図である。2 is a block diagram showing a configuration example of the MEC management system of FIG. 1. 図2のMEC制御部の機能的構成例を示すブロック図である。3 is a block diagram showing an example of a functional configuration of an MEC control unit in FIG. 2. ノードテーブルの例を示す図である。FIG. 13 is a diagram illustrating an example of a node table. ポッド移動テーブルの例を示す図である。FIG. 13 illustrates an example of a pod movement table. UE移動テーブルの例を示す図である。FIG. 13 is a diagram illustrating an example of a UE mobility table. 優先度テーブルの例を示す図である。FIG. 13 is a diagram illustrating an example of a priority table. 優先度テーブルの他の第1の例を示す図である。FIG. 11 is a diagram showing a first other example of the priority table. 優先度テーブルの他の第2の例を示す図である。FIG. 11 is a diagram showing a second example of a priority table; 優先度テーブルの他の第3の例を示す図である。FIG. 11 is a diagram showing another third example of the priority table. 移動設定処理の実行時のシーケンスの例を示す図である。FIG. 11 is a diagram illustrating an example of a sequence when a migration setting process is executed. 移動制御処理の実行時のシーケンスの例を示す図である。FIG. 11 is a diagram illustrating an example of a sequence when a movement control process is executed. 移動設定処理の流れを説明するフローチャートである。13 is a flowchart illustrating the flow of a movement setting process. 移動制御処理の流れを説明するフローチャートである。13 is a flowchart illustrating the flow of a movement control process. コンピュータの構成例を示すブロック図である。FIG. 1 is a block diagram illustrating an example of the configuration of a computer.
<システム構成>
 図1は、本開示を適用したシステムの構成例を示す図である。図1において、本開示を適用したシステムは、MEC(Multi-access Edge Computing)管理システム11、MEC12、キャリア網13、クラウド・データセンタ14、ネームサーバ15、及び無線端末16から構成される。
<System Configuration>
Fig. 1 is a diagram showing an example of the configuration of a system to which the present disclosure is applied. In Fig. 1, the system to which the present disclosure is applied is composed of a MEC (Multi-access Edge Computing) management system 11, an MEC 12, a carrier network 13, a cloud data center 14, a name server 15, and a wireless terminal 16.
 MEC管理システム11は、図1のシステムを構成する機器を制御する機能を有する情報処理装置を含んで構成される。情報処理装置は、サーバ等のコンピュータで構成される。MEC管理システム11は、オンプレミス型で提供されるほか、クラウド型で提供されてもよい。 The MEC management system 11 includes an information processing device that has a function of controlling the devices that make up the system in FIG. 1. The information processing device is composed of a computer such as a server. The MEC management system 11 may be provided as an on-premise type or as a cloud type.
 MEC管理システム11は、MEC12やクラウド・データセンタ14に配置されるノード、ノードで動作(稼働)するポッド、及び3GPP(Third Generation Partnership Project)(登録商標)で規定されたDNAI(DN Access Identifier)によりノードに接続する際のキャリア網13の出口を管理する。 The MEC management system 11 manages the nodes placed in the MEC 12 and the cloud data center 14, the pods that operate (run) on the nodes, and the exit of the carrier network 13 when connecting to the nodes using DNAI (DN Access Identifier) defined by 3GPP (Third Generation Partnership Project) (registered trademark).
 MEC管理システム11は、少なくとも1つ以上のエッジサイトに配置されたノードと、少なくとも1以上のグローバルサイトに配置されたノードとの間で制御信号(C-plain)のやり取りをして管理を行う。この通信経路には、例えばVPN(Virtual Private Network)を利用することができる。エッジサイトのノードは、MEC12に配置されたノードであり、グローバルサイトのノードは、クラウド・データセンタ14に配置されたノードである。MEC管理システム11の詳細な構成は、図2で説明する。また、ポッドの詳細についても後述する。 The MEC management system 11 performs management by exchanging control signals (C-plain) between nodes located at at least one edge site and nodes located at at least one global site. For example, a VPN (Virtual Private Network) can be used for this communication path. The nodes at the edge sites are nodes located in the MEC 12, and the nodes at the global sites are nodes located in the cloud data center 14. The detailed configuration of the MEC management system 11 is explained in Figure 2. Details of the pods will also be explained later.
 MEC12は、無線端末16の近くにエッジサーバを分散配置して、エッジサーバで可能な限り処理を実施するための機器で構成される。MEC12には、エッジサイトのノード21Aが配置される。ノード21Aは、サーバ(エッジサーバ)等のコンピュータで構成される。 MEC12 is configured with devices that distribute edge servers near wireless terminals 16 and perform as much processing as possible at the edge servers. Edge site node 21A is placed in MEC12. Node 21A is configured with computers such as servers (edge servers).
 キャリア網13は、移動体通信事業者(通信キャリア)により構築された移動通信ネットワークであって、コアネットワーク等から構成される。例えば、キャリア網13は、第4世代移動通信システム(4G)に対応した4G網や、第5世代移動通信システム(5G)に対応した5G網などを含む。キャリア網13のコアネットワークを構成するサーバ群には、NEF(Network Exposure Function)サーバが含まれる。なお、図1では、キャリア網13の外部にMEC12を配置した例を示しているが、MEC12は、キャリア網13に接続されていればよく、キャリア網13の内部に配置されてもよい。キャリア網13は、インターネットと接続可能であり、インターネットを介してクラウド・データセンタ14と接続される。 The carrier network 13 is a mobile communication network constructed by a mobile communication operator (communication carrier) and is composed of a core network, etc. For example, the carrier network 13 includes a 4G network corresponding to the fourth generation mobile communication system (4G) and a 5G network corresponding to the fifth generation mobile communication system (5G). The group of servers that make up the core network of the carrier network 13 includes a NEF (Network Exposure Function) server. Note that while FIG. 1 shows an example in which the MEC 12 is placed outside the carrier network 13, the MEC 12 only needs to be connected to the carrier network 13 and may be placed inside the carrier network 13. The carrier network 13 can be connected to the Internet and is connected to the cloud data center 14 via the Internet.
 クラウド・データセンタ14は、サービス提供者がサーバや設備を用意してそれらを利用するためのネットワーク環境を提供するクラウド、又はサーバを一箇所に集積して管理するスペースや施設を提供するデータセンタである。クラウド・データセンタ14は、インターネットを介してキャリア網13と接続される。クラウド・データセンタ14には、グローバルサイトのノード21Bが配置される。ノード21Bは、サーバ(クラウドサーバ)等のコンピュータで構成される。以下、ノード21Aとノード21Bを区別する必要がない場合、ノード21とも称する。 Cloud data center 14 is a cloud that provides a network environment for service providers to prepare and use servers and equipment, or a data center that provides space and facilities for collecting and managing servers in one place. Cloud data center 14 is connected to carrier network 13 via the Internet. Node 21B of the global site is located in cloud data center 14. Node 21B is composed of computers such as servers (cloud servers). Hereinafter, when there is no need to distinguish between node 21A and node 21B, they will also be referred to as node 21.
 ネームサーバ15は、ドメイン名とIPアドレスを変換して名前解決を行うDNS(Domain Name System)の機能を有するサーバである。ネームサーバ15は、MEC管理システム11又は無線端末16と接続可能であれば、いずれの通信ネットワークに設けられてもよく、例えば、クラウドとして提供されてもよい。 The name server 15 is a server that has a DNS (Domain Name System) function that converts domain names and IP addresses to perform name resolution. The name server 15 may be provided in any communication network as long as it can be connected to the MEC management system 11 or the wireless terminal 16, and may be provided as a cloud, for example.
 無線端末16は、スマートフォンやタブレット端末等の移動通信端末である。無線端末16は、キャリア網13を介して通信を行う。また、無線端末16は、ネームサーバ15に接続して名前解決を行うことができる。以下、無線端末16のことを、適宜、UE(User Equipment)とも称する。 The wireless terminal 16 is a mobile communication terminal such as a smartphone or tablet terminal. The wireless terminal 16 communicates via the carrier network 13. The wireless terminal 16 can also connect to the name server 15 to perform name resolution. Hereinafter, the wireless terminal 16 will also be referred to as UE (User Equipment) as appropriate.
 なお、図1においては、MEC12とクラウド・データセンタ14がそれぞれ1箇所だけ存在する場合を示したが、それぞれ複数存在する構成としてもよい。例えば、クラウド・データセンタ14が複数存在する場合は、パブリッククラウド、プライベートクラウド、データセンタ上のインスタンスが複数ある、あるいは複数の異なるパブリッククラウド、プライベートクラウド、データセンタ上にインスタンス(ノード)が配置されている場合である。また、MEC12が複数存在する場合は、異なるエッジサイトが存在する場合である。 Note that while FIG. 1 shows a case where there is only one MEC 12 and one cloud data center 14, there may be multiple of each. For example, when there are multiple cloud data centers 14, this is the case when there are multiple instances on public clouds, private clouds, or data centers, or when instances (nodes) are located on multiple different public clouds, private clouds, or data centers. In addition, when there are multiple MECs 12, this is the case when there are different edge sites.
 また、図1においては、スマートフォン等の無線端末16が複数存在する構成が一般的な構成となるが、説明の簡略化のため、1台の無線端末16が存在する場合を例示している。図1において、ネームサーバ15は、必須の構成要素ではなく、必要に応じて設けられる。 In addition, in FIG. 1, a typical configuration would be one in which there are multiple wireless terminals 16 such as smartphones, but to simplify the explanation, a case in which there is one wireless terminal 16 is shown as an example. In FIG. 1, the name server 15 is not a required component and is provided as needed.
 図2は、図1のMEC管理システム11の構成例を示すブロック図である。図2において、MEC管理システム11は、MEC制御部111、API(Application Programming Interface)部112、AF(Application Function)部113、及びデータベース114から構成される。 FIG. 2 is a block diagram showing an example of the configuration of the MEC management system 11 in FIG. 1. In FIG. 2, the MEC management system 11 is composed of an MEC control unit 111, an API (Application Programming Interface) unit 112, an AF (Application Function) unit 113, and a database 114.
 MEC制御部111は、API部112及びAF部113と連携して、ノード21やノード21で動作(稼働)するポッド31に関する処理、及び無線端末16(UE)の経路制御などを行う。ノード21は、MEC12又はクラウド・データセンタ14に配置される。ノード21は、ハードウェアで構成され、ソフトウェアとして実行されるポッド31を複数動作させることが可能である。 The MEC control unit 111 cooperates with the API unit 112 and the AF unit 113 to perform processes related to the node 21 and the pods 31 operating (running) on the node 21, as well as route control for the wireless terminal 16 (UE). The node 21 is placed in the MEC 12 or the cloud data center 14. The node 21 is configured as hardware, and is capable of operating multiple pods 31 executed as software.
 ポッド31は、複数のコンテナから構成されるアプリケーションプログラム(以下、アプリケーションコンテナともいう)である。換言すれば、ポッド31、複数のコンテナの集合であるとも言える。コンテナは、ソフトウェアをパッケージ化したユニットであり、コンテナ型仮想化の技術を用いることで、アプリケーション環境をコンテナという独立空間に構築して仮想化を実現している。例えば、コンテナには、ライブラリ、システムツール、コード、ランタイムなど、ソフトウェアの実行に必要なものが含まれる。 The pod 31 is an application program (hereinafter also referred to as an application container) that is composed of multiple containers. In other words, the pod 31 can be said to be a collection of multiple containers. A container is a unit in which software is packaged, and by using container-based virtualization technology, virtualization is achieved by constructing an application environment in an independent space called a container. For example, a container contains things necessary for software execution, such as libraries, system tools, code, and runtime.
 また、MEC制御部111は、データベース114と連携して、テーブルの参照や更新、移動設定処理や移動制御処理等の処理を行う。MEC制御部111は、通信I/F(不図示)を制御して、ネームサーバ15に接続することで、ポッド31の移動時にDNSレコードのIPアドレスを差し替える機能を有する。ただし、このIPアドレスの差し替え機能は、必須の機能ではない。 The MEC control unit 111 also works in conjunction with the database 114 to refer to and update tables, perform movement setting processing, movement control processing, and other processing. The MEC control unit 111 has a function of replacing the IP address of the DNS record when the pod 31 moves by controlling a communication I/F (not shown) and connecting to the name server 15. However, this IP address replacement function is not a required function.
 例えば、ネームサーバ15として、AWS(Amazon Web Services)(商標)のRoute53を利用していた場合、aws route53 change-resource-record-setsコマンドに、レコードの内容を記述したjson(JavaScript Object Notation)を利用して書き換えることができる。AWSは、Amazon Web Services, Inc.により提供されるクラウドコンピューティングサービスである。jsonは、データ記述言語の1つであり、テキストベースのデータ変換用のフォーマットである。 For example, if Route53 from AWS (Amazon Web Services) (trademark) is used as the name server 15, the aws route53 change-resource-record-sets command can be rewritten using JSON (JavaScript Object Notation) that describes the contents of the record. AWS is a cloud computing service provided by Amazon Web Services, Inc. JSON is a data description language and a format for text-based data conversion.
 API部112は、APIを利用して、ノード21やポッド31とのコントロールプレーンの通信を行うAPIサーバとして構成される。APIは、kubernetes APIを利用することができる。kubernetesは、コンテナをオーケストレーションするためのオープンソースプラットフォームである。kubernetesでは、作成、管理できるコンピューティングの最小のデプロイ可能なユニットをポッドとして規定している。ポッド31は、kubernetesで規定されたポッドに対応している。 The API unit 112 is configured as an API server that uses the API to communicate with the nodes 21 and the pods 31 on the control plane. The API can use the Kubernetes API. Kubernetes is an open source platform for orchestrating containers. Kubernetes defines the smallest deployable unit of computing that can be created and managed as a pod. The pod 31 corresponds to the pod defined in Kubernetes.
 API部112は、kubernetes APIと連携したkubectlコマンドを用いて、例えば、ノード21やポッド31のコンピューティングリソースを取得したり、ポッド31の詳細情報(qosClass)を取得したり、マニフェストファイルを指定してノード21へのデプロイや削除を行ったりすることができる。マニフェストファイルは、ポッド31のリソース情報を記載したテキストファイルである。 The API unit 112 can use the kubectl command in conjunction with the Kubernetes API to, for example, obtain computing resources of the node 21 and the pod 31, obtain detailed information (qosClass) about the pod 31, and specify a manifest file to deploy to or delete from the node 21. The manifest file is a text file that describes resource information about the pod 31.
 ポッド31のqosClassは、Guaranteed,Burstable,BestEffortの3種類に分類される。Guaranteedは、ポッド内のすべてのコンテナのCPU(Central Processing Unit)とメモリにRequests/Limitsが設定されており、かつ、Requests = Limitsである場合に該当する。Burstableは、ポッド内の1つ以上のコンテナのCPU又はメモリにRequestsが設定されており、かつ、Guaranteedを満たさない場合に該当する。BestEffortは、ポッド内のすべてのコンテナにRequests/Limitsが設定されていない場合に該当する。 The qosClass of Pod 31 is classified into three types: Guaranteed, Burstable, and BestEffort. Guaranteed applies when Requests/Limits are set for the CPU (Central Processing Unit) and memory of all containers in the pod, and Requests = Limits. Burstable applies when Requests are set for the CPU or memory of one or more containers in the pod, and Guaranteed is not met. BestEffort applies when Requests/Limits are not set for all containers in the pod.
 Requestsは、ポッド31が最低限必要とするリソースの容量(CPUとメモリの容量)を表す。Limitsは、ポッド31の負荷が高まった際に使用できるリソースの最大容量(CPUとメモリの最大容量)を表す。本開示では、BurstableとBestEffortのポッド31を移動対象(オフロードの対象)としている。 Requests represents the minimum resource capacity (CPU and memory capacity) required by pod 31. Limits represents the maximum resource capacity (maximum CPU and memory capacity) that can be used when the load on pod 31 increases. In this disclosure, Burstable and BestEffort pods 31 are targets for movement (targets for offloading).
 AF部113は、3GPP APIを利用して、NEFサーバ22と連携した処理を行う。3GPP APIは、通信キャリアがサービスプロバイダなどのサードパーティに、キャリア網13のコアノード機能をAPIとして提供する際に、そのAPIと連携する3GPP TS 29.517に記載の機能である。本開示では、3GPP TS 29.522 5.4に記載のTraffic Influence APIと連携することで、予めDNAIとマッピングしたノードに対してアクセスするUEの経路を制御する。 The AF unit 113 performs processing in cooperation with the NEF server 22 using the 3GPP API. The 3GPP API is a function described in 3GPP TS 29.517 that cooperates with an API when a communications carrier provides a core node function of the carrier network 13 to a third party such as a service provider as an API. In this disclosure, the AF unit 113 cooperates with the Traffic Influence API described in 3GPP TS 29.522 5.4 to control the route of a UE accessing a node that has been mapped in advance to a DNAI.
 NEFサーバ22は、キャリア網13の機能(NF:Network Function)を外部のアプリケーションプログラムから利用できるように公開し、ネットワークからの情報をアプリケーションにて把握できるようにするほか、アプリケーションプログラムからNFを制御できるようにするためのAPIを提供している。 The NEF server 22 exposes the functions of the carrier network 13 (NF: Network Function) so that they can be used by external application programs, allowing applications to grasp information from the network, and also provides an API that allows application programs to control the NF.
 データベース114は、HDD(Hard Disk Drive)等の記憶装置に記録される。データベース114は、DB(Data Base)サーバ等により提供されてもよい。データベース114は、MEC制御部111による処理で用いられる各種テーブルを管理している。 The database 114 is recorded in a storage device such as a hard disk drive (HDD). The database 114 may be provided by a database (DB) server or the like. The database 114 manages various tables used in processing by the MEC control unit 111.
 図3は、図2のMEC制御部111の機能的構成例を示すブロック図である。図3において、MEC制御部111は、管理部121、設定部122、及び制御部123を有する。 FIG. 3 is a block diagram showing an example of the functional configuration of the MEC control unit 111 in FIG. 2. In FIG. 3, the MEC control unit 111 has a management unit 121, a setting unit 122, and a control unit 123.
 管理部121は、データベース114で管理される各種テーブルにより、ノード21やポッド31を管理する。また、管理部121は、データベース114で管理されるテーブルによりDNAIとノード21を予めマッピングしておくことで、ノード21に接続する際のキャリア網13の出口を管理する。 The management unit 121 manages the nodes 21 and pods 31 using various tables managed in the database 114. The management unit 121 also manages the exit of the carrier network 13 when connecting to the node 21 by mapping the DNAI and the node 21 in advance using the tables managed in the database 114.
 設定部122は、ポッドとUEの移動に関する設定のための移動設定処理を行う。この移動設定処理では、ポッドの移動の優先度(移動優先度)が決定され、移動対象の候補となるポッドとUEが、決定された優先度とともに設定される。以下、移動対象の候補となるポッドを移動対象候補ポッド、移動対象の候補となるUEを移動対象候補UEともいう。 The setting unit 122 performs a movement setting process for setting the movement of pods and UEs. In this movement setting process, the priority (movement priority) of the pod movement is determined, and the pods and UEs that are candidates for movement are set with the determined priority. Hereinafter, a pod that is a candidate for movement is also referred to as a candidate pod for movement, and a UE that is a candidate for movement is also referred to as a candidate UE for movement.
 設定部122は、移動設定部131、及び優先度決定部132を有する。移動設定部131は、移動対象候補ポッドと移動対象候補UEの移動に関する設定を行う。優先度決定部132は、ポッドの移動の優先度を決定する。 The setting unit 122 has a movement setting unit 131 and a priority determination unit 132. The movement setting unit 131 performs settings related to the movement of candidate pods to be moved and candidate UEs to be moved. The priority determination unit 132 determines the priority of the movement of pods.
 制御部123は、ポッドとUEの移動に関する制御のための移動制御処理を行う。この移動制御処理では、移動対象候補ポッドと移動対象候補UEの中から、移動の優先度(移動優先度)に応じて移動対象ポッドと移動対象UEが決定され、決定された移動対象ポッドと移動対象UEが移動される。以下、移動対象候補ポッドのうち、実際に移動対象となったポッドを移動対象ポッドともいう。移動対象候補UEのうち、実際に移動対象となったUEを移動対象UEともいう。 The control unit 123 performs a movement control process for controlling the movement of pods and UEs. In this movement control process, a pod to be moved and a UE to be moved are determined from among the candidate pods to be moved and the candidate UEs to be moved according to the movement priority (movement priority), and the determined pod to be moved and UE to be moved are moved. Hereinafter, the pod that is actually selected as the movement target among the candidate pods to be moved is also referred to as the movement target pod. The UE that is actually selected as the movement target among the candidate UEs to be moved is also referred to as the movement target UE.
 制御部123は、移動制御部141、及び経路制御部142を有する。移動制御部141は、API部112を制御して、移動対象ポッドの移動を制御する。経路制御部142は、AF部113を制御して、移動対象UEの経路を制御する。 The control unit 123 has a movement control unit 141 and a route control unit 142. The movement control unit 141 controls the API unit 112 to control the movement of the pod to be moved. The route control unit 142 controls the AF unit 113 to control the route of the UE to be moved.
 データベース114は、ノードテーブル151、ポッド移動テーブル152、UE移動テーブル153、及び優先度テーブル154を管理している。 The database 114 manages a node table 151, a pod movement table 152, a UE movement table 153, and a priority table 154.
 ノードテーブル151は、3GPPで規定されたDNAIと、kubernetes上のノード21とをマッピングして管理している。すなわち、キャリア網13から、kubernetesで管理されるノード21を、ポッド31がデプロイされ得るノード(DN:DataNet)として識別させるためにマッピングしている。また、ノードテーブル151は、ノード21ごとに、サイトの情報、サイトがエッジかグローバルかの識別情報、位置を示す緯度経度、IPアドレスなどの情報を管理している。 The node table 151 maps and manages the DNAI defined by 3GPP and the nodes 21 on Kubernetes. In other words, the mapping is performed so that the carrier network 13 can identify the nodes 21 managed by Kubernetes as nodes (DN: DataNet) to which pods 31 can be deployed. In addition, the node table 151 manages information such as site information, identification information as to whether the site is edge or global, latitude and longitude indicating the location, and IP address for each node 21.
 ポッド移動テーブル152は、移動対象候補ポッドの情報を管理している。ポッド移動テーブル152では、移動元となるエッジサイトのノード21Aである移動元ノード、移動先のグローバルサイトのノード21Bである移動先ノード、及び優先度に基づき、移動対象候補ポッドの移動の優先度を管理している。また、ポッド移動テーブル152は、移動対象候補ポッドごとに、移動元ノードの通信量から決定される移動予定時刻などの情報を管理している。 The pod movement table 152 manages information on pods that are candidates for movement. The pod movement table 152 manages the movement priority of the candidate pods for movement based on the source node, which is node 21A of the edge site from which the movement originates, the destination node, which is node 21B of the global site to which the movement originates, and the priority. The pod movement table 152 also manages information such as the scheduled movement time, which is determined from the communication volume of the source node, for each candidate pod for movement.
 UE移動テーブル153は、移動対象候補ポッドにのみアクセスしているUE(移動対象候補UE)の情報を管理している。UE移動テーブル153では、移動元となるDNAIである移動元DNAI、移動先となるDNAIである移動先DNAI、及び優先度に基づき、移動対象候補UEの移動の優先度を管理している。また、UE移動テーブル153は、移動対象候補UEごとに、移動予定時刻などの情報を管理している。UE移動テーブル153のライフサイクルは、ポッド移動テーブル152と同じである。以下、ポッド移動テーブル152とUE移動テーブル153を区別する必要がない場合、移動テーブルとも称する。 UE movement table 153 manages information on UEs (candidate UEs) that are only accessing candidate pods to be moved. UE movement table 153 manages the movement priority of candidate UEs to be moved based on the source DNAI, which is the DNAI from which the movement will be made, the destination DNAI, which is the DNAI to which the movement will be made, and the priority. UE movement table 153 also manages information such as the scheduled movement time for each candidate UE to be moved. The life cycle of UE movement table 153 is the same as that of pod movement table 152. Hereinafter, when there is no need to distinguish between pod movement table 152 and UE movement table 153, they will also be referred to as movement tables.
 優先度テーブル154は、移動対象候補ポッドの優先度を決定するための情報として、ポッド31の詳細情報(qosClass)と、ポッド31のコンピューティングリソースの使用率に応じた優先度を管理している。例えば、優先度テーブル154では、制限がないBesteffort型のポッドほど移動対象としての優先度が高く、またその中でも、コンピューティングリソースの使用率が中央値以上のものほど優先度は高くなるようにする。これにより、より優先度が高い移動対象候補ポッドほど、移動対象ポッド(オフロードの対象)になるようにしている。 The priority table 154 manages detailed information (qosClass) of the pod 31 and a priority according to the utilization rate of the computing resources of the pod 31 as information for determining the priority of candidate pods to be moved. For example, in the priority table 154, the more unrestricted a Besteffort type pod is, the higher its priority as a pod to be moved, and among those, the higher its priority is set for pods with utilization rates of computing resources equal to or above the median. This makes it possible for a pod to be moved with a higher priority as a candidate pod to be moved (a target for offloading).
 なお、移動対象候補UEには、アクセスしている移動対象候補ポッドに対応した優先度が設定されるため、移動対象候補ポッドの優先度を決定することで、自身にアクセスしている移動対象候補UEの優先度を決定しているとも言える。 In addition, since a priority is set for a candidate UE to be moved that corresponds to the candidate pod to be moved that it is accessing, it can be said that by determining the priority of the candidate pod to be moved, the priority of the candidate UE to be moved that is accessing the UE to be moved is also determined.
<テーブル構成>
 MEC管理システム11において、データベース114により管理されるテーブルの構成について説明する。
<Table configuration>
The configuration of a table managed by the database 114 in the MEC management system 11 will be described.
 図4は、図3のノードテーブル151の例を示す図である。図4において、ノードテーブル151は、ノードを識別するノードIDをキーにして、ノードのサイトに関する情報と、ノードとマッピングしたDNAIと、ノードの位置を示す緯度経度と、ノードのIPアドレスとを対応付けて管理している。 FIG. 4 is a diagram showing an example of the node table 151 in FIG. 3. In FIG. 4, the node table 151 uses the node ID that identifies the node as a key to manage information about the node's site, the DNAI mapped to the node, the latitude and longitude that indicate the node's location, and the node's IP address in association with each other.
 例えば、ノードID 1が割り当てられたノードは、Sendai Edge Siteであるエッジサイトに存在し、DNAI1にマッピングされ、緯度と経度がAA,BBで、IPアドレスがaa.aa.aa.aaとなる。また、ノードID 5が割り当てられたノードは、Tokyo Global Siteであるグローバルサイトに存在し、DNAI5にマッピングされ、緯度と経度がII,JJで、IPアドレスがee.ee.ee.eeとなる。ここでは、ノードID 1,5のレコードを例示したが、他のレコードについても同様に管理されている。 For example, a node assigned node ID 1 exists at an edge site, the Sendai Edge Site, is mapped to DNAI1, has latitude and longitude AA,BB, and an IP address aa.aa.aa.aa. A node assigned node ID 5 exists at a global site, the Tokyo Global Site, is mapped to DNAI5, has latitude and longitude II,JJ, and an IP address ee.ee.ee.ee. Here, records with node IDs 1 and 5 are shown as examples, but other records are managed in the same way.
 図5は、図3のポッド移動テーブル152の例を示す図である。図5において、ポッド移動テーブル152は、移動対象候補ポッドを識別するポッドIDをキーにして、移動元となるサイト(移動元ノード)と、移動先となるサイト(移動先ノード)と、移動時刻を示す移動予定時刻と、ドメイン名を示すネームと、移動の優先度を示す優先度とを対応付けて管理している。 FIG. 5 is a diagram showing an example of the pod movement table 152 in FIG. 3. In FIG. 5, the pod movement table 152 uses a pod ID that identifies a candidate pod to be moved as a key, and manages the source site (source node), destination site (destination node), scheduled movement time indicating the time of movement, name indicating the domain name, and priority indicating the priority of the movement in association with each other.
 例えば、ポッドID 1が割り当てられたポッドは、2021/11/1 1:00に、Sendai Edge SiteのノードからTokyo Global Siteのノードに移動予定であり、優先度1であるため、移動の優先度が最も高い。また、ポッドID 3が割り当てられたポッドは、2021/11/1 2:00に、Fukuoka Edge SiteのノードからOsaka Global Siteのノードに移動予定であり、優先度1であるため、移動の優先度が最も高い。ここでは、ポッドID 1,3のレコードを例示したが、他のレコードについても同様に管理されている。 For example, a pod assigned pod ID 1 is scheduled to move from a node in the Sendai Edge Site to a node in the Tokyo Global Site at 1:00 on 1 November 2021, and has the highest priority for movement because it has priority 1. Similarly, a pod assigned pod ID 3 is scheduled to move from a node in the Fukuoka Edge Site to a node in the Osaka Global Site at 2:00 on 1 November 2021, and has the highest priority for movement because it has priority 1. Here, records with pod IDs 1 and 3 are shown as examples, but other records are managed in the same way.
 図6は、図3のUE移動テーブル153の例を示す図である。図6において、UE移動テーブル153は、移動対象候補UEを識別するUEIDをキーにして、移動元DNAIと、移動先DNAIと、移動時刻を示す移動予定時刻と、IPアドレスと、移動の優先度を示す優先度とを対応付けて管理している。なお、IPアドレスの代わりに、キャリア網13でUEを識別するための識別情報であるSUPI(Subscription Permanent Identifier)を用いても構わない。 FIG. 6 is a diagram showing an example of the UE movement table 153 in FIG. 3. In FIG. 6, the UE movement table 153 uses the UEID that identifies the candidate UE to be moved as a key, and manages the source DNAI, destination DNAI, scheduled movement time indicating the time of movement, IP address, and priority indicating the priority of the movement in association with each other. Note that instead of the IP address, a Subscription Permanent Identifier (SUPI), which is identification information for identifying the UE in the carrier network 13, may be used.
 例えば、UEID 1が割り当てられたUEは、2021/11/1 1:00に、DNAI1からDNAI5に移動予定であり、IPアドレスがaa.aa.aa.1で、優先度1であるため、移動の優先度が最も高い。また、UEID 5が割り当てられたUEは、2021/11/1 2:00に、DNAI4からDNAI6に移動予定であり、IPアドレスがbb.bb.bb.1で、優先度1であるため、移動の優先度が最も高い。ここでは、UEID 1,5のレコードを例示したが、他のレコードについても同様に管理されている。 For example, a UE assigned UEID 1 is scheduled to move from DNAI1 to DNAI5 at 1:00 on 1 November 2021, and has the highest priority for movement because its IP address is aa.aa.aa.1 and its priority is 1. Similarly, a UE assigned UEID 5 is scheduled to move from DNAI4 to DNAI6 at 2:00 on 1 November 2021, and has the highest priority for movement because its IP address is bb.bb.bb.1 and its priority is 1. Here, records for UEID 1 and 5 are shown as examples, but other records are managed in the same way.
 図7は、図3の優先度テーブル154の例を示す図である。図7において、優先度テーブル154は、移動対象候補となるポッドやUEの移動の優先度を決定するために用いられるテーブルであって、ポッドの詳細情報(qosClass)と、ポッドのコンピューティングリソースの使用率(ポッドリソース使用率)に応じた優先度を管理している。図7の例では、優先度の数字が低いものほど、優先度が高いことを表している。ポッドリソース使用率は、ポッドのCPU使用率やメモリ使用率を含む。 FIG. 7 is a diagram showing an example of the priority table 154 in FIG. 3. In FIG. 7, the priority table 154 is a table used to determine the priority of the movement of pods and UEs that are candidates for movement, and manages the priority according to the detailed information of the pod (qosClass) and the utilization rate of the pod's computing resources (pod resource utilization rate). In the example of FIG. 7, the lower the priority number, the higher the priority. The pod resource utilization rate includes the CPU utilization rate and memory utilization rate of the pod.
 例えば、本開示では、Burstable型とBestEffort型のポッドを移動対象とし、制限がないBesteffort型のポッドほど移動対象としての優先度が高く、またその中でも、ポッドリソース使用率が中央値以上のものほど優先度は高くするので、BestEffort型のポッドで、かつ、ポッドリソース使用率が中央値以上のものが、優先度1となって移動の優先度が最も高くなるようにする。また、BestEffort型のポッドで、かつ、ポッドリソース使用率が中央値未満のものが、優先度2となって移動の優先度が優先度1の次に高くなるようにする。 For example, in this disclosure, Burstable and BestEffort type pods are targeted for movement, and the more unlimited the Besteffort type pod is, the higher the priority for being targeted for movement. Among these, pods with pod resource utilization rates equal to or greater than the median are given a higher priority, so that BestEffort type pods with pod resource utilization rates equal to or greater than the median are given priority 1, making them the highest priority for movement. Also, BestEffort type pods with pod resource utilization rates less than the median are given priority 2, making them the second highest priority for movement after priority 1.
 また、Burstable型のポッドで、かつ、ポッドリソース使用率が中央値以上のものは、優先度3となって移動の優先度が優先度2の次に高くなるようにする。Burstable型のポッドで、かつ、ポッドリソース使用率が中央値未満のものは、優先度4となって移動の優先度が最も低くなるようにする。なお、ここでは、対象のノードでポッドが複数動作している場合に、それらのポッドにおけるポッドリソース使用率の中央値を基準として用いることができる。 Furthermore, for burstable pods with pod resource utilization rates equal to or greater than the median, they are assigned a priority of 3, making them the second highest priority for migration after priority 2. For burstable pods with pod resource utilization rates below the median, they are assigned a priority of 4, making them the lowest priority for migration. Note that here, if multiple pods are running on the target node, the median pod resource utilization rates of those pods can be used as the standard.
 このように、優先度テーブル154の優先度は、ポッドの詳細情報(qosClass)とポッドリソース使用率の2軸で優先度を決定することができる。また、これらの2軸に限らず、より状況に適した優先度決めを実現するために、軸を増やしても構わない。図8乃至図10は、優先度テーブルの他の例を示す図である。 In this way, the priority of the priority table 154 can be determined based on two axes: detailed information about the pod (qosClass) and the pod resource usage rate. Furthermore, it is not limited to these two axes, and more axes can be added to achieve a more suitable priority determination for the situation. Figures 8 to 10 show other examples of priority tables.
 図7の優先度テーブル154では、ポッドリソース使用率の中央値を基準にして上位又は下位になるかで優先度を決めているが、ポッドリソース使用率が中央値近辺に固まっているケースも想定される。そこで、ポッドリソース使用率の分散値が閾値未満となる場合、所定期間のアクセスログから、UE等からのアクセス数が低いポッドの優先度を高くする。 In the priority table 154 in FIG. 7, the priority is determined based on whether the pod resource utilization rate is high or low, based on the median value of the pod resource utilization rate, but cases in which the pod resource utilization rates are concentrated near the median value are also assumed. Therefore, when the variance value of the pod resource utilization rate is less than the threshold value, the priority of a pod with a low number of accesses from UEs, etc., based on the access log for a specified period of time is increased.
 図8は、ポッドの詳細情報(qosClass)とポッドのアクセス数に応じた優先度を管理する優先度テーブル154Aを示している。優先度テーブル154Aでは、移動対象のBurstable型とBestEffort型のポッドのうち、BestEffort型のポッドで、かつ、アクセス数が中央値未満のものが、優先度1となって移動の優先度が最も高くなるようにする。また、BestEffort型のポッドで、かつ、アクセス数が中央値以上のものを、優先度2にする。同様にして、Burstable型のポッドであって、アクセス数が中央値未満のものを優先度3、アクセス数が中央値以上のものを優先度4にする。 Figure 8 shows a priority table 154A that manages the priority according to the detailed information (qosClass) of a pod and the number of accesses to the pod. In the priority table 154A, of the Burstable and BestEffort type pods to be moved, BestEffort type pods with an access count below the median are assigned priority 1, making them the highest priority for movement. Also, BestEffort type pods with an access count equal to or greater than the median are assigned priority 2. Similarly, Burstable type pods with an access count below the median are assigned priority 3, and those with an access count equal to or greater than the median are assigned priority 4.
 このように、ポッドリソース使用率の分散値が閾値未満となる場合、すなわち、ポッドリソース使用率が中央値に固まっている場合に、優先度テーブル154Aを用いて、UE等からのアクセス数が少ないにもかかわらず、リソース使用率が高いポッドの優先度を高くして、優先的に移動させることで、移動時に発生する通信断等の影響を小さくすることができる。 In this way, when the variance value of the pod resource utilization rate is less than the threshold value, i.e., when the pod resource utilization rate is fixed at the median value, the priority table 154A is used to increase the priority of pods with high resource utilization rates despite a low number of accesses from UEs, etc., and to move them preferentially, thereby reducing the impact of communication interruptions, etc. that occur during the move.
 ところで、ポッドをグローバルサイトのノードに移動させることにより、MEC12(エッジサイトのノード)での動作時に比べて、5G網として構成されるキャリア網13におけるネットワーク上のホップ数や他エリアのトラフィックの影響が増えることで、遅延や揺らぎが増える傾向にある。ここでは、3GPP TS 23.501のTable 5.7.4-1の「Standardized 5QI to QoS characteristics mapping」に記載のGBR(Guaranteed Bit Rate)のResource TypeのPDU sessionが利用するアプリケーションプログラムが影響を受けないようにするために、可能な限り対象のアプリケーションプログラムを、MEC12(エッジサイトのノード)に残したほうが、ネットワーク上の品質を最小限に抑えることができる。 By the way, moving a pod to a global site node tends to increase delays and fluctuations compared to when it is operating on MEC12 (edge site node) due to the increased number of hops on the network in the carrier network 13 configured as a 5G network and the influence of traffic in other areas. Here, in order to prevent application programs used by PDU sessions of Resource Type GBR (Guaranteed Bit Rate) described in "Standardized 5QI to QoS characteristics mapping" in Table 5.7.4-1 of 3GPP TS 23.501 from being affected, it is better to leave the target application programs on MEC12 (edge site node) as much as possible, in order to minimize quality on the network.
 よって、上記の3GPPにおけるQoSについて、MEC12(エッジサイトのノード)で動作するアプリケーションプログラムに接続するUEがどのQoSを利用したPDU sessionを構築するか事前に知っている場合、あるいは、3GPP TS 29.122の5.14の「AsSessionWithQoS」と、本開示の3GPP APIが相互接続しており事前に設定されたQoSsubscription(qosReference)を取得できる場合(API経由で上記5.7.4-1のTableに記載のQoSを認識し得る場合)には、移動の優先順位について、Non-GBRの優先度を高くするようにする。 Therefore, with regard to the QoS in 3GPP mentioned above, if the UE connected to the application program running on MEC12 (edge site node) knows in advance which QoS will be used to construct a PDU session, or if "AsSessionWithQoS" in 5.14 of 3GPP TS 29.122 is interconnected with the 3GPP API disclosed herein and a pre-configured QoSsubscription (qosReference) can be obtained (if the QoS listed in the Table in 5.7.4-1 above can be recognized via the API), then the priority of Non-GBR will be given a higher priority for movement.
 図9は、ポッドの詳細情報(qosClass)と、ポッドが利用する通信(UEのPDU session)における5GのGBR/Non-GBRの割り当てに応じた優先度を管理する優先度テーブル154Bを示している。優先度テーブル154Bでは、移動対象のBurstable型とBestEffort型のポッドのうち、BestEffort型のポッドで、かつ、Non-GBRであるものが、優先度1となって移動の優先度が最も高くなるようにする。また、BestEffort型のポッドで、かつ、GBRであるものを、優先度2にする。同様にして、Burstable型のポッドであって、Non-GBRであるものを優先度3、GBRであるものを優先度4にする。 Figure 9 shows priority table 154B, which manages detailed pod information (qosClass) and priority according to the 5G GBR/Non-GBR allocation in the communications used by the pod (UE PDU session). In priority table 154B, among the Burstable and BestEffort pods to be moved, BestEffort pods that are both Non-GBR are assigned priority 1, making them the highest priority for movement. Also, BestEffort pods that are GBR are assigned priority 2. Similarly, Burstable pods that are Non-GBR are assigned priority 3, and GBR pods are assigned priority 4.
 このように、GBRのアクセスとそれを利用するポッドほど、優先度を下げて、MEC12(エッジサイトのノード)に残りやすくしている。この優先度決めの軸は、上述したポッドリソース使用率の軸や、アクセス数の軸の後に入れることを想定している。 In this way, the more GBR access and the pods that use it, the lower the priority, making it more likely that they will remain in MEC12 (the edge site node). This axis for determining priorities is intended to be placed after the axis of pod resource utilization and the axis of number of accesses mentioned above.
 上述した説明では、CPUを利用するポッドを想定した場合を示したが、GPU(Graphics Processing Unit)等の他のプロセッサを利用するポッドを対象としてもよい。GPUに対するリソースについては、現状では、CPUほどの細かい制御は効かないため、limited又は非limitedの二値とすることができる。limitedは、ポッドにリソースに関する制限がある場合が該当し、非limitedは、ポッドにリソースに関する制限がない場合が該当する。 The above explanation assumes a pod that uses a CPU, but it may also be the case that the target is a pod that uses other processors such as a GPU (Graphics Processing Unit). Currently, GPU resources cannot be controlled as precisely as CPU resources, so they can be set to two values: limited or non-limited. Limited applies when there are resource restrictions on the pod, and non-limited applies when there are no resource restrictions on the pod.
 図10は、GPUを利用するポッドを対象とした場合における非limitedのポッドのコンピューティングリソースの使用率(ポッドリソース使用率)に応じた優先度を管理する優先度テーブル154Cを示している。ここでのポッドリソース使用率は、GPU使用率となる。優先度テーブル154Cでは、非limitedのポッドのうち、GPU使用率が中央値以上のポッドが優先度1となって移動の優先度が最も高くなるようにする。また、GPU使用率が中央値未満のポッドを、優先度2にする。 FIG. 10 shows a priority table 154C that manages priorities according to the utilization rate of computing resources (pod resource utilization rate) of non-limited pods when targeting pods that use GPUs. The pod resource utilization rate here is the GPU utilization rate. In the priority table 154C, among non-limited pods, pods with GPU utilization rates equal to or greater than the median are assigned priority 1, making them the highest priority for migration. Additionally, pods with GPU utilization rates less than the median are assigned priority 2.
 このように、ポッドリソース使用率は、CPU使用率とメモリ使用率に限らず、ポッドが利用するリソースに応じてGPU使用率などのリソース使用率を用いても構わない。例えば、GPUを利用するポッドを対象とする場合には、非limitedのポッドに対してGPU使用率の中央値以上か又は未満かで同様の優先度を決定することができる。 In this way, pod resource utilization is not limited to CPU utilization and memory utilization, and resource utilization such as GPU utilization may be used depending on the resources used by the pod. For example, when targeting pods that use GPUs, a similar priority can be determined for non-limited pods based on whether their GPU utilization is above or below the median value.
<シーケンス>
 図11は、MEC管理システム11における移動設定処理の実行時のシーケンスの例を示す図である。図11においては、装置間のデータ(信号)のやり取りを双方向の矢印で示している。移動設定処理の実行時において、MEC管理システム11は、MEC12に配置されるエッジサイトのノード21Aと、クラウド・データセンタ14に配置されるグローバルサイトのノード21Bのうち、移動元ノードとなるノード21Aとの間でデータをやり取りする。
<Sequence>
Fig. 11 is a diagram showing an example of a sequence when a migration setting process is executed in the MEC management system 11. In Fig. 11, the exchange of data (signals) between devices is indicated by bidirectional arrows. When executing the migration setting process, the MEC management system 11 exchanges data between a node 21A at an edge site arranged in the MEC 12 and a node 21B at a global site arranged in the cloud data center 14, which is a migration source node.
 MEC管理システム11においては、API部112が、kubectl top nodeコマンドを実行することで、ノード21Aのコンピューティングリソース(ノードリソース使用率)を取得する(S11)。そして、MEC制御部111が、取得したノードリソース使用率を閾値と比較する閾値確認を行う(S12)。ノードリソース使用率は、ノード21AのCPU使用率やメモリ使用率等を含む。 In the MEC management system 11, the API unit 112 executes the kubectl top node command to obtain the computing resources (node resource usage) of node 21A (S11). Then, the MEC control unit 111 performs a threshold check to compare the obtained node resource usage with a threshold (S12). The node resource usage includes the CPU usage and memory usage of node 21A, etc.
 MEC管理システム11では、閾値確認(S12)で、ノードリソース使用率が閾値を超えると判定された場合、以降の処理が行われ、移動の優先度が決定される。 If the MEC management system 11 determines in the threshold check (S12) that the node resource usage rate exceeds the threshold, the following process is carried out and the migration priority is determined.
 すなわち、API部112が、kubectl describe podコマンドを実行することで、ノード21Aで動作している各ポッド31の詳細情報(qosClass)を取得し(S13)、kubectl top podコマンドを実行することで、ポッド31のコンピューティングリソース(ポッドリソース使用率)を取得する(S14)。ポッドリソース使用率は、ポッド31のCPU使用率やメモリ使用率等を含む。そして、MEC制御部111が、取得した詳細情報(qosClass)とポッドリソース使用率を用いて、移動の優先度を決定する(S15)。これにより、優先度テーブル154が作成される。 In other words, the API unit 112 executes the kubectl describe pod command to obtain detailed information (qosClass) for each pod 31 running on node 21A (S13), and executes the kubectl top pod command to obtain the computing resources (pod resource utilization) of the pod 31 (S14). The pod resource utilization includes the CPU utilization and memory utilization of the pod 31. The MEC control unit 111 then uses the obtained detailed information (qosClass) and pod resource utilization to determine the migration priority (S15). This creates the priority table 154.
 MEC管理システム11では、移動の優先度が決定されると、移動対象候補となるポッドとUEが、優先度とともに、ポッド移動テーブル152とUE移動テーブル153にそれぞれ設定される。 In the MEC management system 11, once the priority of the movement is determined, the pods and UEs that are candidates for movement are set in the pod movement table 152 and the UE movement table 153, respectively, along with their priorities.
 すなわち、MEC制御部111が、ノードテーブル151を参照して、緯度経度の情報を用いた所定の距離計算を行い、移動先のノード21B(グローバルサイトのノード)を決定する(S16)。また、移動元のノード21Aにおける通信ログから所定期間の通信量を取得する(S17)ことで、MEC制御部111が、通信量に基づいて移動時刻(移動予定時刻)を決定する(S18)。そして、MEC制御部111は、移動対象候補ポッドに関する情報として、決定した優先度、移動先ノード、移動時刻などの情報を、ポッド移動テーブル152に設定して更新する(S19)。 In other words, the MEC control unit 111 refers to the node table 151, performs a predetermined distance calculation using latitude and longitude information, and determines the destination node 21B (node at the global site) (S16). The MEC control unit 111 also obtains the communication volume for a predetermined period from the communication log of the source node 21A (S17), and determines the movement time (scheduled movement time) based on the communication volume (S18). The MEC control unit 111 then sets and updates the pod movement table 152 with information about the candidate pod to be moved, such as the determined priority, destination node, and movement time (S19).
 また、MEC制御部111は、移動元のノード21Aにおけるアクセスログから、移動対象候補ポッドにのみアクセスしているUEのIPアドレス(UEIP)を抽出して、抽出したUEIPのUEを、移動対象候補UEとする(S20)。そして、MEC制御部111は、移動対象候補UEに関する情報として、移動元DNAI、移動先DNAI、移動時刻、優先度などの情報を、UE移動テーブル153に設定して更新する(S21)。 The MEC control unit 111 also extracts the IP address (UEIP) of the UE that is accessing only the candidate pod to be moved from the access log in the source node 21A, and sets the UE with the extracted UEIP as a candidate UE to be moved (S20). The MEC control unit 111 then sets and updates the UE movement table 153 with information about the candidate UE to be moved, such as the source DNAI, destination DNAI, movement time, and priority (S21).
 このように、移動設定処理の実行時に、図11のシーケンスの流れで処理が行われ、その後、移動設定処理で設定された情報に基づいた移動制御処理が実行される。図12は、MEC管理システム11における移動制御処理の実行時のシーケンスの例を示す図である。移動制御処理の実行時において、MEC管理システム11は、クラウド・データセンタ14に配置されるノード21Bと、ネームサーバ15と、NEFサーバ22との間でもデータをやり取りする。 In this way, when the mobility setting process is executed, the process is performed according to the sequence flow of FIG. 11, and then the mobility control process is executed based on the information set in the mobility setting process. FIG. 12 is a diagram showing an example of the sequence when the mobility control process is executed in the MEC management system 11. When the mobility control process is executed, the MEC management system 11 also exchanges data between the node 21B placed in the cloud data center 14, the name server 15, and the NEF server 22.
 MEC管理システム11においては、MEC制御部111が、時刻を確認し(S31)、対象時刻になったとき、以降の処理が行われる。すなわち、MEC制御部111が、図11のステップS19で更新したポッド移動テーブル152から、最も優先度の高いレコードを特定し、特定したレコードの移動対象候補ポッドを、移動対象ポッドに決定する(S32)。また、MEC制御部111が、図11のステップS21で更新したUE移動テーブル153から、最も優先度の高いレコードを特定し、特定したレコードの移動対象候補UEを、移動対象UEに決定する(S33)。 In the MEC management system 11, the MEC control unit 111 checks the time (S31), and when the target time arrives, the subsequent processing is performed. That is, the MEC control unit 111 identifies the record with the highest priority from the pod movement table 152 updated in step S19 of FIG. 11, and determines the candidate pod to be moved of the identified record as the pod to be moved (S32). The MEC control unit 111 also identifies the record with the highest priority from the UE movement table 153 updated in step S21 of FIG. 11, and determines the candidate UE to be moved of the identified record as the UE to be moved (S33).
 API部112が、MEC制御部111からの制御に従い、移動元のノード21Aで動作していた移動対象ポッドを、移動先のノード21Bにデプロイする(S34)。ここでのデプロイは、kubectlコマンドにてマニフェストファイルを指定することで行われる。さらに、AF部113が、TrafficInfluence APIを利用して、移動対象UEの経路を切り替える(S36)。TrafficInfluence APIの利用に際しては、AF部113が、MEC制御部111からの制御に従い、移動対象UEのUEIPと移動先DNAIを指定することで、NEFサーバ22と連携して、UEの経路制御が実施される。このようにして、優先度に応じた移動対象ポッドと移動対象UEの移動が実現される。 The API unit 112, under control of the MEC control unit 111, deploys the pod to be moved that was running on the source node 21A to the destination node 21B (S34). The deployment here is performed by specifying a manifest file with the kubectl command. Furthermore, the AF unit 113 uses the TrafficInfluence API to switch the route of the UE to be moved (S36). When using the TrafficInfluence API, the AF unit 113, under control of the MEC control unit 111, specifies the UEIP and destination DNAI of the UE to be moved, and performs route control of the UE in cooperation with the NEF server 22. In this way, the movement of the pod to be moved and the UE to be moved according to priority is realized.
 その後、MEC管理システム11では、MEC制御部111が、ポッド移動テーブル152における移動対象ポッドのネームに対してDNSレコードの更新を行い(S37)、移動先のノード21BのIPアドレスを、新IPアドレスとしてネームサーバ15に通知する(S38)。これにより、ネームサーバ15では、移動対象ポッドのネームが登録されている場合に、IPアドレスを、移動先のノード21BのIPアドレスに差し替えて更新することができる。 Then, in the MEC management system 11, the MEC control unit 111 updates the DNS record for the name of the pod to be moved in the pod movement table 152 (S37), and notifies the name server 15 of the IP address of the destination node 21B as the new IP address (S38). As a result, if the name of the pod to be moved is registered in the name server 15, it can update the IP address by replacing it with the IP address of the destination node 21B.
 さらに、API部112が、MEC制御部111からの制御に従い、移動元のノード21Aから移動対象ポッドを削除する(S39)。ここでの削除は、kubectlコマンドにてマニフェストファイルを指定することで行われる。最後に、MEC制御部111が、ポッド移動テーブル152から移動対象ポッドのレコードを削除する(S40)。また、MEC制御部111が、UE移動テーブル153から移動対象UEのレコードを削除する(S40)。 Furthermore, the API unit 112, under control of the MEC control unit 111, deletes the pod to be moved from the source node 21A (S39). The deletion here is performed by specifying the manifest file with the kubectl command. Finally, the MEC control unit 111 deletes the record of the pod to be moved from the pod movement table 152 (S40). In addition, the MEC control unit 111 deletes the record of the UE to be moved from the UE movement table 153 (S40).
<処理の流れ>
 次に、図13のフローチャートを参照して、MEC管理システム11のMEC制御部111により実行される移動設定処理の流れを説明する。図13の説明では、ノードテーブル151で管理されているノードのうち、エッジサイトであるSendai Edge SiteとFukuoka Edge Siteにそれぞれ配置されたノード21Aのコンピューティングリソースの使用率(ノードリソース使用率)が閾値を超えた場合を例示する。
<Processing flow>
Next, the flow of the migration setting process executed by the MEC control unit 111 of the MEC management system 11 will be described with reference to the flowchart in Fig. 13. The explanation in Fig. 13 illustrates an example in which the utilization rate of computing resources (node resource utilization rate) of the node 21A, which is located at the edge sites Sendai Edge Site and Fukuoka Edge Site among the nodes managed in the node table 151, exceeds a threshold value.
 移動設定部131は、API部112を制御して、エッジサイトに配置されたノード21Aのノードリソース使用率を取得し(S111)、取得したノードリソース使用率が閾値を超えるかどうかを判定する(S112)。ここでは、API部112が、kubectl top nodeコマンドを実行することで、ノードリソース使用率として、ノード21AのCPU使用率やメモリ使用率が取得され、閾値と比較される。ノードリソース使用率は、特定の時刻(時間帯)における使用率に限らず、例えば、ロードアベレージを用いてもよい。 The movement setting unit 131 controls the API unit 112 to acquire the node resource utilization rate of the node 21A placed at the edge site (S111), and determines whether the acquired node resource utilization rate exceeds a threshold value (S112). Here, the API unit 112 executes the kubectl top node command to acquire the CPU utilization rate and memory utilization rate of the node 21A as the node resource utilization rate, which are compared with the threshold value. The node resource utilization rate is not limited to the utilization rate at a specific time (time period), and for example, the load average may be used.
 ノードリソース使用率が閾値を超えると判定された場合(S112のYes)、移動設定部131が、データベース114における移動テーブルの有無を確認する(S113)。例えば、新規に、ノードリソース使用率が閾値を超えた場合には、ポッド移動テーブル152とUE移動テーブル153は共に存在しておらず(S113のNo)、移動設定部131は、API部112を制御して、ノード21Aで動作する各ポッドの詳細情報を取得する(S114)。この例では、API部112が、kubectl describe podコマンドを実行することで、Sendai Edge Siteのノード21AとFukuoka Edge Siteのノード21Aで動作している各ポッドの詳細情報(qosClass)を取得する。そして、移動設定部131は、取得したqosClassの種類を確認して、BestEffort型又はBurstable型であると判定されたポッドに対し、ステップS116乃至S122の処理を行う(S115のYes)。 If it is determined that the node resource usage exceeds the threshold (Yes in S112), the migration setting unit 131 checks whether or not there is a migration table in the database 114 (S113). For example, if the node resource usage exceeds the threshold for the first time, neither the pod migration table 152 nor the UE migration table 153 exists (No in S113), and the migration setting unit 131 controls the API unit 112 to acquire detailed information about each pod running on the node 21A (S114). In this example, the API unit 112 executes the kubectl describe pod command to acquire detailed information (qosClass) about each pod running on the node 21A of the Sendai Edge Site and the node 21A of the Fukuoka Edge Site. The migration setting unit 131 then checks the type of qosClass acquired, and performs the processes of steps S116 to S122 for pods determined to be of the BestEffort type or Burstable type (Yes in S115).
 すなわち、移動設定部131は、API部112を制御して、ポッドのコンピューティングリソースの使用率(ポッドリソース使用率)を取得する(S116)。ここでは、API部112が、kubectl top podコマンドを実行することで、ポッドリソース使用率として、ポッドのCPU使用率やメモリ使用率が取得される。ポッドリソース使用率は、特定の時刻(時間帯)における使用率に限らず、例えば、ロードアベレージを用いてもよい。 In other words, the migration setting unit 131 controls the API unit 112 to obtain the utilization rate of the pod's computing resources (pod resource utilization rate) (S116). Here, the API unit 112 executes the kubectl top pod command to obtain the pod's CPU utilization rate and memory utilization rate as the pod resource utilization rate. The pod resource utilization rate is not limited to the utilization rate at a specific time (time period), and for example, the load average may be used.
 優先度決定部132は、qosClassとポッドリソース使用率に基づいて、ポッドの移動の優先度を決定する(S117)。これにより、優先度テーブル154が作成される。例えば、図7の優先度テーブル154では、制限がかかっていないBestEffort型の移動の優先度がより高く、かつ、その中でもポッドリソースの使用率が中央値以上のポッドを、移動の優先度が最も高い優先度1としている。なお、ここでは、アクセス数を用いた優先度テーブル154A(図8)や、GBR/Non-GBRの割り当てに応じた優先度テーブル154B(図9)が作成されてもよい。 The priority determination unit 132 determines the priority of the pod movement based on the qosClass and the pod resource usage rate (S117). This creates a priority table 154. For example, in the priority table 154 of FIG. 7, the priority of unrestricted BestEffort type movement is higher, and among them, pods with pod resource usage rates equal to or higher than the median are assigned the highest movement priority of 1. Note that here, a priority table 154A (FIG. 8) using the number of accesses or a priority table 154B (FIG. 9) according to the GBR/Non-GBR allocation may also be created.
 次に、移動設定部131は、ノードテーブル151を参照して、ノードテーブル151に格納されたグローバルサイトのノードの中から、緯度経度を用いて計算される移動元ノードからの距離が最も近いノードを、移動先ノードに決定する(S118)。例えば、図4のノードテーブル151では、Sendai Edge Siteの移動元ノードに対し、距離が最も近いグローバルサイト(Global Site)であるTokyo Global Siteのノードを移動先に決定する。また、Fukuoka Edge Siteの移動元ノードに対し、距離が最も近いグローバルサイトであるOsaka Global Siteのノードを移動先に決定する。 Next, the movement setting unit 131 refers to the node table 151 and determines, from among the nodes of global sites stored in the node table 151, the node with the closest distance from the source node calculated using latitude and longitude as the destination node (S118). For example, in the node table 151 of FIG. 4, the node of the Tokyo Global Site, which is the global site with the closest distance to the source node of the Sendai Edge Site, is determined as the destination. Also, the node of the Osaka Global Site, which is the global site with the closest distance to the source node of the Fukuoka Edge Site, is determined as the destination.
 次に、移動設定部131は、移動元ノードの通信量が最小の時間帯を決定する(S119)。例えば、移動元ノードにおける所定期間(例えば一週間)の通信ログを参照して、その期間の通信量から各日を平均化し、通信量が最小となる時間帯を算出する。この例では、Sendai Edge Siteのノードでは1:00,Fukuoka Edge Siteのノードでは2:00が、通信量が最も少ない時間帯として算出される。なお、決定される時間帯は、基本的には夜間を想定しているが、例えば、グローバルサイトが海外に及ぶ場合には、時差の関係で昼間になる可能性もある。ここで決定される時間帯に対応した日時が、ポッドの移動時刻(移動予定時刻)となる。 Next, the movement setting unit 131 determines the time period with the least amount of communication on the source node (S119). For example, by referring to the communication log for a specified period (e.g., one week) on the source node, the communication volume for that period is averaged for each day to calculate the time period with the least amount of communication. In this example, 1:00 is calculated as the time period with the least amount of communication on the node at the Sendai Edge Site, and 2:00 is calculated as the time period with the least amount of communication on the node at the Fukuoka Edge Site. Note that the time period to be determined is basically assumed to be at night, but if the global site extends overseas, for example, it may be daytime due to the time difference. The date and time corresponding to the time period determined here becomes the time of movement of the pod (scheduled time of movement).
 移動設定部131は、ステップS117乃至S119で決定された優先度、移動先ノード、及び移動時刻などの情報を、移動対象候補ポッドに関する情報として、ポッド移動テーブル152に設定して更新する(S120)。例えば、図5のポッド移動テーブル152においては、ポッドID 1,2のレコードとして、2021/11/1 1:00である移動予定時刻に、Sendai Edge SiteからTokyo Global Siteに移動されるポッドに関する情報、ポッドID 3,4,5,6,7のレコードとして、2021/11/1 2:00である移動予定時刻に、Fukuoka Edge SiteからOsaka Global Siteに移動されるポッドに関する情報が設定されている。また、各ポッドには優先度が設定されるが、例えば、ポッドID 1,3,4 である移動対象候補ポッドは、優先度1となるため、該当する移動予定時刻に最初に切り替える移動対象ポッドになる。グローバルサイトのノードである移動先ノードは、ノードテーブル151にてDNAIとマッピングされている。 The movement setting unit 131 sets and updates the information, such as the priority, destination node, and movement time determined in steps S117 to S119, in the pod movement table 152 as information about the pod candidate to be moved (S120). For example, in the pod movement table 152 of FIG. 5, information about the pod to be moved from the Sendai Edge Site to the Tokyo Global Site at the scheduled movement time of 2021/11/1 1:00 is set as records for pod IDs 1 and 2, and information about the pod to be moved from the Fukuoka Edge Site to the Osaka Global Site at the scheduled movement time of 2021/11/1 2:00 is set as records for pod IDs 3, 4, 5, 6, and 7. In addition, a priority is set for each pod, and for example, the pods candidate to be moved with pod IDs 1, 3, and 4 have a priority of 1, and therefore become the pods to be moved first at the corresponding scheduled movement times. The destination node, which is a node in the global site, is mapped to DNAI in node table 151.
 次に、移動設定部131は、所定期間(例えば一週間)のアクセスログを参照し、各優先度の移動対象候補ポッドにのみアクセスしているUEのIPアドレス(UEIP)を抽出する(S121)。移動設定部131は、移動対象候補ポッドにのみアクセスしているUEを、移動対象候補UEとして、移動元DNAI、移動先DNAI、移動時刻、優先度などの情報を、UE移動テーブル153に設定して更新する(S122)。 Next, the mobility setting unit 131 refers to the access log for a predetermined period (e.g., one week) and extracts the IP addresses (UEIP) of UEs that are accessing only the candidate pods to be moved for each priority (S121). The mobility setting unit 131 sets and updates the UE mobility table 153 with information such as the source DNAI, destination DNAI, mobility time, and priority as the candidate UEs to be moved (S122).
 ここでの移動時刻と優先度は、移動対象候補UEがアクセスしている移動対象候補ポッドの移動時刻と優先度に対応している。また、移動元DNAIと移動先DNAIは、ノードテーブル151においてノードとマッピングしたDNAIから設定することができる。例えば、図6のUE移動テーブル153においては、UEID 1,2,3,4のレコードとして、2021/11/1 1:00である移動予定時刻に、DNAI1からDNAI5に経路が切り替えられるUE(Sendai Edge SiteのポッドにのみアクセスするUE)に関する情報と、UEID 5,6,7,8,9のレコードとして、2021/11/1 2:00である移動予定時刻に、DNAI4からDNAI6に経路が切り替えられるUE(Fukuoka Edge SiteのポッドにのみアクセスするUE)に関する情報が設定されている。また、各UEには、自身がアクセスするポッドの優先度に応じた優先度が設定されるが、例えば、UEID 1,2,3,5,6 である移動対象候補UEは、優先度1となるため、該当する移動予定時刻に最初に切り替える移動対象UEになる。 The movement time and priority here correspond to the movement time and priority of the candidate pod to be moved that the candidate UE to be moved is accessing. In addition, the source DNAI and destination DNAI can be set from the DNAI mapped to the node in the node table 151. For example, in the UE movement table 153 of FIG. 6, information on UEs (UEs that only access pods at the Sendai Edge Site) whose routes will be switched from DNAI1 to DNAI5 at the scheduled movement time of 1:00 on November 1, 2021 is set as records for UEIDs 1, 2, 3, and 4, and information on UEs (UEs that only access pods at the Fukuoka Edge Site) whose routes will be switched from DNAI4 to DNAI6 at the scheduled movement time of 2:00 on November 1, 2021 is set as records for UEIDs 5, 6, 7, 8, and 9. Additionally, a priority is set for each UE according to the priority of the pod it accesses. For example, candidate UEs for movement with UEIDs 1, 2, 3, 5, and 6 will have a priority of 1, and will be the first UEs to be switched to at the scheduled movement time.
 ステップS122が終了すると、一連の処理が終了する。なお、ノードリソース使用率が閾値未満であると判定された場合(S112のNo)に、移動テーブルがあるとき(S123のYes)、移動テーブルをリセット(S124)して処理を終了する。一方で、移動テーブルがないとき(S123のNo)には、そのまま処理を終了する。また、ノードリソース使用率が閾値を超えると判定された場合(S112のYes)であっても、移動テーブルが存在するとき(S113のYes)、処理を終了する。さらに、各ノードのqosClassの種類が、BestEffort又はBurstableでないと判定された場合(S115のNo)、つまり、Guaranteed型のポッドである場合、本開示の移動対象ポッドとはならないため、ステップS116乃至S122をスキップして、処理を終了する。 When step S122 ends, the series of processes ends. If it is determined that the node resource utilization rate is below the threshold (No in S112), and there is a migration table (Yes in S123), the migration table is reset (S124) and the process ends. On the other hand, if there is no migration table (No in S123), the process ends as it is. Also, even if it is determined that the node resource utilization rate exceeds the threshold (Yes in S112), if there is a migration table (Yes in S113), the process ends. Furthermore, if it is determined that the type of qosClass of each node is not BestEffort or Burstable (No in S115), that is, if it is a Guaranteed type pod, it is not a pod to be moved as disclosed herein, so steps S116 to S122 are skipped and the process ends.
 上述した移動設定処理によって、ポッド移動テーブル152とUE移動テーブル153を更新して、移動対象候補ポッドと移動対象候補UEの情報を優先度とともに設定した後に、移動設定処理で設定された情報に基づいて、優先度に応じて移動対象ポッドと移動対象UEを移動させる移動制御処理が行われる。次に、図14のフローチャートを参照して、MEC管理システム11のMEC制御部111により実行される移動制御処理の流れを説明する。 The above-mentioned movement setting process updates the pod movement table 152 and the UE movement table 153, and sets the information of the movement target candidate pods and the movement target candidate UEs together with their priorities. Then, based on the information set in the movement setting process, a movement control process is performed to move the movement target pods and the movement target UEs according to the priorities. Next, the flow of the movement control process executed by the MEC control unit 111 of the MEC management system 11 will be described with reference to the flowchart in FIG. 14.
 制御部123は、ポッド移動テーブル152を参照し(S131)、現在時刻が、移動予定時刻が示す対象時刻になったか否かを判定する(S132)。そして、制御部123は、対象時刻になったと判定した場合(S132のYes)、ステップS133乃至S139の処理を行う。 The control unit 123 refers to the pod movement table 152 (S131) and determines whether the current time has reached the target time indicated by the scheduled movement time (S132). If the control unit 123 determines that the target time has reached (Yes in S132), it performs the processes of steps S133 to S139.
 すなわち、制御部123は、ポッド移動テーブル152を参照して、最も高い優先度のレコードの移動対象候補ポッドを、移動対象ポッドとする(S133)。また、制御部123は、UE移動テーブル153を参照して、最も高い優先度のレコードの移動対象候補UEを、移動対象UEとする(S134)。この例では、2021/11/1 1:00になったと仮定すれば、図5のポッド移動テーブル152において、移動予定時刻が2021/11/1 1:00で、かつ、優先度1に設定されたポッドID 1のポッドが移動対象ポッドとなる。また、また、図6のUE移動テーブル153において、移動予定時刻が2021/11/1 1:00で、かつ、優先度1に設定されたUEID 1,2,3のUEが移動対象UEとなる。 In other words, the control unit 123 refers to the pod movement table 152 and sets the candidate pod to be moved of the record with the highest priority as the pod to be moved (S133). The control unit 123 also refers to the UE movement table 153 and sets the candidate UE to be moved of the record with the highest priority as the UE to be moved (S134). In this example, assuming that it is 1:00 on November 1, 2021, the pod with pod ID 1, which has a scheduled movement time of 1:00 on November 1, 2021 and is set to priority 1 in the pod movement table 152 of FIG. 5, becomes the pod to be moved. Also, in the UE movement table 153 of FIG. 6, the UEs with UE IDs 1, 2, and 3, which have a scheduled movement time of 1:00 on November 1, 2021 and is set to priority 1, become the UEs to be moved.
 まず、移動制御部141は、API部112を制御して、移動対象ポッドを移動先ノードにデプロイする(S135)。この例では、ポッドID 1の移動対象ポッドの移動先は、Tokyo Global Siteのノードとなる。デプロイ方法はいくつかあるが、代表的なものとして、kubectlコマンドにてマニフェストファイルを指定して行うことができる。移動先ノードであるTokyo Global Siteのノードは、図4のノードテーブル151にてDNAI5であるDNAIとマッピングされている。 First, the movement control unit 141 controls the API unit 112 to deploy the pod to be moved to the destination node (S135). In this example, the destination of the pod to be moved with pod ID 1 is a node in the Tokyo Global Site. There are several deployment methods, but a typical one is to specify a manifest file with the kubectl command. The node in the Tokyo Global Site, which is the destination node, is mapped to DNAI, which is DNAI5, in the node table 151 in Figure 4.
 次に、経路制御部142は、AF部113を制御して、TrafficInfluence APIにより経路制御を実施し、移動対象UEの経路を切り替える(S136)。TrafficInfluence APIの利用に際しては、移動対象UEのUEIPと移動先DNAIを指定することで、NEFサーバ22と連携した経路制御が実施される。例えば、図6のUE移動テーブル153において、優先度1に該当するUEID 1,2,3に対応するaa.aa.aa.1,aa.aa.aa.2,aa.aa.aa.3であるIPアドレスと、DNAI5である移動先DNAIを指定することで、UEID 1,2,3のUEの経路が切り替えられる。 Next, the route control unit 142 controls the AF unit 113 to perform route control using the TrafficInfluence API and switch the route of the UE to be moved (S136). When using the TrafficInfluence API, route control is performed in cooperation with the NEF server 22 by specifying the UEIP and destination DNAI of the UE to be moved. For example, in the UE movement table 153 of Figure 6, the routes of the UEs with UEIDs 1, 2, and 3 are switched by specifying the IP addresses aa.aa.aa.1, aa.aa.aa.2, and aa.aa.aa.3 corresponding to UEIDs 1, 2, and 3 corresponding to priority 1, and the destination DNAI of DNAI 5.
 また、必須の処理ではないが、制御部123は、ネームサーバ15で管理されている移動対象ポッドのネームのIPアドレスを、移動先ノードのIPアドレスに更新する(S137)。例えば、図5のポッド移動テーブル152におけるポッドID 1の移動対象ポッドのネームであるaaa.co.jpが、ネームサーバ15に登録されていた場合、制御部123は、ネームサーバ15に対し、Tokyo Global Siteである移動先ノードのIPアドレスであるee.ee.ee.eeを通知することで、移動対象ポッドのネームのIPアドレスを、移動先ノードのIPアドレスに差し替えることができる。 In addition, although this is not a required process, the control unit 123 updates the IP address of the name of the pod to be moved, which is managed by the name server 15, to the IP address of the destination node (S137). For example, if aaa.co.jp, which is the name of the pod to be moved with pod ID 1 in the pod movement table 152 of Figure 5, is registered in the name server 15, the control unit 123 can replace the IP address of the name of the pod to be moved with the IP address of the destination node by notifying the name server 15 of ee.ee.ee.ee, which is the IP address of the destination node, which is the Tokyo Global Site.
 次に、移動制御部141は、API部112を制御して、移動元ノードから移動対象ポッドを削除する(S138)。この例では、Sendai Edge Siteのノードから、ポッドID 1の移動対象ポッドを削除する。削除方法はいくつかあるが、kubectlコマンドにてマニフェストファイルを指定することで行うことができる。 Next, the movement control unit 141 controls the API unit 112 to delete the pod to be moved from the source node (S138). In this example, the pod to be moved with pod ID 1 is deleted from the node of the Sendai Edge Site. There are several ways to delete the pod, but it can be done by specifying a manifest file with the kubectl command.
 最後に、制御部123は、ポッド移動テーブル152とUE移動テーブル153から、移動対象のレコードを削除する(S139)。例えば、図5のポッド移動テーブル152において、移動対象ポッドとされたポッドID 1のレコードを削除する。また、図6のUE移動テーブル153において、移動対象UEとされたUEID 1,2,3のレコードを削除する。ステップS139が終了すると、一連の処理が終了する。なお、対象時刻になっていないと判定された場合(S132のNo)、ステップS133乃至S139をスキップして、処理を終了する。 Finally, the control unit 123 deletes the records to be moved from the pod movement table 152 and the UE movement table 153 (S139). For example, in the pod movement table 152 of FIG. 5, the record of pod ID 1, which is the pod to be moved, is deleted. Also, in the UE movement table 153 of FIG. 6, the records of UE IDs 1, 2, and 3, which are the UEs to be moved, are deleted. When step S139 ends, the series of processes ends. Note that if it is determined that the target time has not yet arrived (No in S132), steps S133 to S139 are skipped and the process ends.
 上述の説明では、優先度1の移動として、Sendai Edge SiteからTokyo Global Siteへの移動を例示したが、その後、2021/11/1 2:00になったとき、Fukuoka Edge SiteからOsaka Global Siteへの移動が行われ、ポッドID 3,4とUEID 5,6について同様の処理を行う。 In the above explanation, a move from the Sendai Edge Site to the Tokyo Global Site was given as an example of a priority 1 move, but then, at 2:00 on November 1st, 2021, a move from the Fukuoka Edge Site to the Osaka Global Site will occur, and the same process will be performed for pod IDs 3 and 4 and UEIDs 5 and 6.
 また、優先度1の移動以降の処理は、次のような処理が行われる。すなわち、優先度1の移動制御処理が行われて、ポッド移動テーブル152とUE移動テーブル153では共に、優先度1のレコードが削除された状態になる。そして、図13の移動設定処理が再度実行され、ノードリソース使用率の閾値確認が行われる(S111,S112)。この例では、Sendai Edge SiteとFukuoka Edge Siteのノードリソース使用率が閾値未満であると判定された場合(S112のNo)、ポッド移動テーブル152とUE移動テーブル153は不要となり、それらの移動テーブルをリセットする(S124)。つまり、優先度2乃至4のレコードも削除されるため、残りの移動対象候補ポッドと移動対象候補UEを移動対象とした移動制御処理は実施されずに終了する。 The following processing is performed after the movement of priority 1. That is, the movement control processing of priority 1 is performed, and the priority 1 records are deleted from both the pod movement table 152 and the UE movement table 153. Then, the movement setting processing of FIG. 13 is executed again, and the node resource utilization threshold is checked (S111, S112). In this example, if it is determined that the node resource utilization rates of the Sendai Edge Site and the Fukuoka Edge Site are below the threshold (No in S112), the pod movement table 152 and the UE movement table 153 become unnecessary, and these movement tables are reset (S124). That is, since the records of priorities 2 to 4 are also deleted, the movement control processing for the remaining candidate pods and candidate UEs to be moved is not performed and ends.
 一方で、ノードリソース使用率が閾値を超えると判定された場合(S112のYes)、ポッド移動テーブル152とUE移動テーブル153には、優先度2乃至4のレコードが残っているので、例えば、翌日以降の移動予定時刻(例えば、2021/11/2 1:00, 2021/11/2 2:00)に、図14の移動制御処理で、対象時刻になったと判定され(S132のYes)、優先度2の移動対象候補ポッドと移動対象候補UEが、移動対象ポッドと移動対象UEとされ(S133,S134)、移動制御が行われる(S135,S136)。そして、ノードリソース使用率が閾値未満と判定される(S112のNo)まで、優先度3と優先度4のレコードについても同様の処理が行われる。つまり、この例では、最短で1日(例えば、2021/11/1)で処理が終了し、最長で4日(例えば、2021/11/1 ~ 11/4)かかることになる。 On the other hand, if it is determined that the node resource utilization rate exceeds the threshold (Yes in S112), records with priorities 2 to 4 remain in the pod migration table 152 and the UE migration table 153, so for example, at the scheduled migration time on the next day or later (for example, 1:00 on 2021/11/2, 2:00 on 2021/11/2), it is determined that the target time has arrived in the migration control process of FIG. 14 (Yes in S132), and the migration candidate pod and migration candidate UE with priority 2 are set as the migration target pod and migration target UE (S133, S134), and migration control is performed (S135, S136). Then, the same process is performed on the records with priority 3 and priority 4 until it is determined that the node resource utilization rate is less than the threshold (No in S112). In other words, in this example, the processing will be completed in as little as one day (for example, 2021/11/1) and at most four days (for example, 2021/11/1 - 11/4).
 以上のように、本開示では、ノードリソース使用率が閾値を超えたエッジサイトのノードで動作する各ポッドの詳細情報とポッドリソース使用率から、ポッドの移動の優先度を決定して、当該エッジサイトのノードから、グローバルサイトのノードに対し、決定した優先度に応じてポッドを移動させることができる。また、本開示では、移動対象ポッドにのみアクセスしているUEの経路を、決定した優先度に応じて切り替えることができる。これにより、エッジサイトのノードにおいて、特定の負荷のかかるポッド(アプリケーションコンテナ)が、優先的にグローバルサイトのノードに移動されたり、当該ポッドにのみアクセスしているUEの経路が切り替えられたりするため、オフロードの対象とすべきポッド(アプリケーションコンテナ)やUEを適切に決定して、より適切な対象をオフロードすることができる。 As described above, in the present disclosure, the priority of pod movement can be determined from detailed information about each pod operating on an edge site node where node resource utilization has exceeded a threshold and the pod resource utilization, and the pod can be moved from the edge site node to a global site node according to the determined priority. In addition, in the present disclosure, the route of a UE that is only accessing the pod to be moved can be switched according to the determined priority. As a result, at an edge site node, a pod (application container) with a specific load is preferentially moved to a global site node, and the route of a UE that is only accessing that pod is switched, so that the pod (application container) or UE to be offloaded can be appropriately determined, and more appropriate targets can be offloaded.
 また、本開示では、経路変更に伴う体感の品質低下を最小限に収めつつ、コンピューティングリソースが制御可能なポッドが、MEC12(エッジサイトのノード)に集まりやすくなる。これにより、エッジサイトのノードで動作する特定のポッドが、ノードのコンピューティングリソースを占有しすぎることにより、ノードが停止することを未然に防ぐことが可能となる。つまり、エッジサイトのノード上で動作しているすべてのアプリケーションプログラム(ポッド等)が全停止する状態を未然に防ぐことが可能となる。 In addition, in this disclosure, pods whose computing resources can be controlled tend to gather at MEC12 (edge site node) while minimizing the degradation of the quality of experience that accompanies route changes. This makes it possible to prevent a particular pod running on an edge site node from occupying too many of the node's computing resources, causing the node to shut down. In other words, it makes it possible to prevent a situation in which all application programs (pods, etc.) running on an edge site node shut down completely.
<変形例>
 本開示において、システムとは、複数の装置が論理的に集合したものをいうが、MEC管理システム11は、複数の装置(例えばサーバ等)から構成されてもよい。例えば、図2のMEC管理システム11において、MEC制御部111を有する装置と、API部112を有する装置と、AF部113を有する装置と、データベース114を有する装置とが、別々の装置で構成されてもよい。あるいは、MEC管理システム11を、MEC管理装置と捉えても構わない。このMEC管理装置は、MEC制御部111と、API部112と、AF部113と、データベース114を有する装置である。なお、このMEC管理装置は、MEC制御部111を少なくとも有していればよく、API部112と、AF部113と、データベース114は、外部の装置が有していてもよい。
<Modification>
In the present disclosure, a system refers to a logical collection of multiple devices, but the MEC management system 11 may be composed of multiple devices (e.g., servers, etc.). For example, in the MEC management system 11 of FIG. 2, a device having the MEC control unit 111, a device having the API unit 112, a device having the AF unit 113, and a device having the database 114 may be composed of separate devices. Alternatively, the MEC management system 11 may be regarded as an MEC management device. This MEC management device is a device having the MEC control unit 111, the API unit 112, the AF unit 113, and the database 114. Note that the MEC management device only needs to have at least the MEC control unit 111, and the API unit 112, the AF unit 113, and the database 114 may be included in external devices.
 上述した説明では、優先度テーブル154において、1乃至4の4段階の優先度を用いた場合を示したが、優先度は4段階に限らず、それ以上でもよく、2以上であればよい。例えば、本開示では、BurstableとBestEffortのポッドを移動対象としたが、さらに他の分類(Guaranteed等)を移動対象としたり、あるいは、ポッドリソース使用率を中央値以上又は未満ではなく、閾値等を用いてポッドリソース使用率をさらに細かく分けたりすることで、優先度の段階を増やしてもよい。また、ポッドリソース使用率では、中央値を用いたが、中央値に限らず、他の指標を用いても構わない。 In the above explanation, four priority levels from 1 to 4 are used in the priority table 154, but the priority level is not limited to four levels and may be more than four, as long as it is two or more. For example, in this disclosure, Burstable and BestEffort pods are targeted for movement, but other classifications (Guaranteed, etc.) may be targeted for movement, or the pod resource utilization may be further divided into smaller categories using a threshold or the like rather than being above or below the median, thereby increasing the number of priority levels. Also, the median is used for the pod resource utilization, but other indices may be used instead of being limited to the median.
 なお、ノードリソース使用率やポッドリソース使用率等のリソース使用率は、リソース値の一例であり、他の指標を用いてもよい。また、ノードリソース使用率やポッドリソース使用率として、CPU使用率とメモリ使用率などの複数のリソース使用率が取得される場合には、すべてのリソース使用率を用いることは勿論、少なくとも1つのリソース使用率を用いればよい。リソース使用率を閾値や中央値などと比較する際には、取得したリソース使用率をそのまま用いてもよいし、所定の計算を施してから閾値や中央値などと比較しても構わない。 Note that resource utilization rates such as node resource utilization rate and pod resource utilization rate are examples of resource values, and other indices may be used. Furthermore, when multiple resource utilization rates such as CPU utilization rate and memory utilization rate are obtained as node resource utilization rate and pod resource utilization rate, it is sufficient to use at least one of the resource utilization rates, as well as all of the resource utilization rates. When comparing resource utilization rate with a threshold value, median value, etc., the obtained resource utilization rate may be used as is, or a predetermined calculation may be performed before comparing with the threshold value, median value, etc.
 上述した説明では、ノードテーブル151において、エッジサイトのノードとグローバルサイトのノードの位置に関する位置情報として、緯度と経度を用いた場合を例示したが、緯度と経度は位置情報の一例であり、ノードの位置を特定可能な情報であれば、他の位置情報を用いても構わない。 In the above explanation, latitude and longitude are used as location information regarding the positions of edge site nodes and global site nodes in node table 151, but latitude and longitude are only one example of location information, and other location information may be used as long as it is information that can identify the node position.
 上述した説明で用いた用語は、規格によって異なる場合があり、該当する規格の用語に読み替えても構わない。例えば、MEC(図1のMEC12)においてサーバ等で構成されるノード(図1のノード21A)は、kubernetesではワーカノードとも称され、3GPPではデータネット(DN:DataNet)とも称されるが、本開示では、ノードと称している。 The terms used in the above explanation may differ depending on the standard, and may be replaced with the terms of the relevant standard. For example, a node (node 21A in FIG. 1) consisting of a server etc. in an MEC (MEC 12 in FIG. 1) is also called a worker node in Kubernetes and a data net (DN: DataNet) in 3GPP, but is called a node in this disclosure.
<コンピュータの構成>
 上述した一連の処理(例えば、移動設定処理や移動制御処理等の処理)は、ハードウェアにより実行することもできるし、ソフトウェアにより実行することもできる。一連の処理をソフトウェアにより実行する場合には、そのソフトウェアを構成するプログラムが、コンピュータにインストールされる。図15は、上述した一連の処理をプログラムにより実行するコンピュータのハードウェアの構成例を示すブロック図である。
<Computer Configuration>
The above-mentioned series of processes (e.g., the movement setting process, the movement control process, and the like) can be executed by hardware or by software. When the series of processes is executed by software, a program constituting the software is installed in a computer. FIG. 15 is a block diagram showing an example of the hardware configuration of a computer that executes the above-mentioned series of processes by a program.
 コンピュータにおいて、CPU1001、ROM(Read Only Memory)1002、RAM(Random Access Memory)1003は、バス1004により相互に接続されている。バス1004には、さらに、入出力インタフェース1005が接続されている。入出力インタフェース1005には、入力部1006、出力部1007、記憶部1008、通信部1009、及びドライブ1010が接続されている。 In a computer, a CPU 1001, a ROM (Read Only Memory) 1002, and a RAM (Random Access Memory) 1003 are interconnected by a bus 1004. An input/output interface 1005 is further connected to the bus 1004. An input unit 1006, an output unit 1007, a memory unit 1008, a communication unit 1009, and a drive 1010 are connected to the input/output interface 1005.
 入力部1006は、キーボード、マウス、マイクロフォンなどよりなる。出力部1007は、ディスプレイ、スピーカなどよりなる。記憶部1008は、ハードディスクや不揮発性のメモリなどよりなる。通信部1009は、ネットワークインタフェースなどよりなる。ドライブ1010は、半導体メモリ、磁気ディスク、光ディスク、又は光磁気ディスクなどのリムーバブル記録媒体1011を駆動する。 The input unit 1006 includes a keyboard, a mouse, a microphone, etc. The output unit 1007 includes a display, a speaker, etc. The storage unit 1008 includes a hard disk, a non-volatile memory, etc. The communication unit 1009 includes a network interface, etc. The drive 1010 drives a removable recording medium 1011 such as a semiconductor memory, a magnetic disk, an optical disk, or a magneto-optical disk.
 以上のように構成されるコンピュータでは、CPU1001が、ROM1002や記憶部1008に記録されているプログラムを、入出力インタフェース1005及びバス1004を介して、RAM1003にロードして実行することにより、上述した一連の処理が行われる。 In a computer configured as described above, the CPU 1001 loads the programs stored in the ROM 1002 or memory unit 1008 into the RAM 1003 via the input/output interface 1005 and bus 1004, and executes them, thereby carrying out the above-mentioned series of processes.
 コンピュータ(CPU1001)が実行するプログラムは、例えば、パッケージメディア等としてのリムーバブル記録媒体1011に記録して提供することができる。また、プログラムは、ローカルエリアネットワーク、インターネット、デジタル衛星放送といった、有線又は無線の伝送媒体を介して提供することができる。 The program executed by the computer (CPU 1001) can be provided by being recorded on a removable recording medium 1011, such as a package medium, for example. The program can also be provided via a wired or wireless transmission medium, such as a local area network, the Internet, or digital satellite broadcasting.
 コンピュータでは、プログラムは、リムーバブル記録媒体1011をドライブ1010に装着することにより、入出力インタフェース1005を介して、記憶部1008にインストールすることができる。また、プログラムは、有線又は無線の伝送媒体を介して、通信部1009で受信し、記憶部1008にインストールすることができる。その他、プログラムは、ROM1002や記憶部1008に、予めインストールしておくことができる。 In a computer, a program can be installed in the storage unit 1008 via the input/output interface 1005 by inserting the removable recording medium 1011 into the drive 1010. The program can also be received by the communication unit 1009 via a wired or wireless transmission medium and installed in the storage unit 1008. Alternatively, the program can be pre-installed in the ROM 1002 or storage unit 1008.
 ここで、本明細書において、コンピュータがプログラムに従って行う処理は、必ずしもフローチャートとして記載された順序に沿って時系列に行われる必要はない。すなわち、コンピュータがプログラムに従って行う処理は、並列的あるいは個別に実行される処理(例えば、並列処理あるいはオブジェクトによる処理)も含む。また、プログラムは、1のコンピュータ(プロセッサ)により処理されてもよいし、複数のコンピュータによって分散処理されてもよい。 In this specification, the processing performed by a computer according to a program does not necessarily have to be performed chronologically in the order described in the flowchart. In other words, the processing performed by a computer according to a program also includes processing executed in parallel or individually (for example, parallel processing or processing by objects). In addition, a program may be processed by one computer (processor), or may be processed in a distributed manner by multiple computers.
 なお、本開示の実施の形態は、上述した実施の形態に限定されるものではなく、本開示の要旨を逸脱しない範囲において種々の変更が可能である。また、本明細書に記載された効果はあくまで例示であって限定されるものではなく、他の効果があってもよい。 Note that the embodiments of the present disclosure are not limited to the above-described embodiments, and various modifications are possible without departing from the spirit of the present disclosure. Furthermore, the effects described in this specification are merely examples and are not limiting, and other effects may also be present.
 また、本開示は、以下のような構成をとることができる。 Furthermore, this disclosure can be configured as follows:
(1)
 移動通信ネットワークに接続されたエッジサイトのコンピュータである第1のノード、及びインターネットに接続されたグローバルサイトのコンピュータである第2のノードを管理する管理部と、
 前記第1のノードのコンピューティングリソースを示す第1のリソース値が閾値を超えた場合、前記第1のリソース値が閾値を超えた前記第1のノードを移動元ノードとして、前記移動元ノードで動作している複数のコンテナから構成されるアプリケーションコンテナの詳細に関する詳細情報、及び前記アプリケーションコンテナのコンピューティングリソースを示す第2のリソース値に基づいて、前記アプリケーションコンテナの移動の優先度を決定する決定部と、
 前記移動元ノードから、移動先となる前記第2のノードである移動先ノードに対し、前記優先度に応じて前記アプリケーションコンテナを移動させる制御を行う制御部と
 を備える情報処理装置。
(2)
 前記第1のノードと前記第2のノードは、3GPPで規定されたDNAIと予めマッピングされており、
 前記制御部は、前記DNAIとマッピングされた前記第2のノードを前記移動先ノードとして、前記アプリケーションコンテナを移動させる
 前記(1)に記載の情報処理装置。
(3)
 前記決定部は、前記移動元ノードにおける通信に関するログに基づいて、前記移動元ノードにおける通信量が最も少ない時間帯に応じた時刻を、前記アプリケーションコンテナを移動させる移動時刻に決定し、
 前記制御部は、現在時刻が前記移動時刻になった場合、前記優先度に応じて前記アプリケーションコンテナを移動させる
 前記(2)に記載の情報処理装置。
(4)
 前記決定部は、前記第1のノードと前記第2のノードの位置に関する位置情報に基づいて、前記移動元ノードからの距離が最も近い位置にある前記第2のノードを、前記移動先ノードに決定する
 前記(3)に記載の情報処理装置。
(5)
 前記決定部は、前記アプリケーションコンテナへのアクセスに関するログに基づいて、前記移動通信ネットワークを介して通信を行う移動通信端末のうち、移動対象の前記アプリケーションコンテナにのみアクセスしている移動通信端末を、移動対象に決定し、
 前記制御部は、現在時刻が前記移動時刻になった場合、前記優先度に応じて、移動対象となる前記移動通信端末の経路を切り替える制御を行う
 前記(3)又は(4)に記載の情報処理装置。
(6)
 前記詳細情報は、前記アプリケーションコンテナが最低限必要とするリソースの容量と、前記アプリケーションコンテナの負荷が高まったときに使用できるリソースの最大容量とから特定される前記アプリケーションコンテナの分類に関する情報を含む
 前記(1)に記載の情報処理装置。
(7)
 前記第2のリソース値は、前記アプリケーションコンテナによるCPUの使用率及びメモリの使用率を含むリソース使用率である
 前記(6)に記載の情報処理装置。
(8)
 前記詳細情報は、kubernetesで規定されたqosClassを含み、
 前記決定部は、
  BestEffort又はBurstableとなる前記アプリケーションコンテナを移動対象とし、
  Burstableの前記アプリケーションコンテナよりも、BestEffortの前記アプリケーションコンテナの移動の優先度を高くし、
  リソース使用率が中央値未満の前記アプリケーションコンテナよりも、リソース使用率が中央値以上の前記アプリケーションコンテナの移動の優先度を高くする
 前記(7)に記載の情報処理装置。
(9)
 前記決定部は、アクセス数が中央値以上の前記アプリケーションコンテナよりも、アクセス数が中央値未満の前記アプリケーションコンテナの移動の優先度を高くする
 前記(8)に記載の情報処理装置。
(10)
 前記決定部は、3GPPで規定されたGBRの前記アプリケーションコンテナよりも、Non GBRの前記アプリケーションコンテナの移動の優先度を高くする
 前記(9)に記載の情報処理装置。
(11)
 前記第2のリソース値は、前記アプリケーションコンテナによるGPUの使用率を含むリソース使用率であり、
 前記決定部は、リソースに関する制限のない前記アプリケーションコンテナのうち、リソース使用率が中央値未満よりも、リソース使用率が中央値以上のものの移動の優先度を高くする
 前記(1)に記載の情報処理装置。
(12)
 前記アプリケーションコンテナは、kubernetesで規定されたポッドに対応している
 前記(1)に記載の情報処理装置。
(13)
 情報処理装置が、
 移動通信ネットワークに接続されたエッジサイトのコンピュータである第1のノード、及びインターネットに接続されたグローバルサイトのコンピュータである第2のノードを管理し、
 前記第1のノードのコンピューティングリソースを示す第1のリソース値が閾値を超えた場合、前記第1のリソース値が閾値を超えた前記第1のノードを移動元ノードとして、前記移動元ノードで動作している複数のコンテナから構成されるアプリケーションコンテナの詳細に関する詳細情報、及び前記アプリケーションコンテナのコンピューティングリソースを示す第2のリソース値に基づいて、前記アプリケーションコンテナの移動の優先度を決定し、
 前記移動元ノードから、移動先となる前記第2のノードである移動先ノードに対し、前記優先度に応じて前記アプリケーションコンテナを移動させる制御を行う
 情報処理方法。
(1)
a management unit that manages a first node, which is a computer at an edge site connected to a mobile communication network, and a second node, which is a computer at a global site connected to the Internet;
a determination unit that, when a first resource value indicating a computing resource of the first node exceeds a threshold, determines a priority of migration of the application container based on detailed information on details of an application container consisting of a plurality of containers operating on the source node and a second resource value indicating a computing resource of the application container, with the first node whose first resource value exceeds the threshold as a source node;
a control unit that performs control to move the application container from the source node to a destination node, which is the second node that is a destination of the application container, according to the priority.
(2)
The first node and the second node are pre-mapped with DNAI defined in 3GPP,
The information processing device according to (1), wherein the control unit moves the application container to the second node mapped to the DNAI as the destination node.
(3)
the determination unit determines, based on a log related to communication in the source node, a time corresponding to a time period during which communication volume in the source node is the lowest, as a movement time for moving the application container;
The information processing device according to (2), wherein the control unit moves the application container in accordance with the priority when the current time becomes the movement time.
(4)
The information processing device described in (3), wherein the determination unit determines the second node located closest to the source node as the destination node based on location information regarding the locations of the first node and the second node.
(5)
The determination unit determines, as a movement target, a mobile communication terminal that is accessing only the application container to be moved, among the mobile communication terminals that communicate via the mobile communication network, based on a log related to access to the application container;
The information processing device according to (3) or (4), wherein the control unit performs control for switching a route of the mobile communication terminal to be moved according to the priority when the current time becomes the moving time.
(6)
The information processing device described in (1), wherein the detailed information includes information regarding the classification of the application container determined from the minimum resource capacity required by the application container and the maximum resource capacity that can be used when the load on the application container increases.
(7)
The information processing device according to (6), wherein the second resource value is a resource usage rate including a CPU usage rate and a memory usage rate by the application container.
(8)
The detailed information includes a qosClass defined in kubernetes,
The determination unit is
The application container that is BestEffort or Burstable is targeted for migration,
Giving a higher priority to the movement of the BestEffort application container than the Burstable application container;
The information processing device according to (7), wherein a higher priority is given to the movement of the application container whose resource usage rate is equal to or greater than the median than to the application container whose resource usage rate is less than the median.
(9)
The information processing device according to (8), wherein the decision unit assigns a higher priority to the movement of the application container whose access count is less than the median than to the application container whose access count is equal to or greater than the median.
(10)
The information processing device according to (9), wherein the decision unit gives a higher priority to the movement of the application container of Non-GBR than the application container of GBR defined in 3GPP.
(11)
the second resource value is a resource usage rate including a GPU usage rate by the application container;
The information processing device according to (1), wherein the decision unit assigns a higher priority to the movement of application containers with resource usage rates equal to or greater than a median among the application containers with no resource restrictions than to application containers with resource usage rates less than the median.
(12)
The information processing device according to (1), wherein the application container corresponds to a pod defined in Kubernetes.
(13)
An information processing device,
Manage a first node, which is a computer at an edge site connected to a mobile communication network, and a second node, which is a computer at a global site connected to the Internet;
When a first resource value indicating a computing resource of the first node exceeds a threshold, the first node whose first resource value exceeds the threshold is set as a source node, and a priority for migration of the application container is determined based on detailed information on details of an application container consisting of a plurality of containers operating on the source node and a second resource value indicating a computing resource of the application container;
an information processing method for controlling the movement of the application container from the source node to a destination node, which is the second node serving as a destination, according to the priority.
 11 MEC管理システム, 12 MEC, 13 キャリア網, 14 クラウド・データセンタ, 15 ネームサーバ, 16 無線端末, 21,21A,21B ノード, 22 NEFサーバ, 111 MEC制御部, 112 API部, 113 AF部, 114 データベース, 121 管理部, 122 設定部, 123 制御部, 131 移動設定部, 132 優先度決定部, 141 移動制御部, 142 経路制御部, 151 ノードテーブル, 152 ポッド移動テーブル, 153 UE移動テーブル, 154,154A,154B,154C 優先度テーブル, 1001 CPU 11 MEC management system, 12 MEC, 13 Carrier network, 14 Cloud data center, 15 Name server, 16 Wireless terminal, 21, 21A, 21B Node, 22 NEF server, 111 MEC control unit, 112 API unit, 113 AF unit, 114 Database, 121 Management unit, 122 Setting unit, 123 Control unit, 131 Mobility setting unit, 132 Priority determination unit, 141 Mobility control unit, 142 Route control unit, 151 Node table, 152 Pod mobility table, 153 UE mobility table, 154, 154A, 154B, 154C Priority table, 1001 CPU

Claims (13)

  1.  移動通信ネットワークに接続されたエッジサイトのコンピュータである第1のノード、及びインターネットに接続されたグローバルサイトのコンピュータである第2のノードを管理する管理部と、
     前記第1のノードのコンピューティングリソースを示す第1のリソース値が閾値を超えた場合、前記第1のリソース値が閾値を超えた前記第1のノードを移動元ノードとして、前記移動元ノードで動作している複数のコンテナから構成されるアプリケーションコンテナの詳細に関する詳細情報、及び前記アプリケーションコンテナのコンピューティングリソースを示す第2のリソース値に基づいて、前記アプリケーションコンテナの移動の優先度を決定する決定部と、
     前記移動元ノードから、移動先となる前記第2のノードである移動先ノードに対し、前記優先度に応じて前記アプリケーションコンテナを移動させる制御を行う制御部と
     を備える情報処理装置。
    a management unit that manages a first node, which is a computer at an edge site connected to a mobile communication network, and a second node, which is a computer at a global site connected to the Internet;
    a determination unit that, when a first resource value indicating a computing resource of the first node exceeds a threshold, determines a priority of migration of the application container based on detailed information on details of an application container consisting of a plurality of containers operating on the source node and a second resource value indicating a computing resource of the application container, with the first node whose first resource value exceeds the threshold as a source node;
    a control unit that performs control to move the application container from the source node to a destination node, which is the second node that is a destination of the application container, according to the priority.
  2.  前記第1のノードと前記第2のノードは、3GPPで規定されたDNAIと予めマッピングされており、
     前記制御部は、前記DNAIとマッピングされた前記第2のノードを前記移動先ノードとして、前記アプリケーションコンテナを移動させる
     請求項1に記載の情報処理装置。
    The first node and the second node are pre-mapped with DNAI defined in 3GPP,
    The information processing device according to claim 1 , wherein the control unit moves the application container to the second node mapped to the DNAI as the destination node.
  3.  前記決定部は、前記移動元ノードにおける通信に関するログに基づいて、前記移動元ノードにおける通信量が最も少ない時間帯に応じた時刻を、前記アプリケーションコンテナを移動させる移動時刻に決定し、
     前記制御部は、現在時刻が前記移動時刻になった場合、前記優先度に応じて前記アプリケーションコンテナを移動させる
     請求項2に記載の情報処理装置。
    the determination unit determines, based on a log related to communication in the source node, a time corresponding to a time period during which communication volume in the source node is the lowest, as a movement time for moving the application container;
    The information processing device according to claim 2 , wherein the control unit moves the application container in accordance with the priority when the current time becomes the movement time.
  4.  前記決定部は、前記第1のノードと前記第2のノードの位置に関する位置情報に基づいて、前記移動元ノードからの距離が最も近い位置にある前記第2のノードを、前記移動先ノードに決定する
     請求項3に記載の情報処理装置。
    The information processing device according to claim 3 , wherein the determination unit determines the second node located closest to the source node as the destination node based on position information regarding the positions of the first node and the second node.
  5.  前記決定部は、前記アプリケーションコンテナへのアクセスに関するログに基づいて、前記移動通信ネットワークを介して通信を行う移動通信端末のうち、移動対象の前記アプリケーションコンテナにのみアクセスしている移動通信端末を、移動対象に決定し、
     前記制御部は、現在時刻が前記移動時刻になった場合、前記優先度に応じて、移動対象となる前記移動通信端末の経路を切り替える制御を行う
     請求項3に記載の情報処理装置。
    The determination unit determines, as a movement target, a mobile communication terminal that is accessing only the application container to be moved among the mobile communication terminals that communicate via the mobile communication network based on a log related to access to the application container;
    The information processing device according to claim 3 , wherein the control unit performs control for switching a route of the mobile communication terminal to be moved in accordance with the priority when the current time becomes the moving time.
  6.  前記詳細情報は、前記アプリケーションコンテナが最低限必要とするリソースの容量と、前記アプリケーションコンテナの負荷が高まったときに使用できるリソースの最大容量とから特定される前記アプリケーションコンテナの分類に関する情報を含む
     請求項1に記載の情報処理装置。
    The information processing device according to claim 1 , wherein the detailed information includes information regarding a classification of the application container that is identified from a minimum resource capacity required by the application container and a maximum resource capacity that can be used when the load of the application container increases.
  7.  前記第2のリソース値は、前記アプリケーションコンテナによるCPUの使用率及びメモリの使用率を含むリソース使用率である
     請求項6に記載の情報処理装置。
    The information processing apparatus according to claim 6 , wherein the second resource value is a resource usage rate including a CPU usage rate and a memory usage rate by the application container.
  8.  前記詳細情報は、kubernetesで規定されたqosClassを含み、
     前記決定部は、
      BestEffort又はBurstableとなる前記アプリケーションコンテナを移動対象とし、
      Burstableの前記アプリケーションコンテナよりも、BestEffortの前記アプリケーションコンテナの移動の優先度を高くし、
      リソース使用率が中央値未満の前記アプリケーションコンテナよりも、リソース使用率が中央値以上の前記アプリケーションコンテナの移動の優先度を高くする
     請求項7に記載の情報処理装置。
    The detailed information includes a qosClass defined in kubernetes,
    The determination unit is
    The application container that is BestEffort or Burstable is targeted for migration,
    Giving a higher priority to the movement of the BestEffort application container than the Burstable application container;
    The information processing apparatus according to claim 7 , wherein a higher priority is given to the movement of the application container whose resource usage rate is equal to or greater than the median than to the application container whose resource usage rate is less than the median.
  9.  前記決定部は、アクセス数が中央値以上の前記アプリケーションコンテナよりも、アクセス数が中央値未満の前記アプリケーションコンテナの移動の優先度を高くする
     請求項8に記載の情報処理装置。
    The information processing device according to claim 8 , wherein the decision unit assigns a higher priority to the movement of the application container whose access count is less than a median value than to the application container whose access count is equal to or greater than the median value.
  10.  前記決定部は、3GPPで規定されたGBRの前記アプリケーションコンテナよりも、Non GBRの前記アプリケーションコンテナの移動の優先度を高くする
     請求項9に記載の情報処理装置。
    The information processing device according to claim 9 , wherein the decision unit assigns a higher priority to the movement of the non-GBR application container than to the GBR application container defined in 3GPP.
  11.  前記第2のリソース値は、前記アプリケーションコンテナによるGPUの使用率を含むリソース使用率であり、
     前記決定部は、リソースに関する制限のない前記アプリケーションコンテナのうち、リソース使用率が中央値未満よりも、リソース使用率が中央値以上のものの移動の優先度を高くする
     請求項1に記載の情報処理装置。
    the second resource value is a resource usage rate including a GPU usage rate by the application container;
    The information processing device according to claim 1 , wherein the decision unit assigns a higher priority to the movement of application containers with resource usage rates equal to or greater than a median among the application containers with no resource restrictions than to application containers with resource usage rates less than the median.
  12.  前記アプリケーションコンテナは、kubernetesで規定されたポッドに対応している
     請求項1に記載の情報処理装置。
    The information processing device according to claim 1 , wherein the application container corresponds to a pod defined in Kubernetes.
  13.  情報処理装置が、
     移動通信ネットワークに接続されたエッジサイトのコンピュータである第1のノード、及びインターネットに接続されたグローバルサイトのコンピュータである第2のノードを管理し、
     前記第1のノードのコンピューティングリソースを示す第1のリソース値が閾値を超えた場合、前記第1のリソース値が閾値を超えた前記第1のノードを移動元ノードとして、前記移動元ノードで動作している複数のコンテナから構成されるアプリケーションコンテナの詳細に関する詳細情報、及び前記アプリケーションコンテナのコンピューティングリソースを示す第2のリソース値に基づいて、前記アプリケーションコンテナの移動の優先度を決定し、
     前記移動元ノードから、移動先となる前記第2のノードである移動先ノードに対し、前記優先度に応じて前記アプリケーションコンテナを移動させる制御を行う
     情報処理方法。
    An information processing device,
    Manage a first node, which is a computer at an edge site connected to a mobile communication network, and a second node, which is a computer at a global site connected to the Internet;
    When a first resource value indicating a computing resource of the first node exceeds a threshold, the first node whose first resource value exceeds the threshold is set as a source node, and a priority for migration of the application container is determined based on detailed information on details of an application container consisting of multiple containers operating on the source node and a second resource value indicating a computing resource of the application container;
    an information processing method for controlling the movement of the application container from the source node to a destination node, which is the second node serving as a destination, according to the priority.
PCT/JP2023/037327 2022-11-04 2023-10-16 Information processing device, and information processing method WO2024095745A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2022-177130 2022-11-04
JP2022177130A JP2024067227A (en) 2022-11-04 2022-11-04 Information processing device and information processing method

Publications (1)

Publication Number Publication Date
WO2024095745A1 true WO2024095745A1 (en) 2024-05-10

Family

ID=90930219

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2023/037327 WO2024095745A1 (en) 2022-11-04 2023-10-16 Information processing device, and information processing method

Country Status (2)

Country Link
JP (1) JP2024067227A (en)
WO (1) WO2024095745A1 (en)

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113342510A (en) * 2021-08-05 2021-09-03 国能大渡河大数据服务有限公司 Water and power basin emergency command cloud-side computing resource cooperative processing method

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113342510A (en) * 2021-08-05 2021-09-03 国能大渡河大数据服务有限公司 Water and power basin emergency command cloud-side computing resource cooperative processing method

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
MA, ZITONG ET AL.: "Container Migration Mechanism for Load Balancing in Edge Network Under Power Internet of Things", IEEE ACCESS, vol. 8, 24 June 2020 (2020-06-24), pages 118405 - 118416, XP011797151, DOI: 10.1109/ACCESS.2020.3004615 *

Also Published As

Publication number Publication date
JP2024067227A (en) 2024-05-17

Similar Documents

Publication Publication Date Title
US11563713B2 (en) Domain name server allocation method and apparatus
US11297685B2 (en) System and method for session relocation at edge networks
US20240080759A1 (en) On-demand application-driven network slicing
CN105122936B (en) Service distribution processing method and related device
EP3318047B1 (en) Systems and methods for load balancing in a distributed software defined network packet core system
CN108293004B (en) System and method for network slice management
US20220052961A1 (en) Resource discovery in a multi-edge computing network
US11252655B1 (en) Managing assignments of network slices
KR20200062272A (en) Mobile network interaction proxy
US11838273B2 (en) Extending cloud-based virtual private networks to radio-based networks
US8923120B2 (en) Adaptive queue-management
CN112889245B (en) Network system and architecture with multiple load balancers and network access controller
US20160269297A1 (en) Scaling the LTE Control Plane for Future Mobile Access
US20240163178A1 (en) Edge computing topology information exposure
US11871284B1 (en) Integrating multiple access technologies in a radio-based network
US11229004B1 (en) System and method of access point name (APN) dynamic mapping
WO2024095745A1 (en) Information processing device, and information processing method
US10623992B2 (en) System and method for mobile network access load balancing
CN107408058A (en) A kind of dispositions method of virtual resource, apparatus and system
US11563678B1 (en) Orchestration of overlay paths for wide area network virtualization
WO2022266116A1 (en) User plane function selection and hosting for real-time applications
US10986036B1 (en) Method and apparatus for orchestrating resources in multi-access edge computing (MEC) network
Fukushima et al. Determining server locations in server migration service to minimize monetary penalty of dynamic server migration
US12010550B2 (en) Customer-defined capacity limit plans for communication networks
KR102675051B1 (en) Customer-defined capacity limit planning for telecommunications networks

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23885507

Country of ref document: EP

Kind code of ref document: A1