KR102318533B1 - Method and System for GPU-based Embedded Edge Server Configuration and Neural Network Service Utilization - Google Patents

Abstract

Disclosed are a method for configuring a GPU-based embedded edge server and utilizing a neural network service, which implements a single board-based edge cluster equipped with a GPU supporting a container environment, and a system thereof. According to the present invention, the method comprises the following steps: executing a device query application on all edge servers and transmitting update of an extended resource for a GPU specification to an API server as an HTTP patch request to register the update in all nodes; confirming a service request for requesting a desired service from a client to an API server of an edge existing locally; after confirming the service request, receiving, by the edge server, external data for the service request from an IoT data source transmitting/receiving data by using a stream socket; allocating a neural network pod by using Kubernetes scheduling for the received external data; and processing, by the assigned pod, the external data through the neural network.

Description

{Method and System for GPU-based Embedded Edge Server Configuration and Neural Network Service Utilization}

The present invention relates to a method and system for configuring a GPU-based embedded edge server and utilizing a neural network service.

Recently, interest in artificial intelligence (AI) is increasing in various fields. Among these fields, the Internet of Things (IoT) is one domain in which the need for AI is becoming more important as the variety, size, and environment of data available to users increases. Currently, AI services for IoT devices are provided through a central cloud through the Internet, so these AI services are based on considerations regarding performance and management of cloud services, such as low latency, ease of development, database queries, and expansion for resource utilization. are being affected Cloud vendors are exploring new cloud models that work in different ways to handle the massive amounts of data required for AI models. Among these cloud models, edge computing is being discussed as a new processing method that can compensate for the shortcomings of the central cloud structure. US research firm Gartner reported autonomous edge as one of the top 10 strategic technologies announced for 2020, while IBM chose edge computing and leveraged Kubernetes as its methodology for use in network evolution in 2020. Edge computing, which is considered the core of the next-generation cloud, refers to a paradigm that utilizes real-time data according to the surrounding environment and user needs instead of using the existing central server to process all data. Edge computing is based on computing devices in edge location areas, such as micro data centers, cloudlets, and fogs, adjacent to the user plane. The advantages over traditional cloud are low latency, traffic distribution and privacy data protection.

1 is a diagram showing the structure of edge computing.

Research related to edge computing has been conducted from various perspectives. A three-layer (user plane, edge computing plane, cloud computing plane) structure including cloud, mobile edge computing (MEC), and IoT has been proposed in the prior art. In addition, studies suggesting the use of edge computing solutions to organize services, content and functions in areas close to the end user, a hierarchical MEC structure composed of components named as fields, shallow and deep cloudlets, transparent to increase service efficiency The consideration of the configuration problem of edge computing using computing and the proposal of a new approach were carried out. Nebula, an edge computing platform configuration with interoperability of computation and storage to support grid and peer-to-peer systems, has also been proposed.

In terms of planning management of edge computing, capacity planning that allows CPU and GPU resources to meet QoS requirements, average server utilization rate, and latency of application deployment to MEC were studied. In addition, the relationship and effect of edge computing's dynamic computing web content provision was studied.

In terms of resource management and edge computing supply, dynamic service provision of edge cloud through calculation of new service load cost for resource limited nodes, limited node capacity and delivery process request cost has been reported. Migration using the Markov Decision Process (MDP) is based on user movement and response to network performance.

Furthermore, based on the prior art, some studies are being conducted with various test bed types. These studies include evaluation of edge computing based on container environments and three edge cloud implementations of container based PaaS architectures in clusters utilizing Raspberry Pi, a single board computing environment. Performance comparison studies of machine learning packages such as TensorFlow, Caffe2, MXNet, PyTorch, and TensorFlow Lite were performed on edge devices. However, since single boards such as Raspberry Pi identified in previous studies have limitations in terms of hardware specifications for implementing neural networks, it is necessary to implement edge computing in a single board computer (ARM core-based CPU) environment equipped with a GPU (Nvidia GPU). . Moreover, in order to use the GPU resources of the cluster in container units, it is necessary to check the specifications required for the neural network service and monitor the GPU resources at each edge node. These techniques do not take into account the edge device container environment.

The technical problem to be achieved by the present invention is to propose a built-in edge server that builds a container environment with Kubernetes and a new Pod allocation method according to the GPU resources of each node identified through the execution of a device query job. Then, using two neural network models, object detection and driver profiling, a service application composed of Pod units in Kubernetes is constructed, and the default Pod allocation method is compared with the new Pod allocation method.

In one aspect, the GPU-based embedded edge server configuration and neural network service utilization method proposed in the present invention executes a device query application on all edge servers, and updates the extended resources for the GPU specification to the API server with an HTTP patch request. The step is transmitted and registered in all nodes, the step of confirming the service request requesting the desired service from the client to the API server of the edge that exists locally, the IoT data source that transmits and receives data using the stream socket after checking the service request The edge server receives external data for a service request from includes steps.

The step of confirming the service request that requests the desired service from the client to the API server at the edge, which exists locally use

The step of allocating a neural network pod using Kubernetes scheduling for the received external data is set as a new filtering condition in the configuration file of the neural network pod for the newly added GPU resource.

The steps for allocating neural network pods using Kubernetes scheduling for received external data are: volume filtering to check mountable volumes and node validity for conflicts between volumes, resource filtering to check ports requested by pods and sufficient resources, cluster Node filtering phase to identify valid nodes through topology filtering to identify Kubernetes components including permits and selectors to support topology-dependent scheduling, determining and weighting the label conditions of nodes to adjust priorities It includes a node priority calculation step and an actual scheduling step of allocating neural network Pods according to node filtering and node priority calculation.

In another aspect, the GPU-based embedded edge server configuration and neural network service utilization system proposed in the present invention executes a device query application on all edge servers, and updates the extended resources for the GPU specification as an HTTP patch request to the API server. Application execution unit that transmits and registers to all nodes, requests and confirms the desired service to the API server, and then receives external data for the request from the IoT data source through the stream socket. It includes a Pod allocator for allocating neural network Pods using Kubernetes scheduling for external data, and a data processing part for processing data that the allocated Pods use the neural network to transmit externally and stored internally.

According to embodiments of the present invention, it is possible to implement a single board-based edge cluster equipped with a GPU supporting a container environment, a new Pod method considering GPU resources is proposed, and an increase in the number of operation Pods can be confirmed. . In addition, drivers profiling actionable deep learning models for edge devices with limited resources are used to discover service availability.

1 is a diagram showing the structure of edge computing.
2 is a diagram illustrating an image construction step and a process of executing a container-type application.
3 is a diagram illustrating a container runtime interface based on an open container scheme that standardizes a runtime supporting a container environment.
4 is a diagram illustrating a type of container adjustment.
5 is a flowchart illustrating a configuration of a GPU-based embedded edge server and a method of utilizing a neural network service according to an embodiment of the present invention.
6 shows a structure for realizing an edge server according to an embodiment of the present invention.
7 illustrates the operation of an application according to a process of data flow according to an embodiment of the present invention.
8 is a diagram illustrating a configuration of a GPU-based embedded edge server and a system for utilizing a neural network service according to an embodiment of the present invention.

The emerging edge computing technology is presented as a new paradigm to complement the shortcomings of the existing cloud computing. In particular, edge computing uses local data while being used for low-latency service applications. This latest technology requires a neural network approach for running large-scale machine learning on edge servers. The present invention proposes a Pod allocation method by adding various GPU resources to increase the efficiency of configuring a Kubernetes-based edge server using a GPU-based embedded board and a TensorFlow-based neural network service application. According to the experimental results performed on the proposed edge server, the bandwidth according to time and data size change can be inferred in the local (20.4Mbps~42.4Mbps) and Internet environment for service applications (6.31Mbps~25.5Mbps). When running two neural network applications on the edge server, the network time of the object detection application was 112.2 ms (Xavier) to 515.8 ms (Nan), and the network time of the driver profiling application was 321.8 ms (Xavier) to 495.7 ms ( up to nano). The proposed Pod allocation method shows better performance than the default Pod allocation method. According to the embodiment of the present invention, it can be seen that the number of allocable Pods of 3 worker nodes is increased from 5 to 7, and the standard deviation of the response time of the board performance difference is reduced by about 53%. Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

In the emerging 5G network, software networking is a programmable approach that makes extensive use of IT virtualization technologies, including communications infrastructure, functions, and applications. Thus, edge computing has the core technologies and architectural concepts that enable the evolution of 5G. Changes in the network are directly related to the performance of edge computing in terms of low latency, high speed, and structure. These technologies support the reliability of services depending on the network problems of each node in the clustered edge server. In the prior art, edge servers are implemented on various boards to check the stability problem and latency of the user plane.

A container is a virtualization method for running a separate virtual Linux system (container) in Linux. In containers, the layers composed of independent file systems are linked into one image to store the environment using the union file system, and hardware resources are used separately for each container. You can use the saved image to run containerized applications.

2 is a diagram illustrating an image construction step and a process of executing a container-type application.

In this process, Linux container technology is a method of easily setting environmental changes according to the use according to the container runtime, and it is a method of realizing the implementation of microservices composed of small units.

The advantages of containers used in edge servers are as follows. First, due to the hardware limitations of edge servers, programs implemented as hypervisors have significant size limitations, while programs in container environments allow for light memory usage and easy basic configuration such as fast startup. Second, flexible expansion is easy through resource management handled at the OS kernel level in the expansion operation, so it can be efficiently managed by the edge server.

3 is a diagram illustrating Container Runtime Interfaces (CRI) based on the Open Container Initiative (OCI) that standardizes a runtime supporting a container environment.

This list of CRIs includes CRI-O, Docker, and gVisor. Among them, Docker is the most used platform because it is the oldest (mutually) platform used by many people. However, CRI focuses on managing container configuration in a single host environment. In a large cluster server configuration environment, efficient business management in terms of functions is insufficient. Therefore, in the server environment configuration, a platform that provides additional container management is required.

Containers are increasingly becoming the preferred environment for DevOps collaboration. At the same time, container coordination helps manage hardware resources between each terminal, and reliably update and distribute work in a large-scale distributed computing environment such as the cloud. Therefore, the need for a container coordination platform grows.

4 is a diagram illustrating a type of container adjustment.

Container orchestration types include Docker clusters, Kubernetes, and Apache Mesos, and Kubernetes is currently the most popular and well-organized system. First, unlike other platforms, Kubernetes can be deployed in a variety of environments. Second, it has advantages in terms of container scheduling, deployment, and management according to user options. Third, there is an advantage in scheduling service tasks in various ways and utilizing the resources applied to these tasks. Kubernetes is an implementation of an application running on a node. You use a set of Pods for deployment and a set of tasks and daemons for update operations. In addition, Kubernetes internally utilizes a separate virtual network to support each container environment and utilizes an API-server that supports external access from the master node to operate a separate service application according to a client request. At this time, the internal network checks the Pod connection according to each request through the manipulation of the Kube-proxy, and processes the Pod connection using a distribution algorithm.

Edge computing is mainly used for pre-processing in nearby computing areas because it is difficult to integrate each computing resource. Furthermore, as boards with GPU resources available in the IoT area such as Nvidia's Jetson board appear, the utilization of GPU resources in the edge computing area is also being considered. For example, Nvidia provides TensorRT and provides an inference framework such as the DeepStream SDK that simplifies the development of video analytics applications on the Nvidia platform.

In that respect, this framework has advantages over CPUs in artificial neural network processing, which mainly processes through parallel computing rather than raw data from sensors on edge servers due to the utilization of GPU resources and processes media data such as video and photos. While reducing traffic to the central cloud, you need to achieve fast response times and processing times at the same time. NVIDIA, a leading GPU developer, presents a runtime called Nvidia-docker on Github to help manage Docker-based GPU resources according to changes in the container environment. Development tools for AI and neural network models are supported as container images on various platforms through the Nvidia-GPU Cloud (NGC).

5 is a flowchart illustrating a configuration of a GPU-based embedded edge server and a method of utilizing a neural network service according to an embodiment of the present invention.

The proposed GPU-based embedded edge server configuration and neural network service utilization method runs the device query application on all edge servers, and updates the extended resources for GPU specifications are sent to the API server as an HTTP patch request and registered in all nodes. (510), the step of confirming a service request for requesting a desired service from the client to the API server of the locally existing edge (520), after confirming the service request, the edge from the IoT data source that transmits and receives data using a stream socket The server receives external data for the service request ( 530 ), allocating a neural network Pod using Kubernetes scheduling for the received external data ( 540 ), and the allocated Pod is external through the neural network. processing the data (550).

In step 510 , the device query application is executed on all edge servers, and the update of the extended resource for the GPU standard is transmitted to the API server as an HTTP patch request and registered in all nodes.

In step 520, a service request for requesting a desired service from the client to the API server of the edge existing locally is confirmed.

The program that communicates for each request from the client uses a cluster-accessible authentication library and a Kubernetes API that supports internal cloud operations.

In step 530 , after confirming the service request, the edge server receives external data for the service request from an IoT data source that transmits and receives data using a stream socket.

In step 540, a neural network Pod is allocated using Kubernetes scheduling for the received external data. A new filtering condition is set in the configuration file of the neural network pod for the newly added GPU resource.

Received external data uses Kubernetes scheduling to allocate neural network pods, volume filtering checks the mountable volumes and node validity for conflicts between volumes, resource filtering checks the ports requested by the pods and sufficient resources, cluster topology Topology filtering to identify Kubernetes components, including permits and selectors to support scheduling according to Node priority calculation step, which determines the allocation priority and label condition of the node by checking conditions of number, node availability, resource balance, Pod toleration and taints, and adjusts the priority by weighting, and It includes the actual scheduling step of allocating neural network Pods according to node filtering and node priority calculation.

Finally, in step 550, the assigned Pod processes external data through the neural network. A GPU-based embedded edge server configuration and a method of utilizing a neural network service will be described in more detail with reference to FIGS. 6 to 8 .

In the present invention, a native clustering edge server is configured in a wireless environment using Kubernetes that supports a container application environment. Therefore, the implementation of the application utilizing the GPU resource and the new allocation method for the application are checked. First, we propose a built-in edge server structure for test applications in a container environment.

In each area of the edge server environment, IoT data sources must enhance hardware performance for pre-processing of relatively large media data (eg, photos, voices), and reliable transmission methods and data formats for data transmission are required. . The prior art implemented a transport stream socket for stored picture data for the device.

The present invention proposes a structure in which photo data is collected from a camera connected to an IoT device, a format using GStreamer is implemented, and the data is put in a basic transmission queue, connected to a stream socket, and transmitted.

The proposed edge server area includes high-performance processing using neural networks rather than simple data pre-processing. It serves to store the resulting data and transmit the data to the cloud computing plane. Furthermore, in order to judge the usefulness and usability of scaling, the operation is verified through the implementation of a workload in which multiple containers are operated according to additional client transmission.

6 shows a structure for realization of the proposed edge server.

It uses Docker as the CRI for all nodes, and adds Kubernetes for clustering and coordinating each node to run the requested GPU-enabled application on the edge server.

The proposed edge server configuration has the implementation of neural network services for GPU utilization in edge servers. The larger the size of the neural network, the faster the computation speed in the cloud computing plane. However, lightweight neural network models can provide latency advantages even on edge servers if there is no significant difference in accuracy.

In a general container environment, additional configuration is required because the maximum performance for running the neural network cannot be obtained using only CPU and RAM resources. Therefore, the Nvidia-Docker runtime to utilize GPU resources to drive the neural network model is loaded with the graphic driver library used by the host and operated in a container environment. However, the existing image containing the neural network development tool and execution environment is not suitable for the Jetson board, so a new image needs to be implemented. Moreover, object detection-based neural network processing requires the operation of huge GPU resources, so it is better to distribute the application tasks according to the order of the nodes with the best specifications. However, in traditional edge servers, Kubernetes only monitors CPU and RAM resources to deploy Pods. Applications running neural network models need a way to identify GPU resources and assign Pods to appropriate nodes. In desktop environment, a resource called k8s-device-plugin is used, but in Jetson board environment (ARM64 chipset), k8s-device-plugin does not work due to lack of NVML (Nvidia Management Library) library support.

7 illustrates the operation of an application according to a process of data flow according to an embodiment of the present invention.

The present invention proposes a method of allocating GPU application tasks to nodes in the Jetson board environment by adding GPU resources. The proposed method is to check the maximum clock of the GPU within each node of Kubernetes based on Jetson board and add it as an expansion resource along with CPU and RAM capacity to allocate the Pods to the appropriate nodes. This process was implemented by querying the performance of each node as shown in FIG. 7 . Therefore, in the case of Jetson Series boards with different hardware specifications inside the edge server, it is possible to increase the scalability of the neural network application in the edge server and at the same time increase the work efficiency.

To challenge and validate the proposed edge server configuration in deep learning service, various neural networks are deployed for completely different applications, and the deployed neural networks are as follows: Object detection using ssd-mobilenetv2 is based on 2D images, Driver behavior profiling using DeepConvRNN is based on 1D scalar data (driver data, i.e. acceleration, braking, etc.).

As mentioned earlier, the reason for choosing the algorithm for the proposed task is that images from the client's camera are used for real-time environments that require deep learning services (in this case, object detection) of worker nodes under the proposed study configuration. to assume a scenario. Similarly, in the case of 1D scalar data, we assume a connected vehicle environment in which sensor data of an in-vehicle CAN (Cruel Control Area Network) bus requires services of the proposed edge server configuration. In addition, the use of deep learning services enables edge servers to detect the identity of drivers through driver behavior profiling.

Each model will be briefly described to describe the level of complexity of the deep learning algorithm used in the proposed task.

For its detection, a well-known deep learning algorithm called SSD-Mobilenetv2 is deployed, which consists of two models: Single Shot Multi-Box Detector (SSD) and MobilNetv2. On the other hand, MobileNetv2 is used as a shape extractor and achieves competitive accuracy with far fewer parameters (4.3 million parameters). MobileNetv2 is combined with an optimized version of SSD to reduce computing complexity when using the SSD-MobileNetv2 name together. The latest SSD-MobileNetv2 is available for all Jetson series (Xavier, TX1, TX2, Nano) of Jetson inference libraries that are highly optimized using TensorRT. However, in the present invention, frozen models available on the official TensorFlow Github page have been placed. This model has been pre-trained on the COCO dataset and can be easily modified using TensorFlow for development compared to what is available in the Jetson inference library.

In addition, based on the proposed configuration, it was modified as a lightweight deep learning model for a container environment under the protection of edge computing. It is based on the famous multi-model network consisting of convolutional layers, followed by a recurrent neural network. Container environments require a compact network with fewer parameters and memory images. In this regard, we further optimized the network by tuning parameters such as kernel size, kernel depth, window stride, batch size, and number of hidden layers of LSTM, and reducing attention units to reduce the computation and size of the network. In the present invention, we have successfully reduced the size of the network by reducing the accuracy by a subtle amount. The architecture described in Figure 7 uses the Ocslab driven data set for driver profiling and identification. The Ocslab driving dataset contains 51 driving functions acquired using an in-vehicle CAN data bus. However, 15 final list functions corresponding to the driver's personal description are used for driver identification. These 15 shapes are further processed into statistical (mean, median, standard deviation) shapes to create a 45-dimensional shape (15 × 3). An algorithm using TensorFlow 1.15 was implemented in a container environment using a Docker image. The reason for explaining Figure 7 is that, unlike ssd-mobilenetv2, driver profiling is not part of the Jetson inference library, we evaluated different algorithms for the scenarios we propose, and chose this tool for driver profiling, edge server deep This is because it was additionally modified according to the proposed configuration of the learning service.

The structure of GPU resource scheduling according to an embodiment of the present invention is based on a container environment using Kubernetes, and the entire process of running the neural network by the edge server is shown in FIG. 5 .

Referring back to FIG. 5 , first, in step 510 , the device query application is executed with the preset settings of all edge servers, and the update of the extended resource for the GPU specification is transmitted to the API-server as an HTTP patch request to all nodes is registered in

In step 520, a client near the edge server makes a request for a desired service to an API server of a small edge that exists locally. The program that communicates for each request from the client uses an authentication library that can connect to the cluster and the Kubernetes API to support internal cloud operations.

In step 530 , after confirming the request, the edge server operates a data receiving server connected to the IoT data source. The data transferred includes large media files (e.g., photo, video format files) and scalar data (e.g., sensor data, single-valued data) and is sent from the data source to the edge server using stream sockets. .

In step 540, the node assignment of the neural network driven Pod uses the scheduling of the Kube-Scheduler included in Kubernetes. At this time, the newly added GPU resource is set as a new filtering condition in the configuration file of the neural network driving pod, and the pod is assigned.

In step 550, the assigned Pod processes the external data through the neural network. As a result of using the same communication, the edge server uses a separate retransmission pod or monitoring pod inside the edge server through authentication from the outside in that data retransmission affects the performance of the processed neural network pod.

In this process, Kubernetes scheduling in step 540 consists of three steps: 1) node filtering, 2) node priority calculation, and 3) actual scheduling work. The first stage node filtering is a procedure for identifying valid nodes. Filters can be classified into volume filters, resource filters, and topology filters. Volume filters check the validity of nodes, such as mountable volumes and conflicts between volumes. The resource filter checks the ports requested by the Pod and sufficient resources (eg node storage, CPU, RAM, and scaling resources). Topology filters identify Kubernetes components such as permits and selectors to support scheduling according to the cluster topology. The second step, Calculating Node Priority, determines the allocation priority by checking conditions such as the number of Pod replicas of allocatable nodes, node availability, resource balance, Pod-relevance, and taints obtained from previous node filtering. Among them, relevance determines the label condition of the node itself and adjusts its priority by weighting it.

Therefore, the algorithm for adding the proposed extended resource and changing the preference is the same as Algorithm 1.

In this algorithm, the volume, CPU, and RAM capacity of the node suitable for each Pod is compared with the required amount, and the toleration/taints function is used. Then check the node selector settings that affect the Kubernetes system. Finally, the Pod support node is added to the list.

7 shows the operation of the application according to the process of the data flow. In addition, the estimation neural network model that performs object detection in the application uses a pre-trained model from which you can get the ssd-mobilenet-v2 model trained with the COCO dataset from the TensorFlow object detection API.

8 is a diagram illustrating a configuration of a GPU-based embedded edge server and a system for utilizing a neural network service according to an embodiment of the present invention.

The GPU-based embedded edge server configuration and neural network service utilization system 800 according to the present embodiment may include a processor 810 , a bus 820 , a network interface 830 , a memory 840 , and a database 850 . . The memory 840 may include an operating system 841 and a GPV-based embedded edge server configuration and a neural network service utilization routine 842 . The processor 810 may include an application execution unit 811 , a service request confirmation unit 812 , a pod allocator 813 , and a data processing unit 814 . In other embodiments, the GPU-based embedded edge server configuration and neural network service utilization system 800 may include more components than those of FIG. 8 . However, there is no need to clearly show most of the prior art components. For example, the GPU-based embedded edge server configuration and neural network service utilization system 800 may include other components such as a display or a transceiver.

The memory 840 is a computer-readable recording medium and may include a random access memory (RAM), a read only memory (ROM), and a permanent mass storage device such as a disk drive. In addition, program codes for the operating system 841 and the GPV-based embedded edge server configuration and neural network service utilization routine 842 may be stored in the memory 840 . These software components may be loaded from a computer-readable recording medium separate from the memory 840 using a drive mechanism (not shown). The separate computer-readable recording medium may include a computer-readable recording medium (not shown) such as a floppy drive, a disk, a tape, a DVD/CD-ROM drive, and a memory card. In another embodiment, the software components may be loaded into the memory 840 through the network interface 830 instead of a computer-readable recording medium.

The bus 820 may enable communication and data transmission between components of the GPU-based embedded edge server configuration and the neural network service utilization system 800 . Bus 820 may be configured using a high-speed serial bus, parallel bus, storage area network (SAN), and/or other suitable communication technology.

The network interface 830 may be a computer hardware component for connecting the GPU-based embedded edge server configuration and the neural network service utilization system 800 to a computer network. The network interface 830 may connect the GPU-based embedded edge server configuration and the neural network service utilization system 800 to a computer network through a wireless or wired connection.

The database 850 may serve to store and maintain all information necessary for configuring the GPV-based embedded edge server and utilizing the neural network service. In FIG. 8, the GPU-based embedded edge server configuration and the neural network service utilization system 800 are illustrated as being built and included in the database 850, but the present invention is not limited thereto and may be omitted depending on the system implementation method or environment. Alternatively, it is also possible that all or part of the database exists as an external database built on a separate other system.

The processor 810 may be configured to process commands of a computer program by performing basic arithmetic, logic, and GPU-based embedded edge server configuration and input/output operations of the neural network service utilization system 800 . Instructions may be provided to processor 810 by memory 840 or network interface 830 and via bus 820 . The processor 810 may be configured to execute program codes for the application execution unit 811 , the service request confirmation unit 812 , the Pod allocation unit 813 , and the data processing unit 814 . Such program codes may be stored in a recording device such as the memory 840 .

The application execution unit 811 , the service request confirmation unit 812 , the Pod allocator 813 , and the data processing unit 814 may be configured to perform steps 510 to 550 of FIG. 5 .

The GPU-based embedded edge server configuration and neural network service utilization system 800 may include an application execution unit 811 , a service request confirmation unit 812 , a pod assignment unit 813 , and a data processing unit 814 .

The application execution unit 811 executes the device query application in all edge servers, transmits the update of the extended resource for the GPU standard to the API server as an HTTP patch request, and registers it in all nodes.

The service request confirmation unit 812 requests and confirms a desired service from the API server, and then transmits/receives data for receiving external data for the request from the IoT data source to the stream socket.

The program that communicates for each request from the client uses a cluster-accessible authentication library and a Kubernetes API that supports internal cloud operations.

The pod allocator 813 allocates a neural network pod using Kubernetes scheduling for the received external data. The pod allocator 813 sets a new filtering condition in the configuration file of the neural network pod for the newly added GPU resource.

Pod allocator 813 includes volume filtering to check mountable volumes and node validity for conflicts between volumes, resource filtering to check ports requested by Pods and sufficient resources, and allow and selectors to support scheduling according to cluster topology Perform node filtering to identify valid nodes through topology filtering to identify Kubernetes components that

Thereafter, the allocation priority is determined by checking the conditions of the number of Pod replicas of the allocable node, node availability, resource balance, Pod-relevance and taints obtained from node filtering, and the Pod-relevance determines the label condition of the node. It decides and assigns weights to adjust priorities.

Then, the actual scheduling process of allocating neural network Pods according to node filtering and node priority calculation is performed.

Finally, the data processing unit 814 processes the data that the allocated Pod is transmitted externally and stored internally using a neural network.

The device described above may be implemented as a hardware component, a software component, and/or a combination of the hardware component and the software component. For example, the devices and components described in the embodiments may include a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), a programmable logic unit (PLU). ), a microprocessor, or any other device capable of executing and responding to instructions, may be implemented using one or more general purpose computers or special purpose computers. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that can include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

The software may comprise a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or device, to be interpreted by or to provide instructions or data to the processing device. may be embodied in The software may be distributed over networked computer systems, and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and carry out program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

As described above, although the embodiments have been described with reference to the limited embodiments and drawings, various modifications and variations are possible for those skilled in the art from the above description. For example, the described techniques are performed in a different order than the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (8)

executing a device query application on all edge servers, and when GPU resources are added, updating information on the added GPU resources is transmitted to the API server as an HTTP patch request and registered in all nodes;
checking a service request for requesting a desired service from a client to an API server of an edge existing locally;
After confirming the service request, the edge server receiving external data for the service request from an IoT data source that transmits and receives data using a stream socket;
allocating neural network Pods using Kubernetes scheduling for received external data; and
The assigned Pods process external data through the neural network.
including,
Allocating neural network pods using Kubernetes scheduling for received external data includes:
When allocating a newly added GPU resource to a neural network pod, filtering conditions are set according to the information about the added GPU resource,
Allocating the neural network Pod comprises:
Kubernetes components including permissions and selectors to support mountable volumes and node validity checking for conflicts between volumes, resource filtering to check ports requested by Pods and sufficient resources, and scheduling based on cluster topology. a node filtering step of identifying valid nodes through the identifying topology filtering;
a node priority calculation step of determining a label condition of a node and adjusting the priority by assigning weights; and
It includes the actual scheduling step of allocating neural network Pods based on node filtering and node priority calculation.
How to configure embedded edge servers and utilize neural network services. According to claim 1,
The step of confirming the service request requesting the desired service from the client to the API server of the edge existing locally is:
The program that communicates for each request from the client uses the cluster-accessible authentication library and the Kubernetes API to support internal cloud operations.
How to configure embedded edge servers and utilize neural network services. delete delete an application execution unit that executes a device query application on all edge servers and, when a GPU resource is added, transmits an update of information about the added GPU resource to the API server as an HTTP patch request and registers it with all nodes;
After requesting and confirming a desired service to the API server, a service request confirmation unit for transmitting and receiving data for receiving external data for the request from the IoT data source through the stream socket;
Pod allocator for allocating neural network Pods using Kubernetes scheduling for received external data; and
Data processing unit where assigned Pods use neural networks to process data sent externally and stored internally
including,
Pod allocation unit,
When allocating a newly added GPU resource to a neural network pod, a filtering condition is set according to the information about the added GPU resource,
Kubernetes components including permissions and selectors to support mountable volumes and node validity checking for conflicts between volumes, resource filtering to check ports requested by Pods and sufficient resources, and scheduling based on cluster topology. performing node filtering to identify valid nodes through topology filtering to identify;
calculating a node priority for determining a label condition of a node and adjusting the priority by weighting;
Actual scheduling of allocating neural network Pods based on node filtering and node priority calculation
Embedded edge server configuration and neural network service utilization system. 6. The method of claim 5,
The service request confirmation unit,
The program that communicates for each request from the client uses the cluster-accessible authentication library and the Kubernetes API to support internal cloud operations.
Embedded edge server configuration and neural network service utilization system. delete delete

Images