WO2023107128A1 - Système et procédé de gestion et de déploiement de modèles d'apprentissage automatique sur des dispositifs périphériques - Google Patents

Système et procédé de gestion et de déploiement de modèles d'apprentissage automatique sur des dispositifs périphériques Download PDF

Info

Publication number
WO2023107128A1
WO2023107128A1 PCT/US2021/065082 US2021065082W WO2023107128A1 WO 2023107128 A1 WO2023107128 A1 WO 2023107128A1 US 2021065082 W US2021065082 W US 2021065082W WO 2023107128 A1 WO2023107128 A1 WO 2023107128A1
Authority
WO
WIPO (PCT)
Prior art keywords
edge device
models
data
metrics
edge
Prior art date
Application number
PCT/US2021/065082
Other languages
English (en)
Inventor
Eugene DRUZHYNIN
Anton KHOZIAINOV
Kirill Vlasimirovich RYBACHUK
Maksim Goncharov
Original Assignee
Cherry Labs, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Cherry Labs, Inc. filed Critical Cherry Labs, Inc.
Publication of WO2023107128A1 publication Critical patent/WO2023107128A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/08—Configuration management of networks or network elements
    • H04L41/0803—Configuration setting
    • H04L41/0813—Configuration setting characterised by the conditions triggering a change of settings
    • H04L41/0816—Configuration setting characterised by the conditions triggering a change of settings the condition being an adaptation, e.g. in response to network events
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H04L43/0805—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability
    • H04L43/0817—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability by checking functioning

Definitions

  • Edge devices are devices that serve entry points into an enterprise or a service provider’s internal networks.
  • edge devices may include but are not limited to routers, routing switches, integrated access devices (IADs), multiplexers, network access devices, sensors, audio and/or video monitoring devices, and any computing or communication devices capable of processing collected data and providing entries to an internal network.
  • IADs integrated access devices
  • multiplexers network access devices
  • sensors sensors
  • audio and/or video monitoring devices any computing or communication devices capable of processing collected data and providing entries to an internal network.
  • Artificial intelligence (Al) or machine learning (ML) models are increasingly being trained and utilized by various ML applications to analyze collected data and detect various issues, such as abnormalities or attacks to an enterprise. There are significant demands for solutions to deploying the ML models to the edge devices of the enterprise.
  • FIG. 1 depicts an example of a system diagram 100 to support ML model management and deployment on edge devices
  • FIG. 2 depicts an example of a diagram depicting an edge computing environment where two ML models are deployed on an edge device in accordance with one aspect of the present embodiments.
  • FIG. 3(a) depicts an example of a flow of payload data transmitted to the edge devices via an VPN in accordance with one aspect of the present embodiments.
  • FIG. 3(b) depicts an example of a flow of extracted data transmitted from the edge devices via the VPN in accordance with one aspect of the present embodiments.
  • FIG. 3(c) shows an example of a structure of the configuration data package in accordance with one aspect of the present embodiments.
  • FIG. 3(d) shows an example of a structure of the extracted data package with one aspect of the present embodiments.
  • FIG. 4(a) depicts an example of a diagram showing the delivery of the ML models to the edge device via a secure connection in accordance with one aspect of the present embodiments.
  • FIG. 4(b) depicts an example of a diagram illustrating how secured and encrypted data can be used securely in accordance with one aspect of the present embodiments.
  • FIG.5 shows an example of a flow of data through the edge device and the unified interface in accordance with one aspect of the present embodiments.
  • FIG. 6 depicts a flowchart of an example of a process to support ML model management and deployment on edge devices in accordance with one aspect of the present embodiments.
  • a new approach is proposed that contemplates systems and methods to support management and automated deployment of one or more machine learning (ML) models on an edge device with one or more connected sensors.
  • the one or more ML models are trained, optimized, and deployed to the edge device in a particular configuration specific to the edge device, wherein such configuration includes but is not limited to one or more of time frames of the deployment of the ML models, format of the ML models, and the architecture of the edge device.
  • the ML models are executed on the edge device to perform certain ML operations/tasks.
  • various metrics, status, and performance of the deployed ML models as well as the status of the edge device that runs the ML models are collected and monitored in real time.
  • the ML models that haven been deployed on the edge device are re-trained, re-optimized, and re-deployed again to take the change to the edge device into account.
  • the proposed approach enables deploying the right ML models that are best suited to assist performing the ML operations under the current configuration of the edge device, wherein the ML models are optimized at the granularity of user-specific edge device properties, such as a device type, architecture, power, resources available for deployment.
  • ML models with different formats can be deployed on different edge devices and each sensor connected to an edge device may have its own ML models, which can be switched quickly in case of need.
  • the proposed approach enables continuous retraining, optimization, and deployment of the ML models at runtime for different instances of the same or different types of sensor when, e.g., a particular sensor is added to the edge device.
  • the proposed approach maintains a safe environment for running the ML models by keeping all the data at an encrypted location on the edge device and transferring the data with minimum footprint (e.g., only when loading the data into a memory such as RAM).
  • FIG. 1 depicts an example of a system diagram 100 to support ML model management and deployment on edge devices.
  • the diagram depicts components as functionally separate, such depiction is merely for illustrative purposes. It will be apparent that the components portrayed in this figure can be arbitrarily combined or divided into separate software, firmware and/or hardware components. Furthermore, it will also be apparent that such components, regardless of how they are combined or divided, can execute on the same host or multiple hosts, and wherein the multiple hosts can be connected by one or more networks.
  • the system 100 includes an external device 102 where one or more ML models are deployed, a ML model deployment engine 104, and a ML model database (DB) 106.
  • These components in the system 100 each runs on one or more computing units/appliances/devices/hosts (not shown) each having one or more processors and software instructions stored in a storage unit such as a non-volatile memory (also referred to as secondary memory) of the computing unit for practicing one or more processes.
  • a storage unit such as a non-volatile memory (also referred to as secondary memory) of the computing unit for practicing one or more processes.
  • the software instructions are executed by the one or more processors, at least a subset of the software instructions is loaded into memory (also referred to as primary memory) by one of the computing units, which becomes a special purpose one for practicing the processes.
  • each computing unit can be a computing device, a communication device, a storage device, or any computing device capable of running a software component.
  • Each computing unit has a communication interface (not shown), which enables the computing units to communicate with each other, the user, and other devices over one or more communication networks following certain communication protocols, such as TCP/IP, http, https, ftp, and sftp protocols.
  • the communication networks can be but are not limited to, Internet, intranet, wide area network (WAN), local area network (LAN), wireless network, Bluetooth, WiFi, and mobile communication network.
  • the physical connections of the network and the communication protocols are well known to those of skilled in the art.
  • the edge device 102 is located at a boundary of and provides access/entry to the communication networks, wherein the edge device 102 can be but is not limited to a router, a routing switch, an integrated access device, a multiplexers, a network access device, an audio and/or video monitoring device, and any computing or communication devices with processing capabilities and providing entries to a communication network discussed above.
  • the edge device 102 connects/attaches to one or more sensors 103, e.g., audio/video monitoring devices, each configured to collect information from current edge device environment the edge device 102 is located at and provide the collected information to the edge device 102.
  • FIG. 2 depicts an example of a diagram depicting an edge computing environment where two ML models, Model #1 and #2 are deployed on the edge device 102, which is a processing unit that extracts and submits all data collected via the ML models to the ML model deployment engine 104, which, for a non-limiting example, can be a server in a cloud-based platform, for further processing and retraining of the ML models.
  • the ML model deployment engine 104 is configured to label each of the one or more ML models with a set of properties specific to the edge device 102, train or retrain the ML models using previously collected data from the edge device 102, optimize the ML models for executing on the edge device 102, and then deploy the ML models to the edge device 102.
  • each of the one or more ML models follows its own data path to the edge device 102 such that the data collected or produced by each of the ML models is stored separately on the edge device 102, which leads to high security.
  • the ML model deployment engine 104 is configured to store and maintain the ML models in the ML model database 104 and to retrieve the ML models from the ML model database 104 when the ML models are to be deployed to the edge device 102.
  • the ML model deployment engine 104 is configured to transmit the one or more ML models and associated configuration and/or command data to a plurality of edge devices 102s located at the entries of an enterprise’s internal network via a secured Virtual Private Network (VPN) 302 for data security.
  • FIG. 3(a) shows an example of a flow of payload data including the ML models, configuration data and/or command data transmitted from the ML model deployment engine 104 through the VPN 302 to the plurality of edge devices 102s where the one or more ML models are deployed and executed.
  • the deployment of the ML models and the associated configuration and/or command data by the ML model deployment engine 104 is initiated by a user via one or more user interfaces including but not limited to a Command Line Interface (CLI) 304, a Web Interface 306, and a Program Interface 308 as shown by the example of FIG. 3(a).
  • CLI Command Line Interface
  • Web Interface 306
  • Program Interface 308 Program Interface
  • each edge device 102 is configured to extract and process data collected via the one or more sensors and/or audio/video devices (e.g., cameras). The processed data/metrics is then transmitted back to the ML model deployment engine 104 for monitoring and/or further processing.
  • FIG. 3(b) shows an example of a flow of extracted and processed data transmitted from the edge devices 102s to the ML model deployment engine 104 through the VPN 302.
  • the data can be accessed by a user using via one or more user interfaces including but not limited to the Command Line Interface 304, the Web Interface 306, and the Program Interface 308 as shown by the example of FIG. 3(b).
  • the ML model deployment engine 104 is configured to deploy the payload of the one or more ML models and associated configuration and/or command data to a plurality of edge devices 102s in a form of a configuration data package 310.
  • FIG. 3(c) shows an example of a structure of the configuration data package 310.
  • the configuration data package 310 includes at least architecture, format as well as configurations of the one or more ML models.
  • the configuration data package 310 further includes a piece of source code or a binary code such as a plugin or a separate executable code that serves the ML models as well as configuration for such code as shown by the example of FIG. 3(c).
  • each edge device 102 is configured to transmit the extracted and processed data back to the ML model deployment engine 104 for monitoring and/or further processing in a form of extracted data package 312.
  • FIG. 3(d) shows an example of a structure of the extracted data package 312.
  • the extracted data package 312 includes but is not limited to one or more of metrics and prediction output of the ML models, user-defined metrics and output, program logs, system information, audio/video confirmations of the edge devices 102s and their connected sensors.
  • the ML model deployment engine 104 and the edge device 102 communicate with each other only using a secure connection 401 provided by VPN 302 as shown by the example of FIG. 4(a).
  • all the data delivered from the ML model deployment engine 104 in a cloud computing environment is stored in an encrypted storage 402 on the edge device 102 as shown by the example of FIG. 4(a), wherein such data includes but is not limited to the deployed ML models, the configuration data, and the code serving the ML models.
  • Such data is maintained in the encrypted storage 402 until the data is called by the main process and loaded into a memory 404 (e.g., a RAM) of the edge device 102 for execution as shown by the example of FIG. 4(b).
  • FIG. 4(b) shows an example where all the data secured by encryption on the encrypted storage 402 is only decrypted and loaded into the memory 404 managed by a model inference server 406 configured to load and execute the ML models on the edge device 102 and a media streaming pipeline 408 configured to execute pieces of source code or executable plugins that serves the ML models on the edge device 102, respectively.
  • the proposed approach provides maximum security to the ML model deployment engine 104 and the edge device 102 and maintains a minimum footprint of the transferred data.
  • the model inference server 406 and the media streaming pipeline 408 are utilized by the edge device 102 as abstraction layers to provide more flexibility and the ability to use different tools to develop and execute the one or more ML models on the edge device 102.
  • Using separate abstraction layers independent of the edge device 102 allows different ML models (e.g., different types, architecture, formats) to be deployed and executed on different edge devices.
  • the model inference server 406 is utilized as an abstract layer to load and execute the one or more ML models (e.g., model #1, model #2, and model #3) in an efficient and safe manner as shown by the example of FIG. 5.
  • the media processing pipeline 408 serves as an abstraction layer to run source code or executable plugins securely and to save the pipeline from crashing in case the code malfunctions during execution.
  • the media processing pipeline 408 also serves as an interface between sensors connected/attached to the edge device 102 and the one or more ML models, wherein the media streaming pipeline 408 collects, prepares, decodes, and delivers data 501 captured by the sensors (e.g., audio, video, and other sensor data) to the one or more ML models running on the model inference server 406 for processing via a code execution flow 504.
  • the media streaming pipeline 408 is further configured to encode and send the processing output 502 (e.g., the metrics) to the next stages such as collecting, storing, transforming, and transferring to the ML model deployment engine 104.
  • the media processing pipeline 408 is an unified pipeline that enables different ML models to be executed on the edge device 102 in a simple and unified way.
  • FIG.5 shows an example of a flow of data through the edge device 102 and the unified interface that allows the use of practically any ML models on the edge device 102.
  • the ML model deployment engine 104 is configured to continuously monitor the sensor data collected and extracted by the edge device 102 from the sensors attached to the edge device 102 and transmitted to the ML model deployment engine 104. If any change or modification to the configuration of the edge device 102 and/or the sensors connected to the edge device 102 is detected, for a nonlimiting example, a new sensor is attached to or removed from the edge device 102, the ML model deployment engine 104 is configured to retrain and re-optimize the one or more ML models to take such change into account and send the retrained one or more ML models back to the edge device 102 to be re-deployed again so that the edge devicel02 may continue to function under the modified configurations.
  • FIG. 6 depicts a flowchart 600 of an example of a process to support ML model management and deployment on edge devices.
  • FIG. 6 depicts functional steps in a particular order for purposes of illustration, the processes are not limited to any particular order or arrangement of steps.
  • One skilled in the relevant art will appreciate that the various steps portrayed in this figure could be omitted, rearranged, combined and/or adapted in various ways.
  • the flowchart 600 starts at block 602, where one or more machine learning (ML) models are trained, optimized, and deployed to an edge device of a network in a configuration specific to the edge device, wherein the edge device connects to one or more sensors.
  • the flowchart 600 continues to block 604, where the one or more ML models are accepted and executed on the edge device on data collected from the one or more connected sensors.
  • the flowchart 600 continues to block 606, where a set of metrics of the collected data is extracted, processed, and transmitted for further analysis.
  • the flowchart 600 continues to block 608, where the set of metrics transmitted by the edge device is monitored in real time.
  • the flowchart 600 ends at block 610, where the one or more ML models are retrained, re-optimized, and re-deployed to the edge device if a change in the configuration of the edge device is detected at runtime.
  • One embodiment may be implemented using a conventional general purpose or a specialized digital computer or microprocessor(s) programmed according to the teachings of the present disclosure, as will be apparent to those skilled in the computer art.
  • Appropriate software coding can readily be prepared by skilled programmers based on the teachings of the present disclosure, as will be apparent to those skilled in the software art.
  • the invention may also be implemented by the preparation of integrated circuits or by interconnecting an appropriate network of conventional component circuits, as will be readily apparent to those skilled in the art.
  • the methods and system described herein may be at least partially embodied in the form of computer-implemented processes and apparatus for practicing those processes.
  • the disclosed methods may also be at least partially embodied in the form of tangible, non-transitory machine readable storage media encoded with computer program code.
  • the media may include, for example, RAMs, ROMs, CD-ROMs, DVD-ROMs, BD- ROMs, hard disk drives, flash memories, or any other non-transitory machine-readable storage medium, wherein, when the computer program code is loaded into and executed by a computer, the computer becomes an apparatus for practicing the method.
  • the methods may also be at least partially embodied in the form of a computer into which computer program code is loaded and/or executed, such that, the computer becomes a special purpose computer for practicing the methods.
  • the computer program code segments configure the processor to create specific logic circuits.
  • the methods may alternatively be at least partially embodied in a digital signal processor formed of application specific integrated circuits for performing the methods.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Computer And Data Communications (AREA)

Abstract

L'invention concerne une nouvelle approche qui prend en charge la gestion et le déploiement automatisé d'un ou plusieurs modèles d'apprentissage automatique (ML) sur un dispositif périphérique avec un ou plusieurs capteurs connectés. Lesdits modèles ML sont formés, optimisés et déployés sur le dispositif périphérique selon une configuration particulière spécifique au dispositif périphérique. Une fois les modèles ML déployés sur le dispositif périphérique, les modèles ML sont exécutés sur le dispositif périphérique pour effectuer certaines opérations ML. Pendant ce temps, diverses mesures, états et performances des modèles ML déployés ainsi que l'état du dispositif périphérique qui exécute les modèles ML sont recueillis et surveillés en temps réel. S'il y a un quelconque changement apporté au dispositif périphérique, les modèles ML qui ont été déployés sur le dispositif périphérique sont de nouveau formés, optimisés et déployés pour prendre en compte le changement apporté au dispositif périphérique.
PCT/US2021/065082 2021-12-09 2021-12-23 Système et procédé de gestion et de déploiement de modèles d'apprentissage automatique sur des dispositifs périphériques WO2023107128A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202163287906P 2021-12-09 2021-12-09
US63/287,906 2021-12-09

Publications (1)

Publication Number Publication Date
WO2023107128A1 true WO2023107128A1 (fr) 2023-06-15

Family

ID=86730986

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2021/065082 WO2023107128A1 (fr) 2021-12-09 2021-12-23 Système et procédé de gestion et de déploiement de modèles d'apprentissage automatique sur des dispositifs périphériques

Country Status (1)

Country Link
WO (1) WO2023107128A1 (fr)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180054490A1 (en) * 2016-08-22 2018-02-22 fybr System for distributed intelligent remote sensing systems
WO2020209951A1 (fr) * 2019-04-09 2020-10-15 FogHorn Systems, Inc. Plateforme informatique de périphérie intelligent avec capacité d'apprentissage automatique
US20210266225A1 (en) * 2020-02-25 2021-08-26 International Business Machines Corporation Personalized machine learning model management and deployment on edge devices

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180054490A1 (en) * 2016-08-22 2018-02-22 fybr System for distributed intelligent remote sensing systems
WO2020209951A1 (fr) * 2019-04-09 2020-10-15 FogHorn Systems, Inc. Plateforme informatique de périphérie intelligent avec capacité d'apprentissage automatique
US20210266225A1 (en) * 2020-02-25 2021-08-26 International Business Machines Corporation Personalized machine learning model management and deployment on edge devices

Similar Documents

Publication Publication Date Title
US20210058424A1 (en) Anomaly detection for microservices
US11294754B2 (en) System and method for contextual event sequence analysis
US20210067527A1 (en) Structural graph neural networks for suspicious event detection
JP6099793B2 (ja) １つ以上の画像処理アルゴリズムの自動選択のための方法およびシステム
US11243973B2 (en) Automated scalable contextual data collection and extraction system
CN111953641A (zh) 未知网络流量的分类
US20210303984A1 (en) Machine-learning based approach for classification of encrypted network traffic
US11831682B2 (en) Highly scalable distributed connection interface for data capture from multiple network service and cloud-based sources
US20140222813A1 (en) Collecting data in internet of things
US10129114B1 (en) Protocol exposure as network health detection
CN110932910B (zh) 一种软件故障的日志记录方法及装置
US20180248745A1 (en) Method and network node for localizing a fault causing performance degradation of a service
US10628587B2 (en) Identifying and halting unknown ransomware
CN111669281B (zh) 告警分析方法、装置、设备及存储介质
CN104077328B (zh) MapReduce分布式系统的作业诊断方法及设备
US10839311B2 (en) Cognitive computing for servers and mobile devices
CN110062926B (zh) 设备驱动器遥测
US20170255474A1 (en) Heuristic process for inferring resource dependencies for recovery planning
CN105119899B (zh) 非入侵式客户端运行时数据采集方法与系统
KR20220127757A (ko) 오토프로파일링 기반 네트워크 패킷 이상행위 자동 탐지 장치 및 방법
US20160205118A1 (en) Cyber black box system and method thereof
WO2019165456A1 (fr) Système automatisé de collecte et d'extraction de données contextuelles évolutives
CN112367215B (zh) 基于机器学习的网络流量协议识别方法和装置
WO2023107128A1 (fr) Système et procédé de gestion et de déploiement de modèles d'apprentissage automatique sur des dispositifs périphériques
US20220292374A1 (en) Dynamic parameter collection tuning

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21967433

Country of ref document: EP

Kind code of ref document: A1