CN118467181B

CN118467181B - Real-time image processing method and system based on edge calculation

Info

Publication number: CN118467181B
Application number: CN202410920327.4A
Authority: CN
Inventors: 李济民; 谢明军
Original assignee: Shenzhen Xinghe Iot Technology Co ltd; Shenzhen Parca Technology Co ltd
Current assignee: Shenzhen Xinghe Iot Technology Co ltd; Shenzhen Parca Technology Co ltd
Priority date: 2024-07-10
Filing date: 2024-07-10
Publication date: 2024-09-06
Anticipated expiration: 2044-07-10
Also published as: CN118467181A

Abstract

The application relates to the technical field of image processing, and discloses a real-time image processing method and system based on edge calculation. The method comprises the following steps: carrying out heterogeneous resource virtualization on a plurality of edge devices to obtain an edge computing grid; performing real-time monitoring on node performance and network state of a plurality of edge devices based on the edge computing grid, and constructing a multi-dimensional resource portrait model; performing task decomposition and parallelization reconstruction through a multi-dimensional resource portrait model to obtain a task dependency graph; performing topology optimization based on the task dependency graph to obtain a distributed image processing pipeline; dynamically balancing the image data flow and the calculation load of each edge device based on a distributed image processing pipeline to obtain a target task scheduling strategy; according to a target task scheduling strategy, self-adaptive compression transmission and edge cloud cooperative feedback are carried out on image processing results of a plurality of edge devices, and an image processing parameter set is obtained.

Description

Real-time image processing method and system based on edge calculation

Technical Field

The present application relates to the field of image processing technologies, and in particular, to a method and a system for processing a real-time image based on edge calculation.

Background

With the rapid development of internet of things and 5G networks, edge computing is rapidly rising as an emerging distributed computing model. In the field of real-time image processing, the traditional cloud computing mode faces challenges such as high network delay, large bandwidth consumption, privacy security risk and the like, and is difficult to meet the increasingly real-time performance and low-delay requirements. By sinking computing resources to the network edge, the edge computing can effectively reduce data transmission delay, improve processing efficiency and provide a new technical direction for real-time image processing.

However, real-time image processing in an edge computing environment still faces many challenges. Edge devices often have heterogeneous and resource constrained characteristics, and how to efficiently manage and schedule these heterogeneous resources is a critical issue. Secondly, real-time image processing tasks often have computationally intensive features, and how to achieve efficient task decomposition and parallel processing under limited edge resources is also a challenge to be solved. In addition, the cooperative processing between the edge device and the cloud end is also an important issue to be considered how to realize efficient data transmission under the limited bandwidth.

Disclosure of Invention

The application provides a real-time image processing method and a real-time image processing system based on edge calculation, which are used for improving the accuracy of real-time image processing by adopting edge calculation.

In a first aspect, the present application provides a real-time image processing method based on edge calculation, where the real-time image processing method based on edge calculation includes:

Carrying out heterogeneous resource virtualization on a plurality of edge devices to obtain an edge computing grid;

monitoring node performance and network state of the plurality of edge devices in real time based on the edge computing grid, and constructing a multi-dimensional resource portrait model;

Performing task decomposition and parallelization reconstruction on an image processing operator of each edge device through the multi-dimensional resource portrait model to obtain a task dependency graph;

performing topology optimization on the task execution sequence of the plurality of edge devices based on the task dependency graph to obtain a distributed image processing pipeline;

dynamically balancing the image data flow and the calculation load of each edge device based on the distributed image processing pipeline to obtain a target task scheduling strategy;

And carrying out self-adaptive compression transmission and edge cloud cooperative feedback on the image processing results of the plurality of edge devices according to the target task scheduling strategy to obtain an image processing parameter set.

In a second aspect, the present application provides a real-time image processing system based on edge computation, the real-time image processing system based on edge computation comprising:

the virtualization module is used for carrying out heterogeneous resource virtualization on the plurality of edge devices to obtain an edge computing grid;

The real-time monitoring module is used for monitoring node performance and network state of the plurality of edge devices in real time based on the edge computing grid, and constructing a multi-dimensional resource portrait model;

the task decomposition module is used for carrying out task decomposition and parallelization reconstruction on the image processing operators of each edge device through the multi-dimensional resource portrait model to obtain a task dependency graph;

The topology optimization module is used for performing topology optimization on the task execution sequences of the plurality of edge devices based on the task dependency graph to obtain a distributed image processing pipeline;

The dynamic balancing module is used for dynamically balancing the image data flow and the calculation load of each edge device based on the distributed image processing pipeline to obtain a target task scheduling strategy;

and the feedback module is used for carrying out self-adaptive compression transmission and edge cloud cooperative feedback on the image processing results of the plurality of edge devices according to the target task scheduling strategy to obtain an image processing parameter set.

A third aspect of the present application provides a real-time image processing apparatus based on edge calculation, comprising: a memory and at least one processor, the memory having instructions stored therein; the at least one processor invokes the instructions in the memory to cause the edge-calculation based real-time image processing device to perform the edge-calculation based real-time image processing method described above.

A fourth aspect of the present application provides a computer readable storage medium having instructions stored therein which, when run on a computer, cause the computer to perform the above-described edge computation based real-time image processing method.

According to the technical scheme provided by the application, the hardware resources of the plurality of edge devices are subjected to abstract processing and virtualized packaging, so that unified management and scheduling of the heterogeneous edge devices are realized, and the resource utilization efficiency is improved. Based on a multi-dimensional resource portrait model, real-time monitoring of node performance and network state of edge equipment is realized, the parallel processing capability of the edge equipment is fully utilized through task decomposition and parallelization reconstruction of an image processing operator, the task execution sequence of a plurality of edge equipment is reasonably arranged based on the topology optimization of a task dependency graph, the load balancing among the edge equipment is realized through dynamic balancing of image data flow and calculation load, the self-adaptive compression transmission of image processing results is realized according to network bandwidth and image importance, the powerful calculation capability of a cloud is fully utilized through cooperative processing of the edge equipment and the cloud, the iterative optimization of image processing parameters is realized based on the edge cloud fusion analysis result, the self-adaptive adjustment of the processing parameters is realized, and the flexible scheduling and the efficient utilization of edge computing resources are realized through a multi-stage computing resource pool and an elastic network topology structure. Through the node fault prediction model and the load balancing optimization, the reliability and the fault tolerance of the system are improved, the continuity of real-time image processing is guaranteed, and the accuracy of the real-time image processing is further improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings may be obtained based on these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic diagram of an embodiment of a real-time image processing method based on edge computation according to an embodiment of the present application;

FIG. 2 is a schematic diagram of an embodiment of a real-time image processing system based on edge computation in an embodiment of the present application.

Detailed Description

The embodiment of the application provides a real-time image processing method and system based on edge calculation. The terms "first," "second," "third," "fourth" and the like in the description and in the claims and in the above drawings, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments described herein may be implemented in other sequences than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus.

For ease of understanding, a specific flow of an embodiment of the present application is described below, referring to fig. 1, and an embodiment of a real-time image processing method based on edge calculation in an embodiment of the present application includes:

step S101, carrying out heterogeneous resource virtualization on a plurality of edge devices to obtain an edge computing grid;

it will be appreciated that the execution subject of the present application may be a real-time image processing system based on edge computation, and may also be a terminal or a server, which is not limited herein. The embodiment of the application is described by taking a server as an execution main body as an example.

Specifically, hardware resources of a plurality of edge devices are subjected to abstraction processing, and resource descriptors of each edge device are obtained. The resource descriptor describes key performance indicators of computing power, storage capacity, network bandwidth, etc. of each edge device. And carrying out calculation power quantitative evaluation on the plurality of edge devices based on the resource descriptors to obtain device performance indexes. The device performance index provides a standardized measure of the computing power of each edge device, so that the performance of different devices can be directly compared and evaluated. And carrying out hierarchical clustering on a plurality of edge devices based on the device performance index to form a multi-level computing resource pool. Hierarchical clustering builds multiple levels of computing resource pools by aggregating devices of similar performance together. Devices within each resource pool have similar computing power and can provide more refined resource management during the allocation and scheduling of computing tasks. And carrying out virtualization packaging on a plurality of edge devices in the multi-stage computing resource pool to obtain a schedulable computing container. The computing container abstracts computing resources of the physical devices into logical resources through virtualization techniques so that these resources can be flexibly scheduled and managed. And based on the calculation container, carrying out network topology dynamic reconstruction on the plurality of edge devices to obtain an elastic network topology structure. The elastic network topology structure ensures that high-efficiency data transmission can be maintained in various network states by dynamically adjusting network connection between edge devices. And (3) based on the elastic network topology structure, carrying out communication protocol optimization on the computing container to obtain a lightweight data transmission mechanism. The lightweight data transmission mechanism reduces the cost and delay of data transmission and improves the data exchange efficiency between edge devices by simplifying the communication protocol. And according to the lightweight data transmission mechanism, carrying out resource scheduling strategy analysis and full-performance enhancement processing on the computing container to obtain the edge computing grid. The resource scheduling strategy analysis makes an efficient resource scheduling scheme by analyzing the resource use condition and task demand of the computing container, ensures that the computing task can run on the proper computing container, and maximizes the resource utilization rate. The integrity enhancement process ensures the security and integrity of tasks and data running in the edge computing grid by enhancing security protection against the computing container.

Step S102, monitoring node performance and network state of a plurality of edge devices in real time based on an edge computing grid, and constructing a multi-dimensional resource portrait model;

Specifically, resource detection is performed on a plurality of equipment nodes in the edge computing grid, and an original resource data set of each equipment node is obtained. These data sets include basic performance metrics such as computing power, storage capacity, network bandwidth, etc. of the device. And generating a computing capability index of each equipment node according to the original resource data set, and accurately evaluating the processing capability of the equipment. And classifying the storage capacities of the plurality of equipment nodes in a grading manner based on the computing capacity indexes to obtain a storage resource matrix. The matrix describes the storage resource distribution of each device node. And carrying out data access mode analysis on the storage resource matrix to obtain the I/O performance characteristic vector of each equipment node. The I/O performance feature vector reflects the input/output performance of the device under different operations. And carrying out bandwidth test on network connection among a plurality of equipment nodes according to the I/O performance characteristic vector to obtain a network topological graph. The network topology diagram shows the network structure between the device nodes and the connection mode thereof. Based on the network topology diagram, the communication delay among a plurality of equipment nodes is statistically modeled, and a delay distribution function is obtained. The delay profile function provides a probability distribution of communication delays between device nodes that helps optimize network communications. And dynamically evaluating the energy consumption efficiency of each equipment node according to the delay distribution function to obtain an energy consumption efficiency curve. The energy consumption efficiency curve shows the energy consumption of the device under different loads. And performing polynomial fitting on the energy consumption efficiency curve to obtain a relation model of the node load and the energy consumption. And carrying out probability analysis on the reliability of each equipment node based on the relation model to obtain a node fault prediction model. The model can predict the fault probability of the equipment node in future operation, is favorable for taking preventive measures in advance and ensures the stable operation of the system. On the basis, multi-dimensional feature extraction and fusion are carried out, and performance data of different dimensions are synthesized to obtain a comprehensive multi-dimensional resource portrait model.

Step S103, performing task decomposition and parallelization reconstruction on an image processing operator of each edge device through a multi-dimensional resource portrait model to obtain a task dependency graph;

Specifically, the image processing algorithm is divided into functional modules to obtain an initial operator set. The initial operator set comprises all basic functional modules in an image processing algorithm, and an original dependency graph is constructed by analyzing the data dependency relationship among the modules. The original dependency graph shows the dependency relationship among the operators, and is helpful for understanding the overall structure and data flow of the algorithm. And evaluating the computational complexity of the image processing operator based on the original dependency graph to generate an operator weight matrix. The operator weight matrix describes the computational complexity of each operator. And carrying out granularity optimization and combination on the image processing operators of each edge device according to the operator weight matrix to form an optimized operator set. By combining operators with lower computational complexity, the number of operators is reduced, and the overall function of the algorithm is maintained. And carrying out parallelism analysis on the optimization operator set, and identifying potential parallel execution paths. And determining which operators can be executed in parallel by analyzing the independence among the operators to form parallel task blocks. Based on the potential parallel execution paths, task grouping is carried out on the image processing operators of each edge device, so that the computing capability of the device can be fully utilized in actual operation. And carrying out matching analysis on the multi-dimensional resource portrait model according to the parallel task blocks to generate a resource affinity matrix. The resource affinity matrix reflects the degree of matching between the parallel task blocks and the respective edge devices, through which the device most suitable for executing each task block is found. And performing equipment allocation on the parallel task blocks based on the resource affinity matrix to form an initial task mapping scheme. The initial task mapping scheme assigns an appropriate edge device to each task block to achieve efficient task execution. And carrying out load balancing optimization on the initial task mapping scheme. And the load condition of each edge device is analyzed, the task allocation scheme is adjusted, and the calculation load balance distribution of each device is ensured, so that the operation efficiency of the whole system is improved. And generating a task dependency graph according to the balanced task distribution strategy. The task dependency graph shows the dependency relationships among task blocks and the respective execution order.

Step S104, performing topology optimization on task execution sequences of a plurality of edge devices based on the task dependency graph to obtain a distributed image processing pipeline;

Specifically, a critical path analysis is carried out on the task dependency graph, all critical task sequences influencing the overall task execution time are identified, and a task parallelism matrix is constructed. The task parallelism matrix is used for representing the parallel execution capacity of each task under different conditions and helping to identify tasks which can be executed simultaneously so as to maximize the resource utilization rate. And carrying out hierarchical clustering on the tasks based on the task parallelism matrix to obtain a task hierarchical structure diagram. The task hierarchy chart shows the hierarchical relationship and the dependency among the tasks, so that the task organization is clearer. On the basis, the task hierarchy chart is subjected to key resource identification, and a resource competition hot spot list is generated. The resource contention hotspot list marks the contention situation for a particular resource among a plurality of tasks, indicating possible bottlenecks and conflict points. And according to the resource competition hot spot list, sequencing the task execution sequences of the plurality of edge devices in priority, dividing the tasks into different pipeline stages, and constructing an initial pipeline structure. The initial pipeline structure ensures the priority execution of the key tasks through priority ordering and phase division, and simultaneously reduces resource competition and conflict to the greatest extent. Based on the initial pipeline structure, data transmission modeling among tasks is carried out, and a data flow chart is obtained. The data flow diagram shows the data transmission paths and data flow conditions between tasks, helping to optimize the data transmission efficiency. And carrying out load balancing analysis on the initial pipeline structure according to the data flow diagram, and calculating a phase balancing factor. The stage balancing factor is used to evaluate the load balancing condition of each pipeline stage and help identify the problem of load imbalance. And carrying out iterative optimization on the stage balance factors, and gradually improving load distribution through multiple adjustment and optimization to obtain an balanced pipeline configuration scheme. And (3) performing time sequence processing based on the balanced pipeline configuration scheme to obtain a distributed image processing pipeline. The time sequential processing ensures that each task starts and executes at the appropriate point in time, maximizing resource utilization efficiency and reducing latency.

Step 105, dynamically balancing the image data flow and the calculation load of each edge device based on the distributed image processing pipeline to obtain a target task scheduling strategy;

specifically, task decomposition is performed on the distributed image processing pipeline to obtain an atomic task set. The atomic task set represents the smallest executable unit of the entire image processing task, ensuring that each task can be processed independently. And according to the atomic task set, performing slicing processing on the image data stream of each edge device, and generating a data block allocation scheme of each edge device. The data block allocation scheme aims at reasonably allocating image data to each edge device, and ensuring the high efficiency and the balance of data processing. According to the data block allocation scheme, a data transmission cost model of each edge device is generated. The data transfer cost model evaluates the resources and time costs required by each device during the data transfer process, helping to optimize data flow and task allocation. On the basis, an atomic task set is initially distributed, and a task initial distribution diagram is obtained. The task initiation profile shows the distribution of atomic tasks across different edge devices. And calculating the load balance degree of each edge device based on the task initial distribution diagram to obtain a load unbalance degree index. The load unbalance index reflects the load condition of each device under the current task allocation and helps to identify devices which are overloaded or overloaded. And judging the threshold value of the load unbalance index, and determining the task allocation to be adjusted. And migrating part of tasks from equipment with heavy load to equipment with lighter load through task migration, so as to achieve load balancing and obtain a target task distribution scheme. And according to the target task distribution scheme, adjusting the task execution sequence to generate a task scheduling sequence. The task scheduling sequence ensures that tasks are executed in an optimal order, maximizes utilization of device resources and reduces latency. And carrying out resource competition analysis on the task scheduling sequence, identifying possible resource conflict and competition conditions, and generating a target task scheduling strategy based on an analysis result. The target task scheduling strategy ensures that the image processing task runs efficiently and stably in the edge computing environment by optimizing the resource allocation and the task execution sequence.

And S106, carrying out self-adaptive compression transmission and edge cloud cooperative feedback on image processing results of a plurality of edge devices according to a target task scheduling strategy to obtain an image processing parameter set.

Specifically, according to a target task scheduling strategy, quality evaluation is performed on the image processing result of each edge device, and an image quality index matrix is generated. The image quality index matrix describes the quality parameters of the processed image of each edge device. And carrying out image region importance analysis according to the image quality index matrix to obtain a region weight map. The region weight map reflects the importance of each region in the image, helping to identify key regions and secondary regions. And carrying out layered coding and entropy coding on the image based on the region weight map to generate an initial compressed data stream. Layered coding ensures that important areas are preferentially processed and compressed with high quality, while secondary areas are compressed with lower quality, thereby optimizing overall compression efficiency and image quality. And carrying out dynamic estimation on the network bandwidth according to the initial compressed data stream to obtain an available bandwidth prediction model. The available bandwidth prediction model evaluates the available bandwidth of the current network to help determine the optimal transmission strategy. And performing bit allocation optimization on the initial compressed data stream based on the available bandwidth prediction model to form a layered transmission strategy. The layered transmission strategy ensures the best transmission effect under the limited bandwidth by reasonably distributing bit rate, and improves the efficiency and quality of image transmission to the maximum extent. And carrying out transmission error analysis on the layered transmission strategy to obtain a data importance ranking table. The data importance ranking table lists the order of importance of the individual data blocks, which facilitates the priority of sending critical data during transmission. And carrying out priority scheduling on the initial compressed data stream according to the data importance ranking table to generate a target transmission queue. The target transmission queue ensures that the most important data can be transmitted preferentially under the condition of limited bandwidth, and the effectiveness and reliability of image transmission are improved. And carrying out cooperative processing on the data in the target transmission queue and the cloud to obtain an edge cloud fusion analysis result. The edge cloud fusion analysis optimizes the image processing and data transmission process by combining computing resources of the edge device and the cloud. And carrying out iterative optimization on the image processing parameters based on the edge cloud fusion analysis result to finally obtain an image processing parameter set.

In the embodiment of the application, the unified management and scheduling of the heterogeneous edge devices are realized by carrying out abstract processing and virtualized packaging on the hardware resources of the plurality of edge devices, and the resource utilization efficiency is improved. Based on a multi-dimensional resource portrait model, real-time monitoring of node performance and network state of edge equipment is realized, the parallel processing capability of the edge equipment is fully utilized through task decomposition and parallelization reconstruction of an image processing operator, the task execution sequence of a plurality of edge equipment is reasonably arranged based on the topology optimization of a task dependency graph, the load balancing among the edge equipment is realized through dynamic balancing of image data flow and calculation load, the self-adaptive compression transmission of image processing results is realized according to network bandwidth and image importance, the powerful calculation capability of a cloud is fully utilized through cooperative processing of the edge equipment and the cloud, the iterative optimization of image processing parameters is realized based on the edge cloud fusion analysis result, the self-adaptive adjustment of the processing parameters is realized, and the flexible scheduling and the efficient utilization of edge computing resources are realized through a multi-stage computing resource pool and an elastic network topology structure. Through the node fault prediction model and the load balancing optimization, the reliability and the fault tolerance of the system are improved, the continuity of real-time image processing is guaranteed, and the accuracy of the real-time image processing is further improved.

In a specific embodiment, the process of executing step S101 may specifically include the following steps:

(1) Abstracting hardware resources of a plurality of edge devices to obtain a resource descriptor of each edge device, and carrying out calculation power quantitative evaluation on the plurality of edge devices according to the resource descriptor to obtain a device performance index;

(2) Hierarchical clustering is carried out on a plurality of edge devices based on the device performance index to obtain a multi-stage computing resource pool, and virtualization packaging is carried out on the plurality of edge devices in the multi-stage computing resource pool to obtain a schedulable computing container;

(3) Performing network topology dynamic reconstruction on a plurality of edge devices based on a computing container to obtain an elastic network topology structure, and performing communication protocol optimization on the computing container based on the elastic network topology structure to obtain a lightweight data transmission mechanism;

(4) And carrying out resource scheduling strategy analysis and full-property enhancement processing on the computing container according to the lightweight data transmission mechanism to obtain the edge computing grid.

Specifically, the hardware resource of each device is analyzed and abstracted to obtain a resource descriptor. The resource descriptor is a description of hardware resources such as computing capacity, storage capacity, network bandwidth and the like of the equipment, and can uniformly represent resource conditions of different equipment. The resource descriptor may be represented in a vector form, e.gWherein, the method comprises the steps of, wherein,Represent the firstThe resource descriptor of the individual device(s),Representing the computing power of the device, typically measured in instructions per second (MIPS); representing the storage capacity of a device, typically measured in Gigabytes (GB); Representing the network bandwidth of the device, is typically measured in megabits per second (Mbps). And carrying out calculation power quantitative evaluation on the plurality of edge devices according to the resource descriptors to obtain device performance indexes. The device performance index is a comprehensive index that measures the overall performance of each device. To calculate the device performance index, a formula may be defined: wherein, the method comprises the steps of, wherein, Represent the firstPerformance index of individual devices;、 And gamma is the weight coefficient of the computing capacity, the storage capacity and the network bandwidth respectively, and can be adjusted according to the actual application requirements. The formula integrates the computing capacity, the storage capacity and the network bandwidth of the equipment, and can reflect the overall performance of the equipment. And carrying out hierarchical clustering on the plurality of edge devices based on the calculated device performance indexes to form a multi-stage computing resource pool. Hierarchical clustering is the aggregation of devices with similar performance indices together to form a pool of computing resources at different levels. Clustering may be performed by computing the similarity between device performance indices using hierarchical clustering algorithms, such as condensed hierarchical clustering. The multi-level computing resource pool may be represented as Wherein, the method comprises the steps of, wherein,Represent the firstA level computing resource pool comprisingA device resource descriptor having a similar performance index. And carrying out virtualization packaging on a plurality of edge devices in the multi-stage computing resource pool to obtain a schedulable computing container. Virtualization encapsulation is the abstraction of resources of a physical device into logical resources so that these resources can be flexibly scheduled and managed. The computation containers are the result of a virtualized package, each of which may be represented asWherein, the method comprises the steps of, wherein,Represent the firstThe number of computing containers is a number of,Representing the virtual computing power of the device,Representing the capacity of the virtual storage,Representing virtual network bandwidth. Through virtualization encapsulation, a computing container can span different physical devices to realize dynamic allocation and scheduling of resources. And carrying out network topology dynamic reconstruction on the plurality of edge devices based on the calculation container to obtain an elastic network topology structure. The dynamic reconstruction of the network topology is to adjust the network connection between the devices so that the network topology structure can adapt to different application requirements and network states. The elastic network topology can be expressed asWherein, the method comprises the steps of, wherein,A set of device nodes is represented and,Representing a collection of connections between device nodes. By dynamic adjustmentThe transmission efficiency and reliability of the network can be optimized. After the elastic network topology structure is obtained, the communication protocol optimization is carried out on the computing container based on the structure, and a lightweight data transmission mechanism is obtained. Communication protocol optimization is to reduce overhead and delay of data transmission by simplifying the communication protocol. The lightweight data transfer mechanism can be expressed asWherein, the method comprises the steps of, wherein,A simplified communication protocol is represented and is shown,Representing the optimized transmission efficiency. By optimizing, the data transmission efficiency between the edge devices can be obviously improved, and the network delay and bandwidth occupation are reduced. And according to the lightweight data transmission mechanism, carrying out resource scheduling strategy analysis and full-performance enhancement processing on the computing container to obtain the edge computing grid. The analysis of the resource scheduling strategy is to formulate an efficient resource scheduling scheme according to the resource condition and the application requirement of the equipment. The integrity enhancement theory is to ensure the security and integrity of tasks and data running in the power edge computing grid by enhancing the security protection measures of the computing container. The edge computation grid may be represented asWherein, the method comprises the steps of, wherein,Representing a collection of computing containers,Representing the topology of the flexible network,Representing a lightweight data transfer mechanism.

In a specific embodiment, the process of executing step S102 may specifically include the following steps:

(1) Performing resource detection on a plurality of equipment nodes in the edge computing grid to obtain an original resource data set of each equipment node, and generating a computing capability index of each equipment node according to the original resource data set;

(2) Classifying the storage capacities of the plurality of equipment nodes according to the computing capacity indexes to obtain a storage resource matrix, and analyzing the data access mode of the storage resource matrix to obtain an I/O performance feature vector;

(3) Performing bandwidth test on network connection among a plurality of equipment nodes according to the I/O performance characteristic vector to obtain a network topology diagram, and performing statistical modeling on communication delay among the plurality of equipment nodes based on the network topology diagram to obtain a delay distribution function;

(4) Dynamically evaluating the energy consumption efficiency of each equipment node according to the delay distribution function to obtain an energy consumption efficiency curve, and performing polynomial fitting on the energy consumption efficiency curve to obtain a relation model of node load and energy consumption;

(5) And carrying out probability analysis on the reliability of each equipment node according to the relation model to obtain a node fault prediction model, and carrying out multi-dimensional feature extraction and fusion based on the node fault prediction model to obtain a multi-dimensional resource portrait model.

Specifically, resource detection is performed on each equipment node, and an original resource data set is obtained. The resource detection process comprises the key indexes of the computing capacity, the storage capacity, the network bandwidth and the like of the acquisition equipment. Some performance testing tools are used to measure per-node CPU performance (e.g., MIPS, mega instructions per second), memory capacity (e.g., GB, gigabytes), storage 1/O performance (e.g., IOPS, number of input/output operations per second), and network bandwidth (e.g., mbps, megabits per second). These raw data can be represented asWherein, the method comprises the steps of, wherein,Represent the firstThe original resource data set of the individual device nodes,Is the performance of the CPU and,Is the capacity of the memory and,Is the memory I/O performance that is to be stored,Is the network bandwidth. After the original resource data set for each device node is obtained, a computational capability index is generated. The computing power index is used to quantify the computing power of each node, and may be calculated by comprehensively considering various resource indexes. For example, the calculation formula isWherein, the method comprises the steps of, wherein,Represent the firstComputing capability index of individual device nodes;、、 And The weight coefficients are CPU performance, memory capacity, storage 1/O performance and network bandwidth, respectively. The weight coefficients can be adjusted according to actual demands so as to reflect the importance of various resources in different application scenes. And classifying the storage capacities of the plurality of equipment nodes in a grading manner based on the computing capacity indexes to obtain a storage resource matrix. The storage resource matrix is used for describing the storage resource distribution condition of each equipment node, and is expressed asWherein, the method comprises the steps of, wherein,Represent the firstThe individual node is at the firstCapacity in the individual storage tiers. And carrying out data access mode analysis on the storage resource matrix. The 1/O performance characteristic vector of each node is obtained by monitoring and recording the frequency and mode of the storage 1/O operation. The 1/O performance feature vector is expressed asWherein, the method comprises the steps of, wherein,AndRespectively represent the firstPerformance of read and write operations for individual nodes. And carrying out bandwidth test on network connection among a plurality of equipment nodes according to the 1/O performance characteristic vector to obtain a network topological graph. The network topology diagram shows the connection relation and bandwidth condition among the nodes, expressed asWherein, the method comprises the steps of, wherein,Is a collection of nodes that are configured to be connected,Is a collection of connections that are made,Is a set of bandwidths. On the basis, the communication delay among a plurality of equipment nodes is statistically modeled based on a network topological graph, and a delay distribution function is obtained. The delay profile function is expressed asWherein, the method comprises the steps of, wherein,Is the delay of the communication and,Is a time variable. By analyzing the delay distribution function, the characteristics and distribution of network delays are better understood. And dynamically evaluating the energy consumption efficiency of each equipment node according to the delay distribution function to obtain an energy consumption efficiency curve. The energy consumption efficiency curve describes the energy consumption of the node under different loads, expressed asWherein, the method comprises the steps of, wherein,Is the firstThe energy consumption of the individual nodes is such that,Is a load. In order to simplify and utilize the data, polynomial fitting is performed on the energy consumption efficiency curve to obtain a relation model of node load and energy consumption. The relationship model is expressed asWherein, the method comprises the steps of, wherein,Is a coefficient of fit and is a function of the fitting coefficient,Is the order of the polynomial. And carrying out probability analysis on the reliability of each equipment node according to the relation model of the node load and the energy consumption to obtain a node fault prediction model. A failure prediction model is used for predicting the failure probability of a node in future operation and is expressed asWherein, the method comprises the steps of, wherein,Is the firstThe probability of failure of the individual node(s),Is the energy consumption of the device and the method,Is the run time. By means of this model, potential faults can be identified and prevented in advance. And extracting and fusing the multidimensional features based on the node fault prediction model to obtain a multidimensional resource portrait model. The multidimensional resource portrait model integrates information of the computing capacity, the storage performance, the network delay, the energy consumption efficiency, the reliability and the like of the equipment, and provides references for resource scheduling and task allocation. The model is expressed asWherein, the method comprises the steps of, wherein,Is the firstThe resource portrayal model of each node comprises a computing capability index, a storage resource matrix, a network topology, an energy consumption efficiency curve and a fault probability.

In a specific embodiment, the process of executing step S103 may specifically include the following steps:

(1) Performing functional module division on an image processing algorithm to obtain an initial operator set, and analyzing the data dependency relationship among the image processing operators according to the initial operator set to obtain an original dependency graph;

(2) Evaluating the calculation complexity of the image processing operators based on the original dependency graph to obtain an operator weight matrix, and performing granularity optimization and combination on the image processing operators of each edge device according to the operator weight matrix to obtain an optimized algorithm subset;

(3) Carrying out parallelism analysis on the optimization operator set to obtain potential parallel execution paths, and carrying out task grouping on the image processing operators of each edge device based on the potential parallel execution paths to obtain parallel task blocks;

(4) Performing matching analysis on the multi-dimensional resource portrait model according to the parallel task blocks to obtain a resource affinity matrix, and performing equipment allocation on the parallel task blocks based on the resource affinity matrix to obtain an initial task mapping scheme;

(5) And carrying out load balancing optimization on the initial task mapping scheme to obtain a balanced task distribution strategy, and generating a task dependency graph according to the balanced task distribution strategy.

Specifically, the image processing algorithm is divided into functional modules, and the functional modules are decomposed into a plurality of independent functional modules or operators to obtain an initial operator set. For example, in an image processing algorithm, the image processing algorithm may be decomposed into a plurality of functional modules such as preprocessing, edge detection, feature extraction, classification, and the like, and each module corresponds to an operator. The resulting initial operator set can be expressed asWherein, the method comprises the steps of, wherein,Represent the firstAnd (5) an operator. And analyzing the data dependency relationship among the image processing operators according to the initial operator set, and constructing an original dependency graph. The original dependency graph shows the data dependency relationship among the operators, so that the execution sequence and data transmission of the operators are ensured to be correct in the actual execution process. The original dependency graph may be represented as a directed graphWherein, the method comprises the steps of, wherein,Is a collection of nodes, representing image processing operators,Is a collection of edges representing data dependencies between operators. And evaluating the computational complexity of the image processing operator based on the original dependency graph to obtain an operator weight matrix. The computational complexity can be obtained by actual test or theoretical analysis, and the operator weight matrix is expressed asWherein, the method comprises the steps of, wherein,Representing slave operatorsTo operatorData transmission and computational complexity of (a). And carrying out granularity optimization and merging on the image processing operators of each edge device according to the operator weight matrix to form an optimized operator subset. By combining operators with lower computational complexity, the number of operators is reduced, and the overall function of the algorithm is maintained. The subset of optimization algorithms may be expressed asWherein, the method comprises the steps of, wherein,Is an optimized operator. And carrying out parallelism analysis on the optimization operator set, and determining potential parallel execution paths. Parallelism analysis determines the set of operators that can be executed in parallel by identifying independent or weakly dependent operators. The potentially parallel execution paths may be represented asWherein, the method comprises the steps of, wherein,Represent the firstParallel execution paths. And based on the potential parallel execution paths, task grouping is carried out on the image processing operators of each edge device, and parallel task blocks are obtained. Parallel task blocks are represented asWherein, the method comprises the steps of, wherein,Represent the firstParallel task blocks. Each task block contains several operators, which can be executed in parallel on the same device. And carrying out matching analysis on the multi-dimensional resource portrait model according to the parallel task blocks to obtain a resource affinity matrix. The multidimensional resource portrait model integrates information of the computing capacity, the storage performance, the network delay, the energy consumption efficiency, the reliability and the like of the equipment, and the resource affinity matrix represents the matching degree between the equipment and the task block and is expressed asWherein, the method comprises the steps of, wherein,Representation deviceAnd task blockAffinity of (c) to the substrate. And performing equipment allocation on the parallel task blocks based on the resource affinity matrix to obtain an initial task mapping scheme. The initial task mapping scheme is expressed asWherein, the method comprises the steps of, wherein,Representing task blocksDistribution to devices. This process ensures that each task block can be allocated to the most appropriate device to optimize overall computing efficiency and resource utilization. And carrying out load balancing optimization on the initial task mapping scheme to obtain a balanced task distribution strategy. The goal of load balancing optimization is to ensure that the load of each device is as balanced as possible to avoid overloading certain devices to affect overall system performance. The equalization task distribution strategy may be implemented by an optimization algorithm, such as a genetic algorithm or a simulated annealing algorithm. And generating a task dependency graph according to the balanced task distribution strategy. The task dependency graph shows a final task allocation scheme and an execution sequence thereof, ensures that each task can be executed according to an optimal sequence, maximizes equipment resources and reduces waiting time.

In a specific embodiment, the process of executing step S104 may specifically include the following steps:

(1) Carrying out critical path analysis on the task dependency graph to obtain an initial critical task sequence, and constructing a task parallelism matrix according to the initial critical task sequence;

(2) Task hierarchical clustering is carried out based on the task parallelism matrix to obtain a task hierarchical structure diagram, and key resource identification is carried out on the task hierarchical structure diagram to obtain a resource competition hot spot list;

(3) According to the resource competition hot spot list, carrying out priority ordering and pipeline stage division on the task execution sequences of the plurality of edge devices to obtain an initial pipeline structure;

(4) Carrying out data transmission modeling among tasks based on an initial pipeline structure to obtain a data flow chart, and carrying out load balancing analysis on the initial pipeline structure according to the data flow chart to obtain a phase balancing factor;

(5) And performing iterative optimization on the stage balance factors to obtain an equalization pipeline configuration scheme, and performing time sequence processing based on the equalization pipeline configuration scheme to obtain a distributed image processing pipeline.

Specifically, a critical path analysis is performed on the task dependency graph. The task dependency graph may be represented as a directed acyclic graph, wherein nodes represent tasks and edges represent dependencies between tasks. The objective of critical path analysis is to find the longest path from the start point to the end point in the task dependency graph, which determines the shortest time for task completion. Critical path analysis may be achieved by calculating the earliest and latest start times for each task. For each task nodeIts earliest start timeThe calculation can be performed by the following recursive formula:

；

Wherein, Is a taskIs set of all of the pre-tasks of (c),Is a taskIs not shown, is not shown. Likewise, the latest start timeThe calculation can be performed by the following recursive formula:

；

Wherein, Is a taskIs a set of all subsequent tasks. By calculating each taskAndCritical paths, i.e. allIs a path formed by tasks. After the critical path is obtained, an initial critical task sequence is constructed. The critical task sequence is the basis of the task execution order, which determines the priority and parallelism of the tasks. And constructing a task parallelism matrix according to the initial key task sequence. The task parallelism matrix represents the parallel execution capacity of each task under different conditions. Assuming that there isTask-by-task parallelism matrixCan be expressed as:

；

And performing task hierarchical clustering based on the task parallelism matrix to obtain a task hierarchical structure diagram. Task hierarchical clustering forms task groups of different levels by aggregating tasks with similar parallelism features. The task hierarchy chart shows the hierarchical relationship and parallelism between tasks. And on the basis of the task hierarchy chart, carrying out key resource identification to obtain a resource competition hot spot list. The resource contention hotspot list marks the contention situation for a particular resource among a plurality of tasks, indicating possible bottlenecks and conflict points. And according to the resource competition hot spot list, sequencing the task execution sequences of the plurality of edge devices in priority, dividing the tasks into different pipeline stages, and constructing an initial pipeline structure. The initial pipeline structure ensures the priority execution of the key tasks through priority ordering and phase division, and simultaneously reduces resource competition and conflict to the greatest extent. Based on the initial pipeline structure, data transmission modeling among tasks is carried out, and a data flow chart is obtained. The data flow diagram shows the data transmission paths and data flow conditions between tasks, helping to optimize the data transmission efficiency. And carrying out load balancing analysis on the initial pipeline structure according to the data flow diagram, and calculating a phase balancing factor. The stage balancing factor is used to evaluate the load balancing condition of each pipeline stage and help identify the problem of load imbalance. And carrying out iterative optimization on the stage balance factors, and gradually improving load distribution through multiple adjustment and optimization to obtain an balanced pipeline configuration scheme. The balanced pipeline configuration scheme ensures the load balance of each stage and maximizes the system performance by optimizing the resource allocation and the task execution sequence. And performing time sequence processing based on the balanced pipeline configuration scheme to obtain a distributed image processing pipeline. The time sequential processing ensures that each task starts and executes at the appropriate point in time, maximizing resource utilization efficiency and reducing latency. The distributed image processing pipeline improves the efficiency and performance of the overall system by executing tasks in parallel.

In a specific embodiment, the process of executing step S105 may specifically include the following steps:

(1) Performing task decomposition on the distributed image processing pipeline to obtain an atomic task set, and performing slicing processing on the image data stream of each edge device according to the atomic task set to obtain a data block allocation scheme of each edge device;

(2) Generating a data transmission cost model of each edge device according to a data block allocation scheme, and carrying out initial allocation on an atomic task set according to the data transmission cost model to obtain a task initial distribution diagram;

(3) Calculating the load balance degree of each edge device based on the task initial distribution diagram to obtain a load imbalance degree index, and carrying out threshold judgment and task migration on the load imbalance degree index to obtain a target task distribution scheme;

(4) And adjusting the task execution sequence according to the target task distribution scheme to obtain a task scheduling sequence, performing resource competition analysis on the task scheduling sequence and generating a target task scheduling strategy.

Specifically, task decomposition is performed on the distributed image processing pipeline to obtain an atomic task set. Atomic tasks are the smallest units of image processing tasks that can be performed independently and that have a well-defined dependency relationship with each other. The image processing pipeline is assumed to comprise the steps of preprocessing, feature extraction, classification and the like, and can be decomposed into a plurality of atomic task sets. And performing slicing processing on the image data stream of each edge device to generate a data block allocation scheme of each edge device. The slicing process is to divide the large-scale image data into a plurality of small blocks, and each small block can be independently processed, so that the processing efficiency is improved. According to the data block allocation scheme, a data transmission cost model of each edge device is generated. The data transfer cost model evaluates the time and bandwidth required for data transfer from one device to another. Assume thatRepresenting slave devicesTo the deviceTransmitting data blocksCan be expressed by the formula:

；

Wherein, Is the size of the data block and,Is the bandwidth between devices. Based on the model, the total data transmission cost of each edge device is calculated, and the initial distribution of the atomic task set is carried out according to the total data transmission cost, so that a task initial distribution diagram is obtained. The initial distribution diagram of the task shows the distribution condition of each atomic task on different devices, so that the execution path of the task and the required resources can be identified. And calculating the load balance degree of each edge device based on the task initial distribution diagram to obtain a load unbalance degree index. The load balance is used for measuring whether the load distribution of each device is balanced, and the load unbalance index is expressed asWhereinIs an apparatusIs not balanced in terms of load. Assume a deviceIs loaded asThe load imbalance may be calculated by the following formula:

；

Wherein, Is the average load of all devices. By calculating the load imbalance for each device, devices with uneven load distribution can be identified and optimized. And judging a threshold value and performing task migration according to the load unbalance index to obtain a target task distribution scheme. The threshold value judgment is used for determining whether the load unbalance degree exceeds an allowable range, and if so, task migration is needed. Task migration achieves load balancing by transferring a portion of the tasks from an overloaded device to a less loaded device. The target task distribution scheme shows the optimized task distribution condition, and ensures that the load of each device is balanced as much as possible. And adjusting the task execution sequence according to the target task distribution scheme to obtain a task scheduling sequence. The task scheduling sequence ensures that tasks are executed in an optimal order, maximizes utilization of device resources and reduces latency. And carrying out resource competition analysis on the task scheduling sequence and generating a target task scheduling strategy. Resource contention analysis is used to identify resource conflicts that may occur during task execution and resolve these conflicts by adjusting the task execution order or allocating more resources.

In a specific embodiment, the process of executing step S106 may specifically include the following steps:

(1) According to the target task scheduling strategy, carrying out quality evaluation on the image processing result of each edge device to obtain an image quality index matrix;

(2) Carrying out image region importance analysis according to the image quality index matrix to obtain a region weight map, and carrying out image layered coding and entropy coding based on the region weight map to obtain an initial compressed data stream;

(3) Carrying out dynamic estimation of network bandwidth according to the initial compressed data stream to obtain an available bandwidth prediction model, and carrying out bit allocation optimization on the initial compressed data stream based on the available bandwidth prediction model to obtain a layered transmission strategy;

(4) Performing transmission error analysis on the layered transmission strategy to obtain a data importance ranking table, and performing priority scheduling on the initial compressed data stream according to the data importance ranking table to obtain a target transmission queue;

(5) And carrying out cooperative processing on the data in the target transmission queue and the cloud to obtain an edge cloud fusion analysis result, and carrying out iterative optimization on the image processing parameters based on the edge cloud fusion analysis result to obtain an image processing parameter set.

Specifically, the quality evaluation is performed on the image processing result of each edge device, and an image quality index matrix is obtained. The image quality index matrix is used to quantify the quality of the processed image of each device, for example, by calculating the peak signal-to-noise ratio and structural similarity index. Assume that there are three edge devicesThe image quality index matrix thereof can be expressed as:

；

Wherein, Represent the firstPersonal device Process NoQuality index of the sheet image. And carrying out image region importance analysis according to the image quality index matrix to obtain a region weight map. The region weight map reflects the importance of each region in the image and can be determined by analyzing the edge, texture, brightness, etc. characteristics of the image. Assuming that a certain image is divided into several regions, the weight of each region can be expressed as a matrix:

；

Wherein, Represent the firstLine 1The weight of the column region. And carrying out layered coding and entropy coding on the image based on the region weight map to obtain an initial compressed data stream. Layered coding is to layer the image according to importance, ensuring that important areas are preferentially processed and compressed with high quality, while secondary areas are compressed with lower quality. Entropy coding further reduces redundancy of data by exploiting the statistical properties of the data. For example, assuming that an image is divided into three layers, the encoded data stream for each layer can be expressed as:

；

Wherein, Represent the firstEncoded data of the layer. And carrying out dynamic estimation on the network bandwidth according to the initial compressed data stream to obtain an available bandwidth prediction model. The available bandwidth prediction model predicts future available bandwidth conditions by analyzing the current network state. Assuming that network bandwidth varies with time, it can be expressed as a function:

；

Wherein, Is the time ofThe bandwidth of the time-point and,Is the amplitude of the bandwidth and,Is the frequency of the bandwidth variation and,Is the phase shift of the phase of the light,Is the bandwidth base value. And carrying out bit allocation optimization on the initial compressed data stream based on the available bandwidth prediction model to obtain a layered transmission strategy. The bit allocation optimization is to reasonably allocate the bit number of each layer of data stream according to the bandwidth prediction result so as to ensure that the optimal transmission effect is realized under the limited bandwidth. For example, assume that the total bandwidth isThe bit allocation for each layer of data stream can be expressed as:

；

Wherein, Is the firstNumber of data bits of a layer. And carrying out transmission error analysis on the layered transmission strategy to obtain a data importance ranking table. Transmission error analysis is used to evaluate data loss or errors that may occur during transmission and to rank the data according to its importance. Assume that there are three data blocksThe importance ranking table thereof can be expressed as:

；

Wherein, Represent the firstOrdering of the individual data blocks. And carrying out priority scheduling on the initial compressed data stream according to the data importance ranking table to obtain a target transmission queue. Priority scheduling ensures that the most important data can be transmitted preferentially under the condition of limited bandwidth, and the effectiveness and reliability of image transmission are improved. And carrying out cooperative processing on the data in the target transmission queue and the cloud to obtain an edge cloud fusion analysis result. The edge cloud fusion analysis optimizes the image processing and data transmission process by combining computing resources of the edge device and the cloud. And carrying out iterative optimization on the image processing parameters based on the edge cloud fusion analysis result to obtain an image processing parameter set. Iterative optimization is to continuously adjust processing parameters to improve the overall effect of image processing and transmission.

The method for processing a real-time image based on edge computation in the embodiment of the present application is described above, and the following describes a real-time image processing system based on edge computation in the embodiment of the present application, please refer to fig. 2, and one embodiment of the real-time image processing system based on edge computation in the embodiment of the present application includes:

a virtualization module 201, configured to virtualize heterogeneous resources for a plurality of edge devices to obtain an edge computing grid;

The real-time monitoring module 202 is used for monitoring node performance and network state of a plurality of edge devices in real time based on the edge computing grid, and constructing a multi-dimensional resource portrait model;

The task decomposition module 203 is configured to perform task decomposition and parallelization reconstruction on the image processing operators of each edge device through the multidimensional resource portrait model, so as to obtain a task dependency graph;

the topology optimization module 204 is configured to perform topology optimization on task execution sequences of a plurality of edge devices based on the task dependency graph, so as to obtain a distributed image processing pipeline;

the dynamic balancing module 205 is configured to dynamically balance an image data stream and a computing load of each edge device based on the distributed image processing pipeline, so as to obtain a target task scheduling policy;

And the feedback module 206 is configured to perform adaptive compression transmission and edge cloud cooperative feedback on image processing results of the plurality of edge devices according to the target task scheduling policy, so as to obtain an image processing parameter set.

Through the cooperation of the components, the unified management and scheduling of heterogeneous edge devices are realized by carrying out abstract processing and virtualized packaging on hardware resources of a plurality of edge devices, and the resource utilization efficiency is improved. Based on a multi-dimensional resource portrait model, real-time monitoring of node performance and network state of edge equipment is realized, the parallel processing capability of the edge equipment is fully utilized through task decomposition and parallelization reconstruction of an image processing operator, the task execution sequence of a plurality of edge equipment is reasonably arranged based on the topology optimization of a task dependency graph, the load balancing among the edge equipment is realized through dynamic balancing of image data flow and calculation load, the self-adaptive compression transmission of image processing results is realized according to network bandwidth and image importance, the powerful calculation capability of a cloud is fully utilized through cooperative processing of the edge equipment and the cloud, the iterative optimization of image processing parameters is realized based on the edge cloud fusion analysis result, the self-adaptive adjustment of the processing parameters is realized, and the flexible scheduling and the efficient utilization of edge computing resources are realized through a multi-stage computing resource pool and an elastic network topology structure. Through the node fault prediction model and the load balancing optimization, the reliability and the fault tolerance of the system are improved, the continuity of real-time image processing is guaranteed, and the accuracy of the real-time image processing is further improved.

The present application also provides an edge-calculation-based real-time image processing apparatus, which includes a memory and a processor, wherein the memory stores computer-readable instructions that, when executed by the processor, cause the processor to execute the steps of the edge-calculation-based real-time image processing method in the above embodiments.

The present application also provides a computer readable storage medium, which may be a non-volatile computer readable storage medium, and may also be a volatile computer readable storage medium, where instructions are stored in the computer readable storage medium, when the instructions are executed on a computer, cause the computer to perform the steps of the edge calculation-based real-time image processing method.

It will be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, systems and units may refer to the corresponding processes in the foregoing method embodiments, which are not repeated herein.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied essentially or in part or all of the technical solution or in part in the form of a software product stored in a storage medium, including instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a random access memory (random acceS memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

The above embodiments are only for illustrating the technical solution of the present application, and not for limiting the same; although the application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for real-time image processing based on edge computation, the method comprising:

2. The method for processing the real-time image based on the edge computing according to claim 1, wherein the heterogeneous resource virtualization is performed on the plurality of edge devices to obtain an edge computing grid, and the method comprises the following steps:

abstracting hardware resources of a plurality of edge devices to obtain a resource descriptor of each edge device, and carrying out calculation capacity quantitative evaluation on the plurality of edge devices according to the resource descriptor to obtain a device performance index;

Hierarchical clustering is carried out on the plurality of edge devices based on the device performance index to obtain a multi-stage computing resource pool, and virtualization packaging is carried out on the plurality of edge devices in the multi-stage computing resource pool to obtain a schedulable computing container;

Performing network topology dynamic reconstruction on the plurality of edge devices based on the computing container to obtain an elastic network topology structure, and performing communication protocol optimization on the computing container based on the elastic network topology structure to obtain a lightweight data transmission mechanism;

And carrying out resource scheduling strategy analysis and integrity enhancement processing on the computing container according to the lightweight data transmission mechanism to obtain an edge computing grid.

3. The method for real-time image processing based on edge computing according to claim 2, wherein the real-time monitoring of node performance and network status of the plurality of edge devices based on the edge computing grid, and constructing a multi-dimensional resource portrait model, comprises:

performing resource detection on a plurality of equipment nodes in the edge computing grid to obtain an original resource data set of each equipment node, and generating a computing capability index of each equipment node according to the original resource data set;

classifying the storage capacities of the plurality of equipment nodes according to the computing capacity indexes in a grading manner to obtain a storage resource matrix, and analyzing a data access mode of the storage resource matrix to obtain an I/O performance characteristic vector;

Performing bandwidth test on network connection among the plurality of equipment nodes according to the I/O performance characteristic vector to obtain a network topology diagram, and performing statistical modeling on communication delay among the plurality of equipment nodes based on the network topology diagram to obtain a delay distribution function;

Dynamically evaluating the energy consumption efficiency of each equipment node according to the delay distribution function to obtain an energy consumption efficiency curve, and performing polynomial fitting on the energy consumption efficiency curve to obtain a relation model of node load and energy consumption;

And carrying out probability analysis on the reliability of each equipment node according to the relation model to obtain a node fault prediction model, and carrying out multi-dimensional feature extraction and fusion based on the node fault prediction model to obtain a multi-dimensional resource portrait model.

4. The method for real-time image processing based on edge computing according to claim 1, wherein performing task decomposition and parallelization reconstruction on an image processing operator of each edge device through the multi-dimensional resource portrait model to obtain a task dependency graph comprises:

Performing functional module division on an image processing algorithm to obtain an initial operator set, and analyzing the data dependency relationship among the image processing operators according to the initial operator set to obtain an original dependency graph;

evaluating the calculation complexity of the image processing operators based on the original dependency graph to obtain an operator weight matrix, and performing granularity optimization and combination on the image processing operators of each edge device according to the operator weight matrix to obtain an optimized calculation subset;

Carrying out parallelism analysis on the optimization algorithm subset to obtain potential parallel execution paths, and carrying out task grouping on image processing operators of each edge device based on the potential parallel execution paths to obtain parallel task blocks;

Performing matching analysis on the multi-dimensional resource portrait model according to the parallel task blocks to obtain a resource affinity matrix, and performing equipment allocation on the parallel task blocks based on the resource affinity matrix to obtain an initial task mapping scheme;

And carrying out load balancing optimization on the initial task mapping scheme to obtain a balanced task distribution strategy, and generating a task dependency graph according to the balanced task distribution strategy.

5. The method for processing the real-time image based on the edge computing according to claim 1, wherein the performing topology optimization on the task execution sequences of the plurality of edge devices based on the task dependency graph to obtain a distributed image processing pipeline comprises:

Carrying out critical path analysis on the task dependency graph to obtain an initial critical task sequence, and constructing a task parallelism matrix according to the initial critical task sequence;

performing task hierarchical clustering based on the task parallelism matrix to obtain a task hierarchical structure diagram, and performing key resource identification on the task hierarchical structure diagram to obtain a resource competition hot spot list;

According to the resource competition hot spot list, carrying out priority ordering and pipeline stage division on the task execution sequences of the plurality of edge devices to obtain an initial pipeline structure;

carrying out inter-task data transmission modeling based on the initial pipeline structure to obtain a data flow chart, and carrying out load balancing analysis on the initial pipeline structure according to the data flow chart to obtain a stage balancing factor;

And performing iterative optimization on the stage balance factors to obtain an equalization pipeline configuration scheme, and performing time sequence processing based on the equalization pipeline configuration scheme to obtain a distributed image processing pipeline.

6. The method for real-time image processing based on edge computing according to claim 1, wherein dynamically balancing an image data stream and a computing load of each edge device based on the distributed image processing pipeline to obtain a target task scheduling policy, comprises:

Performing task decomposition on the distributed image processing pipeline to obtain an atomic task set, and performing slicing processing on the image data stream of each edge device according to the atomic task set to obtain a data block allocation scheme of each edge device;

Generating a data transmission cost model of each edge device according to the data block allocation scheme, and carrying out initial allocation on the atomic task set according to the data transmission cost model to obtain a task initial distribution diagram;

calculating the load balance degree of each edge device based on the task initial distribution diagram to obtain a load unbalance degree index, and carrying out threshold judgment and task migration on the load unbalance degree index to obtain a target task distribution scheme;

And adjusting the task execution sequence according to the target task distribution scheme to obtain a task scheduling sequence, and performing resource competition analysis on the task scheduling sequence to generate a target task scheduling strategy.

7. The method for processing the real-time image based on the edge calculation according to claim 1, wherein the performing adaptive compression transmission and edge cloud collaborative feedback on the image processing results of the plurality of edge devices according to the target task scheduling policy to obtain an image processing parameter set includes:

according to the target task scheduling strategy, performing quality evaluation on the image processing result of each edge device to obtain an image quality index matrix;

Carrying out image region importance analysis according to the image quality index matrix to obtain a region weight map, and carrying out image layered coding and entropy coding based on the region weight map to obtain an initial compressed data stream;

performing network bandwidth dynamic estimation according to the initial compressed data stream to obtain an available bandwidth prediction model, and performing bit allocation optimization on the initial compressed data stream based on the available bandwidth prediction model to obtain a layered transmission strategy;

performing transmission error analysis on the layered transmission strategy to obtain a data importance ranking table, and performing priority scheduling on the initial compressed data stream according to the data importance ranking table to obtain a target transmission queue;

And carrying out cooperative processing on the data in the target transmission queue and the cloud to obtain an edge cloud fusion analysis result, and carrying out iterative optimization on image processing parameters based on the edge cloud fusion analysis result to obtain an image processing parameter set.

8. A real-time image processing system based on edge computation for performing the real-time image processing method based on edge computation according to any one of claims 1 to 7, characterized in that the system comprises:

9. A real-time image processing apparatus based on edge calculation, characterized by comprising: a memory and at least one processor, the memory having instructions stored therein;

The at least one processor invokes the instructions in the memory to cause the edge computation based real-time image processing device to perform the edge computation based real-time image processing method of any of claims 1-7.

10. A computer readable storage medium having instructions stored thereon, which when executed by a processor, implement the edge computation based real-time image processing method of any of claims 1-7.