CN112099950A - Image preprocessing optimization method based on edge image processing system - Google Patents

Image preprocessing optimization method based on edge image processing system Download PDF

Info

Publication number
CN112099950A
CN112099950A CN202010956879.2A CN202010956879A CN112099950A CN 112099950 A CN112099950 A CN 112099950A CN 202010956879 A CN202010956879 A CN 202010956879A CN 112099950 A CN112099950 A CN 112099950A
Authority
CN
China
Prior art keywords
image
edge
edge computing
computing device
preprocessing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010956879.2A
Other languages
Chinese (zh)
Inventor
田文龙
余缘超
董毅
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chongqing Dianzheng Information Technology Co ltd
Original Assignee
Chongqing Dianzheng Information Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chongqing Dianzheng Information Technology Co ltd filed Critical Chongqing Dianzheng Information Technology Co ltd
Priority to CN202010956879.2A priority Critical patent/CN112099950A/en
Publication of CN112099950A publication Critical patent/CN112099950A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172—Classification, e.g. identification

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Software Systems (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Human Computer Interaction (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Engineering & Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses an image preprocessing optimization method based on an edge image processing system, and belongs to the field of image processing and edge calculation. The method comprises the following steps: s1: acquiring the average CPU utilization rate and the action average response time of all edge computing devices in the database for image preprocessing in each period; s2: calculating the state of the next period through a load prediction model; s3: calculating each action response time for image preprocessing in the next period by using a response prediction model; s4: scheduling the image preprocessing action of each edge computing device in the next period; s5: the edge computing device executes image preprocessing according to the instruction of the main control node and feeds back the image preprocessing; s6: and integrating the images preprocessed by all the edge computing devices. The invention realizes the edge calculation of image preprocessing and the optimal configuration of the edge calculation device, improves the utilization rate of calculation resources, saves the transmission time and cost of data and reduces the data delay.

Description

Image preprocessing optimization method based on edge image processing system
Technical Field
The invention relates to an image preprocessing optimization method based on an edge image processing system, and belongs to the field of image processing and edge calculation.
Background
In recent years, with the rapid development of 5G and industrial internet, the requirement of emerging services on edge computing is urgent, and the requirement on edge computing is mainly embodied in three aspects of time delay, bandwidth and safety in emerging services of a plurality of vertical industries. As is well known, edge computing pushes data processing from the cloud to the edge closer to data and applications, reducing system response delay, saving network bandwidth, and protecting data security, so that the requirements of application scenarios in vertical industries, including smart manufacturing, smart cities, live games, car networking, and the like, can be completely met. Meanwhile, with the increasing perfection of security and control projects in the national key construction of safe cities, skynet projects, snow projects and the like, the number of monitoring probes for security and protection monitoring exceeds 5 hundred million until 2021 year ago, a large number of security and protection monitoring probes generate a large amount of image processing requirements, and the requirements of image identification on the cost, speed and precision of image processing in security and protection monitoring scenes are higher and higher.
In the field of edge calculation, in the current terminal products based on the ARM architecture, the system and the method can only perform simple algorithm analysis on a single image, and complex analysis on continuous images cannot be performed on the end or operation results cannot be obtained in real time; and the cloud server has the problems of time delay, bandwidth and safety in calculation. In practical applications, there are usually a lot of spare computing resources that are not utilized, so it is necessary to perform part of the image processing work, i.e. image pre-processing, at the edge.
Disclosure of Invention
In order to fully exert the value of front-end acquired data, improve the utilization rate of computing resources, greatly relieve network transmission pressure and reduce data delay, the invention provides an image preprocessing optimization method based on an edge image processing system, which comprises the following steps:
s1: the method comprises the steps that a main control node obtains the average CPU utilization rate and the action average response time of all edge computing devices in a database for image preprocessing in each period;
s2: the master control node calculates the state of each edge computing device in the next period through a load prediction model;
s3: by utilizing the response prediction model, the main control node calculates each action response time of the edge computing device for image preprocessing in the next period;
s4: the main control node schedules the image preprocessing action of each edge computing device in the next period;
s5: the edge computing device executes image preprocessing according to the instruction of the main control node and feeds back the response time of each action and the CPU utilization rate;
s6: the main control node integrates the images preprocessed by all the edge computing devices.
Further, the preprocessing actions are as follows in sequence: (1) first, whether the image is out of focus is judged: respectively differentiating the horizontal direction and the longitudinal direction of the image, accumulating the differences, judging that the image is out of focus when the total value of the differences exceeds a threshold value, and deleting the image; (2) then, image quality evaluation was performed: evaluating the image quality of all images in a period, and reserving images with better quality; (3) and finally, carrying out image recognition: and calculating the direction gradient characteristics of each area surface through a sliding window, comparing the direction gradient characteristics with the characteristics of the object or the person, classifying, judging whether the person exists, and further extracting the characteristics of the eyes and the nose of the person if the person exists.
Further, the image quality evaluation method may be: according to methods such as Tenengrad gradient, Laplacian gradient, SMD (grayscale variance), SMD2 (grayscale variance product), etc.
Furthermore, the load prediction model is a first-order Markov prediction model which is established according to the average CPU utilization rate of each edge computing device in each image preprocessing period and corresponds to the average CPU utilization rate; the predicted states are divided according to CPU utilization into: overload O, normal N, low load U; the following state transition probability matrix is generated:
Figure BDA0002680202330000021
wherein, PUURepresenting the probability of the transition from the low load state in the last cycle to the low load state in the next cycle, and the other symbols are the same.
Further, the response prediction model may be a response time prediction model established for the motion average response time of each image preprocessing period of the entire edge image processing system, or a response time prediction model established for the motion average response time of each image preprocessing period of each edge computing device.
Further, the response time prediction model may be a kalman filter, a particle filter, an ARMAX model (autoregestive moving-average model with exogenous inputs), and the like.
Further, the step S4 specifically includes: (1) the main control node completely transfers the preprocessing actions of the overload edge computing device in the next period to the no-load edge computing device for processing according to the load prediction model result; (2) the main control node migrates the preprocessing action of responding overtime of the edge computing device in the next period to the edge computing device with surplus response time of the edge computing device according to the response prediction model; (3) and on the premise of ensuring that the migration in the edge device is not overloaded and not overtime, migrating all the preprocessing actions which are not performed by the edge computing device in time to the main control node for processing.
The edge image processing system is formed by connecting a main control node and at least one edge computing device through one or more of a network, Bluetooth or USB, and also comprises a conventional peripheral interface (USB interface), a network module, a power supply module, a Bluetooth module and the like; the main control node can be a cloud server or an edge computing device node; the edge calculating device is provided with an image processor and is connected with an image input device.
Further, the master node includes:
1. and the resource management module is used for collecting node state information uploaded by the edge computing device management module in distributed edge computing, wherein the node state information comprises a node memory state, a CPU state, a GPU state, running task information, response time and the like.
2. And the task management module is used for issuing an image preprocessing task to the edge computing device and finishing scheduling.
3. And the data preprocessing module is used for preprocessing and segmenting the execution task data.
4. And the result aggregation module is used for collecting the analysis results of the operation modules of the edge calculation devices and sorting the result sets according to the sequence numbers.
5. And the output module is used for outputting the analysis result set to the application layer.
Further, the edge calculation means includes:
1. the node management module is used for collecting state information of the edge computing device, wherein the state information comprises a memory state, a CPU state, a GPU state and information of running tasks; and receiving the operation task, starting the operation module and transmitting the specific operation parameters to the operation module.
2. And the operation module is used for specifically executing an image preprocessing algorithm.
The invention has the beneficial effects that: the invention constructs an edge image processing system, extracts the whole simple task of image preprocessing from the whole task of image processing to the edge computing device for processing, and realizes the optimal configuration of the edge computing of the image preprocessing and the edge computing device by establishing a load prediction model and a load prediction model, thereby improving the utilization rate of computing resources, saving the transmission time and cost of data and reducing the data delay.
Drawings
FIG. 1 is a flow chart of an image pre-processing optimization method based on an edge image processing system according to the present invention;
fig. 2 is a structural diagram of an edge image processing system according to an embodiment of the present invention, wherein: the system comprises a cloud server 1, a cloud server database 2, edge computing devices 3 and 7, local databases 4 and 8, monitoring equipment 5 and 6 and a control computer 9.
Detailed Description
In order to make the purpose and technical solution of the present invention more clearly understood, the present invention will be described in detail with reference to the accompanying drawings and examples.
Examples
The method is characterized in that real-time face recognition monitoring is carried out aiming at monitoring with an edge computing device in a cell, and in order to further analyze whether the monitoring is the owner of the cell, because the data volume of video images of the cell every day is large, and the configuration of a cloud server is low, face recognition preprocessing needs to be carried out at the edge end, so that the implementation provides 'a face recognition preprocessing optimization method based on an edge image processing system'.
A face recognition preprocessing optimization method based on an edge image processing system is realized by the edge image processing system, and with reference to fig. 2, the edge image processing system is formed by connecting a main control node of a cloud server 1 and two edge computing devices 3 and 7 which are connected with monitoring equipment 5 and 6 and correspond to each other one by one through a network, and further comprises a conventional peripheral interface, a network module, a power supply module and the like. The cloud server 1 is provided with a cloud server database 2 and used for storing images subjected to face recognition preprocessing; the edge computing devices 3 and 7 are respectively provided with local databases 4 and 8 for storing images shot by the monitoring equipment; the control end computer 9 can facilitate the user to input operation instructions, obtain related information and realize human-computer interaction.
Further, the cloud server 1 includes:
1. and the resource management module is used for collecting node state information uploaded by the edge computing device management module in distributed edge computing, wherein the node state information comprises a node memory state, a CPU state, a GPU state, running task information, response time and the like.
2. And the task management module is used for issuing an image preprocessing task to the edge computing device and finishing scheduling.
3. And the data preprocessing module is used for preprocessing and segmenting the execution task data.
4. And the result aggregation module is used for collecting the analysis results of the operation modules of the edge calculation devices and sorting the result sets according to the sequence numbers.
5. And the output module is used for outputting the analysis result set to the application layer.
Further, the edge calculation means 3 and 7 include:
1. the node management module is used for collecting state information of the edge computing device, wherein the state information comprises a memory state, a CPU state, a GPU state and information of running tasks; and receiving the operation task, starting the operation module and transmitting the specific operation parameters to the operation module.
2. And the operation module is used for specifically executing an image preprocessing algorithm.
With reference to fig. 1, a face recognition preprocessing optimization method based on an edge image processing system includes the following steps:
the method comprises the following steps: the cloud server 1 acquires the average CPU utilization rate and the action average response time of image preprocessing in each period in the local databases 4 and 8 of the edge computing devices 3 and 7;
step two: establishing a load prediction model and determining parameters through training, and calculating the state of the edge computing devices 3 and 7 in the next period through the load prediction model by the cloud server 1;
the load prediction model is a first-order Markov prediction model which is established according to the average CPU utilization rate of each edge computing device in each image preprocessing period; the predicted states are divided according to CPU utilization into: overload O, normal N, low load U; the following state transition probability matrix is generated:
Figure BDA0002680202330000041
wherein, PUURepresenting the probability of the transition from the low load state in the last cycle to the low load state in the next cycle, and the other symbols are the same.
Step three: establishing a response prediction model, determining parameters through training, and calculating each action response time of the edge computing devices 3 and 7 for image preprocessing in the next period by using the response prediction model through the cloud server 1;
the response prediction model may be a response time prediction model established for the motion average response time of each image preprocessing period of the entire edge image processing system, or a response time prediction model established for the motion average response time of each image preprocessing period of each edge computing device.
In order to better realize the prediction, the response time prediction model is suggested to be an ARMAX (automatic moving-average model with exogenous inputs) and its modified version.
Step four: the cloud server 1 schedules the image preprocessing action of the edge computing devices 3 and 7 in the next period;
the method specifically comprises the following steps: (1) the cloud server 1 completely transfers the preprocessing actions of the overload edge computing device in the next period to the no-load edge computing device for processing according to the load prediction model result; (2) the cloud server 1 migrates the preprocessing action of the overtime response of the edge computing device in the next period to the edge computing device with surplus response time of the edge computing device according to the response prediction model; (3) and on the premise of ensuring that the migration in the edge devices is not overloaded and not overtime, migrating all the preprocessing actions which are not performed by the edge computing devices to the cloud server 1 for processing.
Step five: the edge computing devices 3 and 7 execute image preprocessing according to the instruction of the cloud server 1 and feed back each action response time and CPU utilization rate;
step six: the cloud server 1 integrates the images preprocessed by the edge computing devices 3 and 7 and stores the images into the cloud server database 2 for the next face recognition.
The present invention is not limited to the embodiments described above, and it will be apparent to a person skilled in the art that any modifications or variations to the embodiments of the present invention described above are possible without departing from the scope of protection of the embodiments of the present invention and the appended claims, which are given by way of illustration only and are not intended to limit the invention in any way.

Claims (6)

1. The image preprocessing optimization method based on the edge image processing system is characterized by comprising the following steps of:
s1: the method comprises the steps that a main control node obtains the average CPU utilization rate and the action average response time of all edge computing devices in a database for image preprocessing in each period;
s2: the master control node calculates the state of each edge computing device in the next period through a load prediction model;
s3: by utilizing the response prediction model, the main control node calculates each action response time of the edge computing device for image preprocessing in the next period;
s4: the main control node schedules the image preprocessing action of each edge computing device in the next period;
s5: the edge computing device executes image preprocessing according to the instruction of the main control node and feeds back the response time of each action and the CPU utilization rate;
s6: the main control node integrates the images preprocessed by all the edge computing devices.
2. The method for optimizing image preprocessing based on the edge image processing system as claimed in claim 1, wherein the preprocessing operations are sequentially: (1) first, whether the image is out of focus is judged: respectively differentiating the horizontal direction and the longitudinal direction of the image, accumulating the differences, judging that the image is out of focus when the total value of the differences exceeds a threshold value, and deleting the image; (2) then, image quality evaluation was performed: evaluating the image quality of all images in a period, and reserving images with better quality; (3) and finally, carrying out image recognition: and calculating the direction gradient characteristics of each area surface through a sliding window, comparing the direction gradient characteristics with the characteristics of the object or the person, classifying, judging whether the person exists, and further extracting the characteristics of the eyes and the nose of the person if the person exists.
3. The method of claim 1, wherein the load prediction model is a first-order Markov prediction model that is established according to an average CPU utilization rate of each edge computing device in each image preprocessing cycle; the predicted states are divided according to CPU utilization into: overload O, normal N, low load U; the following state transition probability matrix is generated:
Figure FDA0002680202320000011
wherein, PUURepresenting the probability of the transition from the low load state in the last cycle to the low load state in the next cycle, and the other symbols are the same.
4. The method of claim 1, wherein the response prediction models are respectively built for the average response time of each image preprocessing period of the entire edge image processing system, or for the average response time of each image preprocessing period of each edge computing device.
5. The image preprocessing optimization method based on the edge image processing system according to claim 1, wherein the step S4 specifically comprises: (1) the main control node completely transfers the preprocessing actions of the overload edge computing device in the next period to the no-load edge computing device for processing according to the load prediction model result; (2) the main control node migrates the preprocessing action of responding overtime of the edge computing device in the next period to the edge computing device with surplus response time of the edge computing device according to the response prediction model; (3) and on the premise of ensuring that the migration in the edge device is not overloaded and not overtime, migrating all the preprocessing actions which are not performed by the edge computing device in time to the main control node for processing.
6. The edge image processing system applied to the claims 1-5 is characterized in that the edge image processing system is formed by connecting a master control node and at least one edge computing device through a network, Bluetooth or USB; the main control node can be a cloud server or an edge computing device node; the edge calculating device is provided with an image processor and is connected with an image input device.
CN202010956879.2A 2020-09-15 2020-09-15 Image preprocessing optimization method based on edge image processing system Pending CN112099950A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010956879.2A CN112099950A (en) 2020-09-15 2020-09-15 Image preprocessing optimization method based on edge image processing system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010956879.2A CN112099950A (en) 2020-09-15 2020-09-15 Image preprocessing optimization method based on edge image processing system

Publications (1)

Publication Number Publication Date
CN112099950A true CN112099950A (en) 2020-12-18

Family

ID=73752068

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010956879.2A Pending CN112099950A (en) 2020-09-15 2020-09-15 Image preprocessing optimization method based on edge image processing system

Country Status (1)

Country Link
CN (1) CN112099950A (en)

Similar Documents

Publication Publication Date Title
Wu et al. Helmet detection based on improved YOLO V3 deep model
CN108563739B (en) Weather data acquisition method and device, computer device and readable storage medium
CN106845415B (en) Pedestrian fine identification method and device based on deep learning
CN109743356B (en) Industrial internet data acquisition method and device, readable storage medium and terminal
CN111008640A (en) Image recognition model training and image recognition method, device, terminal and medium
CN104391879B (en) The method and device of hierarchical clustering
CN111930526B (en) Load prediction method, load prediction device, computer equipment and storage medium
CN106682702A (en) Deep learning method and system
CN114066073A (en) Power grid load prediction method
CN115495222A (en) Cloud computing system based on deep learning and big data
CN115794578A (en) Data management method, device, equipment and medium for power system
CN115294397A (en) Classification task post-processing method, device, equipment and storage medium
CN112069039A (en) Monitoring and predicting alarm method and device for artificial intelligence development platform and storage medium
CN114612011A (en) Risk prevention and control decision method and device
CN113822110B (en) Target detection method and device
CN110855474B (en) Network feature extraction method, device, equipment and storage medium of KQI data
CN114640669A (en) Edge calculation method and device
CN112099950A (en) Image preprocessing optimization method based on edge image processing system
CN112270207A (en) Smoke and fire detection method in community monitoring scene
CN115954019A (en) Environmental noise identification method and system integrating self-attention and convolution operation
Ashouri et al. Analyzing distributed deep neural network deployment on edge and cloud nodes in IoT systems
Du et al. OctopusKing: A TCT-aware task scheduling on spark platform
CN115482419B (en) Data acquisition and analysis method and system for marine fishery products
CN112464897B (en) Electric power operator screening method and device
Chen et al. Guarding Deep Learning Systems With Boosted Evasion Attack Detection and Model Update

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination