CN113891083A - Video processing method and device in edge intelligent environment and computing equipment - Google Patents

Video processing method and device in edge intelligent environment and computing equipment Download PDF

Info

Publication number
CN113891083A
CN113891083A CN202111273353.5A CN202111273353A CN113891083A CN 113891083 A CN113891083 A CN 113891083A CN 202111273353 A CN202111273353 A CN 202111273353A CN 113891083 A CN113891083 A CN 113891083A
Authority
CN
China
Prior art keywords
coding strategy
video coding
video
processing
attitude estimation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111273353.5A
Other languages
Chinese (zh)
Inventor
王�琦
李康敬
丁凌
王敬祎
刘迅承
张未展
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Mobile Communications Group Co Ltd
Xian Jiaotong University
MIGU Video Technology Co Ltd
MIGU Culture Technology Co Ltd
Original Assignee
China Mobile Communications Group Co Ltd
Xian Jiaotong University
MIGU Video Technology Co Ltd
MIGU Culture Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Mobile Communications Group Co Ltd, Xian Jiaotong University, MIGU Video Technology Co Ltd, MIGU Culture Technology Co Ltd filed Critical China Mobile Communications Group Co Ltd
Priority to CN202111273353.5A priority Critical patent/CN113891083A/en
Publication of CN113891083A publication Critical patent/CN113891083A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/184—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being bits, e.g. of the compressed video stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124—Quantisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/164—Feedback from the receiver or from the transmission channel

Abstract

The invention discloses a video processing method, a video processing device and computing equipment in an edge intelligent environment, wherein the method comprises the following steps: acquiring resource supply information of a current video coding strategy aiming at a terminal and quality index data of attitude estimation processing aiming at current video data sent by the terminal; calculating to obtain a target video coding strategy through a preset video coding strategy model according to the resource supply information and the quality index data of the attitude estimation processing; the preset video coding strategy model is obtained by fitting a quantization relation among a video coding strategy, a quality index of attitude estimation processing and video processing resource consumption; and issuing a target video coding strategy to the terminal so that the terminal can code the subsequently acquired video data according to the target video coding strategy. By the mode, the video coding strategy self-adaptive decision of the attitude estimation task driven by resources is realized, so that the video coding strategy is more reasonable and scientific.

Description

Video processing method and device in edge intelligent environment and computing equipment
Technical Field
The invention relates to the technical field of data processing, in particular to a video processing method and device in an edge intelligent environment and computing equipment.
Background
With the development of mobile internet and multimedia technology, mobile multimedia data has become one of the most important data sources in the mobile internet. Edge intelligence has gradually become one of the most effective paradigms for deploying low-latency mobile video intelligent analysis applications, and an end user can completely or partially offload his intelligent video analysis task to an edge node. Under the edge intelligent environment, a plurality of links related to edge intelligent video analysis, such as video acquisition, coding and decoding, video transmission, reasoning calculation, result feedback and the like, are realized.
In the prior art, objective QoS (Quality of Service) or (QoE) is used for video transmission as an optimization target. However, in the edge intelligent video analysis environment, the video transmission technology takes machine perception as a final purpose, and the video transmission optimization purpose is fundamentally changed, so that the traditional video transmission technology is difficult to adapt to the edge intelligent video analysis application.
Disclosure of Invention
In view of the above, the present invention is proposed to provide a video processing method, apparatus and computing device in a marginal intelligence environment that overcomes or at least partially solves the above mentioned problems.
According to an aspect of the present invention, there is provided a video processing method in an edge smart environment, the method including:
acquiring resource supply information of a current video coding strategy aiming at a terminal and quality index data of attitude estimation processing aiming at current video data sent by the terminal;
calculating to obtain a target video coding strategy through a preset video coding strategy model according to the resource supply information and the quality index data of the attitude estimation processing;
the preset video coding strategy model is obtained by fitting a quantization relation among a video coding strategy, a quality index of attitude estimation processing and video processing resource consumption;
and issuing a target video coding strategy to the terminal so that the terminal can code the subsequently acquired video data according to the target video coding strategy.
According to another aspect of the present invention, there is provided a video processing method in an edge smart environment, the method including:
sending current video data to an edge node; the method comprises the steps that edge nodes calculate to obtain resource supply information corresponding to a current video coding strategy of a terminal, attitude estimation processing is carried out on current video data, quality index data of the attitude estimation processing is detected, and a target video coding strategy is calculated through a preset video coding strategy model according to the resource supply information and the quality index data of the attitude estimation processing;
the preset video coding strategy model is obtained by fitting a feasible video coding strategy, a quality index of attitude estimation processing and a quantization relation among video processing resource consumption;
and receiving a target video coding strategy fed back by the edge node, and coding the subsequently acquired video data according to the target video coding strategy.
According to another aspect of the present invention, there is provided a video processing apparatus in an edge smart environment, including:
the data acquisition module is suitable for acquiring resource supply information of a current video coding strategy aiming at the terminal and quality index data of attitude estimation processing aiming at current video data sent by the terminal;
the strategy processing module is suitable for calculating to obtain a target video coding strategy through a preset video coding strategy model according to the resource supply information and the quality index data of the attitude estimation processing;
the preset video coding strategy model is obtained by fitting a quantization relation among a video coding strategy, a quality index of attitude estimation processing and video processing resource consumption;
and the strategy issuing module is suitable for issuing a target video coding strategy to the terminal so that the terminal can code the subsequently acquired video data according to the target video coding strategy.
According to another aspect of the present invention, there is provided a video processing apparatus in an edge smart environment, including:
the sending module is suitable for sending the current video data to the edge node; the method comprises the steps that edge nodes calculate to obtain resource supply information corresponding to a current video coding strategy of a terminal, attitude estimation processing is carried out on current video data, quality index data of the attitude estimation processing is detected, and a target video coding strategy is calculated through a preset video coding strategy model according to the resource supply information and the quality index data of the attitude estimation processing; the preset video coding strategy model is obtained by fitting a feasible video coding strategy, a quality index of attitude estimation processing and a quantization relation among video processing resource consumption;
the preset video coding strategy model is obtained by fitting a feasible video coding strategy, a quality index of attitude estimation processing and a quantization relation among video processing resource consumption;
the receiving module is suitable for receiving the target video coding strategy fed back by the edge node;
and the video coding module is suitable for coding the subsequently acquired video data according to the target video coding strategy.
According to yet another aspect of the present invention, there is provided a computing device comprising: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus;
the memory is used for storing at least one executable instruction, and the executable instruction enables the processor to execute the operation corresponding to the video processing method in the edge intelligent environment.
According to still another aspect of the present invention, a computer storage medium is provided, where at least one executable instruction is stored in the storage medium, and the executable instruction causes a processor to perform operations corresponding to the video processing method in the above-mentioned edge smart environment.
According to the video processing method, the video processing device and the computing equipment in the edge intelligent environment, the method comprises the following steps: acquiring resource supply information of a current video coding strategy aiming at a terminal and quality index data of attitude estimation processing aiming at current video data sent by the terminal; calculating to obtain a target video coding strategy through a preset video coding strategy model according to the resource supply information and the quality index data of the attitude estimation processing; the preset video coding strategy model is obtained by fitting a quantization relation among a video coding strategy, a quality index of attitude estimation processing and video processing resource consumption; and issuing a target video coding strategy to the terminal so that the terminal can code the subsequently acquired video data according to the target video coding strategy. By the mode, the influence of resource supply on the video coding strategy and the attitude estimation quality is considered, the quality index of attitude estimation processing sensed by a machine is optimized, and the video transmission self-adaptive decision of the attitude estimation task driven by resources is realized, so that the video coding strategy is more reasonable and scientific, the quality of the attitude estimation task is ensured, the time delay required by video transmission is reduced, and the utilization rate of resources is improved.
The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
FIG. 1 is a flow chart illustrating a video processing method in an edge intelligent environment according to an embodiment of the present invention;
FIG. 2 is a flow chart illustrating a video processing method in an edge smart environment according to another embodiment of the present invention;
FIG. 3 is a flow chart illustrating a video processing method in an edge smart environment according to another embodiment of the present invention;
FIG. 4 is a schematic structural diagram of a video processing apparatus in a marginal intelligence environment according to an embodiment of the present invention;
FIG. 5 is a schematic structural diagram of a video processing apparatus in a marginal intelligence environment according to an embodiment of the present invention;
fig. 6 shows a schematic structural diagram of a computing device provided by an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.
Fig. 1 shows a flowchart of a video processing method in an edge intelligent environment according to an embodiment of the present invention, where the method of the present embodiment is applied to a video transmission scene of pose estimation video analysis, and an execution subject is an edge node in the edge intelligent environment, as shown in fig. 1, the method includes the following steps:
step S110, acquiring resource supply information of the current video coding policy for the terminal, and quality index data of the pose estimation process for the current video data transmitted by the terminal.
Specifically, after the terminal collects video data, the collected video data is coded according to a current video coding strategy to obtain current video data, and the current video data is sent to the edge node. In particular, the video data may be transmitted in the form of an RTMP video stream or an HTTP video stream.
The gesture estimation processing aims at outputting human body key point information and can be used in AR substitution application, quality index data of the gesture estimation processing comprises accuracy of gesture estimation analysis and transmission time delay, the accuracy of the gesture estimation analysis is determined by analyzing difference degree between the output human body key point information and actual human body key point information, and the transmission time delay is time delay required by video transmission. The two quality indexes are key indexes for evaluating the quality of the posture estimation task and are machine perception indexes. In the method of this embodiment, after receiving the current video data, the edge node performs the attitude estimation processing, and analyzes to obtain quality index data of the attitude estimation processing. And the depth model of the attitude estimation is deployed in an edge server and used for carrying out resource estimation processing on the video.
The current video coding strategy refers to a coding strategy adopted by a current terminal for coding, and the coding strategy specifically comprises a video resolution dimension, a quantization parameter dimension and a coding region. The resource supply information includes supply information of the computing resource and supply information of the bandwidth resource, and is obtained by performing scheduling calculation on resource information owned by the current terminal.
And step S120, calculating to obtain a target video coding strategy through a preset video coding strategy model according to the resource supply information and the quality index data of the attitude estimation processing.
The preset video coding strategy model is obtained by fitting a quantization relation among a video coding strategy, a quality index of attitude estimation processing and video processing resource consumption.
The video encoding strategy model is constructed by determining the quantization relationship among the video encoding strategy, the quality live broadcast index of attitude estimation analysis and the video processing resource consumption in advance through a data analysis technology, determining the internal association and transfer relationship among the resource supply, the video encoding strategy and the quality of attitude estimation processing. In the subsequent process, a better video coding strategy under the given resource supply can be calculated through the video coding strategy model, and the acquired video data is coded according to the target video coding strategy, so that the attitude estimation analysis quality is high, the transmission delay is low, and the resource utilization rate is high.
And step S130, issuing a target video coding strategy to the terminal so that the terminal can code the subsequently acquired video data according to the target video coding strategy.
And after the edge node calculates the target video coding strategy, returning the target video coding strategy to the terminal, wherein similarly, the target video coding strategy also comprises video resolution, quantization parameters and video parameters of three dimensions of a coding region. And the terminal receives the target video coding strategy fed back by the edge node, and the subsequently acquired video data is coded according to the video parameters contained in the target video coding strategy.
According to the video processing method under the edge intelligent environment provided by the embodiment, a preset video coding strategy model is constructed and obtained by fitting the feasible coding video strategy, the quality index of attitude estimation processing and the quantization relation among video processing consumed resources, a target video coding strategy under the current scene is obtained by processing the preset coding strategy model, and then the collected video data is coded by using the target video coding strategy. By the mode, the influence of resource supply on the video coding strategy and the attitude estimation quality is considered, the quality index of attitude estimation processing sensed by a machine is optimized, and the video transmission self-adaptive decision of the attitude estimation task driven by resources is realized, so that the video coding strategy is more reasonable and scientific, the quality of the attitude estimation task is ensured, the time delay required by video transmission is reduced, and the utilization rate of resources is improved.
Fig. 2 is a flowchart illustrating a video processing method in an edge smart environment according to another embodiment of the present invention, and as shown in fig. 2, the method includes the following steps:
step S210, fitting a first quantization relation between the video coding strategy and the quality index of the attitude estimation processing.
The video coding strategy is obtained by combining video parameters of all dimensions, and the video parameters comprise: video resolution multiple, quantization parameter, and coding region. Wherein the video resolution multiple indicates a multiple of adjusting the resolution of the original video, for example, up to 1.25 times, 1.5 times, 2 times, and down to 0.8 times, 0.75 times, 0.66 times, 0.5 times the resolution of the original video. The encoding region indicates a region in the video to be encoded, and setting the encoding region includes, for example, in consideration of the specificity of the pose estimation task: detecting human body parts and high-quality coding-free areas in the complete video frame and the video frame. For example, if the video coding strategy is {1.25, 0.8, full video frame }, it means that the coding process is: raising the resolution to 1.23 times the original resolution, setting the quantization parameter to 75%, and encoding the complete video frame.
And aiming at each video coding strategy, coding video data by adopting the video coding strategy, sending the coded video data to an edge node for attitude estimation processing, and detecting quality indexes of the attitude estimation processing to obtain the accuracy of attitude estimation analysis and transmission delay under the coding strategy. And performing multiple tests to obtain a large amount of sample data, analyzing and fitting the sample data to obtain a first quantization relation between the video coding strategy and the quality index of the attitude estimation processing.
Step S220, a second quantization relationship between the feasible video coding strategy and the video processing resource consumption is fitted.
Similarly, for each video coding strategy, the video coding strategy is adopted to code video data, the coded video data are sent to an edge node, and computing resources consumed during video coding, including CPU utilization rate and memory utilization rate, and bandwidth resources during video transmission, including network traffic, are detected, namely resource consumption under the video coding strategy is obtained. And performing multiple tests to obtain a large amount of sample data, analyzing and fitting the sample data to obtain a second quantitative relation between the video coding strategy and the video processing resource consumption.
Step S230, according to the first quantization relationship and the second quantization relationship, fitting to obtain a third quantization relationship among the video processing resource consumption, the video coding strategy, and the quality index of the attitude estimation process, and according to the third quantization relationship, constructing a depth-enhanced learning frame to obtain a video coding strategy model.
According to the quantization relationship between the video coding strategy obtained in the step S210 and the quality index of the pose estimation process, and the relationship between the video coding strategy obtained in the step S220 and the resource consumption of the video processing, the intrinsic relationship and the transfer relationship between the resource consumption, the video coding strategy, and the quality index of the pose estimation process are defined. And then, according to a third quantization relation among the fitted resource consumption, the feasible video coding strategy and the quality index of the attitude estimation processing, constructing a depth reinforcement learning framework to obtain a video coding strategy model. In the subsequent process, a better target video coding strategy under the given resource supply can be calculated through the video coding strategy model, and the acquired video data is coded according to the target video coding strategy, so that the attitude estimation analysis quality is high, the transmission delay is low, and the resource utilization rate is high.
In aIn an alternative approach, multivariate functions are used
Figure BDA0003328556030000081
Fitting the relation between the video coding strategy x and the quality index of the attitude estimation processing, specifically as follows:
Figure BDA0003328556030000082
where MSE represents the mean square error and D is the set of video coding strategies.
Fitting the computational resources required for a video coding strategy x using a function g
Figure BDA0003328556030000083
And bandwidth resources
Figure BDA0003328556030000084
The quantization relationship of (2) is as follows:
Figure BDA0003328556030000085
and finally, fitting the quantitative relation among the video processing resource consumption, the video coding strategy and the quality index of the attitude estimation processing, and constructing a resource-driven low-delay video analysis reasoning quality model under the edge intelligent environment on the basis, wherein the model is specifically as follows:
Figure BDA0003328556030000086
wherein X is an intermediate variable representing the set of video coding schemes, the independent variable of the Q function is the quality index of the attitude estimation processing, and the dependent variable is QoI, Q (r)c,rn) Representing a reward function, rc,rnDenoted as resource provisioning.
Equivalently, according to a large amount of experimental data, fitting to obtain a quantization relation among the resource consumption, the video coding strategy and the quality index of attitude estimation processing, constructing a video coding strategy model on the basis of the quantization relation, and determining a proper target video coding strategy through the model based on the current resource supply in the subsequent process.
Optionally, for a deep strong chemical framework, setting the QoI as a reward function, and setting PDRLThe decision function of the deep reinforcement learning framework representing the edge nodes is represented as follows:
Figure BDA0003328556030000087
wherein x denotes the decision-derived video coding strategy, the decision function PDRLThe larger the function value of (a) is, the more advantageous the corresponding video coding scheme is.
Step S240, determining resource supply information according to the current video coding strategy and the second quantization relation of the terminal; wherein the resource provisioning information comprises: computing resource provisioning information and network resource provisioning information.
And calculating the calculation resources required by the video coding under the current video coding strategy and the network resources required by the video transmission according to the quantization relation between the video coding strategy and the resource consumption determined in the step and the current video coding strategy, and then obtaining resource supply information, namely a resource scheduling step.
And step S250, receiving the current video data sent by the terminal, performing attitude estimation processing on the current video data, and analyzing to obtain quality index data of attitude estimation.
The terminal processes the collected video data according to three video parameters contained in the current video coding strategy, the coded video data are sent to the edge node, the edge node carries out attitude estimation processing after receiving the coded video data, and the accuracy of the attitude estimation processing and the transmission delay are obtained through analysis.
And step S260, calculating to obtain a target video coding strategy through a preset video coding strategy model according to the resource supply information and the quality index data of the attitude estimation processing.
And inputting the calculated resource supply information and the quality index data of the attitude estimation processing into a deep chemical framework for processing, and extracting a corresponding video coding strategy when the value of a decision function of the deep reinforcement learning framework is maximum, thereby obtaining a target video coding strategy.
And step S270, issuing a target video coding strategy to the terminal so that the terminal can code the subsequently acquired video data according to the target video coding strategy.
And after the target video coding strategy is obtained through calculation, the target video coding strategy is issued to the terminal, and the terminal carries out coding processing on subsequent video data according to the target video coding strategy.
In an alternative manner, the method of this embodiment is executed every predetermined time interval, that is, the video coding strategy for the terminal to code the video data is updated every certain time interval.
In another alternative, the method of this embodiment is performed in real time, that is, a video encoding policy for encoding video data by the terminal is detected and updated in real time.
According to the video processing method under the edge intelligent environment provided by the embodiment, a preset video coding strategy model is constructed and obtained by fitting the feasible coding video strategy, the quality index of attitude estimation processing and the quantization relation among video processing consumed resources, a target video coding strategy under the current scene is obtained by processing the preset coding strategy model, and then the collected video data is coded by using the target video coding strategy. By the mode, the influence of resources on video parameters and attitude estimation quality is considered, the quality index of attitude estimation processing sensed by a machine is optimized, the video transmission self-adaptive decision of the attitude estimation task driven by the resources is realized, the quality of the attitude estimation task is ensured, the time delay required by video transmission is reduced, and the utilization rate of the resources is improved.
Fig. 3 is a flowchart illustrating a video processing method in an edge smart environment according to another embodiment of the present invention, where the method of this embodiment is applied to a terminal side, as shown in fig. 3, and the method includes:
step S310, sending the current video data to the edge node; and calculating by using an edge node to obtain resource supply information corresponding to the current video coding strategy of the terminal, performing attitude estimation processing on the current video data, detecting quality index data of the attitude estimation processing, and calculating by using a preset video coding strategy model according to the resource supply information and the quality index data of the attitude estimation processing to obtain a target video coding strategy.
The preset video coding strategy model is obtained by fitting a feasible video coding strategy, a quality index of attitude estimation processing and a quantization relation among video processing resource consumption;
and step S320, receiving the target video coding strategy fed back by the edge node, and coding the subsequently acquired video data according to the target video coding strategy.
The method applied to the terminal side in this embodiment corresponds to the method applied to the edge node side in the foregoing embodiment, and for specific embodiments, reference may be made to the description in the foregoing embodiment, which is not described herein again.
According to the video processing method under the edge intelligent environment provided by the embodiment, a preset video coding strategy model is constructed and obtained by fitting the feasible coding video strategy, the quality index of attitude estimation processing and the quantization relation among video processing consumed resources, a target video coding strategy under the current scene is obtained by processing the preset coding strategy model, and then the collected video data is coded by using the target video coding strategy. By the mode, the influence of resource supply on video parameters and attitude estimation quality is considered, the quality index of attitude estimation processing sensed by a machine is optimized, the video transmission self-adaptive decision of the attitude estimation task driven by resources is realized, the quality of the attitude estimation task is ensured, the time delay required by video transmission is reduced, and the utilization rate of resources is improved.
Fig. 4 is a schematic structural diagram of a video processing apparatus in an edge smart environment according to an embodiment of the present invention, and as shown in fig. 4, the apparatus includes:
a data acquisition module 41 adapted to acquire resource supply information for a current video coding policy of the terminal and quality index data for pose estimation processing of current video data transmitted by the terminal;
the strategy processing module 42 is adapted to calculate a target video coding strategy through a preset video coding strategy model according to the resource supply information and the quality index data of the attitude estimation processing;
the preset video coding strategy model is obtained by fitting a quantization relation among a video coding strategy, a quality index of attitude estimation processing and video processing resource consumption;
and the strategy issuing module 43 is adapted to issue the target video coding strategy to the terminal, so that the terminal can perform coding processing on subsequently acquired video data according to the target video coding strategy.
In an alternative approach, the accuracy of the analysis and the propagation delay are estimated.
In an alternative approach, the target video coding strategy includes the following video parameters: video resolution multiple, quantization parameter, coding area.
In an optional manner, the apparatus further comprises:
the model building module is suitable for fitting a first quantization relation between a video coding strategy and a quality index of attitude estimation processing and fitting a second quantization relation between the video coding strategy and video processing resource consumption; fitting to obtain a third quantization relation among the video processing resource consumption, the video coding strategy and the quality index of the attitude estimation processing according to the first quantization relation and the second quantization relation; and constructing a depth reinforcement learning framework according to the third quantization relation to obtain a video coding strategy model.
In an optional manner, the apparatus further comprises:
the scheduling module is suitable for receiving resource state information sent by a terminal, performing resource scheduling processing through the second quantitative relation and determining the resource supply information; wherein the resource provisioning information comprises: computing resource provisioning information and network resource provisioning information.
In an optional manner, the apparatus further comprises:
and the video analysis module is suitable for receiving the current video data sent by the terminal, performing attitude estimation processing on the current video data, and analyzing to obtain quality index data subjected to attitude estimation processing.
Fig. 5 is a schematic structural diagram of a video processing apparatus in an edge smart environment according to an embodiment of the present invention, and as shown in fig. 5, the apparatus includes:
a sending module 51 adapted to send the current video data to the edge node; the method comprises the steps that edge nodes calculate to obtain resource supply information corresponding to a current video coding strategy of a terminal, attitude estimation processing is carried out on current video data, quality index data of the attitude estimation processing is detected, and a target video coding strategy is calculated through a preset video coding strategy model according to the resource supply information and the quality index data of the attitude estimation processing; the preset video coding strategy model is obtained by fitting a feasible video coding strategy, a quality index of attitude estimation processing and a quantization relation among video processing resource consumption;
the preset video coding strategy model is obtained by fitting a feasible video coding strategy, a quality index of attitude estimation processing and a quantization relation among video processing resource consumption;
a receiving module 52, adapted to receive the target video coding strategy fed back by the edge node;
and the video coding module 53 is adapted to perform coding processing on subsequently acquired video data according to the target video coding strategy.
In an alternative manner, the quality indicators of the attitude estimation process include: accuracy of attitude estimation analysis and transmission delay.
In an alternative approach, the target video coding strategy includes the following video parameters: video resolution multiple, quantization parameter, coding area.
An embodiment of the present invention provides a non-volatile computer storage medium, where the computer storage medium stores at least one executable instruction, and the computer executable instruction may execute a video processing method in an edge intelligent environment in any of the above method embodiments.
Fig. 6 is a schematic structural diagram of an embodiment of a computing device according to the present invention, and a specific embodiment of the present invention does not limit a specific implementation of the computing device.
As shown in fig. 6, the computing device may include: a processor (processor)602, a communication Interface 604, a memory 606, and a communication bus 608.
Wherein: the processor 602, communication interface 604, and memory 606 communicate with one another via a communication bus 608. A communication interface 604 for communicating with network elements of other devices, such as clients or other servers. The processor 602 is configured to execute the program 610, and may specifically perform relevant steps in the above-described embodiment of the video processing method used in the edge intelligence environment of the computing device.
In particular, program 610 may include program code comprising computer operating instructions.
The processor 602 may be a central processing unit CPU or an application Specific Integrated circuit asic or one or more Integrated circuits configured to implement embodiments of the present invention. The computing device includes one or more processors, which may be the same type of processor, such as one or more CPUs; or may be different types of processors such as one or more CPUs and one or more ASICs.
And a memory 606 for storing a program 610. Memory 606 may comprise high-speed RAM memory, and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.
The algorithms or displays presented herein are not inherently related to any particular computer, virtual system, or other apparatus. Various general purpose systems may also be used with the teachings herein. The required structure for constructing such a system will be apparent from the description above. In addition, embodiments of the present invention are not directed to any particular programming language. It is appreciated that a variety of programming languages may be used to implement the teachings of the present invention as described herein, and any descriptions of specific languages are provided above to disclose the best mode of the invention.
In the description provided herein, numerous specific details are set forth. It is understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the embodiments of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the invention and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: that the invention as claimed requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
Those skilled in the art will appreciate that the modules in the device in an embodiment may be adaptively changed and disposed in one or more devices different from the embodiment. The modules or units or components of the embodiments may be combined into one module or unit or component, and furthermore they may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Furthermore, those skilled in the art will appreciate that while some embodiments herein include some features included in other embodiments, rather than other features, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments may be used in any combination.
The various component embodiments of the invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that a microprocessor or Digital Signal Processor (DSP) may be used in practice to implement some or all of the functionality of some or all of the components according to embodiments of the present invention. The present invention may also be embodied as apparatus or device programs (e.g., computer programs and computer program products) for performing a portion or all of the methods described herein. Such programs implementing the present invention may be stored on computer-readable media or may be in the form of one or more signals. Such a signal may be downloaded from an internet website or provided on a carrier signal or in any other form.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names. The steps in the above embodiments should not be construed as limiting the order of execution unless specified otherwise.

Claims (10)

1. A method for video processing in an edge smart environment, the method comprising:
acquiring resource supply information of a current video coding strategy aiming at a terminal and quality index data of attitude estimation processing aiming at current video data sent by the terminal;
calculating to obtain a target video coding strategy through a preset video coding strategy model according to the resource supply information and the quality index data processed by the attitude estimation;
the preset video coding strategy model is obtained by fitting a quantization relation among a video coding strategy, a quality index of attitude estimation processing and video processing resource consumption;
and issuing the target video coding strategy to the terminal so that the terminal can code the subsequently acquired video data according to the target video coding strategy.
2. The method of claim 1, wherein the quality indicators of the pose estimation process comprise: accuracy of attitude estimation analysis and transmission delay.
3. The method of claim 1, wherein the target video coding strategy comprises the following video parameters: video resolution multiple, quantization parameter, coding area.
4. The method of claim 1, wherein prior to performing the method, further comprising:
fitting a first quantization relation between a video coding strategy and a quality index of attitude estimation processing, and fitting a second quantization relation between the video coding strategy and video processing resource consumption;
fitting to obtain a third quantization relation among the video processing resource consumption, the video coding strategy and the quality index of the attitude estimation processing according to the first quantization relation and the second quantization relation;
and constructing a depth reinforcement learning framework according to the third quantization relation to obtain the video coding strategy model.
5. The method of claim 4, wherein prior to performing the method, further comprising:
receiving resource state information sent by a terminal, performing resource scheduling processing through the second quantitative relation, and determining the resource supply information; wherein the resource provisioning information comprises: computing resource provisioning information and network resource provisioning information.
6. The method of claim 5, wherein prior to performing the method, further comprising:
and receiving the current video data sent by the terminal, performing attitude estimation processing on the current video data, and analyzing to obtain quality index data subjected to attitude estimation processing.
7. A method for video processing in an edge smart environment, the method comprising:
sending current video data to an edge node; the method comprises the steps that the edge nodes calculate to obtain resource supply information corresponding to a current video coding strategy of a terminal, attitude estimation processing is carried out on current video data, quality index data of the attitude estimation processing is detected, and a target video coding strategy is calculated through a preset video coding strategy model according to the resource supply information and the quality index data of the attitude estimation processing;
the preset video coding strategy model is obtained by fitting a feasible video coding strategy, a quality index of attitude estimation processing and a quantization relation among video processing resource consumption;
and receiving a target video coding strategy fed back by the edge node, and coding the subsequently acquired video data according to the target video coding strategy.
8. Video processing apparatus in a marginal intelligence environment, comprising:
the data acquisition module is suitable for acquiring resource supply information of a current video coding strategy aiming at the terminal and quality index data of attitude estimation processing aiming at current video data sent by the terminal;
the strategy processing module is suitable for calculating to obtain a target video coding strategy through a preset video coding strategy model according to the resource supply information and the quality index data processed by the attitude estimation;
the preset video coding strategy model is obtained by fitting a quantization relation among a video coding strategy, a quality index of attitude estimation processing and video processing resource consumption;
and the strategy issuing module is suitable for issuing the target video coding strategy to the terminal so that the terminal can code the subsequently acquired video data according to the target video coding strategy.
9. A computing device, comprising: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus;
the memory is used for storing at least one executable instruction, and the executable instruction causes the processor to execute the operation corresponding to the video processing method in the edge intelligent environment according to any one of claims 1-6.
10. A computer storage medium having at least one executable instruction stored therein, the executable instruction causing a processor to perform operations corresponding to the video processing method in the edge smart environment according to any one of claims 1 to 6.
CN202111273353.5A 2021-10-29 2021-10-29 Video processing method and device in edge intelligent environment and computing equipment Pending CN113891083A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111273353.5A CN113891083A (en) 2021-10-29 2021-10-29 Video processing method and device in edge intelligent environment and computing equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111273353.5A CN113891083A (en) 2021-10-29 2021-10-29 Video processing method and device in edge intelligent environment and computing equipment

Publications (1)

Publication Number Publication Date
CN113891083A true CN113891083A (en) 2022-01-04

Family

ID=79015028

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111273353.5A Pending CN113891083A (en) 2021-10-29 2021-10-29 Video processing method and device in edge intelligent environment and computing equipment

Country Status (1)

Country Link
CN (1) CN113891083A (en)

Similar Documents

Publication Publication Date Title
CN110807109A (en) Data enhancement strategy generation method, data enhancement method and device
CN110557633B (en) Compression transmission method, system and computer readable storage medium for image data
CN109743356B (en) Industrial internet data acquisition method and device, readable storage medium and terminal
JP7356581B2 (en) Information processing methods, devices, equipment and computer readable storage media
CN114757352B (en) Intelligent agent training method, cross-domain heterogeneous environment task scheduling method and related device
CN112437022B (en) Network traffic identification method, device and computer storage medium
CN112399177B (en) Video coding method, device, computer equipment and storage medium
CN113891083A (en) Video processing method and device in edge intelligent environment and computing equipment
CN108900865B (en) Server, and scheduling method and execution method of transcoding task
CN111726656A (en) Transcoding method, device, server and storage medium for live video
CN116390221A (en) Periodic registration method, system, device and medium for satellite communication user terminal
CN104168274A (en) Data obtaining request processing method, client sides and server
CN114238658A (en) Link prediction method and device of time sequence knowledge graph and electronic equipment
KR101932130B1 (en) Apparatus and method for improving quality of experience of remote display
CN114374841A (en) Optimization method and device for video coding rate control and electronic equipment
CN112633198A (en) Picture processing method and device, storage medium and electronic device
CN114692888A (en) System parameter processing method, device, equipment and storage medium
CN111901631A (en) Transcoding method, device, server and storage medium for live video
CN113822454A (en) Prediction method and device for slice complaint processing resources
JP6915362B2 (en) Transfer device, transfer method and program
CN113949633A (en) 5G network slice disaster recovery pool resource management method and device based on machine learning
CN111614700B (en) Real-scene interactive driving simulation method and device and driving simulation system platform
CN113784115B (en) Multimedia quality evaluation method, device, equipment and storage medium
CN113993148B (en) 5G network slice disaster recovery switching method and device based on machine learning
CN116233443A (en) Quality evaluation method, quality evaluation device, quality evaluation computer, quality evaluation storage medium, and quality evaluation program product

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination