CN116089043A - Heterogeneous application system video analysis task scheduling method, device, terminal and medium - Google Patents
Heterogeneous application system video analysis task scheduling method, device, terminal and medium Download PDFInfo
- Publication number
- CN116089043A CN116089043A CN202310076808.7A CN202310076808A CN116089043A CN 116089043 A CN116089043 A CN 116089043A CN 202310076808 A CN202310076808 A CN 202310076808A CN 116089043 A CN116089043 A CN 116089043A
- Authority
- CN
- China
- Prior art keywords
- video
- task
- scheduling
- calculation
- decoding
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention discloses a heterogeneous application system video analysis task scheduling method, a heterogeneous application system video analysis task scheduling device, a heterogeneous application system video analysis task scheduling terminal and a heterogeneous application system video analysis task scheduling medium, wherein the heterogeneous application system video analysis task scheduling method comprises the following steps: acquiring calculation force parameters and video decoding parameters required by a video analysis task; determining a scheduling strategy of the video analysis task and a corresponding computing module according to the required computing power parameter and video decoding parameter and the computing power parameter and video encoding and decoding parameter of each computing module; and scheduling the video decoding task and the computing task in the video analysis task to the corresponding computing module according to the scheduling strategy so as to execute the corresponding task. According to the invention, through video data communication transmission among the multiple computing modules in the heterogeneous application system, the multiple computing modules are scheduled to complete video analysis tasks, so that the maximum utilization of computing power and video coding and decoding capacity of the heterogeneous application system is realized, the processing path number of the video analysis tasks supported by the system is improved, and the video analysis efficiency is improved.
Description
Technical Field
The invention relates to the technical field of video processing, in particular to a method, a device, a terminal and a medium for scheduling video analysis tasks of a heterogeneous application system.
Background
For the fields of intelligent transportation, intelligent security, intelligent construction site or industrial intelligent manufacturing, a large amount of camera video data is accessed, and then detection and analysis are carried out on multiple paths of videos through edge computing equipment, such as detection of whether traffic intersections are congested, detection of vehicle characteristics (such as vehicle counting, license plate numbers, vehicle types and vehicle body color detection), detection and tracking of mud head vehicles, detection of personnel characteristics (such as detection of people counting, gender, age, wearing, mask and the like), face recognition, detection of safety helmets of construction sites, detection of abnormal product appearance and the like. The number of cameras which are generally accessed is relatively large, and each edge computing device is required to analyze and process relatively multiple paths of video data. While computing modules of a particular hardware platform have limited processing power, the design of multiple computing modules may be employed within an edge computing device. Meanwhile, according to the specific service and cost requirements, a plurality of computing modules of the edge computing device can be designed in a heterogeneous mode to form a heterogeneous application system.
For video detection and analysis tasks, the edge computing device or heterogeneous application system in the industry generally dispatches a certain path of video analysis task to a certain computing module according to the computing power and video decoding capability of each built-in computing module and the computing power and video resources required by the video analysis task. This path of video analysis task may fail to schedule if the remaining computing power and video resources of all computing modules in the edge computing device are insufficient. While there may be a substantial amount of computing power on some computing modules and substantial video processing power on other computing modules, this may result in wasted hardware resources in the edge computing device.
In addition, aiming at the calculation modules of different hardware platforms in the current industry, some calculation modules have high calculation power, but the video coding and decoding processing capacity is lower. Some computing modules have strong video encoding and decoding processing capability, but have lower computing power. Different computing power configurations and video decoding processing capacity parameter configurations, different cost prices and different power consumption of the computing modules. In order to be able to handle the video detection analysis tasks of more cameras, higher computational power and more video decoding processing capacity may be required at the same time, which may result in very high cost of the whole edge computing device; therefore, for the computing modules of different hardware platforms, the problem of low processing efficiency of the video analysis task also exists.
Accordingly, there is a need in the art for improvement.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a method, a device, a terminal and a medium for scheduling video analysis tasks of a heterogeneous application system, so as to solve the technical problem of low processing efficiency of the existing video analysis tasks.
The technical scheme adopted for solving the technical problems is as follows:
in a first aspect, the present invention provides a method for scheduling video analysis tasks of a heterogeneous application system, including:
Acquiring calculation force parameters and video decoding parameters required by a video analysis task;
determining a scheduling strategy of the video analysis task and a corresponding computing module according to the required computing power parameter and video decoding parameter and the computing power parameter and video encoding and decoding parameter of each computing module;
and scheduling the video decoding task and the computing task in the video analysis task to the corresponding computing module according to the scheduling strategy so as to execute the corresponding task.
In one implementation, the acquiring the computational power parameters and the video decoding parameters required for the video analysis task includes:
and configuring the calculation force parameters and the video coding and decoding parameters of each calculation module according to the preset parameters.
In one implementation, the acquiring the computational power parameter and the video decoding parameter required by the video analysis task further includes:
and acquiring the calculation force parameters and the video coding and decoding parameters reported by each calculation module.
In one implementation, the acquiring the computational power parameters and the video decoding parameters required for the video analysis task includes:
acquiring video data information acquired by each video acquisition device, and configuring and obtaining the video analysis task according to the video data information;
And analyzing the calculation force parameters and the video decoding parameters required by the video analysis task.
In one implementation manner, the determining the scheduling policy of the video analysis task and the corresponding computing module according to the required computing power parameter and the video decoding parameter and the computing power parameter and the video encoding and decoding parameter of each computing module includes:
according to the required calculation power parameters and video decoding parameters and the calculation power parameters and video coding and decoding parameters of each calculation module, judging whether each calculation module meets the video decoding requirement and the calculation requirement at the same time;
if each computing module cannot meet the video decoding requirement and the computing requirement at the same time, determining a first computing module meeting the video decoding requirement and a second computing module meeting the computing requirement.
In one implementation, the scheduling the video decoding task and the computing task in the video analysis task to the corresponding computing module according to the scheduling policy includes:
scheduling a video decoding task in the video analysis task to the first computing module for decoding to obtain decoded video data;
and dispatching the calculation task in the video analysis task to the second calculation module, sending the decoded video data to the second calculation module for analysis and detection, and outputting the analysis result of the video analysis task.
In one implementation manner, the determining the scheduling policy of the video analysis task and the corresponding computing module according to the required computing power parameter and the video decoding parameter and the computing power parameter and the video encoding and decoding parameter of each computing module further includes:
according to the required calculation power parameters and video decoding parameters and the calculation power parameters and video coding and decoding parameters of each calculation module, judging whether each calculation module meets the video decoding requirements or not in sequence;
if each computing module cannot meet the video decoding requirement, converting the video data compression format into a preset compression format through a corresponding video acquisition device controlled by a protocol;
a third calculation module for determining that the decoding requirement of the video data in the preset format is met;
a fourth calculation module is determined that meets the calculation requirements.
In one implementation manner, the scheduling the video decoding task and the computing task in the video analysis task to the corresponding computing module according to the scheduling policy further includes:
scheduling a video decoding task in the video analysis task to the third computing module for decoding to obtain decoded video data;
and dispatching the calculation task in the video analysis task to the fourth calculation module, sending the decoded video data to the fourth calculation module for analysis and detection, and outputting the analysis result of the video analysis task.
In one implementation, the decoded video data is data in an original YUV format or an RGB format, or the decoded video data is data meeting the size required by an analysis algorithm, or the decoded video data is compressed image data of H264/H265/JPGE after recompression.
In one implementation manner, the scheduling the video decoding task and the computing task in the video analysis task to the corresponding computing module according to the scheduling policy further includes:
determining a degradation scheduling strategy according to the lowest processing frame rate and the lowest computational power requirement in the task parameters;
and gradually reducing the processing frame rate and the calculation power parameter requirements according to the degradation scheduling strategy until the task scheduling is successful or the minimum processing frame rate and the minimum calculation power requirements are reached.
In one implementation, the method further comprises:
receiving batch video analysis tasks, and acquiring calculation force parameters and video decoding parameters required by each video analysis task;
acquiring calculation force parameters and video coding and decoding parameters of each calculation module, and acquiring video coding compression parameters supported by each video acquisition device;
comprehensively calculating optimal scheduling strategy parameters according to calculation power parameters and video decoding parameters required by each video analysis task, calculation power parameters and video coding and decoding parameters of each calculation module and video coding and compression parameters supported by each video acquisition device;
And scheduling a corresponding calculation module according to the optimal scheduling policy parameters, and executing video decoding tasks and calculation tasks of the batch video analysis tasks.
In a second aspect, the present invention provides a video analysis task scheduling device for a heterogeneous application system, including:
the task parameter acquisition module is used for acquiring calculation force parameters and video decoding parameters required by the video analysis task;
the scheduling strategy module is used for determining the scheduling strategy of the video analysis task and the corresponding calculation module according to the required calculation power parameter and video decoding parameter as well as the calculation power parameter and video encoding and decoding parameter of each calculation module;
and the task execution module is used for dispatching the video decoding task and the calculation task in the video analysis task to the corresponding calculation module according to the dispatching strategy so as to execute the corresponding task.
In a third aspect, the present invention provides a terminal comprising: the system comprises a processor and a memory, wherein the memory stores a heterogeneous application system video analysis task scheduler, and the heterogeneous application system video analysis task scheduler is used for realizing the operation of the heterogeneous application system video analysis task scheduling method according to the first aspect when being executed by the processor.
In a fourth aspect, the present invention further provides a medium, where the medium is a computer readable storage medium, where the medium stores a heterogeneous application system video analysis task scheduler, where the heterogeneous application system video analysis task scheduler is configured to implement the operation of the heterogeneous application system video analysis task scheduling method according to the first aspect when executed by a processor.
The technical scheme adopted by the invention has the following effects:
according to the invention, by acquiring the calculation power parameters and the video decoding parameters required by the video analysis task, the scheduling strategy of the video analysis task and the corresponding calculation modules can be determined according to the required calculation power parameters and the video decoding parameters as well as the calculation power parameters and the video encoding and decoding parameters of each calculation module, so that the video decoding task and the calculation task in the video analysis task are scheduled to the corresponding calculation modules according to the scheduling strategy to execute the corresponding tasks; according to the invention, through video data communication transmission among the multiple computing modules in the heterogeneous application system, the multiple computing modules are scheduled to complete video analysis tasks, so that the maximum utilization of computing power and video coding and decoding capacity of the heterogeneous application system is realized, the processing path number of the video analysis tasks supported by the system is improved, and the video analysis efficiency is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to the structures shown in these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a method for scheduling tasks for video analytics of a heterogeneous application system in one implementation of the present invention.
FIG. 2 is a schematic diagram of a first heterogeneous application system in one implementation of the present invention.
FIG. 3 is a schematic diagram of a second heterogeneous application system in one implementation of the present invention.
FIG. 4 is a schematic diagram of a first way of obtaining the calculation force parameters and the video codec parameters in one implementation of the invention.
FIG. 5 is a schematic diagram of a second way of obtaining the calculation force parameters and the video codec parameters in one implementation of the invention.
FIG. 6 is a schematic diagram of the scheduling of a single video analytics task in one implementation of the present invention.
Fig. 7 is a schematic diagram of a first transmission of decoded video data in one implementation of the invention.
Fig. 8 is a second transmission schematic of decoded video data in one implementation of the invention.
Fig. 9 is a schematic diagram of the configuration of video encoding compression parameters of a video acquisition device in one implementation of the invention.
Fig. 10 is a schematic diagram illustrating reporting of video coding compression parameters of a video acquisition device in an implementation of the present invention.
FIG. 11 is a schematic diagram of a coded compression format by protocol switching in one implementation of the invention.
FIG. 12 is a schematic diagram of the scheduling of batch video analytics tasks in one implementation of the present invention.
Fig. 13 is a functional schematic of a terminal in one implementation of the invention.
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more clear and clear, the present invention will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
Exemplary method
Aiming at the calculation modules of different hardware platforms in the current industry, some calculation modules have high calculation power, but the video coding and decoding processing capacity is lower. Some computing modules have strong video encoding and decoding processing capability, but have lower computing power. Different computing power configurations and video decoding processing capacity parameter configurations, different cost prices and different power consumption of the computing modules. In order to be able to process video detection and analysis tasks of more cameras, higher computation power and more video decoding processing capacity may be required at the same time, which may result in very high cost of the whole edge computing device and also unable to maximize the capacity of each computing module, so that there is a problem that processing efficiency of video analysis tasks is low for computing modules of different hardware platforms.
Aiming at the technical problems, the embodiment provides a heterogeneous application system video analysis task scheduling method, in the embodiment, the plurality of calculation modules are scheduled to complete video analysis tasks through video data communication transmission among the plurality of calculation modules in the heterogeneous application system, so that the maximum utilization of calculation power and video coding and decoding capacity of the heterogeneous application system is realized, the processing path number of the video analysis tasks supported by the system is improved, and the video analysis efficiency is improved.
As shown in fig. 1, an embodiment of the present invention provides a method for scheduling tasks for video analysis of a heterogeneous application system, including the following steps:
step S100, obtaining calculation power parameters and video decoding parameters required by a video analysis task.
In this embodiment, the method for scheduling video analysis tasks of the heterogeneous application system is applied to a terminal, where the terminal includes but is not limited to: a computer, etc.
In this embodiment, the terminal may be an independent control module (i.e. a management module) in the whole heterogeneous application system, as shown in fig. 2, where the control module is a module connected to multiple computing modules, and is used to control the multiple computing modules; within the heterogeneous application system in the system, a plurality of computing modules and a control module are included. Each computing module has a certain computing power or video encoding and decoding capability. The control module is responsible for scheduling all video analysis tasks. Data communication channels, such as network channels, exist between all computing modules and the control module, and between all computing modules. All video acquisition equipment such as cameras are connected and communicated with the heterogeneous application system through channels such as a network.
In this embodiment, the terminal may also be a certain computing module in the whole heterogeneous application system, as shown in fig. 3, where the control module is one of the computing modules and is used for the remaining computing modules; in this system, the function of the control module may be performed by one of the computing modules.
In this embodiment, as shown in fig. 4, the computing power, video encoding and decoding format capability, etc. of all the computing modules in the heterogeneous application system can be configured to the control module side through parameters.
Specifically, in one implementation of the present embodiment, the step S100 includes the following steps before:
step S101a, the calculation power parameters and the video coding and decoding parameters of each calculation module are configured according to preset parameters.
In this embodiment, in the process of configuring the power calculation parameters and the video encoding and decoding parameters of each calculation module, the hardware information, for example, the CPU information and the GPU information, of each calculation module may be obtained, and then the preset parameters are determined according to the hardware information of each calculation module, so as to obtain the power calculation parameters and the video encoding and decoding parameters of each calculation module.
Specifically, in one implementation of the present embodiment, the following steps are further included before step S100:
step S101b, obtaining the calculation force parameters and the video coding and decoding parameters reported by each calculation module.
In this embodiment, as shown in fig. 5, the computing power, the video encoding and decoding format capability and the like of all the computing modules in the heterogeneous application system may also be actively reported to the control module by each computing module, where the control module may obtain the computing power parameters and the video encoding and decoding parameters of each computing module.
Specifically, in one implementation of the present embodiment, step S100 includes the steps of:
step S101, acquiring video data information acquired by each video acquisition device, and configuring and obtaining the video analysis task according to the video data information;
step S102, analyzing the calculation power parameters and the video decoding parameters required by the video analysis task.
In this embodiment, the video data information acquired by the control module is derived from the video acquisition device connected to each path of computing module, or may be derived from the video acquisition device directly connected to the control module; and determining the information such as the size, resolution, format and the like of the video data acquired by each video acquisition device after the video data information is acquired, configuring a video analysis task according to the information, and analyzing and obtaining calculation power parameters and video decoding parameters required by the video analysis task so as to determine a corresponding task scheduling strategy according to the required parameters.
As shown in fig. 1, in an implementation manner of the embodiment of the present invention, the method for scheduling video analysis tasks of a heterogeneous application system further includes the following steps:
step 200, determining a scheduling strategy of the video analysis task and a corresponding computing module according to the required computing power parameter and video decoding parameter and the computing power parameter and video encoding and decoding parameter of each computing module.
In this embodiment, the control module obtains the calculation power and video decoding capability requirements required by each path of video analysis task according to the calculation power, video encoding and decoding capability and other parameter conditions of each calculation module in the system, and dispatches the video analysis task to a certain calculation module meeting the calculation power and video decoding capability requirements; specific scheduling strategies include, but are not limited to, the following three ways.
Specifically, in one implementation of the present embodiment, step S200 includes the steps of:
step S211, judging whether each calculation module meets the video decoding requirement and the calculation requirement at the same time according to the required calculation power parameter and the video decoding parameter as well as the calculation power parameter and the video encoding and decoding parameter of each calculation module;
in step S212, if each computing module cannot meet the video decoding requirement and the computing requirement at the same time, determining a first computing module meeting the video decoding requirement and a second computing module meeting the computing requirement.
In this embodiment, the first scheduling policy is: if the control module finds out the calculation power and video decoding capability required by a certain path of video analysis task, all calculation modules in the system can not meet the corresponding requirements at the same time, the calculation modules meeting the calculation power and the calculation modules meeting the video decoding capability are detected respectively. If a computing module A (i.e., a first computing module) satisfying the video decoding capability and a computing module B (i.e., a second computing module) satisfying the computing power are found, the computing module A is set as an executing module of the video decoding task, and the computing module B is set as an executing module of the computing task.
Specifically, in one implementation of the present embodiment, step S200 further includes the following steps:
step S221, judging whether each calculation module meets the video decoding requirement according to the required calculation power parameter and video decoding parameter, and the calculation power parameter and video coding and decoding parameter of each calculation module;
step S222, if each computing module can not meet the video decoding requirement, converting the video data compression format into a preset compression format through the corresponding video acquisition equipment controlled by a protocol;
step S223, determining a third calculation module meeting the decoding requirement of the video data in the preset format;
In step S224, a fourth computing module satisfying the computing requirement is determined.
In this embodiment, the second scheduling policy is: if the control module finds out the calculation power and video decoding capability required by a certain path of video analysis task, all calculation modules in the system cannot meet the corresponding requirements at the same time, and also cannot meet the calculation modules of the video decoding capability, but detect the calculation modules meeting the calculation power. The control module can control the corresponding video acquisition equipment and other video acquisition equipment through a protocol, and switch the video data compression format of the video acquisition equipment into a preset compression format; the preset compression format is a format of which the current whole system meets the video decoding capability requirement. If the video compression format of the video acquisition equipment is successfully switched, the control module detects one or more calculation modules (namely a third calculation module) meeting the new video decoding capability requirement and sets the calculation modules as execution modules of video decoding tasks; and a computing module (namely a fourth computing module) for detecting the requirement of the video analysis task and setting the computing module as an executing module of the computing task.
In the second scheduling strategy, the video data encoding format of the video acquisition device is modified, so that various video image formats such as H264, H265, JPEG and the like can be switched, and video or original image data with lower resolution and lower frame rate can be directly transmitted according to algorithm requirements.
In this embodiment, the third scheduling policy is: if the control module finds out the calculation power and video decoding capability required by a certain path of video analysis task, one or more calculation modules in the system can meet the corresponding requirements at the same time, the calculation module is set as the execution module of the video analysis task, or the calculation module with the optimal capability is set as the execution module of the video analysis task according to the calculation power and the strength of the video decoding capability.
In the embodiment, through video data communication transmission among the multiple computing modules in the heterogeneous application system, the multiple computing modules are scheduled to complete a video analysis task, so that the maximum utilization of computing power and video coding and decoding capacity of the heterogeneous application system is realized, and the processing path number of the video analysis task supported by the system is improved.
As shown in fig. 1, in an implementation manner of the embodiment of the present invention, the method for scheduling video analysis tasks of a heterogeneous application system further includes the following steps:
and step S300, scheduling the video decoding task and the computing task in the video analysis task to the corresponding computing module according to the scheduling strategy so as to execute the corresponding task.
In this embodiment, for the three different scheduling policies, different tasks are executed in the corresponding computing modules, and specific execution processes are as follows.
Specifically, in one implementation of the present embodiment, step S300 includes the steps of:
step S311, scheduling a video decoding task in the video analysis task to the first computing module for decoding, to obtain decoded video data;
step S312, scheduling a calculation task in the video analysis task to the second calculation module, and sending the decoded video data to the second calculation module for analysis and detection, and outputting an analysis result of the video analysis task.
In this embodiment, for the first scheduling policy, the control module controls the computing module a to complete video decoding, and after obtaining decoded video data, the control module sends the video data to the computing module B through a communication channel between the computing modules, and the computing module B completes video analysis and detection to obtain a final video analysis result.
Specifically, in one implementation of the present embodiment, step S300 further includes the following steps:
step S321, a video decoding task in the video analysis task is scheduled to the third computing module for decoding, and decoded video data are obtained;
step S322, scheduling a calculation task in the video analysis task to the fourth calculation module, sending the decoded video data to the fourth calculation module for analysis and detection, and outputting an analysis result of the video analysis task.
In this embodiment, for the second scheduling policy, the control module controls the third computing module to complete video decoding, and after obtaining decoded video data, the control module sends the video data to the fourth computing module through a communication channel between the computing modules, and the fourth computing module completes video analysis and detection to obtain a final video analysis result.
In this embodiment, for the third scheduling policy, the control module may control the computing module that satisfies the conditions at the same time to directly execute the video decoding task and the video analysis task, or the computing module with the optimal control capability directly executes the video decoding task and the video analysis task.
In this embodiment, in the execution of the first scheduling policy or the second scheduling policy, the decoded video data is data in an original YUV format or an RGB format, or the decoded video data is data meeting a size required by an analysis algorithm, or the decoded video data is compressed image data for recompression of H264/H265/JPGE.
Taking the first scheduling policy as an example, as shown in fig. 7, the transmission of video data between the computing module a and the computing module B may use the original image YUV or RGB data (including image data in other formats such as RGBA and ARGB) after video decoding, or may compress the original image YUV or RGB data to the size required by the algorithm such as 416×416 or 224×224 according to the requirement of the video analysis algorithm, and then transmit the compressed video data through a network, so as to reduce the amount of transmission data. The video data sent from the computing module A to the computing module B can be transmitted according to the frame rate required by the video analysis algorithm, and each frame of data is not required to be sent.
As shown in fig. 8, video data may be transmitted between the computing module a and the computing module B, or video compression may be performed on the computing module a according to a coding format of the computing module B meeting the capability requirement according to the coding/decoding capability conditions of multiple video/image formats such as H264, H265, JPEG, etc. of the two computing modules, that is, the transmitted video image data is compressed by the computing module a and then transmitted to the computing module B. And the computing module B decompresses the received video/image data after receiving the video/image data and then performs video analysis and computation. When the computing module a compresses the video image data, the video image data can be compressed to a size of 416×416 or 224×224 and the like according to the requirement of a video analysis algorithm and the frame rate required by the algorithm.
As shown in fig. 9, for the second scheduling policy, the video coding compression capability parameters supported by each video acquisition device may be uniformly configured on the control module by the parameters.
As shown in fig. 10, for the second scheduling policy, the video coding compression capability parameters supported by each video capturing device may also be actively reported to the control module by each video capturing device.
As shown in fig. 11, for the second scheduling policy, the control module may set video data of a certain video capture device to a specified encoding compression format through a protocol. The communication protocol can refer to standard protocols such as GB/T-28181 or digital retina or carry out corresponding expansion support.
Specifically, in one implementation of the present embodiment, step S300 further includes the following steps:
step S331, determining a degradation scheduling strategy according to the lowest processing frame rate and the lowest calculation power requirement in the task parameters;
step S332, gradually reducing the processing frame rate and the computational power parameter requirements according to the degradation scheduling policy until the task scheduling is successful or the minimum processing frame rate and the minimum computational power requirements are reached.
In another implementation manner of the embodiment of the present invention, for the foregoing video analysis task, if configured as a video analysis task that can be scheduled in a downgrade manner, parameters of the task may be included in the parameters of the task, such as a normal processing frame rate, a normal computing power requirement, a minimum processing frame rate, a minimum computing power requirement, and the like. Under the condition that the task normally processes the frame rate and the normal calculation power requires parameters, if the task scheduling fails, the degradation scheduling can be gradually performed from high to low especially under the condition that the calculation power cannot meet the requirements. And (3) carrying out task scheduling again by gradually reducing the processing frame rate and calculation power parameter requirements until the task scheduling is successful or the lowest processing frame rate and calculation power requirements of the task are reached.
The present embodiment is directed to a specific video analysis task, if none of the computing modules in the system can meet the requirement of video decoding capability, but none of the computing modules can meet the requirement of computing power. The video data coding format sent by the video acquisition equipment can be set by controlling the accessed video acquisition equipment such as a camera through a certain communication protocol, and is modified into a video decoding capacity format available for the system, so that the scheduling of video analysis tasks is finally completed, and the hardware requirement of video decoding on the whole system is reduced by a protocol modification mode.
In an implementation manner of the embodiment of the present invention, the method for scheduling video analysis tasks of a heterogeneous application system further includes the following steps:
step S400, receiving batch video analysis tasks, and obtaining calculation force parameters and video decoding parameters required by each video analysis task;
step S500, acquiring calculation force parameters and video coding and decoding parameters of each calculation module, and acquiring video coding compression parameters supported by each video acquisition device;
step S600, comprehensively calculating optimal scheduling strategy parameters according to calculation power parameters and video decoding parameters required by each video analysis task, calculation power parameters and video coding and decoding parameters of each calculation module and video coding and decoding compression parameters supported by each video acquisition device;
and step S700, scheduling a corresponding calculation module according to the optimal scheduling policy parameters, and executing video decoding tasks and calculation tasks of the batch video analysis tasks.
In this embodiment, for the optimal policy calculation during batch scheduling, the video decoding capability and calculation power requirements required by all video analysis tasks can be evaluated and calculated, and the video acquisition device is preferentially controlled to switch to the video decoding capability supported by the system according to the maximum video decoding capability situation supported by the heterogeneous system. And secondly, according to the demands of computing power and video decoding capacity, a video analysis task is scheduled to the same computing module as much as possible. And finally, completing the scheduling of a video analysis task by a plurality of computing modules.
According to the embodiment, the optimal scheduling strategy parameters can be comprehensively calculated aiming at batch video analysis tasks, scheduling of the video analysis tasks is finished to the greatest extent, and consumption of system resources is reduced.
The following describes a single video analysis task and a batch of video analysis tasks through actual application scenes respectively.
As shown in fig. 6, in a single video analysis task scenario, the following steps are included:
step S11, acquiring parameters such as calculation power, video decoding capability and the like required by a video analysis task;
step S12, judging whether a computing module meeting the requirements of computing power and video decoding capability exists; if yes, executing step S13; if not, executing step S15;
step S13, dispatching the video analysis task to a computing module meeting the requirements of computing power and video decoding capability for execution;
step S14, the task scheduling is successful.
The above steps S13 to S14 are the third scheduling policy described in the embodiment.
Step S15, judging whether a computing module meeting the computing power requirement and a computing module meeting the video decoding capability requirement exist; if yes, go to step S16; if not, executing step S18;
step S16, the video decoding function is scheduled to be executed on a computing module A meeting the video decoding capability requirement, and the video analysis computing task is scheduled to be executed on a computing module B meeting the computing capability requirement; the computing module A sends the video data to the computing module B;
Step S17, the scheduling task is successful.
The steps S15 to S17 are the first scheduling policy described in the embodiment.
Step S18, judging whether a computing module meeting the calculation force requirement exists; if yes, go to step S20; if not, executing step S19;
step S19, the task scheduling fails.
Step S20, the video data format of the video acquisition equipment is controlled to be switched into a video decoding capacity format which meets the current requirement of the system through a protocol;
step S21, whether the video format of the video acquisition equipment is successfully switched or not; if yes, returning to the step S12; if not, executing step S22;
step S22, the task scheduling fails.
The steps S20 to S21 are the second scheduling policy described in the embodiment.
As shown in fig. 12, in the batch video analysis task scenario, the following steps are included:
step S31, receiving a batch video analysis task;
step S32, obtaining parameters such as calculation power, video decoding capability and the like required by each video analysis task;
step S33, obtaining the computing power and video coding and decoding capability resource conditions of each computing module in the current system;
step S34, obtaining video coding compression capacity parameters supported by each video acquisition device;
Step S35, comprehensively calculating optimal scheduling strategy parameters;
and step S36, completing control and function scheduling of each calculation module and video acquisition equipment.
The following technical effects are achieved through the technical scheme:
according to the embodiment, by acquiring the calculation power parameters and the video decoding parameters required by the video analysis task, the scheduling strategy of the video analysis task and the corresponding computing modules can be determined according to the required calculation power parameters and the video decoding parameters as well as the calculation power parameters and the video encoding and decoding parameters of each computing module, so that the video decoding task and the computing task in the video analysis task are scheduled to the corresponding computing modules according to the scheduling strategy to execute the corresponding tasks; according to the embodiment, the video data communication transmission among the multiple computing modules in the heterogeneous application system is utilized, the multiple computing modules are scheduled to complete the video analysis task, the maximum utilization of computing power and video coding and decoding capacity of the heterogeneous application system is realized, the processing path number of the video analysis task supported by the system is improved, and the video analysis efficiency is improved.
Exemplary apparatus
Based on the above embodiment, the present invention further provides a task scheduling device for video analysis of a heterogeneous application system, including:
Receiving batch video analysis tasks, and acquiring calculation force parameters and video decoding parameters required by each video analysis task;
acquiring calculation force parameters and video coding and decoding parameters of each calculation module, and acquiring video coding compression parameters supported by each video acquisition device;
comprehensively calculating optimal scheduling strategy parameters according to calculation power parameters and video decoding parameters required by each video analysis task, calculation power parameters and video coding and decoding parameters of each calculation module and video coding and compression parameters supported by each video acquisition device;
and scheduling a corresponding calculation module according to the optimal scheduling policy parameters, and executing video decoding tasks and calculation tasks of the batch video analysis tasks.
Based on the above embodiment, the present invention also provides a terminal, and a functional block diagram thereof may be shown in fig. 13.
The terminal comprises: the system comprises a processor, a memory, an interface, a display screen and a communication module which are connected through a system bus; wherein the processor of the terminal is configured to provide computing and control capabilities; the memory of the terminal comprises a storage medium and an internal memory; the storage medium stores an operating system and a computer program; the internal memory provides an environment for the operation of the operating system and computer programs in the storage medium; the interface is used for connecting external equipment such as mobile terminals, computers and other equipment; the display screen is used for displaying corresponding information; the communication module is used for communicating with a cloud server or a mobile terminal.
The computer program, when executed by the processor, is operative to implement a method for scheduling video analytics tasks for heterogeneous applications.
It will be appreciated by those skilled in the art that the functional block diagram shown in fig. 13 is merely a block diagram of some of the structures associated with the present inventive arrangements and is not limiting of the terminal to which the present inventive arrangements may be applied, and that a particular terminal may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.
In one embodiment, a terminal is provided, including: the system comprises a processor and a memory, wherein the memory stores a heterogeneous application system video analysis task scheduler, and the heterogeneous application system video analysis task scheduler is used for realizing the operation of the heterogeneous application system video analysis task scheduling method when being executed by the processor.
In one embodiment, a storage medium is provided, wherein the storage medium stores a heterogeneous application system video analytics task scheduler that when executed by a processor is configured to implement the operations of the heterogeneous application system video analytics task scheduling method as described above.
Those skilled in the art will appreciate that implementing all or part of the above-described methods may be accomplished by way of a computer program comprising instructions for the relevant hardware, the computer program being stored on a non-volatile storage medium, the computer program when executed comprising the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include non-volatile and/or volatile memory.
In summary, the invention provides a method, a device, a terminal and a medium for scheduling video analysis tasks of a heterogeneous application system, wherein the method comprises the following steps: acquiring calculation force parameters and video decoding parameters required by a video analysis task; determining a scheduling strategy of the video analysis task and a corresponding computing module according to the required computing power parameter and video decoding parameter and the computing power parameter and video encoding and decoding parameter of each computing module; and scheduling the video decoding task and the computing task in the video analysis task to the corresponding computing module according to the scheduling strategy so as to execute the corresponding task. According to the invention, through video data communication transmission among the multiple computing modules in the heterogeneous application system, the multiple computing modules are scheduled to complete video analysis tasks, so that the maximum utilization of computing power and video coding and decoding capacity of the heterogeneous application system is realized, the processing path number of the video analysis tasks supported by the system is improved, and the video analysis efficiency is improved.
It is to be understood that the invention is not limited in its application to the examples described above, but is capable of modification and variation in light of the above teachings by those skilled in the art, and that all such modifications and variations are intended to be included within the scope of the appended claims.
Claims (14)
1. The method for scheduling the video analysis tasks of the heterogeneous application system is characterized by comprising the following steps of:
acquiring calculation force parameters and video decoding parameters required by a video analysis task;
determining a scheduling strategy of the video analysis task and a corresponding computing module according to the required computing power parameter and video decoding parameter and the computing power parameter and video encoding and decoding parameter of each computing module;
and scheduling the video decoding task and the computing task in the video analysis task to the corresponding computing module according to the scheduling strategy so as to execute the corresponding task.
2. The method for scheduling video analysis tasks of a heterogeneous application system according to claim 1, wherein the acquiring the computational power parameters and the video decoding parameters required for the video analysis tasks previously comprises:
and configuring the calculation force parameters and the video coding and decoding parameters of each calculation module according to the preset parameters.
3. The method for scheduling video analysis tasks of claim 1, wherein the acquiring the computational power parameters and the video decoding parameters required for the video analysis tasks further comprises:
And acquiring the calculation force parameters and the video coding and decoding parameters reported by each calculation module.
4. The method for scheduling video analysis tasks of a heterogeneous application system according to claim 1, wherein the obtaining the computational power parameters and the video decoding parameters required for the video analysis tasks comprises:
acquiring video data information acquired by each video acquisition device, and configuring and obtaining the video analysis task according to the video data information;
and analyzing the calculation force parameters and the video decoding parameters required by the video analysis task.
5. The method for scheduling video analysis tasks of heterogeneous application system according to claim 4, wherein determining the scheduling policy of the video analysis tasks and the corresponding computing modules according to the required computing power parameter and video decoding parameter and the computing power parameter and video encoding and decoding parameter of each computing module comprises:
according to the required calculation power parameters and video decoding parameters and the calculation power parameters and video coding and decoding parameters of each calculation module, judging whether each calculation module meets the video decoding requirement and the calculation requirement at the same time;
if each computing module cannot meet the video decoding requirement and the computing requirement at the same time, determining a first computing module meeting the video decoding requirement and a second computing module meeting the computing requirement.
6. The method for scheduling video analysis tasks of heterogeneous application system according to claim 5, wherein the scheduling video decoding tasks and computing tasks in the video analysis tasks to corresponding computing modules according to the scheduling policy comprises:
scheduling a video decoding task in the video analysis task to the first computing module for decoding to obtain decoded video data;
and dispatching the calculation task in the video analysis task to the second calculation module, sending the decoded video data to the second calculation module for analysis and detection, and outputting the analysis result of the video analysis task.
7. The method for scheduling video analysis tasks of heterogeneous application system according to claim 4, wherein determining the scheduling policy of the video analysis tasks and the corresponding computing modules according to the required computing power parameter and video decoding parameter and the computing power parameter and video encoding and decoding parameter of each computing module further comprises:
according to the required calculation power parameters and video decoding parameters and the calculation power parameters and video coding and decoding parameters of each calculation module, judging whether each calculation module meets the video decoding requirements or not in sequence;
If each computing module cannot meet the video decoding requirement, converting the video data compression format into a preset compression format through a corresponding video acquisition device controlled by a protocol;
a third calculation module for determining that the decoding requirement of the video data in the preset format is met;
a fourth calculation module is determined that meets the calculation requirements.
8. The method for scheduling video analysis tasks of heterogeneous application system according to claim 7, wherein the scheduling video decoding tasks and computing tasks in the video analysis tasks to corresponding computing modules according to the scheduling policy further comprises:
scheduling a video decoding task in the video analysis task to the third computing module for decoding to obtain decoded video data;
and dispatching the calculation task in the video analysis task to the fourth calculation module, sending the decoded video data to the fourth calculation module for analysis and detection, and outputting the analysis result of the video analysis task.
9. The method for scheduling task of video analysis of heterogeneous application system according to claim 6 or 8, wherein the decoded video data is data in original YUV format or RGB format, or the decoded video data is data meeting the size required by analysis algorithm, or the decoded video data is compressed image data of H264/H265/JPGE after recompression.
10. The method for scheduling video analysis tasks of a heterogeneous application system according to claim 1, wherein the scheduling video decoding tasks and computing tasks in the video analysis tasks to corresponding computing modules according to the scheduling policy further comprises:
determining a degradation scheduling strategy according to the lowest processing frame rate and the lowest computational power requirement in the task parameters;
and gradually reducing the processing frame rate and the calculation power parameter requirements according to the degradation scheduling strategy until the task scheduling is successful or the minimum processing frame rate and the minimum calculation power requirements are reached.
11. The heterogeneous application system video analysis task scheduling method of claim 1, further comprising:
receiving batch video analysis tasks, and acquiring calculation force parameters and video decoding parameters required by each video analysis task;
acquiring calculation force parameters and video coding and decoding parameters of each calculation module, and acquiring video coding compression parameters supported by each video acquisition device;
comprehensively calculating optimal scheduling strategy parameters according to calculation power parameters and video decoding parameters required by each video analysis task, calculation power parameters and video coding and decoding parameters of each calculation module and video coding and compression parameters supported by each video acquisition device;
And scheduling a corresponding calculation module according to the optimal scheduling policy parameters, and executing video decoding tasks and calculation tasks of the batch video analysis tasks.
12. A heterogeneous application system video analysis task scheduling device, comprising:
the task parameter acquisition module is used for acquiring calculation force parameters and video decoding parameters required by the video analysis task;
the scheduling strategy module is used for determining the scheduling strategy of the video analysis task and the corresponding calculation module according to the required calculation power parameter and video decoding parameter as well as the calculation power parameter and video encoding and decoding parameter of each calculation module;
and the task execution module is used for dispatching the video decoding task and the calculation task in the video analysis task to the corresponding calculation module according to the dispatching strategy so as to execute the corresponding task.
13. A terminal, comprising: a processor and a memory storing a heterogeneous application system video analytics task scheduler that, when executed by the processor, is configured to implement the operations of the heterogeneous application system video analytics task scheduling method of any of claims 1-11.
14. A medium, characterized in that the medium is a computer readable storage medium, the medium storing a heterogeneous application system video analysis task scheduler, the heterogeneous application system video analysis task scheduler when executed by a processor being configured to implement the operations of the heterogeneous application system video analysis task scheduling method according to any one of claims 1 to 11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310076808.7A CN116089043A (en) | 2023-01-13 | 2023-01-13 | Heterogeneous application system video analysis task scheduling method, device, terminal and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310076808.7A CN116089043A (en) | 2023-01-13 | 2023-01-13 | Heterogeneous application system video analysis task scheduling method, device, terminal and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116089043A true CN116089043A (en) | 2023-05-09 |
Family
ID=86206089
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310076808.7A Pending CN116089043A (en) | 2023-01-13 | 2023-01-13 | Heterogeneous application system video analysis task scheduling method, device, terminal and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116089043A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116886880A (en) * | 2023-09-08 | 2023-10-13 | 中移（杭州）信息技术有限公司 | Method, device, equipment and computer program product for adjusting surveillance video |
-
2023
- 2023-01-13 CN CN202310076808.7A patent/CN116089043A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116886880A (en) * | 2023-09-08 | 2023-10-13 | 中移（杭州）信息技术有限公司 | Method, device, equipment and computer program product for adjusting surveillance video |
| CN116886880B (en) * | 2023-09-08 | 2023-12-26 | 中移（杭州）信息技术有限公司 | Method, device, equipment and computer program product for adjusting surveillance video |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110049324B (en) | Video encoding method, system, device, and computer-readable storage medium | |
| CN113259451B (en) | Cluster processing architecture and method for intelligent analysis of large-scale monitoring nodes | |
| CN115914649A (en) | Data transmission method and system for medical video | |
| CN116089043A (en) | Heterogeneous application system video analysis task scheduling method, device, terminal and medium | |
| CN104041016A (en) | Camera device, server device, image monitoring system, control method of image monitoring system, and control program of image monitoring system | |
| CN110708507A (en) | Monitoring video data transmission method and device and terminal equipment | |
| JP7429756B2 (en) | Image processing method, device, electronic device, storage medium and computer program | |
| CN112116551A (en) | Camera shielding detection method and device, electronic equipment and storage medium | |
| CN104105006A (en) | Video image processing method and system | |
| CN113299245A (en) | Method and device for adjusting local backlight of display equipment, display equipment and storage medium | |
| CN111314702A (en) | Vehicle real-time monitoring system, method and equipment based on image recognition | |
| CN114970899A (en) | Intelligent park operation and maintenance system, method, medium and electronic equipment | |
| CN114187541A (en) | Intelligent video analysis method and storage device for user-defined service scene | |
| CN114125400A (en) | Multi-channel video analysis method and device | |
| CN113660465A (en) | Image processing method, image processing device, readable medium and electronic equipment | |
| CN117492989A (en) | AI calculates power adjustable intelligent video analysis system | |
| JP2023002773A (en) | Image processing method, device, electronic equipment, storage medium, and computer program | |
| CN115086572A (en) | Image transmission method and device and electronic equipment | |
| CN116016937A (en) | Sample self-adaptive compensation method and device in video coding | |
| CN110572712A (en) | decoding method and device | |
| CN112866332A (en) | System, method and device for realizing emergency recognition and early warning based on cloud edge fusion, processor and storage medium thereof | |
| Sakaushi et al. | Edge-centric video surveillance system based on event-driven rate adaptation for 24-hour monitoring | |
| CN113115072A (en) | Video target detection tracking scheduling method and system based on end cloud cooperation | |
| CN111953986A (en) | Image processing method and device | |
| CN113706573B (en) | Method and device for detecting moving object and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |