CN114697705B - Video stream object processing method and device, video stream processing system and electronic equipment - Google Patents
Video stream object processing method and device, video stream processing system and electronic equipment Download PDFInfo
- Publication number
- CN114697705B CN114697705B CN202011596589.8A CN202011596589A CN114697705B CN 114697705 B CN114697705 B CN 114697705B CN 202011596589 A CN202011596589 A CN 202011596589A CN 114697705 B CN114697705 B CN 114697705B
- Authority
- CN
- China
- Prior art keywords
- task
- processing
- list
- processing object
- video stream
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 18
- 238000000034 method Methods 0.000 claims abstract description 37
- 238000000605 extraction Methods 0.000 claims description 23
- 238000004590 computer program Methods 0.000 claims description 11
- 238000009826 distribution Methods 0.000 claims description 8
- 238000000354 decomposition reaction Methods 0.000 claims description 7
- 239000000284 extract Substances 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 14
- 238000004458 analytical method Methods 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000009827 uniform distribution Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000000523 sample Substances 0.000 description 1
- 230000000391 smoking effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/239—Interfacing the upstream path of the transmission network, e.g. prioritizing client content requests
- H04N21/2393—Interfacing the upstream path of the transmission network, e.g. prioritizing client content requests involving handling client requests
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/262—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists
- H04N21/26258—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists for generating a list of items to be played back in a given order, e.g. playlist, or scheduling item distribution according to such list
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Television Signal Processing For Recording (AREA)
Abstract
The embodiment of the invention provides a video stream object processing method, a device, a video stream processing system, electronic equipment and a storage medium, wherein the method comprises the following steps: the method comprises the steps of obtaining a video stream object, decomposing a processing task of the video stream object to obtain a first task processing object, a second task processing object and a third task processing object, wherein the first task processing object comprises a first processing time, the second task processing object comprises a second processing time and the third task processing object comprises a third processing time; according to the first processing time, the second processing time and the third processing time, the first task processing object, the second task processing object and the third task processing object are respectively added into a task list; and distributing the first task processing object, the second task processing object and the third task processing object based on the task list. The embodiment of the invention can improve the processing speed of the video stream data, thereby reducing the processing time of the video stream data.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a method and apparatus for processing a video stream object, a video stream processing system, an electronic device, and a storage medium.
Background
With the development of artificial intelligence technology, object intelligent analysis and recognition technology based on video stream data such as faces, human bodies, vehicles and the like is increasingly widely applied to various fields, such as scenes of behavior analysis (smoking at gas stations, making calls, people wearing no masks, vendors swaying to occupy channels and the like), vehicle analysis (long-time illegal parking) and the like.
In the process of analyzing based on video stream data, a plurality of bottlenecks exist, firstly, because of the limitation of network bandwidth, the intelligent analysis and identification equipment cannot bear a large number of video stream access and analysis processes at the same time; secondly, as the video coding and decoding task itself consumes very much resources, the processing speed of video stream data is slower due to the use of single server resources, single process or single thread processing mode, and a large number of video processing tasks have higher requirements on task scheduling, but usually, a task object is traversed after the task is scanned regularly by using a single thread, and the task reaching time is processed.
Disclosure of Invention
The embodiment of the invention provides a video stream object processing method which can improve the processing speed of video stream data and further reduce the processing time of the video stream data.
In a first aspect, an embodiment of the present invention provides a method for processing a video stream object, including the following steps:
the method comprises the steps of obtaining a video stream object, decomposing a processing task of the video stream object to obtain a first task processing object, a second task processing object and a third task processing object, wherein the first task processing object comprises a first processing time, the second task processing object comprises a second processing time and the third task processing object comprises a third processing time;
according to the first processing time, the second processing time and the third processing time, the first task processing object, the second task processing object and the third task processing object are respectively added into a task list;
and distributing the first task processing object, the second task processing object and the third task processing object based on the task list.
Optionally, the task list includes a scheduling time axis, a first sub-list and a second sub-list, and the first sub-list is located before the second sub-list on the scheduling time axis.
Optionally, the adding the first task processing object, the second task processing object, and the third task processing object to the task list according to the first processing time, the second processing time, and the third processing time includes:
if the first processing time, the second processing time and the third processing time are smaller than a preset time threshold, adding the corresponding task processing object to the first sub-list;
and if the first processing time, the second processing time and the third processing time are greater than a preset time threshold, adding the corresponding task processing object to the second sub-list.
Optionally, the second sub-list includes at least one list, and if one of the first processing time, the second processing time, and the third processing time is greater than a preset time threshold, adding the corresponding task processing object to the second sub-list includes:
acquiring a difference value between the processing time and the preset time threshold;
and if the difference value is greater than zero, adding the corresponding task processing object to the list according to the difference value.
Optionally, the distributing the first task processing object, the second task processing object, and the third task processing object based on the task list includes:
Acquiring a task processing object with the minimum processing time from the second sub-list, and transferring the task processing object to the first sub-list;
acquiring a task processing object with the minimum processing time from the first sub-list, and distributing the task processing object to a task queue to be processed;
repeating the steps until the last task processing object in the second sub-list.
Optionally, the obtaining the task processing object with the smallest processing time from the second sub-list and transferring the task processing object to the first sub-list includes:
traversing each list of the second sub-list to obtain a task processing object with the minimum processing time in each list;
and acquiring the one with the minimum processing time from the obtained task processing object and transferring the one to the first sub-list.
In a second aspect, an embodiment of the present invention provides a video stream object processing apparatus, including:
the system comprises an acquisition and decomposition module, a processing module and a processing module, wherein the acquisition and decomposition module is used for acquiring a video stream object and decomposing a processing task of the video stream object to obtain a first task processing object, a second task processing object and a third task processing object, the first task processing object comprises a first processing time, the second task processing object comprises a second processing time and the third task processing object comprises a third processing time;
The adding module is used for adding the first task processing object, the second task processing object and the third task processing object to a task list respectively according to the first processing time, the second processing time and the third processing time;
and the distribution module is used for distributing the first task processing object, the second task processing object and the third task processing object based on the task list.
In a third aspect, an embodiment of the present invention provides a video stream processing system, including: the video stream processing device executes the video stream object processing method, acquires a video stream object from the video stream access module and decomposes a video stream object processing task, and then distributes the video stream processing task to the video stream frame extraction module; and the video stream frame extraction module reads, decapsulates and decodes the video stream data according to the video stream processing task, extracts frames after decoding, and inputs the obtained image frames to the image processing module for corresponding image processing.
In a fourth aspect, an embodiment of the present invention provides an electronic device, including: the video stream object processing method comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the steps in the video stream object processing method provided by the embodiment of the invention are realized when the processor executes the computer program.
In a fifth aspect, an embodiment of the present invention provides a computer readable storage medium, where a computer program is stored, where the computer program when executed by a processor implements steps in a video stream object processing method provided by an embodiment of the present invention.
In the embodiment of the invention, a video stream object is obtained, and processing tasks of the video stream object are decomposed to obtain a first task processing object, a second task processing object and a third task processing object, wherein the first task processing object comprises a first processing time, the second task processing object comprises a second processing time and the third task processing object comprises a third processing time; according to the first processing time, the second processing time and the third processing time, the first task processing object, the second task processing object and the third task processing object are respectively added into a task list; and distributing the first task processing object, the second task processing object and the third task processing object based on the task list. The processing tasks of the video stream objects are decomposed to obtain corresponding task processing objects, and then the task processing objects are added to a task list according to the processing time of the task processing objects for uniform distribution scheduling, so that the processing speed of the video stream data can be improved, and the processing time of the video stream data is reduced.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of a video stream object processing method according to an embodiment of the present invention;
FIG. 2 is a flow chart of a task adding method provided by an embodiment of the present invention;
FIG. 3 is a flow chart of another task adding method provided by an embodiment of the present invention;
FIG. 4 is a flowchart of a task processing object distribution method provided by an embodiment of the present invention;
FIG. 5 is a flowchart of a task processing object dump method according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a video stream object processing apparatus according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of a task adding device according to an embodiment of the present invention;
FIG. 8 is a schematic structural diagram of another task adding device according to an embodiment of the present invention;
Fig. 9 is a schematic structural diagram of a task processing object distribution device according to an embodiment of the present invention;
fig. 10 is a schematic structural diagram of an acquisition and dump module according to an embodiment of the present invention;
fig. 11 is a schematic structural diagram of a video stream processing system according to an embodiment of the present invention;
fig. 12 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used in the description of the applications herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having" and any variations thereof in the description and claims of the present application and in the description of the figures above are intended to cover non-exclusive inclusions. The terms first, second and the like in the description and in the claims or in the above-described figures, are used for distinguishing between different objects and not necessarily for describing a sequential or chronological order.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the present application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 1, fig. 1 is a flowchart of a video stream object processing method according to an embodiment of the present invention, as shown in fig. 1, including the following steps:
101. the method comprises the steps of obtaining a video stream object, and decomposing processing tasks of the video stream object to obtain a first task processing object, a second task processing object and a third task processing object.
The first task processing object includes a first processing time, the second task processing object includes a second processing time, and the third task processing object includes a third processing time.
In the embodiment of the present invention, the video stream object may be corresponding video information obtained from video stream data, for example, the video stream data may be subjected to preliminary analysis, and information such as a size, a storage location, an encoding format, etc. of the video stream data may be obtained as the video stream object. The video stream data can be obtained through equipment with a camera shooting function, such as a traffic probe, a computer camera, a mobile phone camera and the like, automatically obtained in real time, or obtained from a network through other visual interfaces such as a GUI (graphical user interface) or a WEB page, a command line and the like, and then stored in a local disk or a memory. And then the video stream data can be obtained from the local and analyzed to obtain the video stream object, and then the frame extraction processing is carried out on the video stream data corresponding to the video stream object according to the service frame extraction requirement.
Specifically, the frame extraction processing process comprises three steps of decapsulation, decoding and frame extraction, wherein the frame extraction processing process can be decomposed in a task processing mode to obtain a task processing object corresponding to each step, namely a first task processing object corresponding to the decapsulation, a second task processing object corresponding to the decoding and a third task processing object corresponding to the frame extraction; and then, independently and parallelly reading video stream data in a multi-process or multi-thread mode according to each task processing object and carrying out corresponding processing, namely, respectively starting a corresponding process or thread for each task processing object to asynchronously carry out data processing.
The first processing time, the second processing time, and the third processing time are used for identifying the emergency degree of the corresponding task processing, and if the processing time is smaller, the corresponding task processing is indicated to be more emergency, and the corresponding task processing object is scheduled to execute preferentially. For example, when one video stream data is larger, the data may be divided into multiple segments and then frame extraction processing is performed on each segment of data, so that each segment of video stream data has a corresponding first task processing object, a corresponding second task processing object, and a corresponding third task processing object, where the frame extraction process of the whole video stream data may generate a large number of task processing objects, including a plurality of first processing times, a second processing time, and a third processing time, where the task processing object with the smallest processing time is preferentially scheduled and executed, and corresponding data processing is performed, and if the second processing time of the second task processing object of a certain segment of data is the smallest in all processing times, the decoding thread is scheduled according to information in the second task processing object to read the data to be decoded and then perform data decoding, so as to obtain decoded data.
The frame extraction processing process of the video stream object is subjected to task decomposition, and each task is asynchronously and concurrently processed through multiple processes or multiple threads, so that the processing speed of video stream data can be improved, and the processing time of the video stream data can be reduced.
102. And respectively adding the first task processing object, the second task processing object and the third task processing object to a task list according to the first processing time, the second processing time and the third processing time.
In the embodiment of the present invention, the task list includes a scheduling time axis, a first sub-list and a second sub-list, where the scheduling time axis includes at least one routine processing time T, where the routine processing time T is used to represent a scheduling time period of the task processing object, that is, scheduling the task processing object once every other routine processing time T on the scheduling time axis, that is, a task processing object to be scheduled is preferentially acquired from the first sub-list, and a new task processing object is acquired from the decomposition of a video stream data processing task in the step 101 and then added to the first sub-list or the second sub-list, so as to implement scheduling all processing tasks of the video stream data periodically. The first sub-list is used for storing the task processing objects which are more urgent in processing tasks, the second sub-list is used for storing the task processing objects which are less urgent in processing tasks, the first sub-list is positioned before the second sub-list on the scheduling time axis, the task processing objects in the first sub-list are scheduled before the task processing objects in the second sub-list, and all the task processing objects in the second sub-list can be further urgently distinguished on the scheduling time axis according to the corresponding processing time, and the task processing objects which are less urgent are more urgent are spaced by the routine processing time T on the scheduling time axis.
Optionally, referring to fig. 2, fig. 2 is a flowchart of a task adding method provided by an embodiment of the present invention, as shown in fig. 2, adding, according to a first processing time, a second processing time, and a third processing time, a first task processing object, a second task processing object, and a third task processing object to a task list respectively includes:
201. and if the first processing time, the second processing time and the third processing time are smaller than the preset time threshold, adding the corresponding task processing object into the first sub-list.
In the embodiment of the present invention, the first sub-list is used to store the urgent task processing object in the video stream data processing task, and may be any one or more of the first task processing object, the second task processing object, and the third task processing object, specifically, the processing time in each task processing object may be compared with the preset time threshold t, for example, if there is a first processing time, a second processing time, and a third processing time in the task processing object that is less than the preset time threshold t, such as 0.5t, the first task processing object, the second task processing object, and the third task processing object are correspondingly added to the first sub-list, and specifically, the data fields in the linked list nodes of the first sub-list may be sequentially inserted.
202. And if the first processing time, the second processing time and the third processing time are greater than a preset time threshold, adding the corresponding task processing object into the second sub-list.
In the embodiment of the present invention, the second sub-list is used to store relatively non-urgent task processing objects in the video stream data processing task, and may be any one or more of the first task processing object, the second task processing object, and the third task processing object, specifically, the processing time in each task processing object may be compared with the preset time threshold t, for example, if any one of the first processing time, the second processing time, and the third processing time in the task processing object is greater than (including equal to) the preset time threshold t, such as t, 2t, and so on, the first task processing object, the second task processing object, and the third task processing object are correspondingly added to the second sub-list, and specifically, may be sequentially inserted into the data fields in the linked list nodes of the second sub-list.
Optionally, the second sub-list includes at least one list, referring to fig. 3, fig. 3 is a flowchart of another task adding method provided in an embodiment of the present invention, and as shown in fig. 3, if one of the first processing time, the second processing time, and the third processing time is greater than a preset time threshold, adding the corresponding task processing object to the second sub-list includes:
301. And obtaining a difference value between the processing time and a preset time threshold.
In the embodiment of the present invention, the processing time of each of the first task processing object, the second task processing object, and the third task processing object, that is, the first processing time, the second processing time, and the third processing time, may be compared with the preset time threshold, and the corresponding difference may be obtained.
302. And if the difference value is greater than zero, adding the corresponding task processing object to the list according to the difference value.
In this embodiment of the present invention, a difference between the processing time of the first task processing object, the second task processing object, and the third task processing object and the preset time threshold may be obtained in step 301, and if the difference is greater than zero, a multiple of the corresponding processing time and the preset time threshold may be further calculated, and the multiple may be used as an offset to offset the corresponding task processing object to the list corresponding to the second sub-list.
Specifically, the above list may be implemented by a data structure of a linked list, where the linked list is a non-continuous, non-sequential storage structure on a physical storage unit, and the linked list is formed by a series of nodes, where each node includes two parts: one is a data field storing data, and the other is a pointer field storing the address of the next node; nodes can be dynamically generated as needed at runtime and added to the tail of the linked list. The access logic of the linked list node is realized through pointers in the linked list pointer domain, namely, other nodes are accessed through addresses pointed by the pointers. The first sub-list and the second sub-list may include one or more of the linked lists (i.e., the list). For example, the second sub-list may include 3 linked lists, a linked list head of each linked list (i.e. an entry node of the corresponding linked list) may be recorded in the second sub-list and the order of the linked lists, i.e. the 1 st linked list, the 2 nd linked list and the 3 rd linked list may be marked, and the first sub-list may also operate as such; and then storing the task processing objects with the processing time multiple smaller than 1 with the preset time threshold value into a 1 st linked list of the first sub-list, storing the task processing objects with the processing time multiple of 1 to 2 (including 1) into a 1 st linked list of the second sub-list, storing the task processing objects with the processing time multiple of 2 to 3 into a 2 nd linked list of the second sub-list, storing the task processing objects with the processing time multiple of 3 to 4 into a 3 rd linked list of the second sub-list, and so on, so that the second sub-list can arrange all the task processing objects in the sub-list on the scheduling time axis according to the corresponding processing time, and the 1 st linked list to the 3 rd linked list of the second sub-list are separated by two preset time thresholds T on the scheduling time axis, wherein the preset time threshold T is not smaller than the routine processing time T, such as t=2t.
103. And distributing the first task processing object, the second task processing object and the third task processing object based on the task list.
In the embodiment of the invention, the second sub-list of the task list can be traversed, the task processing object with the minimum processing time is obtained from the second sub-list, and the task processing object is transferred to the first sub-list; then, acquiring a task processing object with the minimum processing time from the first sub-list, and distributing the task processing object to a task queue to be processed, wherein the task queue to be processed comprises task processing objects which are orderly arranged from small to large according to time, and the task processing object is about to be processed with the minimum time; and finally repeating the steps until the last task processing object in the second sub-list is distributed to the task queue to be processed, and finishing all task scheduling of the video stream data processing.
Optionally, referring to fig. 4, fig. 4 is a flowchart of a task processing object distribution method provided by an embodiment of the present invention, and as shown in fig. 4, the distributing the first task processing object, the second task processing object, and the third task processing object based on the task list includes:
401. And acquiring the task processing object with the minimum processing time from the second sub-list, and transferring the task processing object to the first sub-list.
In the embodiment of the invention, the second sub-list of the task list can be traversed, and the task processing object with the minimum processing time is acquired from the second sub-list and then transferred to the first sub-list;
optionally, referring to fig. 5, fig. 5 is a flowchart of a task processing object transferring method provided by an embodiment of the present invention, as shown in fig. 5, the obtaining, from the second sub-list, a task processing object with a minimum processing time, and transferring the task processing object to the first sub-list includes:
501. traversing each list of the second sub-list to obtain a task processing object with the minimum processing time in each list;
502. and acquiring the one with the minimum processing time from the obtained task processing object and transferring the one to the first sub-list.
In this embodiment of the present invention, the task processing objects may be obtained from the data fields of each node in the list by traversing the list (i.e., the linked list) in the second sub-list in turn, and then sorting the processing times corresponding to all the task processing objects, and transferring the task processing object with the smallest processing time to the linked list in the first sub-list.
402. And acquiring a task processing object with the minimum processing time from the first sub-list, and distributing the task processing object to a task queue to be processed.
In the embodiment of the invention, the linked list in the first sub-list can be traversed, the data field of each node is obtained from the linked list, then the corresponding task processing object is obtained, the processing time of all the task processing objects is ordered, the task processing object with the smallest processing time is transferred to the task queue to be processed, and the task queue to be processed comprises task processing objects which are ordered from small to large according to time; the task queue to be processed can be specifically realized through a data structure of a FIFO (first in first out), namely, the task processing object which is firstly entered into the queue is firstly taken out, and then the corresponding thread is called for data processing after analysis, for example, after the decoding task processing object is analyzed, the information such as the storage position, the size, the coding mode and the like of the data to be decoded is obtained, and thus, the corresponding decoding thread is called for decoding the video stream data according to the information.
403. Repeating the steps until the last task processing object in the second sub-list.
In the embodiment of the present invention, the steps 401 and 402 are repeated to transfer and distribute the task processing objects of the first sub-list and the second sub-list in the task list until the last task processing object in the second sub-list is transferred to the first sub-list, which indicates that all task scheduling of the video stream data processing is finished and then distributed to the task queue to be processed for processing, which indicates that all task scheduling of the video stream data processing is finished, and when the last task processing object in the task queue to be processed is fetched and analyzed, which indicates that the processing of the video stream data is finished.
In summary, a large number of corresponding task processing objects are obtained by decomposing the processing tasks of the video stream objects, then all the task processing objects are added to a task list according to the processing time of the task processing objects to perform unified distribution scheduling, and then each task processing object is executed asynchronously and concurrently by a corresponding thread, so that the processing speed of the video stream data is improved, and the processing time of the video stream data is reduced.
It should be noted that, the video stream object processing method provided by the embodiment of the present invention may be applied to devices such as a mobile phone, a monitor, a computer, and a server, which may perform video stream object processing.
Referring to fig. 6, fig. 6 is a schematic structural diagram of a video stream object processing apparatus according to an embodiment of the present invention, as shown in fig. 6, the apparatus 600 includes:
the acquiring and decomposing module 601 is configured to acquire a video stream object, decompose a processing task of the video stream object, and obtain a first task processing object, a second task processing object, and a third task processing object, where the first task processing object includes a first processing time, the second task processing object includes a second processing time, and the third task processing object includes a third processing time;
An adding module 602, configured to add the first task processing object, the second task processing object, and the third task processing object to a task list according to the first processing time, the second processing time, and the third processing time, respectively;
and the distributing module 603 is configured to distribute the first task processing object, the second task processing object, and the third task processing object based on the task list.
Optionally, as shown in fig. 7, fig. 7 is a schematic structural diagram of a first task adding device provided in an embodiment of the present invention, where the first task adding device 700 includes:
a first task adding module 701, configured to add a corresponding task processing object to the first sub-list if any of the first processing time, the second processing time, and the third processing time is less than a preset time threshold;
the second task adding module 702 is configured to add the corresponding task processing object to the second sub-list if any of the first processing time, the second processing time, and the third processing time is greater than a preset time threshold.
Optionally, the second sub-list includes at least one list, as shown in fig. 8, fig. 8 is a schematic structural diagram of a second task adding device provided in an embodiment of the present invention, where the second task adding device 800 includes:
An obtaining module 801, configured to obtain a difference between a processing time and a preset time threshold;
the adding module 802 is configured to add the corresponding task processing object to the list according to the difference value if the difference value is greater than zero.
Alternatively, referring to fig. 9, fig. 9 is a schematic structural diagram of a task processing object distribution device according to an embodiment of the present invention, as shown in fig. 9, the task processing object distribution device 900 includes:
the acquiring and transferring module 901 is configured to acquire a task processing object with the smallest processing time from the second sub-list, and transfer the task processing object to the first sub-list;
an acquiring and distributing module 902, configured to acquire a task processing object with the smallest processing time from the first sub-list, and distribute the task processing object to a task queue to be processed;
a repeating module 903, configured to repeat the above steps until the last task in the second sub-list processes an object.
Alternatively, referring to fig. 10, fig. 10 is a schematic structural diagram of an acquiring and transferring module according to an embodiment of the present invention, as shown in fig. 10, the acquiring and transferring module 901 includes:
an obtaining sub-module 9011, configured to traverse each list of the second sub-list, and obtain a task processing object with the minimum processing time in each list;
The transferring sub-module 9012 is configured to obtain, from the obtained task processing object, the one with the smallest processing time, and transfer the one to the first sub-list.
It should be noted that the video stream object processing device provided by the embodiment of the present invention may be applied to devices such as a mobile phone, a monitor, a computer, and a server, which may perform video stream object processing.
The video stream object processing device provided by the embodiment of the invention can realize each process realized by the video stream object processing method in the method embodiment, and can achieve the same beneficial effects. In order to avoid repetition, a description thereof is omitted.
Referring to fig. 11, fig. 11 is a schematic structural diagram of a video stream processing system according to an embodiment of the present invention, and as shown in fig. 11, the video stream processing system 1100 includes: the video stream access module 1101, the video stream object processing device 600, the video stream frame extraction module 1102 and the image processing module 1103, wherein the video stream access module 1101 is configured to acquire and cache original video stream data, the video stream object processing device 600 executes the video stream object processing method, acquires a video stream object from the video stream access module 1101, decomposes a video stream object processing task, and distributes the video stream processing task to the video stream frame extraction module 1102; the video stream frame extraction module 1102 performs frame extraction processing after reading, decapsulating and decoding on the video stream data according to the video stream processing task, and inputs the obtained image frame to the image processing module 1103 for performing corresponding image processing.
The processing tasks of the video stream data objects acquired by the video stream access module 1101 are decomposed by the video stream object processing device 600 to obtain a large number of task processing objects corresponding to the unpacking, decoding and frame extracting tasks, all the task processing objects are added to a task list according to the processing time of the task processing objects to perform uniform distribution scheduling, and then each task processing object is asynchronously and concurrently executed by a corresponding thread in the video stream frame extracting module 1102, so that a corresponding data processing process is realized, the processing speed of the video stream data is improved, and the processing time of the video stream data is reduced; finally, the image frames obtained by frame extraction in the video stream frame extraction module 1102 are input to the image processing module 1103 for image processing, so that the video stream is converted into a picture stream through the structuring process, and the occupation of bandwidth and the consumption of computing resources are effectively reduced.
Referring to fig. 12, fig. 12 is a schematic structural diagram of an electronic device according to an embodiment of the present invention, as shown in fig. 12, including: a memory 1202, a processor 1201 and a computer program stored on the memory 1202 and executable on the processor 1201, wherein:
The processor 1201 is configured to call a computer program stored in the memory 1202, and perform the following steps:
the method comprises the steps of obtaining a video stream object, decomposing a processing task of the video stream object to obtain a first task processing object, a second task processing object and a third task processing object, wherein the first task processing object comprises a first processing time, the second task processing object comprises a second processing time and the third task processing object comprises a third processing time;
according to the first processing time, the second processing time and the third processing time, the first task processing object, the second task processing object and the third task processing object are respectively added into a task list;
and distributing the first task processing object, the second task processing object and the third task processing object based on the task list.
Optionally, the adding, by the processor 1201, the first task processing object, the second task processing object, and the third task processing object to the task list according to the first processing time, the second processing time, and the third processing time includes:
if the first processing time, the second processing time and the third processing time are smaller than a preset time threshold, adding the corresponding task processing object to the first sub-list;
And if the first processing time, the second processing time and the third processing time are greater than a preset time threshold, adding the corresponding task processing object to the second sub-list.
Optionally, the second sub-list includes at least one list, and the adding, by the processor 1201, the corresponding task processing object to the second sub-list if any of the first processing time, the second processing time, and the third processing time is greater than a preset time threshold includes:
acquiring a difference value between the processing time and the preset time threshold;
and if the difference value is greater than zero, adding the corresponding task processing object to the list according to the difference value.
Optionally, the distributing, by the processor 1201, the first task processing object, the second task processing object, and the third task processing object based on the task list includes:
acquiring a task processing object with the minimum processing time from the second sub-list, and transferring the task processing object to the first sub-list;
acquiring a task processing object with the minimum processing time from the first sub-list, and distributing the task processing object to a task queue to be processed;
Repeating the steps until the last task processing object in the second sub-list.
Optionally, the acquiring, by the processor 1201, the task processing object with the smallest processing time from the second sub-list, and transferring the task processing object to the first sub-list includes:
traversing each list of the second sub-list to obtain a task processing object with the minimum processing time in each list;
and acquiring the one with the minimum processing time from the obtained task processing object and transferring the one to the first sub-list.
The electronic device may be a mobile phone, a monitor, a computer, a server, or the like, which can be used for processing a video stream object.
The electronic device provided by the embodiment of the invention can realize each process realized by the video stream object processing method in the embodiment of the method, can achieve the same beneficial effects, and is not repeated here for avoiding repetition.
The embodiment of the invention also provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor, implements each process of the video stream object processing method provided by the embodiment of the invention, and can achieve the same technical effect, so that repetition is avoided, and no further description is provided herein.
Those skilled in the art will appreciate that implementing all or part of the above-described methods in accordance with the embodiments may be accomplished by way of a computer program stored on a computer readable storage medium, which when executed may comprise the steps of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM) or the like.
The foregoing disclosure is illustrative of the present invention and is not to be construed as limiting the scope of the invention, which is defined by the appended claims.
Claims (6)
1. A method for processing a video stream object, comprising the steps of:
the method comprises the steps of obtaining a video stream object, decomposing a processing task of the video stream object to obtain a first task processing object, a second task processing object and a third task processing object, wherein the first task processing object comprises a first processing time, the second task processing object comprises a second processing time and the third task processing object comprises a third processing time, and the first task processing object, the second task processing object and the third task processing object respectively correspond to the unpacking, decoding and frame extraction of the processing task;
According to the first processing time, the second processing time and the third processing time, the first task processing object, the second task processing object and the third task processing object are respectively added into a task list;
distributing the first task processing object, the second task processing object and the third task processing object based on the task list;
the task list includes a scheduling time axis, a first sub-list and a second sub-list, the first sub-list is located before the second sub-list on the scheduling time axis, and the adding the first task processing object, the second task processing object and the third task processing object to the task list according to the first processing time, the second processing time and the third processing time includes:
if the first processing time, the second processing time and the third processing time are smaller than a preset time threshold, adding the corresponding task processing object to the first sub-list;
if the first processing time, the second processing time and the third processing time are greater than a preset time threshold, adding the corresponding task processing object to the second sub-list;
The second sub-list at least comprises a list, the list is realized by a data structure of a linked list, and is composed of a series of nodes, each node comprises a data field for storing data and a pointer field for storing the address of the next node, and if one of the first processing time, the second processing time and the third processing time is greater than a preset time threshold, adding the corresponding task processing object to the second sub-list comprises:
acquiring a difference value between the processing time and the preset time threshold;
if the difference is greater than zero, calculating the multiple of the corresponding processing time and the preset time threshold, and taking the multiple as an offset to offset the corresponding task processing object to the data field of the list corresponding to the second sub-list, wherein different multiples correspond to different lists;
the distributing the first task processing object, the second task processing object, and the third task processing object based on the task list includes:
acquiring a task processing object with the minimum processing time from the second sub-list, and transferring the task processing object to the first sub-list;
acquiring a task processing object with the minimum processing time from the first sub-list, and distributing the task processing object to a task queue to be processed;
Repeating the steps until the last task processing object in the second sub-list.
2. The method of claim 1, wherein the obtaining the task processing object with the smallest processing time from the second sub-list and transferring to the first sub-list comprises:
traversing each list of the second sub-list to obtain a task processing object with the minimum processing time in each list;
and acquiring the one with the minimum processing time from the obtained task processing object and transferring the one to the first sub-list.
3. A video stream object processing apparatus, comprising:
the system comprises an acquisition and decomposition module, a processing module and a frame extraction module, wherein the acquisition and decomposition module is used for acquiring a video stream object and decomposing a processing task of the video stream object to obtain a first task processing object, a second task processing object and a third task processing object, the first task processing object comprises a first processing time, the second task processing object comprises a second processing time and the third task processing object comprises a third processing time, and the first task processing object, the second task processing object and the third task processing object respectively correspond to the unpacking, the decoding and the frame extraction of the processing task;
The adding module is used for adding the first task processing object, the second task processing object and the third task processing object to a task list respectively according to the first processing time, the second processing time and the third processing time;
the distribution module is used for distributing the first task processing object, the second task processing object and the third task processing object based on the task list; the distributing the first task processing object, the second task processing object, and the third task processing object based on the task list includes: acquiring a task processing object with the minimum processing time from the second sub-list, and transferring the task processing object to the first sub-list; acquiring a task processing object with the minimum processing time from the first sub-list, and distributing the task processing object to a task queue to be processed; repeating the steps until the last task processing object in the second sub-list;
the task list comprises a scheduling time axis, a first sub-list and a second sub-list, the first sub-list is positioned in front of the second sub-list on the scheduling time axis, and the adding module is further used for adding a corresponding task processing object to the first sub-list if one of the first processing time, the second processing time and the third processing time is smaller than a preset time threshold value; if the first processing time, the second processing time and the third processing time are greater than a preset time threshold, adding the corresponding task processing object to the second sub-list; the second sub-list at least comprises a list, the list is realized by a data structure of a linked list and consists of a series of nodes, each node comprises a data field for storing data and a pointer field for storing the address of the next node, and the adding module is further used for obtaining the difference value between the processing time and the preset time threshold; if the difference is greater than zero, calculating the multiple of the corresponding processing time and the preset time threshold, and taking the multiple as an offset to offset the corresponding task processing object to the data field of the list corresponding to the second sub-list, wherein different positions correspond to different lists.
4. A video stream processing system, comprising: the video stream processing device comprises a video stream access module, a video stream object processing device according to claim 3, a video stream frame extraction module and an image processing module, wherein the video stream access module is used for acquiring and caching original video stream data, the video stream object processing device executes the video stream object processing method according to claim 1 or 2, acquires a video stream object from the video stream access module and carries out decomposition of a video stream object processing task, and then distributes the video stream processing task to the video stream frame extraction module; and the video stream frame extraction module reads, decapsulates and decodes the video stream data according to the video stream processing task, extracts frames after decoding, and inputs the obtained image frames to the image processing module for corresponding image processing.
5. An electronic device, comprising: memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps in the video stream object processing method according to claim 1 or 2 when the computer program is executed.
6. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of the video stream object processing method according to claim 1 or 2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011596589.8A CN114697705B (en) | 2020-12-29 | 2020-12-29 | Video stream object processing method and device, video stream processing system and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011596589.8A CN114697705B (en) | 2020-12-29 | 2020-12-29 | Video stream object processing method and device, video stream processing system and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114697705A CN114697705A (en) | 2022-07-01 |
| CN114697705B true CN114697705B (en) | 2024-03-22 |
Family
ID=82131492
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011596589.8A Active CN114697705B (en) | 2020-12-29 | 2020-12-29 | Video stream object processing method and device, video stream processing system and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114697705B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115134328B (en) * | 2022-07-22 | 2024-03-19 | 平安银行股份有限公司 | Corner mark configuration method and device, computer equipment and readable storage medium |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6182120B1 (en) * | 1997-09-30 | 2001-01-30 | International Business Machines Corporation | Method and system for scheduling queued messages based on queue delay and queue priority |
| CN101261592A (en) * | 2007-03-07 | 2008-09-10 | 国际商业机器公司 | Method and apparatus for scheduling task request |
| KR20100060408A (en) * | 2008-11-27 | 2010-06-07 | 한국전자통신연구원 | Apparatus and method for decoding video using multiprocessor |
| CN104133724A (en) * | 2014-04-03 | 2014-11-05 | 腾讯科技（深圳）有限公司 | Concurrent task scheduling method and concurrent task scheduling device |
| CN104159142A (en) * | 2014-08-06 | 2014-11-19 | 乐视网信息技术（北京）股份有限公司 | Video soft decoding method and device of equipment |
| CN104581423A (en) * | 2013-10-12 | 2015-04-29 | 北京航天长峰科技工业集团有限公司 | Comprehensive scheduling and load balancing method for intelligent massive video analysis system |
| CN106713944A (en) * | 2016-12-30 | 2017-05-24 | 北京奇虎科技有限公司 | Method and apparatus for processing streaming data task |
| CN110196775A (en) * | 2019-05-30 | 2019-09-03 | 苏州浪潮智能科技有限公司 | A kind of calculating task processing method, device, equipment and readable storage medium storing program for executing |
| CN110213636A (en) * | 2018-04-28 | 2019-09-06 | 腾讯科技（深圳）有限公司 | Video frame generating method, device, storage medium and the equipment of Online Video |
| CN111327921A (en) * | 2018-12-17 | 2020-06-23 | 深圳市炜博科技有限公司 | Video data processing method and device |
| CN111866525A (en) * | 2020-09-23 | 2020-10-30 | 腾讯科技（深圳）有限公司 | Multi-view video playing control method and device, electronic equipment and storage medium |
| CN112114973A (en) * | 2020-09-29 | 2020-12-22 | 中国银行股份有限公司 | Data processing method and device |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9826011B2 (en) * | 2014-07-31 | 2017-11-21 | Istreamplanet Co. | Method and system for coordinating stream processing at a video streaming platform |
| CN111832366B (en) * | 2019-04-22 | 2024-04-02 | 富联精密电子(天津)有限公司 | Image recognition apparatus and method |
-
2020
- 2020-12-29 CN CN202011596589.8A patent/CN114697705B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6182120B1 (en) * | 1997-09-30 | 2001-01-30 | International Business Machines Corporation | Method and system for scheduling queued messages based on queue delay and queue priority |
| CN101261592A (en) * | 2007-03-07 | 2008-09-10 | 国际商业机器公司 | Method and apparatus for scheduling task request |
| KR20100060408A (en) * | 2008-11-27 | 2010-06-07 | 한국전자통신연구원 | Apparatus and method for decoding video using multiprocessor |
| CN104581423A (en) * | 2013-10-12 | 2015-04-29 | 北京航天长峰科技工业集团有限公司 | Comprehensive scheduling and load balancing method for intelligent massive video analysis system |
| CN104133724A (en) * | 2014-04-03 | 2014-11-05 | 腾讯科技（深圳）有限公司 | Concurrent task scheduling method and concurrent task scheduling device |
| CN104159142A (en) * | 2014-08-06 | 2014-11-19 | 乐视网信息技术（北京）股份有限公司 | Video soft decoding method and device of equipment |
| CN106713944A (en) * | 2016-12-30 | 2017-05-24 | 北京奇虎科技有限公司 | Method and apparatus for processing streaming data task |
| CN110213636A (en) * | 2018-04-28 | 2019-09-06 | 腾讯科技（深圳）有限公司 | Video frame generating method, device, storage medium and the equipment of Online Video |
| CN111327921A (en) * | 2018-12-17 | 2020-06-23 | 深圳市炜博科技有限公司 | Video data processing method and device |
| CN110196775A (en) * | 2019-05-30 | 2019-09-03 | 苏州浪潮智能科技有限公司 | A kind of calculating task processing method, device, equipment and readable storage medium storing program for executing |
| CN111866525A (en) * | 2020-09-23 | 2020-10-30 | 腾讯科技（深圳）有限公司 | Multi-view video playing control method and device, electronic equipment and storage medium |
| CN112114973A (en) * | 2020-09-29 | 2020-12-22 | 中国银行股份有限公司 | Data processing method and device |
Non-Patent Citations (2)
| Title |
|---|
| 基于LLF算法的双队列TinyOS调度策略;周艳;;辽东学院学报(自然科学版)(04);全文 * |
| 基于流媒体技术的智能视频监视系统;吴宁;罗安;雷震;;自动化仪表(07);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114697705A (en) | 2022-07-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110321958B (en) | Training method of neural network model and video similarity determination method | |
| CN110349161B (en) | Image segmentation method, image segmentation device, electronic equipment and storage medium | |
| CN109934142B (en) | Method and apparatus for generating feature vectors of video | |
| US20210201501A1 (en) | Motion-based object detection method, object detection apparatus and electronic device | |
| CN110457974B (en) | Image superposition method and device, electronic equipment and readable storage medium | |
| CN114697705B (en) | Video stream object processing method and device, video stream processing system and electronic equipment | |
| CN111935663B (en) | Sensor data stream processing method, device, medium and electronic equipment | |
| CN110633434A (en) | Page caching method and device, electronic equipment and storage medium | |
| CN112235598B (en) | Video structured processing method and device and terminal equipment | |
| CN110633433A (en) | Page caching method and device, electronic equipment and storage medium | |
| CN112069950B (en) | Method, system, electronic device and medium for extracting hotwords | |
| CN111783632A (en) | Face detection method and device for video stream, electronic equipment and storage medium | |
| CN111209311B (en) | Method and device for processing data | |
| CN112637538B (en) | Smart tag method, system, medium, and terminal for optimizing video analysis | |
| CN114866706A (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| CN113936271A (en) | Text recognition method and device, readable medium and electronic equipment | |
| CN114399814A (en) | Deep learning-based obstruction removal and three-dimensional reconstruction method | |
| CN112732979A (en) | Information writing method, information writing device, electronic equipment and computer readable medium | |
| CN112686314A (en) | Target detection method and device based on long-distance shooting scene and storage medium | |
| CN112488204A (en) | Training sample generation method, image segmentation method, device, equipment and medium | |
| CN111901561A (en) | Video data processing method, device and system in monitoring system and storage medium | |
| CN112364682A (en) | Case searching method and device | |
| CN112312205A (en) | Video processing method and device, electronic equipment and computer storage medium | |
| CN115375208B (en) | Camera data analysis method and device, electronic equipment and storage medium | |
| CN113762266B (en) | Target detection method, device, electronic equipment and computer readable medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |