CN115514970A - Image frame pushing method and system, electronic equipment and readable storage medium - Google Patents
Image frame pushing method and system, electronic equipment and readable storage medium Download PDFInfo
- Publication number
- CN115514970A CN115514970A CN202211339277.8A CN202211339277A CN115514970A CN 115514970 A CN115514970 A CN 115514970A CN 202211339277 A CN202211339277 A CN 202211339277A CN 115514970 A CN115514970 A CN 115514970A
- Authority
- CN
- China
- Prior art keywords
- frame
- image frame
- algorithm
- video analysis
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 62
- 238000012545 processing Methods 0.000 claims abstract description 20
- 238000004422 calculation algorithm Methods 0.000 claims description 210
- 238000004458 analytical method Methods 0.000 claims description 136
- 238000004590 computer program Methods 0.000 claims description 22
- 238000006243 chemical reaction Methods 0.000 claims description 8
- 230000004044 response Effects 0.000 claims description 7
- 238000009432 framing Methods 0.000 claims 2
- 238000004891 communication Methods 0.000 description 11
- 238000010586 diagram Methods 0.000 description 10
- 230000006870 function Effects 0.000 description 8
- 238000012369 In process control Methods 0.000 description 7
- 210000004544 dc2 Anatomy 0.000 description 7
- 238000010191 image analysis Methods 0.000 description 7
- 238000004190 ion pair chromatography Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 230000003287 optical effect Effects 0.000 description 5
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000003703 image analysis method Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention relates to the technical field of image processing, and discloses an image frame pushing method, an image frame pushing system, electronic equipment and a readable storage medium.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to an image frame pushing method, an image frame pushing system, an electronic device, and a readable storage medium.
Background
Currently, IPCs are classified into smart IPCs and non-smart IPCs. The intelligent IPC can directly carry out intelligent analysis on the image after the video data acquired by the sensor is subjected to image processing. The non-intelligent IPC can also perform intelligent analysis on the acquired video code stream through a post-intelligent analysis function of an edge intelligent product (such as NVR), wherein the compressed code stream received by the NCR is in a format of H.264/H.265, the compressed code stream needs to be decoded first through a decoder, and then image processing and intelligent analysis can be performed subsequently.
However, since the video code stream in the same path may activate multiple intelligent analysis algorithms, and the image frame code rate required by each intelligent analysis algorithm is different, in order to adapt to each intelligent analysis algorithm, the code stream needs to be decoded for multiple times according to different image frame code rates, and then each decoding path is subjected to image processing and intelligent analysis, so that the decoding times and the image processing times are more, and the image frame for intelligent analysis occupies a larger memory, thereby increasing the resource consumption of the NVR with weak decoding capability and small memory capacity on the image frame.
Disclosure of Invention
The following presents a simplified summary in order to provide a basic understanding of some aspects of the disclosed embodiments. This summary is not an extensive overview nor is intended to identify key/critical elements or to delineate the scope of such embodiments but rather as a prelude to the more detailed description that is presented later.
In view of the above drawbacks of the prior art, the present invention discloses an image frame pushing method, system, electronic device and readable storage medium, so as to reduce the memory occupied by image frames, thereby reducing resource consumption.
The invention discloses an image frame pushing method, which comprises the following steps: acquiring an original video code stream and a video analysis algorithm set, wherein the video analysis algorithm set comprises a plurality of video analysis algorithms and algorithm required frame rates corresponding to the video analysis algorithms; calculating the maximum common multiple between the required frame rates of the algorithms to obtain a reference image frame rate, and decoding the original video code stream according to the reference image frame rate to obtain a decoded frame sequence, wherein the decoded frame sequence comprises a plurality of intermediate image frames; determining a plurality of target image frames from the intermediate image frame according to frame sending intervals corresponding to the video analysis algorithms, and determining a target pushing algorithm corresponding to each target image frame from each video analysis algorithm, wherein the frame sending intervals are determined according to the reference image frame rate and the algorithm required frame rate; and sequentially pushing each target image frame to each corresponding target pushing algorithm, so that the target pushing algorithm performs video analysis on the received target image frames.
Optionally, the frame sending interval is determined by the following formula:
in the formula, gap (n) is a frame sending interval corresponding to the nth video analysis algorithm, FPS is the reference image frame rate, and FPS (n) is the algorithm required frame rate corresponding to the nth video analysis algorithm.
Optionally, determining a plurality of target image frames from the intermediate image frame according to the frame sending interval corresponding to each video analysis algorithm, and determining a target pushing algorithm corresponding to each target image frame from each video analysis algorithm, includes: setting an interval threshold corresponding to the video analysis algorithm according to a frame sending interval corresponding to the video analysis algorithm and an integral multiple interval corresponding to the frame sending interval; determining a frame sequence number corresponding to each intermediate image frame according to the sequence of the decoded frame sequence, and respectively matching the frame sequence number with an interval threshold corresponding to each video analysis algorithm; if the frame number corresponding to any intermediate image frame is equal to the interval threshold corresponding to at least one video analysis algorithm, determining the intermediate image frame as a target image frame, and determining all matched video analysis algorithms as target push algorithms corresponding to the target image frame.
Optionally, after determining a plurality of target image frames from the intermediate image frame according to the frame sending interval corresponding to each video analysis algorithm, the method further includes: and performing format conversion on each target image frame according to a preset image processing rule, wherein the image processing rule comprises a color format and/or an image size.
Optionally, decoding the original video code stream according to the reference image frame rate to obtain a decoded frame sequence, including: transmitting the reference image frame rate into a preset hardware decoder, and configuring the reference image frame rate into a decoding output frame rate of the hardware decoder; and decoding the original video code stream by using the hardware decoder to obtain a decoded frame sequence corresponding to the maximum image frame.
Optionally, after calculating a greatest common multiple between the required frame rates of the algorithms to obtain a reference image frame rate, the method further includes: in response to the fact that a new video analysis algorithm is activated or any one of the video analysis algorithms is closed, recalculating the maximum common multiple according to the updated required frame rates of the algorithms to obtain an updated image frame rate; if the updated image frame rate is the same as the reference image frame rate, maintaining the decoding output frame rate of the hardware decoder; and if the updated image frame rate is different from the reference image frame rate, updating the decoding output frame rate of the hardware decoder according to the updated image frame rate.
Optionally, the method further comprises: determining any intermediate image frame as a current image frame, and acquiring a frame reference parameter corresponding to the current image frame, wherein the frame reference parameter is set as a preset initial value; in response to sending the current image frame to any corresponding target push algorithm, increasing the frame reference parameter by one unit; reducing the frame reference parameter by one unit in response to the current image frame completing video analysis of any target push algorithm; and if the obtained frame reference parameter is equal to the preset initial value, releasing the memory of the current image frame.
The invention discloses an image frame pushing system, which comprises: the video analysis method comprises the steps that an acquisition module is used for acquiring an original video code stream and a video analysis algorithm set, wherein the video analysis algorithm set comprises a plurality of video analysis algorithms and algorithm required frame rates corresponding to the video analysis algorithms; the decoding module is used for calculating the maximum common multiple between the required frame rates of the algorithms to obtain a reference image frame rate, and decoding the original video code stream according to the reference image frame rate to obtain a decoded frame sequence, wherein the decoded frame sequence comprises a plurality of intermediate image frames; a determining module, configured to determine a plurality of target image frames from the intermediate image frame according to a frame sending interval corresponding to each video analysis algorithm, and determine a target pushing algorithm corresponding to each target image frame from each video analysis algorithm, where the frame sending interval is determined according to the reference image frame rate and the algorithm required frame rate; and the pushing module is used for sequentially pushing each target image frame to each corresponding target pushing algorithm, so that the target pushing algorithm performs video analysis on the received target image frames.
The invention discloses an electronic device, comprising: a processor and a memory; the memory is used for storing computer programs, and the processor is used for executing the computer programs stored by the memory so as to make the electronic equipment execute the method.
The invention discloses a computer-readable storage medium, on which a computer program is stored: which when executed by a processor implements the method described above.
The invention has the beneficial effects that:
the method comprises the steps of obtaining algorithm required frame rates corresponding to all video analysis algorithms, decoding an original video code stream according to a reference image frame rate between the algorithm required frame rates to obtain a decoded frame sequence, and determining target image frames and target pushing algorithms corresponding to the target image frames from intermediate image frames of the decoded frame sequence according to frame sending intervals, so that all the target image frames are pushed to the corresponding target pushing algorithms. Therefore, compared with the decoding channel for obtaining each analysis algorithm by decoding the video code stream for multiple times, the original video code stream only needs to be decoded once by the maximum common multiple between the algorithm required frame rates, the target image is determined from the intermediate image frame according to the frame sending interval and is pushed to the corresponding video analysis algorithm for video analysis, the decoding times are reduced, the memory occupied by the image frame for intelligent analysis is reduced, and the resource consumption is reduced.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application. It is obvious that the drawings in the following description are only some embodiments of the application, and that for a person skilled in the art, other drawings can be derived from them without inventive effort. In the drawings:
fig. 1 is a schematic flowchart of an image frame pushing method according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of an existing image analysis method according to an embodiment of the present invention;
fig. 3 is a flowchart illustrating another image frame pushing method according to an embodiment of the present invention;
FIG. 4 is a flowchart illustrating a frame rate control method according to an embodiment of the present invention;
FIG. 5 is a flowchart illustrating a method for decoding a video bitstream according to an embodiment of the present invention;
FIG. 6 is a flowchart illustrating an output frame rate updating method of a hardware decoder according to an embodiment of the present invention;
FIG. 7 is a flow chart illustrating a method for determining a target image frame according to an embodiment of the present invention;
FIG. 8 is a flow chart illustrating a method of image analysis in accordance with an embodiment of the present invention;
fig. 9 is a schematic structural diagram of an image frame pushing system according to an embodiment of the present invention;
fig. 10 is a schematic structural diagram of an electronic device in an embodiment of the invention.
Detailed Description
The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It should be noted that, in the following embodiments and examples, subsamples may be combined without conflict.
It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present invention, and the components related to the present invention are only shown in the drawings rather than drawn according to the number, shape and size of the components in actual implementation, and the type, quantity and proportion of the components in actual implementation may be changed freely, and the layout of the components may be more complicated.
In the following description, numerous details are set forth to provide a more thorough explanation of embodiments of the present invention, however, it will be apparent to one skilled in the art that embodiments of the present invention may be practiced without these specific details, and in other embodiments, well-known structures and devices are shown in block diagram form, rather than in detail, in order to avoid obscuring embodiments of the present invention.
The terms "first," "second," and the like in the description and in the claims, and the above-described drawings of embodiments of the present disclosure, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It should be understood that the data so used may be interchanged under appropriate circumstances such that embodiments of the present disclosure described herein may be made. Furthermore, the terms "comprising" and "having," as well as any variations thereof, are intended to cover non-exclusive inclusions.
The term "plurality" means two or more, unless otherwise specified.
In the embodiment of the present disclosure, the character "/" indicates that the preceding and following objects are in an or relationship. For example, A/B represents: a or B.
The term "and/or" is an associative relationship that describes objects, meaning that three relationships may exist. E.g., a and/or B, represents: a or B, or A and B.
With reference to fig. 1, an embodiment of the present disclosure provides an image frame pushing method, including:
step S101, acquiring an original video code stream and a video analysis algorithm set;
the video analysis algorithm set comprises a plurality of video analysis algorithms and algorithm required frame rates corresponding to the video analysis algorithms;
step S102, calculating the maximum common multiple between the required frame rates of all algorithms to obtain a reference image frame rate, and decoding an original video code stream according to the reference image frame rate to obtain a decoded frame sequence;
wherein the decoded frame sequence comprises a plurality of intermediate image frames;
step S103, determining a plurality of target image frames from the intermediate image frame according to the frame sending intervals corresponding to the video analysis algorithms, and determining the target pushing algorithm corresponding to each target image frame from each video analysis algorithm;
the frame sending interval is determined according to the reference image frame rate and the algorithm required frame rate;
and step S104, sequentially pushing each target image frame to each corresponding target pushing algorithm, so that the target pushing algorithm performs video analysis on the received target image frame.
By adopting the image frame pushing method provided by the embodiment of the disclosure, the algorithm required frame rates corresponding to the video analysis algorithms are obtained, the original video code stream is decoded according to the reference image frame rate between the calculation algorithm required frame rates to obtain a decoded frame sequence, and then the target image frame and the target pushing algorithm corresponding to the target image frame are determined from the intermediate image frame of the decoded frame sequence according to the frame sending interval, so that each target image frame is pushed to each corresponding target pushing algorithm. Compared with the decoding channel for obtaining each analysis algorithm by decoding the video code stream for multiple times, the decoding channel for obtaining each analysis algorithm only needs to decode the original video code stream once according to the maximum common multiple between the algorithm required frame rates, then determines the target image from the intermediate image frame according to the frame sending interval, and pushes the target image to the corresponding video analysis algorithm for video analysis, so that the decoding times are reduced, the memory occupied by the image frame for intelligent analysis is reduced, and the resource consumption is reduced.
As shown in fig. 2, an embodiment of the present disclosure provides an existing image analysis method, including:
step S201, acquiring a video code stream through acquisition equipment;
wherein, the acquisition equipment comprises NVR, IPC, network equipment and the like;
step S202, decoding the video code stream according to the decoding rules corresponding to the analysis algorithms respectively to obtain video frames corresponding to the analysis algorithms;
step S203, format conversion and size scaling are respectively carried out on each decoded video frame to obtain a YUV image;
and step S204, intelligently analyzing the YUV image to obtain an intelligent analysis result.
With reference to fig. 3, an embodiment of the present disclosure provides an image frame pushing method, including:
step S301, acquiring a video code stream through acquisition equipment;
the acquisition equipment comprises NVR, IPC, network equipment and the like;
step S302, calculating the maximum common multiple between the required frame rates of all algorithms to obtain a reference image frame rate, and decoding an original video code stream according to the reference image frame rate to obtain a decoded frame sequence;
step S303, determining a plurality of target image frames from the intermediate image frame according to the frame sending intervals corresponding to the video analysis algorithms, and determining the target pushing algorithm corresponding to each target image frame from each video analysis algorithm;
step S304, carrying out frame format conversion and size scaling on the target image frame to obtain a YUV image frame;
step S305, sending the YUV image frame to at least one image analysis algorithm based on the corresponding target push algorithm so as to perform image analysis on the YUV image frame;
and step S306, releasing the data memory after all the YUV image frames finish image analysis.
By adopting the image frame pushing method provided by the embodiment of the disclosure, the algorithm required frame rates corresponding to the video analysis algorithms are obtained, the original video code stream is decoded according to the reference image frame rate between the calculation algorithm required frame rates to obtain a decoded frame sequence, and then the target image frame and the target pushing algorithm corresponding to the target image frame are determined from the intermediate image frame of the decoded frame sequence according to the frame sending interval, so that each target image frame is pushed to each corresponding target pushing algorithm, and the method has the following advantages:
compared with a decoding channel for obtaining each analysis algorithm by decoding a video code stream for multiple times, the original video code stream is decoded for only one time by the maximum common multiple between algorithm required frame rates, a target image is determined from an intermediate image frame according to a frame sending interval and is pushed to a corresponding video analysis algorithm for video analysis, decoding times are reduced, and an internal memory occupied by the image frame for intelligent analysis is reduced, so that resource consumption is reduced;
secondly, managing each target image frame according to the algorithm required frame rate corresponding to each video analysis algorithm to realize self-adaptive frame sending;
and thirdly, decoding resources and memory resources are saved while the consistency of the intelligent analysis effect is ensured, so that new decoding equipment does not need to be replaced along with the increase of the decoding resources, and the equipment cost is saved.
Optionally, decoding the original video code stream according to the reference image frame rate to obtain a decoded frame sequence, including: transmitting the reference image frame rate into a preset hardware decoder, and configuring the reference image frame rate into a decoding output frame rate of the hardware decoder; and decoding the original video code stream by using a hardware decoder to obtain a decoded frame sequence corresponding to the maximum image frame.
In some embodiments, the hardware decoder changes the frame rate of the original video stream by a uniform frame loss manner, so as to obtain a decoded frame sequence.
With reference to fig. 4, a frame rate control method is provided in an embodiment of the present disclosure, where a video original frame rate of an original video code stream is 25Fps, and since an intelligent a frame rate of an intelligent analysis algorithm a is 6Fps and an intelligent B frame rate of an intelligent analysis algorithm B is 4Fps, a reference video frame rate (LCM frame rate) is determined to be 12Fps, the original video code stream is decoded and uniformly lost frames are performed at the reference video frame rate to obtain a decoded frame sequence, and then the decoded frame sequence is uniformly lost frames according to the intelligent a frame rate and the intelligent B frame rate, so as to obtain 6Fps video frames and 4Fps video frames.
With reference to fig. 5, an embodiment of the present disclosure provides a method for decoding a video code stream, including:
step S501, acquiring an algorithm required frame rate corresponding to a video analysis algorithm;
step S502, calculating the least common multiple according to the required frame rate of each algorithm to obtain the frame rate of a reference image;
step S503, transmitting the reference image frame rate into a hardware decoder, and configuring the reference image frame rate into a decoding output frame rate;
step S504, decoding the original video code stream by using a hardware decoder to obtain a decoding frame sequence corresponding to the maximum image frame.
In some embodiments, the target frame rates of 1 to n image analysis algorithms for opening the video channel are counted, a least common multiple of all the target frame rates is calculated through a least common multiple algorithm, the least common multiple is subjected to range protection to obtain a reference image frame rate, the frame rate is configured to a hardware decoder as an output frame rate, and the decoded video frame is output at the frame rate after decoding operation. Considering that different image analysis algorithms may have different input frame rate requirements, the least common multiple algorithm is skillfully used for calculating the reference image frame rate, and the reference image frame rate is configured to the hardware decoder as the output frame rate.
Optionally, after calculating the greatest common multiple between the required frame rates of the algorithms to obtain the reference image frame rate, the method further includes: responding to the fact that a new video analysis algorithm is activated or any video analysis algorithm is closed, and recalculating the maximum common multiple according to the updated required frame rates of the algorithms to obtain an updated image frame rate; if the updated image frame rate is the same as the reference image frame rate, maintaining the decoding output frame rate of the hardware decoder; and if the updated image frame rate is different from the reference image frame rate, updating the decoding output frame rate of the hardware decoder according to the updated image frame rate.
As shown in fig. 6, an embodiment of the present disclosure provides an output frame rate updating method for a hardware decoder, including:
step S601, if the video analysis algorithm is identified to be turned on or turned off, selecting a decoding channel of the video analysis algorithm;
step S602, updating a frame rate table of a decoding channel according to the algorithm required frame rate of the video analysis algorithm;
the frame rate table comprises algorithm required frame rates of all video analysis algorithms in a decoding channel;
step S603, recalculating the maximum common multiple according to the updated frame rate table to obtain an updated image frame rate;
step S604, judging whether the updated image frame rate is the same as the reference image frame rate, if so, skipping to step S605, otherwise, skipping to step S606;
step S605, maintaining the decoding output frame rate of the hardware decoder;
step S606, updating the decoding output frame rate of the hardware decoder according to the updated image frame rate.
Optionally, the frame sending interval is determined by the following formula:
in the formula, gap (n) is a frame sending interval corresponding to the nth video analysis algorithm, FPS is a reference image frame rate, and FPS (n) is an algorithm required frame rate corresponding to the nth video analysis algorithm.
Optionally, determining a plurality of target image frames from the intermediate image frame according to the frame sending interval corresponding to each video analysis algorithm, and determining a target pushing algorithm corresponding to each target image frame from each video analysis algorithm, includes: setting an interval threshold corresponding to the video analysis algorithm according to a frame sending interval corresponding to the video analysis algorithm and an integral multiple interval corresponding to the frame sending interval; determining the frame number corresponding to each intermediate image frame according to the sequence of the decoded frame sequence, and respectively matching the frame number with the interval threshold corresponding to each video analysis algorithm; and if the frame sequence number corresponding to any one intermediate image frame is equal to the interval threshold corresponding to at least one video analysis algorithm, determining the intermediate image frame as a target image frame, and determining all matched video analysis algorithms as target push algorithms corresponding to the target image frame.
Optionally, after determining a plurality of target image frames from the intermediate image frame according to the frame sending interval corresponding to each video analysis algorithm, the method further includes: and carrying out format conversion on each target image frame according to a preset image processing rule, wherein the image processing rule comprises a color format and/or an image size.
With reference to fig. 7, an embodiment of the present disclosure provides a method for determining a target image frame, including:
step S701, acquiring a decoding frame sequence output by a hardware decoder;
step S702, sequentially selecting intermediate image frames from the decoded frame sequence, and cumulatively adding 1 to the frame number when selecting one image frame;
step S703, determining whether the frame number of the selected intermediate image frame matches an interval threshold corresponding to any video analysis algorithm, if so, skipping to step S704, and if not, skipping to step S702;
step S704, determining the intermediate image frame as a target image frame, and determining all matched video analysis algorithms as target push algorithms corresponding to the target image frame;
step S705, performing format conversion on the target image frame according to a preset image processing rule to obtain a YUV image frame.
In some embodiments, the hardware decoder outputs decoded video frames, and channel frame number is accumulated for each output of a video frame; calculating whether the current output frame is matched with an interval threshold corresponding to at least one video analysis algorithm or not through the frame sequence number and the interval threshold; and when the video frame is matched with the interval threshold corresponding to one or more video analysis algorithms, recording the matched video analysis algorithms, and sending the video analysis algorithms to a preprocessing module for frame format conversion and size conversion according to a preset image processing rule to obtain a YUV image frame.
Optionally, the method further comprises: determining any intermediate image frame as a current image frame, and acquiring a frame reference parameter corresponding to the current image frame, wherein the frame reference parameter is set as a preset initial value; in response to the current image frame being sent to any corresponding target push algorithm, increasing the frame reference parameter by one unit; reducing the frame reference parameter by one unit in response to the completion of video analysis of any target push algorithm of the current image frame; and if the obtained frame reference parameter is equal to the preset initial value, releasing the memory of the current image frame.
With reference to fig. 8, an embodiment of the present disclosure provides an image analysis method, including:
step S801, acquiring a YUV image frame and a push list corresponding to the YUV image frame, and skipping to step S802 and step S804;
step S802, if the target pushing algorithm which is not taken out exists in the pushing list, the target pushing algorithm is taken out in sequence to serve as the current pushing algorithm;
step S803, the frame reference parameter +1 is added, the YUV image frame is pushed to the current pushing algorithm, and the step S802 is skipped;
step S804, if any target push algorithm finishes the image analysis of the YUV image frame, the frame is quoted to a parameter-1;
in step S805, if the frame reference parameter is equal to 0, the memory of the YUV image frame is released.
In some embodiments, in an intelligent analysis process of YUV image frames, a frame rate matched intelligent list is carried along with the frames, the intelligent type in the list is a target push algorithm to be issued by the image frame, a target push algorithm n (n =0,1,2 \8230;) is taken out from the target push algorithm n, 1 is added to a frame reference parameter, the frame is sent to the target push algorithm n for analysis, then a target push algorithm n +1 is taken from the list, and under a normal condition, the frame reference parameter is equal to the number of the matched intelligent types in the intelligent list; and after the YUV image frame is analyzed by the target pushing algorithm n, calling a frame release interface, firstly subtracting 1 from the frame reference parameter, judging whether the reference count is 0, when the reference count reaches 0, indicating that the intelligence for referencing the frame is completely analyzed, releasing the YUV image frame, and otherwise, waiting for the next intelligent release.
As shown in fig. 9, an embodiment of the present disclosure provides an image frame pushing system, which includes an obtaining module 901, a decoding module 902, a determining module 903, and a pushing module 904. The acquiring module 901 is configured to acquire an original video code stream and a video analysis algorithm set, where the video analysis algorithm set includes a plurality of video analysis algorithms and algorithm required frame rates corresponding to the video analysis algorithms; the decoding module 902 is configured to calculate a maximum common multiple between frame rates required by each algorithm to obtain a reference image frame rate, and decode an original video code stream according to the reference image frame rate to obtain a decoded frame sequence, where the decoded frame sequence includes a plurality of intermediate image frames; the determining module 903 is configured to determine a plurality of target image frames from the intermediate image frame according to a frame sending interval corresponding to each video analysis algorithm, and determine a target pushing algorithm corresponding to each target image frame from each video analysis algorithm, where the frame sending interval is determined according to a reference image frame rate and an algorithm required frame rate; the pushing module 904 is configured to sequentially push each target image frame to a corresponding target pushing algorithm, so that the target pushing algorithm performs video analysis on the received target image frame.
By adopting the image frame pushing system provided by the embodiment of the disclosure, the algorithm required frame rates corresponding to the video analysis algorithms are obtained, the original video code stream is decoded according to the reference image frame rate between the calculation algorithm required frame rates to obtain a decoded frame sequence, and then the target image frame and the target pushing algorithm corresponding to the target image frame are determined from the intermediate image frame of the decoded frame sequence according to the frame sending interval, so that each target image frame is pushed to each corresponding target pushing algorithm. Compared with the decoding channel for obtaining each analysis algorithm by decoding the video code stream for multiple times, the decoding channel for obtaining each analysis algorithm only needs to decode the original video code stream once according to the maximum common multiple between the algorithm required frame rates, then determines the target image from the intermediate image frame according to the frame sending interval, and pushes the target image to the corresponding video analysis algorithm for video analysis, so that the decoding times are reduced, the memory occupied by the image frame for intelligent analysis is reduced, and the resource consumption is reduced.
FIG. 10 illustrates a schematic structural diagram of a computer system suitable for use in implementing the electronic device of an embodiment of the present application. It should be noted that the computer system 1000 of the electronic device shown in fig. 10 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.
As shown in fig. 10, the computer system 1000 includes a Central Processing Unit (CPU) 1001 that can perform various appropriate actions and processes, such as performing the methods in the above-described embodiments, according to a program stored in a Read-only memory (ROM) 1002 or a program loaded from a storage portion 1008 into a Random Access Memory (RAM) 1003. In the RAM1003, various programs and data necessary for system operation are also stored. The CPU1001, ROM1002, and RAM1003 are connected to each other by a bus 1004. An Input/Output (I/O) interface 1005 is also connected to the bus 1004.
The following components are connected to the I/O interface 1005: an input section 1006 including a keyboard, a mouse, and the like; an output portion 1007 including a display panel such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and a speaker; a storage portion 1008 including a hard disk and the like; and a communication section 1009 including a network interface card such as a LAN (local area network) card, a modem, or the like. The communication section 1009 performs communication processing via a network such as the internet. The driver 1010 is also connected to the I/O interface 1005 as necessary. A removable medium 1011 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 1010 as necessary, so that a computer program read out therefrom is mounted into the storage section 1008 as necessary.
In particular, according to embodiments of the application, the processes described above with reference to the flow diagrams may be implemented as computer software programs. For example, embodiments of the present application include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising a computer program for performing the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from the network through the communication part 1009 and/or installed from the removable medium 1011. When the computer program is executed by a Central Processing Unit (CPU) 1001, various functions defined in the system of the present application are executed.
It should be noted that the computer readable medium shown in the embodiments of the present application may be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may be, for example, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an Erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer-readable signal medium may include a propagated data signal with a computer program embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.
The disclosed embodiments also provide a computer-readable storage medium on which a computer program is stored, which when executed by a processor implements any of the methods in the embodiments.
The computer-readable storage medium in the embodiments of the present disclosure may be understood by those skilled in the art as follows: all or a portion of the steps of implementing the above-described method embodiments may be performed by hardware associated with a computer program. The aforementioned computer program may be stored in a computer readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.
The electronic device disclosed in this embodiment includes a processor, a memory, a transceiver, and a communication interface, where the memory and the communication interface are connected to the processor and the transceiver and perform mutual communication, the memory is used for storing a computer program, the communication interface is used for performing communication, and the processor and the transceiver are used for operating the computer program, so that the electronic device performs the steps of the above method.
In this embodiment, the memory may include a Random Access Memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.
The processor may be a general-purpose processor, and includes a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a Network Processor (NP), and the like; the integrated circuit may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete gate or transistor logic device, or discrete hardware components.
The above description and drawings sufficiently illustrate embodiments of the disclosure to enable those skilled in the art to practice them. Other embodiments may incorporate structural, logical, electrical, process, and other changes. The examples merely typify possible variations. Individual components and functions are optional unless explicitly required, and the sequence of operations may vary. Portions and subsamples of some embodiments may be included in or substituted for portions and subsamples of other embodiments. Furthermore, the words used in the specification are words of description only and are not intended to limit the claims. As used in the description of the embodiments and the claims, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. Similarly, the term "and/or" as used in this application is meant to encompass any and all possible combinations of one or more of the associated listed. Furthermore, the terms "comprises," "comprising," and variations thereof, when used in this application, specify the presence of stated sub-samples, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other sub-samples, integers, steps, operations, elements, components, and/or groups thereof. Without further limitation, an element defined by the phrase "comprising one of 8230," does not exclude the presence of another identical element in a process, method or device comprising the element. In this document, each embodiment may be described with emphasis on differences from other embodiments, and the same and similar parts between the respective embodiments may be referred to each other. For methods, products, etc. of the embodiment disclosures, reference may be made to the description of the method section for relevance if it corresponds to the method section of the embodiment disclosure.
Those of skill in the art would appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software may depend upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the disclosed embodiments. It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the system, the apparatus and the unit described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the embodiments disclosed herein, the disclosed methods, products (including but not limited to devices, apparatuses, etc.) may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of a unit may be merely a division of a logical function, and an actual implementation may have another division, for example, a plurality of units or components may be combined or may be integrated into another system, or some subsamples may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form. The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one position, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to implement the present embodiment. In addition, functional units in the embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. In the description corresponding to the flowcharts and block diagrams in the figures, operations or steps corresponding to different blocks may also occur in different orders than disclosed in the description, and sometimes there is no specific order between the different operations or steps. For example, two sequential operations or steps may in fact be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. Each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
Claims (10)
1. An image frame pushing method, comprising:
acquiring an original video code stream and a video analysis algorithm set, wherein the video analysis algorithm set comprises a plurality of video analysis algorithms and algorithm required frame rates corresponding to the video analysis algorithms;
calculating the maximum common multiple between the required frame rates of the algorithms to obtain a reference image frame rate, and decoding the original video code stream according to the reference image frame rate to obtain a decoded frame sequence, wherein the decoded frame sequence comprises a plurality of intermediate image frames;
determining a plurality of target image frames from the intermediate image frame according to frame sending intervals corresponding to the video analysis algorithms, and determining a target pushing algorithm corresponding to each target image frame from each video analysis algorithm, wherein the frame sending intervals are determined according to the reference image frame rate and the algorithm required frame rate;
and sequentially pushing each target image frame to each corresponding target pushing algorithm, so that the target pushing algorithm performs video analysis on the received target image frames.
2. The method of claim 1, wherein the framing interval is determined by the formula:
in the formula, gap (n) is a frame sending interval corresponding to the nth video analysis algorithm, FPS is the reference image frame rate, and FPS (n) is an algorithm required frame rate corresponding to the nth video analysis algorithm.
3. The method of claim 1, wherein determining a plurality of target image frames from the intermediate image frames according to the frame sending interval corresponding to each of the video analysis algorithms, and determining a target pushing algorithm corresponding to each of the target image frames from each of the video analysis algorithms comprises:
setting an interval threshold value corresponding to the video analysis algorithm according to the frame sending interval corresponding to the video analysis algorithm and the integral multiple interval corresponding to the frame sending interval;
determining a frame sequence number corresponding to each intermediate image frame according to the sequence of the decoded frame sequence, and respectively matching the frame sequence number with an interval threshold corresponding to each video analysis algorithm;
if the frame number corresponding to any intermediate image frame is equal to the interval threshold corresponding to at least one video analysis algorithm, determining the intermediate image frame as a target image frame, and determining all matched video analysis algorithms as target push algorithms corresponding to the target image frame.
4. The method of claim 1, wherein after determining a plurality of target image frames from the intermediate image frames according to a framing interval corresponding to each of the video analysis algorithms, the method further comprises:
and performing format conversion on each target image frame according to a preset image processing rule, wherein the image processing rule comprises a color format and/or an image size.
5. The method of claim 1, wherein decoding the original video bitstream according to the reference video frame rate to obtain a decoded frame sequence comprises:
transmitting the reference image frame rate into a preset hardware decoder, and configuring the reference image frame rate into a decoding output frame rate of the hardware decoder;
and decoding the original video code stream by using the hardware decoder to obtain a decoded frame sequence corresponding to the maximum image frame.
6. The method of claim 5, wherein after calculating a greatest common multiple between the required frame rates of the algorithms to obtain a reference image frame rate, the method further comprises:
responding to the fact that a new video analysis algorithm is activated or any video analysis algorithm is closed, and recalculating the maximum common multiple according to the updated required frame rates of the algorithms to obtain an updated image frame rate;
if the updated image frame rate is the same as the reference image frame rate, maintaining the decoding output frame rate of the hardware decoder;
and if the updated image frame rate is different from the reference image frame rate, updating the decoding output frame rate of the hardware decoder according to the updated image frame rate.
7. The method according to any one of claims 1 to 6, further comprising:
determining any intermediate image frame as a current image frame, and acquiring a frame reference parameter corresponding to the current image frame, wherein the frame reference parameter is set as a preset initial value;
in response to sending the current image frame to any corresponding target push algorithm, increasing the frame reference parameter by one unit;
reducing the frame reference parameter by one unit in response to the current image frame completing video analysis of any target push algorithm;
and if the obtained frame reference parameter is equal to the preset initial value, releasing the memory of the current image frame.
8. An image frame push system, comprising:
the video analysis method comprises the steps that an acquisition module is used for acquiring an original video code stream and a video analysis algorithm set, wherein the video analysis algorithm set comprises a plurality of video analysis algorithms and algorithm required frame rates corresponding to the video analysis algorithms;
the decoding module is used for calculating the maximum common multiple between the required frame rates of the algorithms to obtain a reference image frame rate, and decoding the original video code stream according to the reference image frame rate to obtain a decoded frame sequence, wherein the decoded frame sequence comprises a plurality of intermediate image frames;
the determining module is configured to determine a plurality of target image frames from the intermediate image frame according to a frame sending interval corresponding to each video analysis algorithm, and determine a target pushing algorithm corresponding to each target image frame from each video analysis algorithm, where the frame sending interval is determined according to the reference image frame rate and the algorithm required frame rate;
and the pushing module is used for sequentially pushing each target image frame to each corresponding target pushing algorithm, so that the target pushing algorithm performs video analysis on the received target image frames.
9. An electronic device, comprising: a processor and a memory;
the memory is configured to store a computer program and the processor is configured to execute the computer program stored by the memory to cause the electronic device to perform the method of any of claims 1 to 7.
10. A computer-readable storage medium having stored thereon a computer program, characterized in that:
the computer program, when executed by a processor, implements the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211339277.8A CN115514970A (en) | 2022-10-28 | 2022-10-28 | Image frame pushing method and system, electronic equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211339277.8A CN115514970A (en) | 2022-10-28 | 2022-10-28 | Image frame pushing method and system, electronic equipment and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115514970A true CN115514970A (en) | 2022-12-23 |
Family
ID=84513186
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211339277.8A Pending CN115514970A (en) | 2022-10-28 | 2022-10-28 | Image frame pushing method and system, electronic equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115514970A (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1289509A (en) * | 1998-09-28 | 2001-03-28 | 松下电器产业株式会社 | Data processing method, data processor, and program recorded medium |
| CN111698555A (en) * | 2020-06-19 | 2020-09-22 | 浙江大华技术股份有限公司 | Video frame extraction processing method and device |
| CN112738418A (en) * | 2020-12-25 | 2021-04-30 | 北京百度网讯科技有限公司 | Video acquisition method and device and electronic equipment |
| CN112969049A (en) * | 2021-01-29 | 2021-06-15 | 南京长江油运有限公司 | Intelligent detection system for ship violation behaviors |
| CN114598834A (en) * | 2022-05-10 | 2022-06-07 | 中国铁塔股份有限公司 | Video processing method and device, electronic equipment and readable storage medium |
| WO2022141115A1 (en) * | 2020-12-29 | 2022-07-07 | 深圳市大疆创新科技有限公司 | Video processing method and apparatus, system on chip, and storage medium |
| WO2022143688A1 (en) * | 2020-12-31 | 2022-07-07 | 百果园技术(新加坡)有限公司 | Video frame extraction processing method, apparatus and device, and medium |
-
2022
- 2022-10-28 CN CN202211339277.8A patent/CN115514970A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1289509A (en) * | 1998-09-28 | 2001-03-28 | 松下电器产业株式会社 | Data processing method, data processor, and program recorded medium |
| CN111698555A (en) * | 2020-06-19 | 2020-09-22 | 浙江大华技术股份有限公司 | Video frame extraction processing method and device |
| CN112738418A (en) * | 2020-12-25 | 2021-04-30 | 北京百度网讯科技有限公司 | Video acquisition method and device and electronic equipment |
| WO2022141115A1 (en) * | 2020-12-29 | 2022-07-07 | 深圳市大疆创新科技有限公司 | Video processing method and apparatus, system on chip, and storage medium |
| WO2022143688A1 (en) * | 2020-12-31 | 2022-07-07 | 百果园技术(新加坡)有限公司 | Video frame extraction processing method, apparatus and device, and medium |
| CN112969049A (en) * | 2021-01-29 | 2021-06-15 | 南京长江油运有限公司 | Intelligent detection system for ship violation behaviors |
| CN114598834A (en) * | 2022-05-10 | 2022-06-07 | 中国铁塔股份有限公司 | Video processing method and device, electronic equipment and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10182095B2 (en) | Method and system for video call using two-way communication of visual or auditory effect | |
| AU2014253499B2 (en) | Space-optimized display of multi-column tables with selective text truncation based on a combined text width | |
| CN109194960A (en) | A kind of picture frame rendering method, device and electronic equipment | |
| EP2593862B1 (en) | Out-of-order command execution in a multimedia processor | |
| CN103348320B (en) | Computing resource in general graphical processing unit is Pipelining | |
| CN114615510B (en) | Live broadcast interface display method and equipment | |
| CN115883899B (en) | Image on-screen playing method and system, electronic equipment and readable storage medium | |
| EP4064711A1 (en) | Method and apparatus for providing video stream based on machine learning | |
| CN110852801B (en) | Information processing method, device and equipment | |
| CN111783632B (en) | Face detection method and device for video stream, electronic equipment and storage medium | |
| CN110633597A (en) | Driving region detection method and device | |
| CN115514970A (en) | Image frame pushing method and system, electronic equipment and readable storage medium | |
| US9645637B2 (en) | Managing a free list of resources to decrease control complexity and reduce power consumption | |
| CN108765503B (en) | Skin color detection method, device and terminal | |
| CN110809166A (en) | Video data processing method and device and electronic equipment | |
| CN111010606B (en) | Video processing method and device | |
| CN113271494B (en) | Video frame processing method and device and electronic equipment | |
| CN112218003B (en) | Desktop image acquisition method and device and electronic equipment | |
| CN116501832A (en) | Comment processing method and comment processing equipment | |
| CN108335706B (en) | Seamless playing method of multimedia file, terminal equipment and storage medium | |
| US20230362386A1 (en) | Estimated macroblock distortion co-optimization | |
| CN116684610B (en) | Method and device for determining reference state of long-term reference frame and electronic equipment | |
| US6018367A (en) | Frame display dilation in video decoding | |
| CN115761595A (en) | Low-quality video detection and video understanding model training method and device | |
| CN114727111A (en) | Picture decoding method and device, set top box and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |