CN114339156A - Video stream frame rate adjusting method, device, equipment and readable storage medium - Google Patents
Video stream frame rate adjusting method, device, equipment and readable storage medium Download PDFInfo
- Publication number
- CN114339156A CN114339156A CN202111639965.1A CN202111639965A CN114339156A CN 114339156 A CN114339156 A CN 114339156A CN 202111639965 A CN202111639965 A CN 202111639965A CN 114339156 A CN114339156 A CN 114339156A
- Authority
- CN
- China
- Prior art keywords
- video
- frame
- time information
- frames
- video stream
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention provides a video stream frame rate adjusting method, a video stream frame rate adjusting device, video stream frame rate adjusting equipment and a readable storage medium. The method comprises the following steps: inputting the video frames of the original video stream into the time identification model frame by frame to obtain the time information of each video frame of the input time identification model; detecting whether a time abnormal event exists according to the time information of each video frame; if the time abnormal event exists, processing the time abnormal event to obtain a repaired video stream; and carrying out fixed frame rate processing on the repaired video stream to obtain the video stream with fixed frame rate. According to the invention, the time abnormal event existing in the original video stream is processed to obtain the repaired video stream, and then the fixed frame rate processing is carried out on the repaired video stream to obtain the video stream with the fixed frame rate, so that the problems of time error and unstable frame rate in the original video stream are solved.
Description
Technical Field
The present invention relates to the field of video monitoring technologies, and in particular, to a method, an apparatus, a device, and a readable storage medium for adjusting a frame rate of a video stream.
Background
With the improvement of the intelligent degree of the security industry, more and more security products need to utilize computer vision technologies (such as pedestrian detection, multi-target tracking, pedestrian re-identification and the like) to analyze monitoring pictures.
Due to server performance limitations, one server is usually used to store the video stream as an offline video, and another server is used to perform the analysis process. However, due to data transmission, network delay and the camera itself, the frame rate of the stored video stream may not be consistent with the set frame rate, thereby affecting the subsequent analysis processing. Particularly, when the stored video stream has a video frame with abnormal time information, the frame rate of the stored video stream cannot be accurately adjusted, and thus the problem that the frame rate of the stored video stream is inconsistent with the set frame rate cannot be solved.
Disclosure of Invention
In order to solve the above technical problem, the present invention provides a method, an apparatus, a device and a readable storage medium for adjusting a frame rate of a video stream.
In a first aspect, the present invention provides a method for adjusting a frame rate of a video stream, where the method for adjusting the frame rate of the video stream includes:
inputting the video frames of the original video stream into the time identification model frame by frame to obtain the time information of each video frame of the input time identification model;
detecting whether a time abnormal event exists according to the time information of each video frame;
if the time abnormal event exists, processing the time abnormal event to obtain a repaired video stream;
and carrying out fixed frame rate processing on the repaired video stream to obtain the video stream with fixed frame rate.
Optionally, the step of detecting whether a temporal anomaly event exists according to the temporal information of each video frame includes:
detecting whether a video frame with empty time information exists according to the time information of each video frame;
detecting whether two adjacent video frames with discontinuous time information exist according to the time information of each video frame;
detecting whether a video frame with frozen minute hands exists or not according to the time information of each video frame;
if the time information is an empty video frame, determining that a first time abnormal event exists;
if two adjacent video frames with discontinuous time information exist, determining that a second time abnormal event exists;
and if the video frame with the frozen minute hand exists, determining that a third time abnormal event exists.
Optionally, if there is a time exception, the step of processing the time exception includes:
when a first time abnormal event exists, acquiring time information t1 of a video frame with time information in the front of the video frame with empty time information and time information t2 of a video frame with time information in the back of the video frame with time information, and calculating the value delta t of t2-t 1;
when Δ t is 1, adding t1 to a specific position of a video frame whose temporal information is empty;
when Deltat is more than 1, detecting whether the number n of video frames with empty time information can be divided by (Deltat-1) in an integer way;
if n can be evenly divided by (delta t-1), adding (t1+1) to specific positions of the previous m video frames with empty time information, adding (t1+2) to specific positions of the next m video frames with empty time information, and so on according to the acquisition sequence of the video frames with empty time information, until adding (t2-1) to specific positions of the last m video frames with empty time information, wherein m is a quotient of n evenly divided by (delta t-1);
if n cannot be evenly divided by (Deltat-1), determining two adjacent frames with lowest similarity in the video frames with empty time information, and performing frame interpolation processing between the two adjacent frames with lowest similarity until the number n 'of the video frames with empty time information can be evenly divided by (Deltat-1), adding (t1+1) to the specific positions of the first m' video frames with empty time information, adding (t1+2) to the specific positions of the next m 'video frames with empty time information according to the acquisition sequence of the video frames with empty time information, and so on until (t2-1) is added to the specific positions of the last m' video frames with empty time information, and m 'is the quotient of n' evenly divided by (Deltat-1).
Optionally, if there is a time exception, the step of processing the time exception includes:
when a second time abnormal event exists, performing frame interpolation processing between two adjacent video frames with discontinuous time information, wherein the frame interpolation number is N (t4-t3-1), t3 is the time information of the previous frame in the two adjacent video frames with discontinuous time information, t4 is the time information of the next frame in the two adjacent video frames with discontinuous time information, and N is a preset value;
add (t3+1) to the particular location of the first N video frames inserted, (t3+2) to the particular location of the next N video frames inserted, and so on until (t4-1) is added to the particular location of the last N video frames inserted.
Optionally, if there is a time exception, the step of processing the time exception includes:
when a third time abnormal event exists, acquiring time information t5 of a video frame which is not frozen in the previous minute and is not frozen in the next minute of the frozen video frames and time information t6 of a video frame which is not frozen in the next minute;
dividing the video frame frozen by the pointer into (t6-t5-1) parts;
add (t5+1) to the particular position of the video frame in the first share, (t5+2) to the particular position of the video frame in the second share, and so on until (t6-1) is added to the particular position of the video frame in the last share.
Optionally, the step of performing fixed frame rate processing on the repaired video stream to obtain a video stream with a fixed frame rate includes:
dividing video frames with the same time in the repaired video stream into a group to obtain a plurality of video frame groups;
comparing the number of video frames contained in each video frame group with a preset number of frames;
if the number of the video frames contained in one video frame group is the same as the number of the preset frames, no processing is carried out;
if the number of video frames contained in a video frame group is greater than the preset number of frames, deleting the adjacent frames with the highest similarity in the video frame group until the number of video frames contained in the video frame group is the same as the preset number of frames;
if the number of the video frames contained in one video frame group is less than the preset number of frames, performing frame interpolation between adjacent frames with the lowest similarity in the video frame group until the number of the video frames contained in the video frame group is the same as the preset number of frames;
and obtaining the video stream with fixed frame rate based on all the video frame groups after traversing.
Optionally, the time identification model is a convolutional neural network, and the convolutional neural network includes a convolutional layer, a pooling layer, and a full-link layer.
In a second aspect, the present invention further provides a video stream frame rate adjusting apparatus, including:
the identification module is used for inputting the video frames of the original video stream into the time identification model frame by frame to obtain the time information of each video frame of the input time identification model;
the detection module is used for detecting whether a time abnormal event exists according to the time information of each video frame;
the restoration module is used for processing the time abnormal event if the time abnormal event exists to obtain a restored video stream;
and the frame rate fixing module is used for carrying out fixed frame rate processing on the repaired video stream to obtain the video stream with the fixed frame rate.
In a third aspect, the present invention further provides a video stream frame rate adjusting apparatus, which includes a processor, a memory, and a video stream frame rate adjusting program stored on the memory and executable by the processor, wherein the video stream frame rate adjusting program, when executed by the processor, implements the steps of the video stream frame rate adjusting method as described above.
In a fourth aspect, the present invention further provides a readable storage medium, on which a video stream frame rate adjustment program is stored, wherein the video stream frame rate adjustment program, when executed by a processor, implements the steps of the video stream frame rate adjustment method as described above.
In the invention, the video frames of the original video stream are input into a time identification model frame by frame to obtain the time information of each video frame of the input time identification model; detecting whether a time abnormal event exists according to the time information of each video frame; if the time abnormal event exists, processing the time abnormal event to obtain a repaired video stream; and carrying out fixed frame rate processing on the repaired video stream to obtain the video stream with fixed frame rate. According to the invention, the time abnormal event existing in the original video stream is processed to obtain the repaired video stream, and then the fixed frame rate processing is carried out on the repaired video stream to obtain the video stream with the fixed frame rate, so that the problems of time error and unstable frame rate in the original video stream are solved.
Drawings
Fig. 1 is a schematic diagram of a hardware structure of a video stream frame rate adjusting apparatus according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a video stream frame rate adjustment method according to an embodiment of the present invention;
fig. 3 is a functional block diagram of an embodiment of a video stream frame rate adjustment apparatus according to the invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
In a first aspect, embodiments of the present invention provide a video stream frame rate adjustment apparatus, which may be an apparatus having a data processing function, such as a Personal Computer (PC), a notebook computer, or a server.
Referring to fig. 1, fig. 1 is a schematic diagram of a hardware structure of a video stream frame rate adjusting apparatus according to an embodiment of the present invention. In this embodiment of the present invention, the video stream frame rate adjusting apparatus may include a processor 1001 (e.g., a Central Processing Unit, CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used for realizing connection communication among the components; the user interface 1003 may include a Display screen (Display), an input unit such as a Keyboard (Keyboard); the network interface 1004 may optionally include a standard wired interface, a WIreless interface (e.g., a WI-FI interface, WI-FI interface); the memory 1005 may be a Random Access Memory (RAM) or a non-volatile memory (non-volatile memory), such as a magnetic disk memory, and the memory 1005 may optionally be a storage device independent of the processor 1001. Those skilled in the art will appreciate that the hardware configuration depicted in FIG. 1 is not intended to be limiting of the present invention, and may include more or less components than those shown, or some components in combination, or a different arrangement of components.
With continued reference to fig. 1, a memory 1005, which is one type of computer storage medium in fig. 1, may include an operating system, a network communication module, a user interface module, and a video stream frame rate adjustment program. The processor 1001 may call a video stream frame rate adjustment program stored in the memory 1005, and execute the video stream frame rate adjustment method provided by the embodiment of the present invention.
In a second aspect, an embodiment of the present invention provides a method for adjusting a frame rate of a video stream.
In an embodiment, referring to fig. 2, fig. 2 is a flowchart illustrating a video stream frame rate adjusting method according to an embodiment of the invention. As shown in fig. 2, the video stream frame rate adjustment method includes:
step S10, inputting the video frames of the original video stream into the time identification model frame by frame, and obtaining the time information of each video frame of the input time identification model;
in this embodiment, the time identification model is a pre-trained neural network model, and the time identification model identifies the input video frame in a specific image area of the video frame, so as to obtain time information corresponding to each video frame.
Further, in one embodiment, the time identification model is a convolutional neural network, and the convolutional neural network comprises a convolutional layer, a pooling layer and a fully-connected layer.
In this embodiment, the convolutional neural network comprises a convolutional layer, a pooling layer, and a fully-connected layer. The convolutional layer convolves the input 1 × 28 × 28 single-channel picture to 11 × 24 × 24, the pooling layer maximally pools the input to obtain 11 × 12 × 12 vectors, then expands the vectors into one (1584, 1) vector, and finally classifies the vectors into 11 nodes (R0, R1 … R10) by the fully-connected layer. When the recognition result is R10, it is determined that the time information corresponding to the input video frame is empty.
Step S20, detecting whether there is time abnormal event according to the time information of each video frame;
in this embodiment, it is able to detect whether there is a temporal anomaly according to the temporal information of each video frame, where the temporal anomaly includes: the video frame with empty existence time information, two adjacent video frames with discontinuous existence time information and the video frame with split needle freezing.
Further, in one embodiment, step S20 includes:
detecting whether a video frame with empty time information exists according to the time information of each video frame;
detecting whether two adjacent video frames with discontinuous time information exist according to the time information of each video frame;
detecting whether a video frame with frozen minute hands exists or not according to the time information of each video frame;
if the time information is an empty video frame, determining that a first time abnormal event exists;
if two adjacent video frames with discontinuous time information exist, determining that a second time abnormal event exists;
and if the video frame with the frozen minute hand exists, determining that a third time abnormal event exists.
In this embodiment, whether a video frame whose time information is empty exists is detected according to the time information of each video frame, and if a video frame whose time information is empty exists, it is determined that a first time abnormal event exists.
And detecting whether two adjacent video frames with discontinuous time information exist according to the time information of each video frame, and if two adjacent video frames with discontinuous time information exist (for example, the time information of the two adjacent video frames is 14:17:58 and 14:18:00 respectively), determining that a second time abnormal event exists.
And detecting whether a video frame with a frozen minute hand exists according to the time information of each video frame, and if the video frame with the frozen minute hand exists (for example, if the time information of the video frame subsequent to the video frame with the time information of 22:59 is 22:01, 22:02, 22:03 and 23:04 respectively, the video frame with the time information of 22:01, 22:02 and 22:03 respectively is the video frame with the frozen minute hand), determining that a third time abnormal event exists.
Step S30, if a time abnormal event exists, the time abnormal event is processed to obtain a repaired video stream;
in this embodiment, if there is a time exception, the time exception is processed according to a policy corresponding to the type of the time exception, and after all the time exceptions are processed, a repaired video stream can be obtained.
Further, in an embodiment, if there is a time exception, the step of processing the time exception includes:
when a first time abnormal event exists, acquiring time information t1 of a video frame with time information in the front of the video frame with empty time information and time information t2 of a video frame with time information in the back of the video frame with time information, and calculating the value delta t of t2-t 1;
when Δ t is 1, adding t1 to a specific position of a video frame whose temporal information is empty;
when Deltat is more than 1, detecting whether the number n of video frames with empty time information can be divided by (Deltat-1) in an integer way;
if n can be evenly divided by (delta t-1), adding (t1+1) to specific positions of the previous m video frames with empty time information, adding (t1+2) to specific positions of the next m video frames with empty time information, and so on according to the acquisition sequence of the video frames with empty time information, until adding (t2-1) to specific positions of the last m video frames with empty time information, wherein m is a quotient of n evenly divided by (delta t-1);
if n cannot be evenly divided by (Deltat-1), determining two adjacent frames with lowest similarity in the video frames with empty time information, and performing frame interpolation processing between the two adjacent frames with lowest similarity until the number n 'of the video frames with empty time information can be evenly divided by (Deltat-1), adding (t1+1) to the specific positions of the first m' video frames with empty time information, adding (t1+2) to the specific positions of the next m 'video frames with empty time information according to the acquisition sequence of the video frames with empty time information, and so on until (t2-1) is added to the specific positions of the last m' video frames with empty time information, and m 'is the quotient of n' evenly divided by (Deltat-1).
In the present embodiment, when there is a first time exceptional event, the time information t1 of the video frame having the time information before and the time information t2 of the video frame having the time information after the video frame having the empty time information are acquired, and the value Δ t of t2 to t1 is calculated.
When Δ t is 1, t1 is added to a specific position of the video frame where the time information is empty, where the specific position is a position on the video frame where the time information is displayed.
When Deltat > 1, it is detected whether the number n of video frames whose temporal information is empty can be divided by (Deltat-1) exactly.
For example, if t1 is 14:17:58, t2 is 14:18:05, Δ t is 7, if n is 30, n can be evenly divided by (7-1), and the quotient m is 5, then (14:17:58+1 is 14:17:59) is added to the specific position of the 1 st to 5 th video frames whose time information is null, (14:17:58+2 is 14:18:00) is added to the specific position of the 6 th to 10 th video frames whose time information is null, (14:17:58+3 is 14:18:01) is added to the specific position of the 11 th to 15 th video frames whose time information is null, and so on until (14:18:05-1 is 14:18:04) is added to the specific position of the 26 th to 30 th video frames whose time information is null, in the acquisition order of the video frames whose time information is null.
For example, if t1 is 14:17:58 and t2 is 14:18:05, Δ t is 7, and if n is 29, n cannot be evenly divided by (7-1), two adjacent frames with the lowest similarity in the video frames with empty time information are determined, and frame interpolation processing is performed between the two adjacent frames with the lowest similarity until the number n' of video frames with empty time information is evenly divided by (Δ t-1). The method for calculating the similarity of two adjacent frames is histogram matching, the similarity of every two adjacent frames in the video frames with the empty time information can be obtained through the histogram matching, so that the two adjacent frames with the lowest similarity are determined, and then frame interpolation processing is carried out between the two adjacent frames with the lowest similarity. When n is 29, only one frame needs to be inserted, so that the number n' of video frames with empty time information can be evenly divided by (Δ t-1), according to the acquisition sequence of the video frames with empty time information, (14:17:58+ 1: 14:17:59) is added to the specific position of the 1 st to 5 th video frames with empty time information, (14:17:58+ 2: 14:18:00) is added to the specific position of the 6 th to 10 th video frames with empty time information, (14:17:58+ 3: 14:18:01) is added to the specific position of the 11 th to 15 th video frames with empty time information, and so on until (14:18: 05-1: 14:18:04) is added to the specific position of the 26 th to 30 th video frames with empty time information.
Thus, the first time abnormal event in the original video stream can be solved.
Further, in an embodiment, if there is a time exception, the step of processing the time exception includes:
when a second time abnormal event exists, performing frame interpolation processing between two adjacent video frames with discontinuous time information, wherein the frame interpolation number is N (t4-t3-1), t3 is the time information of the previous frame in the two adjacent video frames with discontinuous time information, t4 is the time information of the next frame in the two adjacent video frames with discontinuous time information, and N is a preset value;
add (t3+1) to the particular location of the first N video frames inserted, (t3+2) to the particular location of the next N video frames inserted, and so on until (t4-1) is added to the particular location of the last N video frames inserted.
In this embodiment, for example, if t3 is 14:17:58 and t4 is 14:18:02, the number of interpolated frames is N (14:18:02-14:17:58-1), i.e., the number of interpolated frames is N × 3. Add (14:17:58+1 ═ 14:17:59) to the specific positions of the first N video frames inserted, (14:17:58+2 ═ 14:18:00) to the specific positions of the next N video frames inserted, and (14:18:02-1 ═ 14:18:01) to the specific positions of the last N video frames inserted. And N is a preset value and is set according to actual needs.
At this point, the second temporal exception in the original video stream is resolved.
Further, in an embodiment, if there is a time exception, the step of processing the time exception includes:
when a third time abnormal event exists, acquiring time information t5 of a video frame which is not frozen in the previous minute and is not frozen in the next minute of the frozen video frames and time information t6 of a video frame which is not frozen in the next minute;
dividing the video frame frozen by the pointer into (t6-t5-1) parts;
add (t5+1) to the particular position of the video frame in the first share, (t5+2) to the particular position of the video frame in the second share, and so on until (t6-1) is added to the particular position of the video frame in the last share.
In this embodiment, for example, t5 is 14:17:58, t6 is 14:18:02, the frame of the video that is frozen by the minute hand is divided into 3 shares, t5+1 is added to the specific position of the video frame in the first share, t5+2 is added to the specific position of the video frame in the second share, and so on until t6-1 is added to the specific position of the video frame in the last share.
Thus, the third time abnormal event in the original video stream can be solved.
Step S40, performing fixed frame rate processing on the repaired video stream to obtain a video stream with a fixed frame rate.
In this embodiment, the repaired video stream is a video stream without a time abnormal event, and based on this, the repaired video stream may be subjected to fixed frame rate processing, so as to obtain a video stream with a fixed frame rate.
Further, in one embodiment, step S40 includes:
dividing video frames with the same time in the repaired video stream into a group to obtain a plurality of video frame groups;
comparing the number of video frames contained in each video frame group with a preset number of frames;
if the number of the video frames contained in one video frame group is the same as the number of the preset frames, no processing is carried out;
if the number of video frames contained in a video frame group is greater than the preset number of frames, deleting the adjacent frames with the highest similarity in the video frame group until the number of video frames contained in the video frame group is the same as the preset number of frames;
if the number of the video frames contained in one video frame group is less than the preset number of frames, performing frame interpolation between adjacent frames with the lowest similarity in the video frame group until the number of the video frames contained in the video frame group is the same as the preset number of frames;
and obtaining the video stream with fixed frame rate based on all the video frame groups after traversing.
In this embodiment, video frames with the same time in the repaired video stream are divided into a group to obtain a plurality of video frame groups, for example, if the starting time and the ending time of the repaired video stream are 18:00:01 and 19:00:00, video frames with time information of 18:00:01 can be divided into a group, video frames with time information of 18:00:02 can be divided into a group, and so on, video frames with time information of 19:00:00 can be divided into a group, so that 60 video frame groups can be obtained. It should be noted that the illustration is only schematic.
And then comparing the number of video frames contained in each video frame group with the preset number of frames in sequence.
If the number of video frames included in a video frame group is the same as the preset number of frames, no processing is performed.
If the number of video frames in a video frame group is 68 and the number of preset frames is 60, determining the adjacent frame with the highest similarity in the video frame group, and then deleting the adjacent frame; at this time, the number of the video frames contained in the video frame group is 66, and is still greater than the preset number of frames, the adjacent frame with the highest similarity in the video frame group is determined again, and then the adjacent frame is deleted; at this time, if the number of the video frames included in the video frame group is 64 and is still greater than the preset number of frames, determining the adjacent frame with the highest similarity in the video frame group again, and then deleting the adjacent frame; at this time, if the number of the video frames included in the video frame group is 62 and is still greater than the preset number of frames, determining the adjacent frame with the highest similarity in the video frame group again, and then deleting the adjacent frame; at this time, the number of video frames included in the video frame group is 60, and the number of video frames is equal to the preset number of frames, the processing of the video frame group is stopped.
If the number of video frames included in a video frame group is 58 and the number of preset frames is 60, determining the adjacent frame with the lowest similarity in the video frame group, and performing frame interpolation between the adjacent frames with the lowest similarity, for example, after 2 frames are interpolated, the number of video frames included in the video frame group is 60 and is the same as the number of preset frames, and stopping processing the video frame group.
And obtaining the video stream with fixed frame rate based on all the video frame groups after traversal is completed.
In the embodiment, the video frames of the original video stream are input into the time identification model frame by frame, so as to obtain the time information of each video frame of the input time identification model; detecting whether a time abnormal event exists according to the time information of each video frame; if the time abnormal event exists, processing the time abnormal event to obtain a repaired video stream; and carrying out fixed frame rate processing on the repaired video stream to obtain the video stream with fixed frame rate. According to the embodiment, the time abnormal event existing in the original video stream is processed to obtain the repaired video stream, and then the fixed frame rate processing is performed on the repaired video stream to obtain the video stream with the fixed frame rate, so that the problems of time error and unstable frame rate in the original video stream are solved.
In a third aspect, an embodiment of the present invention further provides a device for adjusting a frame rate of a video stream.
In an embodiment, referring to fig. 3, fig. 3 is a functional module schematic diagram of an embodiment of a video stream frame rate adjusting device according to the present invention. As shown in fig. 3, the video stream frame rate adjusting apparatus includes:
the identification module 10 is configured to input video frames of an original video stream into the time identification model frame by frame, so as to obtain time information of each video frame of the input time identification model;
a detecting module 20, configured to detect whether a time exception event exists according to the time information of each video frame;
the repair module 30 is configured to, if a time exception event exists, process the time exception event to obtain a repaired video stream;
and a frame rate fixing module 40, configured to perform fixed frame rate processing on the repaired video stream to obtain a video stream with a fixed frame rate.
Further, in an embodiment, the detecting module 20 is configured to:
detecting whether a video frame with empty time information exists according to the time information of each video frame;
detecting whether two adjacent video frames with discontinuous time information exist according to the time information of each video frame;
detecting whether a video frame with frozen minute hands exists or not according to the time information of each video frame;
if the time information is an empty video frame, determining that a first time abnormal event exists;
if two adjacent video frames with discontinuous time information exist, determining that a second time abnormal event exists;
and if the video frame with the frozen minute hand exists, determining that a third time abnormal event exists.
Further, in an embodiment, the repair module 30 is configured to:
when a first time abnormal event exists, acquiring time information t1 of a video frame with time information in the front of the video frame with empty time information and time information t2 of a video frame with time information in the back of the video frame with time information, and calculating the value delta t of t2-t 1;
when Δ t is 1, adding t1 to a specific position of a video frame whose temporal information is empty;
when Deltat is more than 1, detecting whether the number n of video frames with empty time information can be divided by (Deltat-1) in an integer way;
if n can be evenly divided by (delta t-1), adding (t1+1) to specific positions of the previous m video frames with empty time information, adding (t1+2) to specific positions of the next m video frames with empty time information, and so on according to the acquisition sequence of the video frames with empty time information, until adding (t2-1) to specific positions of the last m video frames with empty time information, wherein m is a quotient of n evenly divided by (delta t-1);
if n cannot be evenly divided by (Deltat-1), determining two adjacent frames with lowest similarity in the video frames with empty time information, and performing frame interpolation processing between the two adjacent frames with lowest similarity until the number n 'of the video frames with empty time information can be evenly divided by (Deltat-1), adding (t1+1) to the specific positions of the first m' video frames with empty time information, adding (t1+2) to the specific positions of the next m 'video frames with empty time information according to the acquisition sequence of the video frames with empty time information, and so on until (t2-1) is added to the specific positions of the last m' video frames with empty time information, and m 'is the quotient of n' evenly divided by (Deltat-1).
Further, in an embodiment, the repair module 30 is configured to:
when a second time abnormal event exists, performing frame interpolation processing between two adjacent video frames with discontinuous time information, wherein the frame interpolation number is N (t4-t3-1), t3 is the time information of the previous frame in the two adjacent video frames with discontinuous time information, t4 is the time information of the next frame in the two adjacent video frames with discontinuous time information, and N is a preset value;
add (t3+1) to the particular location of the first N video frames inserted, (t3+2) to the particular location of the next N video frames inserted, and so on until (t4-1) is added to the particular location of the last N video frames inserted.
Further, in an embodiment, the repair module 30 is configured to:
when a third time abnormal event exists, acquiring time information t5 of a video frame which is not frozen in the previous minute and is not frozen in the next minute of the frozen video frames and time information t6 of a video frame which is not frozen in the next minute;
dividing the video frame frozen by the pointer into (t6-t5-1) parts;
add (t5+1) to the particular position of the video frame in the first share, (t5+2) to the particular position of the video frame in the second share, and so on until (t6-1) is added to the particular position of the video frame in the last share.
Further, in an embodiment, the frame rate fixing module 40 is configured to:
dividing video frames with the same time in the repaired video stream into a group to obtain a plurality of video frame groups;
comparing the number of video frames contained in each video frame group with a preset number of frames;
if the number of the video frames contained in one video frame group is the same as the number of the preset frames, no processing is carried out;
if the number of video frames contained in a video frame group is greater than the preset number of frames, deleting the adjacent frames with the highest similarity in the video frame group until the number of video frames contained in the video frame group is the same as the preset number of frames;
if the number of the video frames contained in one video frame group is less than the preset number of frames, performing frame interpolation between adjacent frames with the lowest similarity in the video frame group until the number of the video frames contained in the video frame group is the same as the preset number of frames;
and obtaining the video stream with fixed frame rate based on all the video frame groups after traversing.
Further, in one embodiment, the time identification model is a convolutional neural network, and the convolutional neural network comprises a convolutional layer, a pooling layer and a fully-connected layer.
The function implementation of each module in the video stream frame rate adjusting apparatus corresponds to each step in the video stream frame rate adjusting method embodiment, and the function and implementation process thereof are not described in detail here.
In a fourth aspect, the embodiment of the present invention further provides a readable storage medium.
The readable storage medium of the present invention stores a video stream frame rate adjustment program, wherein the video stream frame rate adjustment program, when executed by a processor, implements the steps of the video stream frame rate adjustment method as described above.
The method for implementing the video stream frame rate adjustment program when executed may refer to various embodiments of the video stream frame rate adjustment method of the present invention, and will not be described herein again.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) as described above and includes instructions for causing a terminal device to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. A method for adjusting the frame rate of a video stream, comprising:
inputting the video frames of the original video stream into the time identification model frame by frame to obtain the time information of each video frame of the input time identification model;
detecting whether a time abnormal event exists according to the time information of each video frame;
if the time abnormal event exists, processing the time abnormal event to obtain a repaired video stream;
and carrying out fixed frame rate processing on the repaired video stream to obtain the video stream with fixed frame rate.
2. The method for adjusting the frame rate of a video stream according to claim 1, wherein the step of detecting whether there is a temporal anomaly event according to the temporal information of each video frame comprises:
detecting whether a video frame with empty time information exists according to the time information of each video frame;
detecting whether two adjacent video frames with discontinuous time information exist according to the time information of each video frame;
detecting whether a video frame with frozen minute hands exists or not according to the time information of each video frame;
if the time information is an empty video frame, determining that a first time abnormal event exists;
if two adjacent video frames with discontinuous time information exist, determining that a second time abnormal event exists;
and if the video frame with the frozen minute hand exists, determining that a third time abnormal event exists.
3. The method for adjusting frame rate of video stream according to claim 2, wherein the step of processing the time exception event if the time exception event exists comprises:
when a first time abnormal event exists, acquiring time information t1 of a video frame with time information in the front of the video frame with empty time information and time information t2 of a video frame with time information in the back of the video frame with time information, and calculating the value delta t of t2-t 1;
when Δ t is 1, adding t1 to a specific position of a video frame whose temporal information is empty;
when Deltat is more than 1, detecting whether the number n of video frames with empty time information can be divided by (Deltat-1) in an integer way;
if n can be evenly divided by (delta t-1), adding (t1+1) to specific positions of the previous m video frames with empty time information, adding (t1+2) to specific positions of the next m video frames with empty time information, and so on according to the acquisition sequence of the video frames with empty time information, until adding (t2-1) to specific positions of the last m video frames with empty time information, wherein m is a quotient of n evenly divided by (delta t-1);
if n cannot be evenly divided by (Deltat-1), determining two adjacent frames with lowest similarity in the video frames with empty time information, and performing frame interpolation processing between the two adjacent frames with lowest similarity until the number n 'of the video frames with empty time information can be evenly divided by (Deltat-1), adding (t1+1) to the specific positions of the first m' video frames with empty time information, adding (t1+2) to the specific positions of the next m 'video frames with empty time information according to the acquisition sequence of the video frames with empty time information, and so on until (t2-1) is added to the specific positions of the last m' video frames with empty time information, and m 'is the quotient of n' evenly divided by (Deltat-1).
4. The method for adjusting frame rate of video stream according to claim 2, wherein the step of processing the time exception event if the time exception event exists comprises:
when a second time abnormal event exists, performing frame interpolation processing between two adjacent video frames with discontinuous time information, wherein the frame interpolation number is N (t4-t3-1), t3 is the time information of the previous frame in the two adjacent video frames with discontinuous time information, t4 is the time information of the next frame in the two adjacent video frames with discontinuous time information, and N is a preset value;
add (t3+1) to the particular location of the first N video frames inserted, (t3+2) to the particular location of the next N video frames inserted, and so on until (t4-1) is added to the particular location of the last N video frames inserted.
5. The method for adjusting frame rate of video stream according to claim 2, wherein the step of processing the time exception event if the time exception event exists comprises:
when a third time abnormal event exists, acquiring time information t5 of a video frame which is not frozen in the previous minute and is not frozen in the next minute of the frozen video frames and time information t6 of a video frame which is not frozen in the next minute;
dividing the video frame frozen by the pointer into (t6-t5-1) parts;
add (t5+1) to the particular position of the video frame in the first share, (t5+2) to the particular position of the video frame in the second share, and so on until (t6-1) is added to the particular position of the video frame in the last share.
6. The method for adjusting the frame rate of a video stream according to claim 1, wherein the step of performing fixed frame rate processing on the repaired video stream to obtain the video stream with a fixed frame rate comprises:
dividing video frames with the same time in the repaired video stream into a group to obtain a plurality of video frame groups;
comparing the number of video frames contained in each video frame group with a preset number of frames;
if the number of the video frames contained in one video frame group is the same as the number of the preset frames, no processing is carried out;
if the number of video frames contained in a video frame group is greater than the preset number of frames, deleting the adjacent frames with the highest similarity in the video frame group until the number of video frames contained in the video frame group is the same as the preset number of frames;
if the number of the video frames contained in one video frame group is less than the preset number of frames, performing frame interpolation between adjacent frames with the lowest similarity in the video frame group until the number of the video frames contained in the video frame group is the same as the preset number of frames;
and obtaining the video stream with fixed frame rate based on all the video frame groups after traversing.
7. The method according to any of claims 1 to 6, wherein the time-recognition model is a convolutional neural network, the convolutional neural network comprising a convolutional layer, a pooling layer and a fully-connected layer.
8. A video stream frame rate adjustment apparatus, the video stream frame rate adjustment apparatus comprising:
the identification module is used for inputting the video frames of the original video stream into the time identification model frame by frame to obtain the time information of each video frame of the input time identification model;
the detection module is used for detecting whether a time abnormal event exists according to the time information of each video frame;
the restoration module is used for processing the time abnormal event if the time abnormal event exists to obtain a restored video stream;
and the frame rate fixing module is used for carrying out fixed frame rate processing on the repaired video stream to obtain the video stream with the fixed frame rate.
9. A video stream frame rate adjustment device, characterized in that the video stream frame rate adjustment device comprises a processor, a memory, and a video stream frame rate adjustment program stored on the memory and executable by the processor, wherein the video stream frame rate adjustment program, when executed by the processor, implements the steps of the video stream frame rate adjustment method according to any one of claims 1 to 7.
10. A readable storage medium, having a video stream frame rate adjustment program stored thereon, wherein the video stream frame rate adjustment program, when executed by a processor, implements the steps of the video stream frame rate adjustment method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111639965.1A CN114339156B (en) | 2021-12-29 | 2021-12-29 | Video stream frame rate adjusting method, device, equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111639965.1A CN114339156B (en) | 2021-12-29 | 2021-12-29 | Video stream frame rate adjusting method, device, equipment and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114339156A true CN114339156A (en) | 2022-04-12 |
| CN114339156B CN114339156B (en) | 2023-03-14 |
Family
ID=81017602
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111639965.1A Active CN114339156B (en) | 2021-12-29 | 2021-12-29 | Video stream frame rate adjusting method, device, equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114339156B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1366271A (en) * | 2000-12-08 | 2002-08-28 | 松下电器产业株式会社 | Data transform device, data coding device and data recording device |
| CN102571588A (en) * | 2007-09-17 | 2012-07-11 | 华为技术有限公司 | Method, system and equipment for transmitting transcode rate adapt unit (TRAU) frame in packet network |
| US20160358537A1 (en) * | 2015-06-02 | 2016-12-08 | Samsung Electronics Co., Ltd. | Method for controlling display in electronic device and electronic device thereof |
| CN109286601A (en) * | 2017-07-20 | 2019-01-29 | 腾讯科技（深圳）有限公司 | A kind of processing method and terminal, computer storage medium of medium stream information |
| CN113052147A (en) * | 2021-04-30 | 2021-06-29 | 北京邮电大学 | Behavior identification method and device |
| CN113673311A (en) * | 2021-07-05 | 2021-11-19 | 浙江大华技术股份有限公司 | Traffic abnormal event detection method, equipment and computer storage medium |
| CN113850718A (en) * | 2021-06-01 | 2021-12-28 | 天翼智慧家庭科技有限公司 | Video synchronization space-time super-resolution method based on inter-frame feature alignment |
-
2021
- 2021-12-29 CN CN202111639965.1A patent/CN114339156B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1366271A (en) * | 2000-12-08 | 2002-08-28 | 松下电器产业株式会社 | Data transform device, data coding device and data recording device |
| CN102571588A (en) * | 2007-09-17 | 2012-07-11 | 华为技术有限公司 | Method, system and equipment for transmitting transcode rate adapt unit (TRAU) frame in packet network |
| US20160358537A1 (en) * | 2015-06-02 | 2016-12-08 | Samsung Electronics Co., Ltd. | Method for controlling display in electronic device and electronic device thereof |
| CN109286601A (en) * | 2017-07-20 | 2019-01-29 | 腾讯科技（深圳）有限公司 | A kind of processing method and terminal, computer storage medium of medium stream information |
| CN113052147A (en) * | 2021-04-30 | 2021-06-29 | 北京邮电大学 | Behavior identification method and device |
| CN113850718A (en) * | 2021-06-01 | 2021-12-28 | 天翼智慧家庭科技有限公司 | Video synchronization space-time super-resolution method based on inter-frame feature alignment |
| CN113673311A (en) * | 2021-07-05 | 2021-11-19 | 浙江大华技术股份有限公司 | Traffic abnormal event detection method, equipment and computer storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 沈航等: "多跳无线网络中具有时间意识的视频流控制协议", 《东南大学学报(自然科学版)》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114339156B (en) | 2023-03-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108734162B (en) | Method, system, equipment and storage medium for identifying target in commodity image | |
| JP6853379B2 (en) | Target person search method and equipment, equipment, program products and media | |
| CN108256479B (en) | Face tracking method and device | |
| EP3882820A1 (en) | Node classification method, model training method, device, apparatus, and storage medium | |
| US11704563B2 (en) | Classifying time series image data | |
| CN109711508B (en) | Image processing method and device | |
| US20180089839A1 (en) | Moving object detection based on motion blur | |
| CN111783665A (en) | Action recognition method and device, storage medium and electronic equipment | |
| US20090322875A1 (en) | Surveillance system, surveillance method and computer readable medium | |
| CN111767750A (en) | Image processing method and device | |
| CN110807110A (en) | Image searching method and device combining local and global features and electronic equipment | |
| CN115984307A (en) | Video object segmentation method and device, electronic equipment and storage medium | |
| CN109145987B (en) | Model generation method and device | |
| JP4572175B2 (en) | Unsteady image detection device, unsteady image detection method, and program implementing the method | |
| US11393091B2 (en) | Video image processing and motion detection | |
| CN114339156B (en) | Video stream frame rate adjusting method, device, equipment and readable storage medium | |
| CN113298852A (en) | Target tracking method and device, electronic equipment and computer readable storage medium | |
| CN111160410B (en) | Object detection method and device | |
| CN110633597A (en) | Driving region detection method and device | |
| CN112950687B (en) | Method and device for determining tracking state, storage medium and electronic equipment | |
| EP4099264B1 (en) | Learning device and learning method | |
| CN114360055A (en) | Behavior detection method, device and storage medium based on artificial intelligence | |
| AU2013263838A1 (en) | Method, apparatus and system for classifying visual elements | |
| CN113886628A (en) | Image retrieval method, device and storage medium | |
| CN112948630A (en) | List updating method, electronic device, storage medium and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |