CN114710637A - Low-delay processing method, device, equipment and medium for WEB terminal monitoring video stream - Google Patents
Low-delay processing method, device, equipment and medium for WEB terminal monitoring video stream Download PDFInfo
- Publication number
- CN114710637A CN114710637A CN202210407454.5A CN202210407454A CN114710637A CN 114710637 A CN114710637 A CN 114710637A CN 202210407454 A CN202210407454 A CN 202210407454A CN 114710637 A CN114710637 A CN 114710637A
- Authority
- CN
- China
- Prior art keywords
- video stream
- video
- preset
- web
- playing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 137
- 238000003672 processing method Methods 0.000 title claims description 21
- 230000007246 mechanism Effects 0.000 claims abstract description 75
- 238000012545 processing Methods 0.000 claims abstract description 36
- 238000000034 method Methods 0.000 claims abstract description 30
- 238000004590 computer program Methods 0.000 claims description 20
- 238000004891 communication Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 4
- 230000003139 buffering effect Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000001133 acceleration Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000005484 gravity Effects 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000010079 rubber tapping Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
- H04N5/91—Television signal processing therefor
- H04N5/93—Regeneration of the television signal or of selected parts thereof
- H04N5/95—Time-base error compensation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
Abstract
The invention discloses a method, a device, equipment and a medium for processing WEB terminal monitoring video stream with low time delay, wherein the method comprises the following steps: integrating the existing playing mechanism and adding a judging mechanism, and judging whether the video delay time of the video streaming unit exceeds the preset delay time through a WEB terminal; if the video delay time exceeds the preset delay time, judging whether the video delay time exceeds a preset frame tracing threshold value; if the frame tracking threshold value exceeds the preset frame tracking threshold value, the playing mechanism of the monitoring video stream is adjusted to be a frame skipping mechanism through the WEB end; and if the frame rate does not exceed the preset frame tracking threshold, adjusting the playing speed of the monitoring video stream through the WEB end. The WEB terminal can better effectively judge the playing mechanism required by the currently displayed video stream unit and dynamically adjust the playing speed in the frame tracking mechanism according to the network condition of the actual use scene, thereby effectively reducing the time delay of monitoring the video stream and improving the impression of users.
Description
Technical Field
The invention relates to the technical field of display of network cameras, in particular to a method and a device for processing low delay of monitoring video streams at a WEB terminal, mobile terminal equipment and a computer readable storage medium.
Background
In some application occasions with higher timeliness requirements on the monitoring video stream, such as the fields of security monitoring, telemedicine or intelligent production and the like, the monitoring video stream displayed on a video processing component at a WEB end is required to be played in real time, but due to the existence of factors such as network delay, a cache mechanism and the like, the monitoring video stream generally has great delay, if the monitoring video stream is not processed specially, the delay of the monitoring video stream displayed at the WEB end can reach a time of several seconds to dozens of seconds and is not equal, and even more, the delay can be calculated in minutes, the existing solutions have two types, one type adopts a frame skipping mechanism, the other type adopts a frame tracing mechanism, although the two types can eliminate the delay of the monitoring video stream to a certain extent, the preset period requirements on the monitoring video stream are great, the preset period is too large, and the displayed monitoring video has large-picture loss by directly adopting the frame skipping mechanism, the preset period is too small, the problems of frequent frame skipping and too fast frame tracing exist, and the watching effect of a user is extremely influenced.
Therefore, the existing scheme for solving the video delay has great limitation on the preset period, and the displayed monitoring video stream cannot be flexibly adjusted, so that the effect of really reducing the video delay is achieved.
Disclosure of Invention
The invention mainly aims to provide a WEB terminal monitoring video stream low-delay processing method, a WEB terminal monitoring video stream low-delay processing device, mobile terminal equipment and a computer readable storage medium, and aims to solve the technical problems that the existing video delay solution is large in limitation, the efficiency of reducing video delay is low, and the watching experience of a user is influenced.
In order to achieve the above object, the present invention provides a low latency processing method for a monitoring video stream at a WEB end, where the low latency processing method for the monitoring video stream at the WEB end includes the following steps:
carrying out time delay judgment on a video streaming unit through a WEB side, and judging whether the video delay time of the video streaming unit exceeds a preset delay time or not;
if the video delay time of the video streaming unit exceeds the preset delay time, judging whether the video delay time exceeds a preset frame tracing threshold value;
if the video delay time exceeds the preset frame tracing threshold, adjusting the playing mechanism of the video streaming unit to be a frame skipping mechanism through the WEB end;
and if the video delay time does not exceed the preset frame chasing threshold, adjusting the playing speed of the video stream unit through the WEB end, wherein the adjustment of the playing speed of the video stream unit refers to the adjustment of the playing mechanism of the video stream unit into a frame chasing mechanism.
Optionally, before the step of performing the delay judgment on the video stream unit through the WEB port, the method further includes:
after creating the link of the monitoring video stream based on the WEB terminal, sending a request for receiving an initial monitoring video stream to a monitoring video stream terminal;
and if the initial monitoring video stream is received based on the link, creating a video stream buffer area based on the WEB side, and writing the initial monitoring video stream into the video stream buffer area in a preset video format.
Optionally, after the step of writing the initial monitoring video stream into the video stream buffer in a preset video format, the method further includes:
extracting the monitoring video stream in the video format through the WEB end, and dividing the monitoring video stream at intervals of a preset period based on a video processing component in the WEB end to obtain a plurality of continuous video stream units;
and displaying the video stream units one by one on the basis of a video display component.
Optionally, after the step of determining whether the video delay time of the video streaming unit exceeds a preset delay time, the method further includes:
and if the video delay time of the video streaming unit does not exceed the preset delay time, not adjusting the playing mechanism of the video streaming unit.
Optionally, the step of adjusting the play speed of the video streaming unit through the WEB port includes:
predicting and calculating the playing speed of the video streaming unit based on the WEB side, and judging whether the playing speed exceeds a first preset playing speed;
if the playing speed exceeds a first preset playing speed, adjusting the playing speed to the first preset playing speed;
and if the playing speed does not exceed a first preset playing speed, not adjusting the playing speed.
Optionally, the step of adjusting the play speed of the video streaming unit through the WEB port includes:
and if the playing speed does not exceed a first preset playing speed and is less than a second preset playing speed, adjusting the playing speed to the second preset playing speed.
Optionally, the low-delay processing method for the monitoring video stream at the WEB end further includes the following steps:
if the WEB terminal detects that the playing of the current video stream unit is finished, detecting whether the monitoring video stream still has a next video stream unit;
if the monitoring video stream still has a next video stream unit, executing the step of performing delay judgment on the video stream unit through the WEB side;
and if the monitoring video stream does not have the next video stream unit, outputting the information of the playing end of the monitoring video stream through the WEB terminal.
In addition, in order to achieve the above object, the present invention further provides a low latency processing apparatus for monitoring a video stream at a WEB end, where the apparatus includes:
the judging module is used for carrying out delay judgment on a preset period of the monitoring video stream through a WEB end and judging whether the video delay time of the preset period exceeds the preset delay time or not;
if the video delay time of the preset period exceeds the preset delay time, judging whether the video delay time exceeds a preset frame tracing threshold value;
the adjusting module is used for adjusting the playing mechanism of the monitoring video stream to a frame skipping mechanism through the WEB end if the video delay time exceeds the preset frame tracing threshold;
and if the video delay time does not exceed the preset frame chasing threshold, adjusting the playing speed of the monitoring video stream through the WEB end, wherein the adjustment of the playing speed of the monitoring video stream refers to the adjustment of the playing mechanism of the monitoring video stream into a frame chasing mechanism.
In addition, in order to achieve the above object, the present invention further provides a mobile terminal device, where the mobile terminal device includes a memory, a processor, and a computer program stored on the memory and capable of running on the processor, and the computer program, when executed by the processor, implements the steps of the method for processing low latency of the monitoring video stream at the WEB end.
In addition, in order to achieve the above object, the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method for processing low latency of a monitoring video stream at a WEB end are implemented.
The scheme of the invention combines the existing video delay solutions, introduces a judgment mechanism on the basis of the combination, enables a WEB end to dynamically select a playing mechanism of a monitoring video stream according to a network environment, processes and displays the monitoring video stream through a video processing component on the WEB end, divides the monitoring video stream into a plurality of video stream units on the basis of a preset period, enables the WEB end to better judge the playing mechanism required by the currently displayed video stream unit through calculating the video delay time of the video stream, avoids the defects of large pictures caused by using a frame skipping mechanism when the preset period is overlarge, and solves the problems of frequent frame skipping and too fast frame tracing when the preset period is overlarge, and enables the WEB end to dynamically adjust the playing speed in the frame tracing mechanism according to the network condition of an actual use scene through introducing a calculation module, the vicious result of frequent video buffering is avoided, and the limitation of reducing video time delay by adopting a playing mechanism is effectively reduced.
Drawings
Fig. 1 is a schematic terminal structure diagram of a hardware operating environment according to an embodiment of the present invention;
fig. 2 is a schematic flowchart of an embodiment of a low-latency processing method for a WEB-side surveillance video stream according to the present invention;
FIG. 3 is a detailed flowchart of step S40 in FIG. 2;
fig. 4 is a schematic flowchart of another embodiment of a low-latency processing method for a monitoring video stream at a WEB end according to the present invention;
FIG. 5 is a diagram illustrating a prediction state of a prediction cycle.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The main solution of the embodiment of the invention is as follows: by combining the existing video delay solutions and introducing a judgment mechanism on the basis of the combination, the WEB end can dynamically select a playing mechanism for monitoring the video stream according to the network environment, and can dynamically adjust the video playing speed in the frame tracking mechanism according to the network condition of an actual use scene when the playing mechanism is the frame tracking mechanism.
In the prior art, due to the singleness and inflexibility of a playing mechanism for playing the monitoring video stream at the WEB end, no matter the frame skipping mechanism or the frame tracking mechanism, if the preset period is set to be small and low delay is pursued, a large number of vicious results such as frame skipping, frequent video buffering and the like can be generated, the vicious results are sensitive to network fluctuation, and if the preset period is set to be high, a large number of pictures of the video are lost, and the problem of video delay cannot be effectively solved.
The invention provides a solution, so that a WEB end can better effectively judge a playing mechanism required by a currently displayed video stream unit, can dynamically adjust the playing speed in a frame tracking mechanism according to the network condition of an actual use scene, effectively reduces the limitation of the playing mechanism, and achieves the effect of really reducing the video time delay.
As shown in fig. 1, fig. 1 is a schematic terminal structure diagram of a hardware operating environment according to an embodiment of the present invention.
The low-delay processing device for monitoring the video stream at the WEB end can be a PC (personal computer), a tablet computer, a portable computer and other mobile terminal equipment with a display function.
As shown in fig. 1, the terminal may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, a communication bus 1002. The communication bus 1002 is used to implement connection communication among these components. The user interface 1003 may include a Display screen (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory such as a disk memory. The memory 1005 may alternatively be a storage device separate from the processor 1001.
Optionally, the low-latency processing device for monitoring the video stream at the WEB end may further include a camera, a Radio Frequency (RF) circuit, a sensor, an audio circuit, a WiFi module, and the like. Such as light sensors, motion sensors, and other sensors. Specifically, the light sensor may include an ambient light sensor that may adjust the brightness of the display screen according to the brightness of ambient light, and a proximity sensor that may turn off the display screen and/or the backlight when the mobile terminal is moved to the ear. As one of the motion sensors, the gravity acceleration sensor can detect the magnitude of acceleration in each direction (generally, three axes), detect the magnitude and direction of gravity when the mobile terminal is stationary, and can be used for applications (such as horizontal and vertical screen switching, related games, magnetometer attitude calibration), vibration recognition related functions (such as pedometer and tapping) and the like for recognizing the attitude of the mobile terminal; of course, the mobile terminal may also be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, and an infrared sensor, which are not described herein again.
Those skilled in the art will understand that the structure of the low-latency processing apparatus for monitoring video stream at the WEB end shown in fig. 1 does not constitute a limitation to the low-latency processing apparatus for monitoring video stream at the WEB end, and may include more or less components than those shown in the figure, or combine some components, or arrange different components.
As shown in fig. 1, a memory 1005, which is a kind of computer storage medium, may include therein an operating system, a network communication module, a user interface module, and a computer program.
In the terminal shown in fig. 1, the network interface 1004 is mainly used for connecting to a backend server and performing data communication with the backend server; the user interface 1003 is mainly used for connecting a client (user side) and performing data communication with the client; and the processor 1001 may be configured to invoke the computer program stored in the memory 1005 and perform the following operations:
carrying out time delay judgment on a video streaming unit through a WEB side, and judging whether the video delay time of the video streaming unit exceeds a preset delay time or not;
if the video delay time of the video streaming unit exceeds the preset delay time, judging whether the video delay time exceeds a preset frame tracing threshold value;
if the video delay time exceeds the preset frame tracing threshold, adjusting the playing mechanism of the video streaming unit to be a frame skipping mechanism through the WEB end;
and if the video delay time does not exceed the preset frame chasing threshold, adjusting the playing speed of the video stream unit through the WEB end, wherein the adjustment of the playing speed of the video stream unit refers to the adjustment of the playing mechanism of the video stream unit into a frame chasing mechanism.
Further, the processor 1001 may call the computer program stored in the memory 1005, and also perform the following operations:
before the step of carrying out delay judgment on a video stream unit through a WEB side, after a link of the monitoring video stream is established based on the WEB side, a request for receiving an initial monitoring video stream is sent to a monitoring video stream side;
and if the initial monitoring video stream is received based on the link, creating a video stream buffer area based on the WEB side, and writing the initial monitoring video stream into the video stream buffer area in a preset video format.
Further, the processor 1001 may call the computer program stored in the memory 1005, and also perform the following operations:
after the step of writing the initial monitoring video stream into the video stream buffer area in a preset video format, extracting the monitoring video stream in the video format through the WEB end, and dividing the monitoring video stream at intervals of a preset period based on a video processing component in the WEB end to obtain a plurality of continuous video stream units;
and displaying the video stream units one by one on the basis of a video display component.
Further, the processor 1001 may call the computer program stored in the memory 1005, and also perform the following operations:
after the step of judging whether the video delay time of the video streaming unit exceeds the preset delay time, if the video delay time of the video streaming unit does not exceed the preset delay time, the playing mechanism of the video streaming unit is not adjusted.
Further, the processor 1001 may call the computer program stored in the memory 1005, and also perform the following operations:
the step of adjusting the playing speed of the video stream unit through the WEB end comprises the following steps: predicting and calculating the playing speed of the video streaming unit based on the WEB side, and judging whether the playing speed exceeds a first preset playing speed;
if the playing speed exceeds a first preset playing speed, adjusting the playing speed to the first preset playing speed;
and if the playing speed does not exceed the first preset playing speed, not adjusting the playing speed.
Further, the processor 1001 may call the computer program stored in the memory 1005, and also perform the following operations:
the step of adjusting the playing speed of the video stream unit through the WEB end comprises the following steps: and if the playing speed does not exceed a first preset playing speed and is less than a second preset playing speed, adjusting the playing speed to the second preset playing speed.
Further, the processor 1001 may call the computer program stored in the memory 1005, and also perform the following operations:
the low-delay processing method of the monitoring video stream at the WEB terminal also comprises the following steps: if the WEB terminal detects that the playing of the current video stream unit is finished, detecting whether the monitoring video stream still has a next video stream unit;
if the monitoring video stream still has a next video stream unit, executing the step of performing delay judgment on the video stream unit through the WEB side;
and if the monitoring video stream does not have the next video stream unit, outputting the information of the playing end of the monitoring video stream through the WEB terminal.
Further, as shown in fig. 1, the low-latency processing apparatus for a monitoring video stream at a WEB end further provides a low-latency processing apparatus for a monitoring video stream at a WEB end, including:
the judging module is used for carrying out delay judgment on a preset period of the monitoring video stream through a WEB end and judging whether the video delay time of the preset period exceeds the preset delay time or not;
if the video delay time of the preset period exceeds the preset delay time, judging whether the video delay time exceeds a preset frame tracing threshold value;
the adjusting module is used for adjusting a playing mechanism of the monitoring video stream into a frame skipping mechanism through the WEB terminal if the video delay time exceeds the preset frame tracing threshold;
and if the video delay time does not exceed the preset frame chasing threshold, adjusting the playing speed of the monitoring video stream through the WEB end, wherein the adjustment of the playing speed of the monitoring video stream refers to the adjustment of the playing mechanism of the monitoring video stream into a frame chasing mechanism.
Referring to fig. 2, an embodiment of the present invention provides a low latency processing method for a monitoring video stream at a WEB end, where the low latency processing method for the monitoring video stream at the WEB end includes:
step S10, performing a delay judgment on the video stream unit through a WEB (World Wide WEB, global Wide area network or World Wide WEB) port, and judging whether the video delay time of the video stream unit exceeds a preset delay time;
in this embodiment, the collection of video images is performed by an IPC (IP Camera), which includes codes, an ISP (Image Signal Processing), and the like, and the format of an initial monitoring video stream pushed to a WEB end is converted into an HTTP (Hyper text transfer Protocol) format by the IPC, because the WEB end for Processing and displaying the monitoring video stream is a distributed graphic information system based on a Hyper text and an HTTP.
After an initial monitoring video stream in an HTTP format is processed through a video processing component on a WEB end, the monitoring video stream is divided into a plurality of groups of processing periods with preset time T, namely video stream units, each video stream unit is subjected to delay judgment, and before the video stream units are subjected to delay judgment, a video frame timestamp C of the current monitoring video stream, a video frame timestamp E of the last frame of the preset period and the current network environment speed S need to be acquired. Since the video delay time of the video stream unit is obtained by calculating the video frame timestamp C of the current monitoring video stream and the video frame timestamp E of the last frame of the preset period, the calculation result of the video delay time of the video stream unit is E-C.
The preset delay time can be set by a user, and can also be the delay time which can be received by the current playing environment and is obtained by the WEB end through calculating the monitoring video stream.
Optionally, before the step of performing the delay judgment on the preset period of the monitoring video stream through the WEB end in step S10, the method further includes:
step S11, after creating the link of the monitoring video stream based on the WEB terminal, sending a request for receiving an initial monitoring video stream to a monitoring video stream terminal;
in this embodiment, after creating the link of the monitoring video stream, the WEB end will make a request for the monitoring video stream to the IPC.
Step S12, if the initial surveillance video stream is received based on the link, creating a video stream buffer area based on the WEB side, and writing the initial surveillance video stream into the video stream buffer area in a preset video format.
After the monitoring video stream sent by the IPC end is detected, the WEB end creates a buffer area for buffering the monitoring video playing data, and writes the HTTP format video of the monitoring video stream sent by the IPC into the buffer area after the creation is finished.
Optionally, after the step of writing the initial monitoring video stream in the preset video format into the video stream buffer in step S12, the method further includes:
step S13, extracting the monitoring video stream in the video format through the WEB end, and dividing the monitoring video stream at intervals of a preset period based on a video processing component in the WEB end to obtain a plurality of continuous video stream units;
step S14, displaying the video stream units one by one based on the video display component.
In the present embodiment, the Video processing components used are the HTML5 Video control and flv.js open source script, but the Video processing components are not limited to the above Video processing components, and a user may use different Video processing components according to different actual usage scenarios.
Secondly, the video image acquisition terminal in this embodiment is not limited to IPC, and any monitoring video stream sent by a terminal capable of video push can be effectively processed and displayed at a WEB end with low delay.
After the monitoring video stream converted into the HTTP format is extracted, the monitoring video stream is further processed by a video processing component carried by the WEB side, and the extracted monitoring video stream is divided at intervals of a preset period T, so that a plurality of video stream units with the preset period T can be obtained.
It should be noted that the video stream units are continuous and uninterrupted.
After the continuous video stream units are obtained through processing, the video stream units are displayed one by one through the video display component, and the delay judgment of the WEB side on the video stream units is judged in the display process.
Step S20, if the video delay time of the video streaming unit exceeds the preset delay time, determining whether the video delay time exceeds a preset frame-chasing threshold;
if the WEB end detects that the video delay time of the currently played video stream unit exceeds the preset delay time, it indicates that the time delay performance of the monitoring video stream is high, playing according to the original playing mechanism may cause a pause of a playing picture.
The preset frame tracking threshold value means that the video delay time within the threshold value range can realize the effect of reducing the delay by tracking frames by adjusting the playing speed.
Optionally, after the step of determining whether the video delay time of the video stream unit exceeds the preset delay time in step S20, the method further includes:
step S21, if the video delay time of the video stream unit does not exceed the preset delay time, the playing mechanism of the video stream unit is not adjusted;
if the WEB end detects that the video delay time of the currently played video stream unit does not exceed the preset delay time, it indicates that the time delay of the currently monitored video stream is low, and playing according to the original playing mechanism does not cause the phenomenon of pause of the played picture, so that excessive adjustment is not performed on the playing mechanism at this time, and unnecessary resource waste is avoided.
Step S30, if the video delay time exceeds the preset frame tracking threshold, the playing mechanism of the video stream unit is adjusted to a frame skipping mechanism through the WEB end;
if the WEB end detects that the video delay time of the currently played video stream unit exceeds the preset frame tracking threshold, it indicates that the current video delay time cannot achieve the reduction of the delay by adjusting the playing speed, for example, when the preset frame tracking threshold is equal to the preset time T, it means that it is possible to track up a real time frame within a preset period only when the playing speed exceeding 2 times speed is used, but the viewing effect at this time is very poor, and it is very likely that the monitoring video stream is buffered due to the excessively fast playing speed. Therefore, in order to avoid frequent buffer status of the monitored video stream caused by low watching quality due to delay and too fast frame tracking, the playing mechanism of the current monitored video stream can only be adjusted to be a frame skipping mechanism, i.e. the current monitored video stream directly skips to the video frame timestamp E of the last frame of the preset period, thereby eliminating the delay of the monitored video stream.
However, the above-mentioned delay condition usually occurs only at the initial stage and special condition of the monitored video stream playing, and after a relatively stable video stream connection is created at the WEB end, there is rarely a condition that the video delay time exceeds the preset frame chase threshold.
Step S40, if the video delay time does not exceed the preset frame chase threshold, adjusting the playing speed of the video stream unit through the WEB port, where adjusting the playing speed of the video stream unit refers to adjusting the playing mechanism of the video stream unit to a frame chase mechanism.
If the WEB end detects that the video delay time of the currently played video stream unit does not exceed the preset frame tracking threshold, it is stated that the current video delay time can achieve the effect of frame-chasing to reduce the delay by adjusting the play speed, for example, when the preset frame tracking threshold is smaller than the preset time T, the speed playing speed below 2 times of speed is adopted, for example, 1.3 times, the real-time frame can be tracked within a preset period, therefore, only the playing mechanism of the current monitoring video stream needs to be adjusted to the frame tracking mechanism, the playing speed of the monitoring video stream is dynamically adjusted according to the result of the playing speed predicted and calculated by the WEB terminal, so that the phenomenon that the playing speed is too small due to the fact that the playing speed cannot be intelligently and dynamically adjusted according to the real-time network environment in the conventional frame tracking mechanism is avoided, the requirement of real-time playing can not be met, and the influence on the user impression is caused when the playing speed is too high.
In this embodiment, by combining the existing video delay solutions, the WEB end can perform automatic switching of the playing mechanism according to the actual playing situation, and by introducing the judgment mechanism on the basis of the combination, the WEB end can better perform effective judgment on the playing mechanism required by the currently displayed video stream unit, thereby avoiding the deletion of a large picture existing in the frame skipping mechanism when the preset period is too large, and when the preset period is too small, the problems of frequent frame skipping and too fast frame tracking exist, namely, the effective reduction of the monitoring video stream delay is realized, and the impression of the user is also improved.
Further, referring to fig. 3 and fig. 5, an embodiment of the present invention provides a low latency processing method for monitoring a video stream at a WEB end, where based on the embodiment shown in step S40, the step of adjusting the play speed of the video stream unit by the WEB end includes:
step S41, based on the WEB end, predicting and calculating the playing speed of the video stream unit, and judging whether the playing speed exceeds a first preset playing speed;
step S42, if the playback speed exceeds a first preset playback speed, adjusting the playback speed to the first preset playback speed;
step S43, if the playing speed does not exceed a first preset playing speed, the playing speed is not adjusted;
optionally, the step of adjusting the playing speed of the video stream unit through the WEB port in step S43 includes:
in step S44, if the playing speed does not exceed a first preset playing speed and is less than a second preset playing speed, the playing speed is adjusted to the second preset playing speed.
Firstly, the WEB side carries out pre-storage calculation of predicted cacheable video time F on the playing speed of the monitoring video stream according to the current network environment speed S and the code rate of the IPC pushed monitoring video stream, and the formula is as follows:
fig. 5 shows the prediction status of a predetermined period, wherein C 'represents the ideal video frame time stamp after a prediction period, and E' represents the predicted buffered video frame time stamp, as can be seen from the following figures:
E’＝E+F
C’＝E’-δ
the play speed of the current monitoring video stream obtained from the calculation result is:
the first preset playing speed is the highest playing speed which is set by a user or calculated by a WEB end and can be accepted by the user, and the second preset playing speed is the lowest playing speed which is set by the user or calculated by the WEB end and can meet the real-time playing requirement. The second preset playing speed value is smaller than the first preset playing speed value.
Assuming that the first preset playing speed is 1.3 and the second preset playing speed is 1, when P exceeds 1.3 but does not exceed the preset frame tracking threshold, the current playing speed is set to 1.3, when P is less than 1.3 but greater than 1, the current playing speed is not changed, and when P is less than 1, the current network speed is blocked and cannot meet the requirement of real-time playing, the current playing speed is set to 1.
It should be noted that the playing speed can be set according to different situations of the actual application scenario, so as to achieve adaptive dynamic adjustment of the playing speed.
In this embodiment, the playing speed of the monitoring video stream is predicted and calculated through the WEB end, so that the WEB end can dynamically adjust the playing speed of the monitoring video stream according to an actual application scene.
Referring to fig. 4, an embodiment of the present invention provides a method for processing a low latency of a monitoring video stream at a WEB end, where the method for processing the low latency of the monitoring video stream at the WEB end further includes:
step S50, if the WEB end detects that the playing of the current video stream unit is finished, it detects whether the monitoring video stream has the next video stream unit;
step S51, if the monitoring video stream still has the next video stream unit, executing the step of performing delay judgment on the video stream unit through the WEB side;
step S52, if the monitored video stream does not have the next video stream unit, outputting the information of the end of playing the monitored video stream through the WEB end.
In this embodiment, because a plurality of video stream units exist in one monitoring video stream, after the operation of the current video stream unit is finished, the WEB side needs to detect whether the monitoring video stream still has a next non-operating video stream unit, so as to avoid playing incompleteness caused by the absence of the video stream unit.
The output information of the end of the playing of the surveillance video stream can be a word of the playing picture output no-surveillance video stream, a prompt of the end of the playing of the voice output, and the like.
Further, an embodiment of the present invention further provides a low latency processing apparatus for a monitoring video stream at a WEB end, where the low latency processing apparatus for a monitoring video stream at a WEB end includes:
the judging module A10 is used for performing delay judgment on a preset period of the monitoring video stream through a WEB end, and judging whether the video delay time of the preset period exceeds the preset delay time;
if the video delay time of the preset period exceeds the preset delay time, judging whether the video delay time exceeds a preset frame tracing threshold value;
an adjusting module a20, configured to adjust, by the WEB end, a playing mechanism of the monitored video stream to a frame skipping mechanism if the video delay time exceeds the preset frame tracing threshold;
and if the video delay time does not exceed the preset frame chasing threshold, adjusting the playing speed of the monitoring video stream through the WEB end, wherein the adjustment of the playing speed of the monitoring video stream refers to the adjustment of the playing mechanism of the monitoring video stream into a frame chasing mechanism.
In addition, an embodiment of the present invention further provides a mobile terminal device, where the mobile terminal device includes a memory, a processor, and a computer program that is stored in the memory and can be run on the processor, and when the computer program is executed by the processor, the steps of the method for processing a low latency of a monitoring video stream at a WEB end are implemented.
In addition, the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method for processing low delay of a monitoring video stream at a WEB end are implemented.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the description of the foregoing embodiments, it is clear to those skilled in the art that the method of the foregoing embodiments may be implemented by software plus a necessary general hardware platform, and certainly may also be implemented by hardware, but in many cases, the former is a better implementation. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) as described above and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention. The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. A low-delay processing method for a monitoring video stream at a WEB terminal is characterized by comprising the following steps:
carrying out time delay judgment on a video streaming unit through a WEB side, and judging whether the video delay time of the video streaming unit exceeds a preset delay time or not;
if the video delay time of the video streaming unit exceeds the preset delay time, judging whether the video delay time exceeds a preset frame tracing threshold value;
if the video delay time exceeds the preset frame tracing threshold, adjusting the playing mechanism of the video streaming unit to be a frame skipping mechanism through the WEB end;
and if the video delay time does not exceed the preset frame chasing threshold, adjusting the playing speed of the video stream unit through the WEB end, wherein the adjustment of the playing speed of the video stream unit refers to the adjustment of the playing mechanism of the video stream unit into a frame chasing mechanism.
2. The WEB-side surveillance video stream low-latency processing method according to claim 1, wherein before the step of performing the latency judgment on the video stream unit through the WEB side, the method further comprises:
after creating the link of the monitoring video stream based on the WEB terminal, sending a request for receiving an initial monitoring video stream to a monitoring video stream terminal;
and if the initial monitoring video stream is received based on the link, creating a video stream buffer area based on the WEB side, and writing the initial monitoring video stream into the video stream buffer area in a preset video format.
3. The WEB-side surveillance video stream low-latency processing method according to claim 2, wherein after the step of writing the initial surveillance video stream into the video stream buffer in a preset video format, the method further comprises:
extracting the monitoring video stream in the video format through the WEB end, and dividing the monitoring video stream at intervals of a preset period based on a video processing component in the WEB end to obtain a plurality of continuous video stream units;
and displaying the video stream units one by one on the basis of a video display component.
4. The WEB-side surveillance video stream low-latency processing method according to claim 1, wherein after the step of determining whether the video latency of the video stream unit exceeds the preset latency, the method further comprises:
and if the video delay time of the video streaming unit does not exceed the preset delay time, not adjusting the playing mechanism of the video streaming unit.
5. The WEB-side monitoring video stream low-latency processing method according to claim 1, wherein the step of adjusting the play speed of the video stream unit through the WEB side includes:
predicting and calculating the playing speed of the video streaming unit based on the WEB side, and judging whether the playing speed exceeds a first preset playing speed;
if the playing speed exceeds a first preset playing speed, adjusting the playing speed to the first preset playing speed;
and if the playing speed does not exceed the first preset playing speed, not adjusting the playing speed.
6. The WEB-side monitoring video stream low-latency processing method according to claim 5, wherein the step of adjusting the play speed of the video stream unit through the WEB side includes:
and if the playing speed does not exceed a first preset playing speed and is less than a second preset playing speed, adjusting the playing speed to the second preset playing speed.
7. The WEB side surveillance video stream low-latency processing method according to claim 1, wherein the WEB side surveillance video stream low-latency processing method further comprises the following steps:
if the WEB terminal detects that the playing of the current video stream unit is finished, detecting whether the monitoring video stream still has a next video stream unit;
if the monitoring video stream still has a next video stream unit, executing the step of performing delay judgment on the video stream unit through the WEB side;
and if the monitoring video stream does not have the next video stream unit, outputting the information of the playing end of the monitoring video stream through the WEB terminal.
8. A low-delay processing device for monitoring video stream at a WEB side is characterized by comprising:
the judging module is used for carrying out delay judgment on the video streaming unit through a WEB side and judging whether the video delay time of the video streaming unit exceeds the preset delay time or not;
if the video delay time of the video streaming unit exceeds the preset delay time, judging whether the video delay time exceeds a preset frame tracing threshold value;
an adjusting module, configured to adjust, if the video delay time exceeds the preset frame tracking threshold, a playing mechanism of the video stream unit to a frame skipping mechanism through the WEB end;
and if the video delay time does not exceed the preset frame chasing threshold, adjusting the playing speed of the video stream unit through the WEB end, wherein the adjustment of the playing speed of the video stream unit refers to the adjustment of the playing mechanism of the video stream unit into a frame chasing mechanism.
9. A mobile terminal device, characterized in that the mobile terminal device comprises a memory, a processor and a computer program stored on the memory and executable on the processor, and the processor implements the steps of the low-latency processing method for the WEB-side surveillance video stream according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when being executed by a processor, the computer program implements the steps of the method for low-latency processing of a WEB-side surveillance video stream according to any one of claims 1 to 7.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210407454.5A CN114710637A (en) | 2022-04-18 | 2022-04-18 | Low-delay processing method, device, equipment and medium for WEB terminal monitoring video stream |
| PCT/CN2023/088477 WO2023202495A1 (en) | 2022-04-18 | 2023-04-14 | Web end monitoring video stream low-delay processing method and apparatus, device, and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210407454.5A CN114710637A (en) | 2022-04-18 | 2022-04-18 | Low-delay processing method, device, equipment and medium for WEB terminal monitoring video stream |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114710637A true CN114710637A (en) | 2022-07-05 |
Family
ID=82175338
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210407454.5A Pending CN114710637A (en) | 2022-04-18 | 2022-04-18 | Low-delay processing method, device, equipment and medium for WEB terminal monitoring video stream |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN114710637A (en) |
| WO (1) | WO2023202495A1 (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115665486A (en) * | 2022-12-26 | 2023-01-31 | 理工全盛(北京)科技有限公司 | Data processing method for playing video stream web end of unmanned aerial vehicle tracking device |
| WO2023202495A1 (en) * | 2022-04-18 | 2023-10-26 | 深圳创维－Rgb电子有限公司 | Web end monitoring video stream low-delay processing method and apparatus, device, and medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107222715A (en) * | 2017-06-15 | 2017-09-29 | 杭州联吉技术有限公司 | A kind of method, device and the computer-readable recording medium of remote playing video |
| CN107371061A (en) * | 2017-08-25 | 2017-11-21 | 普联技术有限公司 | A kind of video stream playing method, device and equipment |
| WO2017210027A1 (en) * | 2016-06-01 | 2017-12-07 | Amazon Technologies, Inc. | Catching up to the live playhead in live streaming |
| CN107846621A (en) * | 2017-09-28 | 2018-03-27 | 北京京东尚科信息技术有限公司 | Delay process method, system and electronic equipment |
| CN111885334A (en) * | 2020-08-26 | 2020-11-03 | 杭州速递科技有限公司 | Method for reducing delay of real-time frame pursuit of audio and video |
| CN114302230A (en) * | 2021-12-30 | 2022-04-08 | 苏州万店掌网络科技有限公司 | Front-end playing component optimization method, system and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114710637A (en) * | 2022-04-18 | 2022-07-05 | 深圳创维－Rgb电子有限公司 | Low-delay processing method, device, equipment and medium for WEB terminal monitoring video stream |
-
2022
- 2022-04-18 CN CN202210407454.5A patent/CN114710637A/en active Pending
-
2023
- 2023-04-14 WO PCT/CN2023/088477 patent/WO2023202495A1/en unknown
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017210027A1 (en) * | 2016-06-01 | 2017-12-07 | Amazon Technologies, Inc. | Catching up to the live playhead in live streaming |
| CN107222715A (en) * | 2017-06-15 | 2017-09-29 | 杭州联吉技术有限公司 | A kind of method, device and the computer-readable recording medium of remote playing video |
| CN107371061A (en) * | 2017-08-25 | 2017-11-21 | 普联技术有限公司 | A kind of video stream playing method, device and equipment |
| CN107846621A (en) * | 2017-09-28 | 2018-03-27 | 北京京东尚科信息技术有限公司 | Delay process method, system and electronic equipment |
| CN111885334A (en) * | 2020-08-26 | 2020-11-03 | 杭州速递科技有限公司 | Method for reducing delay of real-time frame pursuit of audio and video |
| CN114302230A (en) * | 2021-12-30 | 2022-04-08 | 苏州万店掌网络科技有限公司 | Front-end playing component optimization method, system and storage medium |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023202495A1 (en) * | 2022-04-18 | 2023-10-26 | 深圳创维－Rgb电子有限公司 | Web end monitoring video stream low-delay processing method and apparatus, device, and medium |
| CN115665486A (en) * | 2022-12-26 | 2023-01-31 | 理工全盛(北京)科技有限公司 | Data processing method for playing video stream web end of unmanned aerial vehicle tracking device |
| CN115665486B (en) * | 2022-12-26 | 2023-03-21 | 理工全盛(北京)科技有限公司 | Data processing method for playing video stream web end of unmanned aerial vehicle tracking device |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2023202495A1 (en) | 2023-10-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10187668B2 (en) | Method, system and server for live streaming audio-video file | |
| US10447756B2 (en) | Method and apparatus for improving quality of experience in sharing screen among devices, and recording medium thereof | |
| CN107135422B (en) | Information processing apparatus, information processing method, and computer program | |
| CN114710637A (en) | Low-delay processing method, device, equipment and medium for WEB terminal monitoring video stream | |
| KR101526081B1 (en) | System and method for controllably viewing digital video streams captured by surveillance cameras | |
| WO2021143479A1 (en) | Media stream transmission method and system | |
| CN107257510B (en) | Video unified playing method, terminal and computer readable storage medium | |
| CN108737884B (en) | Content recording method and equipment, storage medium and electronic equipment | |
| CN110248245B (en) | Video positioning method and device, mobile terminal and storage medium | |
| CN112399123B (en) | Video definition adjusting method and device, electronic equipment and storage medium | |
| US20140297883A1 (en) | Method and apparatus for altering bandwidth consumption | |
| CN112328941A (en) | Application screen projection method based on browser and related device | |
| CN113542795A (en) | Video processing method and device, electronic equipment and computer readable storage medium | |
| CN113535063A (en) | Live broadcast page switching method, video page switching method, electronic device and storage medium | |
| US9729931B2 (en) | System for managing detection of advertisements in an electronic device, for example in a digital TV decoder | |
| KR101311463B1 (en) | remote video transmission system | |
| WO2021143388A1 (en) | Bitrate switching method and device | |
| CN107484035B (en) | Closed caption display method, apparatus and computer readable storage medium | |
| US20120154678A1 (en) | Receiving device, screen frame transmission system and method | |
| CN109922366B (en) | Equipment parameter adjusting method, device, equipment and medium | |
| CN112543348A (en) | Remote screen recording method, device, equipment and computer readable storage medium | |
| CN111711857A (en) | Multi-channel video playing method, terminal and readable storage medium | |
| CN114647468B (en) | Screen projection image display method and device, electronic equipment and storage medium | |
| CN111541940B (en) | Motion compensation method and device for display equipment, television and storage medium | |
| CN102547204B (en) | Receiving device, screen picture transmission system and screen picture transmission method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |