CN115150675B

CN115150675B - Reverse order playing method, system and reverse order slicing method

Info

Publication number: CN115150675B
Application number: CN202210759187.8A
Authority: CN
Inventors: 康守伟; 黄至春; 李嘉; 张剑; 姜涛
Original assignee: Suzhou Keyuan Software Technology Development Co ltd; Suzhou Keda Technology Co Ltd
Current assignee: Suzhou Keyuan Software Technology Development Co ltd; Suzhou Keda Technology Co Ltd
Priority date: 2022-06-30
Filing date: 2022-06-30
Publication date: 2023-09-26
Anticipated expiration: 2042-06-30
Also published as: CN115150675A

Abstract

The application provides a reverse order playing method, a system and a reverse order slicing method, wherein the reverse order playing method of a webpage playing end comprises the following steps: sending a reverse playing request to a server side, and enabling the server side to carry out reverse sequence slicing on a video stream to be played according to the reverse playing request to obtain a slice sequence; sequentially obtaining reverse sequence fragments from a server side; setting a timestamp offset attribute value of a source buffer of an associated media source extension module for the obtained current reverse-order fragments and buffering the current reverse-order fragments into the source buffer; the media source expansion module regenerates a second timestamp for the frame sequence in the current reverse-order slicing according to the timestamp offset attribute value associated with the current reverse-order slicing, so that the second timestamp of the frame sequence in the current reverse-order slicing monotonically increases according to the arrangement sequence; and decrementing the current time value of the video element according to the step value, so that the current time value advances the reverse order playing of the video stream to be played according to the decrementing direction of the second timestamp. The application realizes the video reverse order playing in the browser.

Description

Reverse order playing method, system and reverse order slicing method

Technical Field

The application relates to the field of video playing, in particular to a reverse order playing method, a reverse order playing system and a reverse order slicing method.

Background

Video monitoring video playback is a basic function in security solution application, and usually, a time period in which video resources can be played back is displayed at a client in a time axis manner, and a user selects a certain time point to start positive-sequence or reverse-sequence playback, or switches playback directions back and forth in the playback process, so that a target image concerned can be flexibly and rapidly found and the change process of the target image can be observed. The reverse playback (simply called reverse playback) is used for positioning security events in video monitoring videos in the security field, and has very practical value.

In the security solution, the application of audio and video media (including video-on-demand) needs to meet the requirements of image retrieval response instantaneity, multiple functions, flexibility of cross-service integration and the like, and at present, the main challenges are as follows:

1) Because the ability support and interface opening of a Web browser to audio and video media are very limited, most of audio and video applications are based on a client-server architecture (CS architecture), but the application is very difficult in meeting the aspects of cross-service integration flexibility and multi-version compatibility, and is not as flexible and efficient as a browser-server architecture (BS architecture);

2) Later evolved pseudo BS architecture based on Web plug-in technology (such as ActiveX (open integrated platform), NPAPI (Netscape Plugin Application Programming Interface, internet protocol plug-in application program interface), PPAPI (PepperPlugin ApplicationProgramming Interface, pepper plug-in application program interface), etc.), the audio and video media core function of the client is essentially implemented in a local (Native) mode, which has obvious defects in terms of plug-in installation experience, version compatibility and security, especially considering security, and currently mainstream browsers do not support such Web plug-in technology any more;

3) With the development of Web browser technology, especially the advent of the 5 th generation HTML technical specification, the browser is currently available to support media playback related technologies such as video (video) tags, webrtc (Web Real-Time Communication ), MSE (Media Source Extensions, media source expansion), WASM (a binary instruction set), webcodes (allowing Web applications to encode and decode audio and video), and the like. The technologies are obviously improved in the aspect of meeting the application of audio and video media, but for special functions such as 'reverse playing', an open and ready-to-use interface can not be supported at present.

In the traditional scheme, video reverse playing is mainly based on a CS architecture, because the local client can completely and autonomously control the decoding, rendering and other processes of the code stream, and can control the forward sequence and reverse sequence playing of the code stream frame by frame.

However, implementing the reverse playback function based on the BS architecture is limited by the capabilities of the browser and its open interface, and faces a few difficulties. For example, the video tag and the Webrtc scheme provided by Web are realized and packaged by a browser in transmission, decoding and display, and an application layer cannot flexibly control the processes, and particularly, the code stream is cached in advance as required when the process is inverted to ensure the fluency; for the WASM scheme, although the decoding and rendering processes of the code stream can be completely and autonomously controlled, the decoding performance is poor, and many scenes cannot be satisfied; for the webcodes scheme, although the frame data before and after decoding can be controlled, only a few types of new browsers are supported at present, and the compatibility cannot meet the requirement. For the MSE scheme, although the transmission mode can be controlled autonomously, the decoding and rendering processes are encapsulated by a browser, and the decoded frame data cannot be directly operated.

Therefore, how to realize the video reverse order playing in the browser is a technical problem to be solved urgently by the person skilled in the art.

Disclosure of Invention

In order to overcome the defects in the prior art, the application provides a reverse order playing method, a system and a reverse order slicing method, so that video reverse order playing is realized in a browser.

According to one aspect of the present application, there is provided a reverse order playing method applied to a web page playing end, including:

the method comprises the steps that the webpage playing end sends a reverse playing request to a server end, the server end carries out reverse slicing on a video stream to be played according to the reverse playing request to obtain a slicing sequence, reverse slicing of the slicing sequence is opposite to acquisition time arrangement, frame sequences in each reverse slicing are forward to the acquisition time arrangement, and first time stamps of the frame sequences of the slicing sequence are monotonically increased according to arrangement sequence after being set;

the webpage playing end sequentially acquires reverse sequence fragments from the server end;

the webpage playing end sets a timestamp offset attribute value of a source buffer of an associated media source expansion module for the obtained current reverse-order fragments and buffers the current reverse-order fragments into the source buffer, so that the timestamp offset attribute value associated with the current reverse-order fragments is the time length of the last reverse-order fragments subtracted from the timestamp offset attribute value associated with the last reverse-order fragments;

The media source expansion module of the webpage playing end regenerates a second timestamp for the frame sequence in the current reverse-order slicing according to the timestamp offset attribute value associated with the current reverse-order slicing, so that the second timestamp of the frame sequence in the current reverse-order slicing monotonically increases according to the arrangement sequence, and the second timestamp decreasing frame interval of the starting frame of the frame sequence in the last reverse-order slicing is the second timestamp of the ending frame of the frame sequence in the current reverse-order slicing;

the webpage playing end plays the video stream to be played in a reverse order by adopting video elements, wherein the webpage playing end decrements the current time value of the video elements according to step values, so that the current time value advances the reverse order playing of the video stream to be played according to the decrementing direction of the second time stamp.

In some embodiments of the present application, the step-down of the current time value of the video element by the web page playing end so that the step-down of the current time value in the second timestamp includes:

and responding to the network transmission parameters between the webpage playing end and the server end being smaller than the set network transmission parameters, wherein the webpage playing end caches a plurality of reverse sequence fragments in the source cache and then decrements the current time value of the video element according to the step value, so that the current time value advances the reverse sequence playing of the video stream to be played according to the decrementing direction of the second timestamp.

In some embodiments of the present application, the initial value of the current time value is set to a maximum timestamp of the reverse-order slice in the source buffer, the step value is less than or equal to a frame interval of the reverse-order slice, the web page playing end decrements the current time value of the video element according to the step value, so that the current time value advances the reverse-order playing of the video stream to be played according to the decrementing direction of the second timestamp includes:

the webpage playing end regularly decrements the current time value of the video element according to the step value, so that the current time value advances the reverse order playing of the video stream to be played according to the decrementing direction of the second timestamp;

and responding to the current time value equal to the minimum time stamp of the reverse sequence fragment in the source cache, and stopping the reverse sequence playing of the video stream to be played by the webpage playing end.

According to still another aspect of the present application, there is also provided a reverse order slicing method, applied to a server, including:

the server receives a reverse playing request sent by a webpage playing end, wherein the reverse playing request comprises an identifier of a video stream to be played and initial playing time, and the video stream to be played corresponding to the identifier of the video stream to be played is a local video file of the server end;

The server determines the current frame position of the video stream to be played based on the initial playing time;

the server searches the key frame closest to the current frame position in the video stream to be played in reverse order;

the server end forms the current original slicing from the current frame position to the previous frame of the searched nearest key frame, and the current original slicing is used for packaging into reversed-order slicing;

taking the key frame position of the current original slice as the current frame position to position the next original slice until the reverse-order slice is finished;

and reversing the reverse sequence slicing to the acquisition time arrangement to generate a slice sequence, wherein the frame sequence in each reverse sequence slicing is forward to the acquisition time arrangement, the first time stamp of the frame sequence of the slice sequence monotonically increases according to the arrangement sequence, and the slice sequence is used for being sent to the webpage playing end to realize reverse sequence playing of the video stream to be played.

In some embodiments of the present application, the current raw tile is for packaging as a reverse order tile; the step of taking the key frame position of the current original slice as the current frame position to locate the next original slice until the reverse-order slice is finished comprises the following steps: in response to the number of original fragments for packaging into the current reverse sequence fragments not reaching a set number threshold, the server takes the key frame position of the current original fragments as the current frame position to position the next original fragments until the reverse sequence fragments are finished;

In response to the number of original tiles for encapsulation into the current reverse order tile reaching a set number threshold:

the server side sequentially reads a frame sequence from the original intra-slice positive sequence of the set quantity threshold according to the acquisition sequence of the original slices;

the server encapsulates each original fragment into a current reverse sequence fragment, generates a first timestamp which monotonically increases according to the arrangement sequence for the read frame sequence of the current reverse sequence fragment, and generates the first timestamp of the frame sequence of the current reverse sequence fragment according to the first timestamp of the frame sequence of the previous reverse sequence fragment;

and the server takes the key frame position of the current original slice as the current frame position to position the original slice of the next reverse-order slice until the reverse-order slice is finished.

the server receives a reverse playing request sent by a webpage playing end, wherein the reverse playing request comprises an identifier of a video stream to be played and a starting playing time, and the video stream to be played corresponding to the identifier of the video stream to be played is stored in a networking system;

the server takes the initial playing time as the request time for generating a positive sequence playing request;

The server side generates a positive sequence playing request and sends the positive sequence playing request to the networking system;

the server receives a video stream to be played, which is sent by the networking system from a keyframe positive sequence closest to the request time, takes the received first frame as a starting keyframe of a current reverse sequence slice, and packages a previous frame from the starting keyframe of the current reverse sequence slice to the starting keyframe of a previous reverse sequence slice into the current reverse sequence slice;

the server takes N seconds before the start key frame of the current reverse sequence slicing as the request time to generate the forward sequence playing request again until the reverse sequence slicing is finished;

the server side reverses the reverse sequence slicing to the acquisition time arrangement to generate a slice sequence, the frame sequence in each reverse sequence slicing is forward to the acquisition time arrangement, the first time stamp of the frame sequence of the slice sequence monotonically increases according to the arrangement sequence, and the slice sequence is used for being sent to the webpage playing side to realize reverse sequence playing of the video stream to be played.

In some embodiments of the present application, when the server side generates the first positive sequence play request, the server side receives a video stream to be played sent by the networking system from a positive sequence of key frames with the closest request time, takes the received first frame as a start key frame of the first reverse sequence slice, and encapsulates the received video frame into the first reverse sequence slice according to the set time/the set number of key frames/a termination frame of the video stream to be played.

In some embodiments of the present application, the server receiving a video stream to be played sent by the networking system in positive sequence from a keyframe with a closest request time, using the received first frame as a starting keyframe of a current reverse-sequence slice, and encapsulating a previous frame from the starting keyframe of the current reverse-sequence slice to a starting keyframe of a previous reverse-sequence slice into the current reverse-sequence slice includes:

the server generates a first time stamp for the frame sequence in the packaged current reverse sequence slicing, and the first time stamp of the frame sequence in the current reverse sequence slicing is generated according to the first time stamp of the frame sequence in the previous reverse sequence slicing, so that the first time stamp of the frame sequence in the slice sequence monotonically increases according to the arrangement sequence.

In some embodiments of the present application, the encapsulating the starting key frame of the current reverse order slice into the starting key frame of the previous reverse order slice comprises:

responding to the current video frame received by the server as a key frame, matching the characteristic data of the key frame with the characteristic data of the starting key frame of the previous reverse sequence segmentation, if not, writing the key frame into the current reverse sequence segmentation and continuing to receive the video frame; if so, identifying the key frame as a starting key frame of a previous reverse-order fragment, and packaging the starting key frame of the current reverse-order fragment to a frame before the key frame into the current reverse-order fragment;

And when the server side encapsulates the current reverse-order fragments, taking the data of the set position of the coding load of the starting key frame of the current reverse-order fragments as the characteristic data of the starting key frame of the current reverse-order fragments.

In some embodiments of the application, the positive sequence play request is a multiple speed positive sequence play request.

In some embodiments of the application, the reverse order slices are FMP4 media slices.

According to still another aspect of the present application, there is also provided a reverse order playing system, including:

the webpage playing end executes the reverse order playing method;

and the server side executes the reverse order slicing method.

In some embodiments of the application, further comprising:

and the networking system is used for communicating with the server side so as to send the video stream to be played to the server side.

According to still another aspect of the present application, there is also provided an electronic apparatus including: a processor; a storage medium having stored thereon a computer program which, when executed by the processor, performs the steps as described above.

According to a further aspect of the present application there is also provided a storage medium having stored thereon a computer program which, when executed by a processor, performs the steps as described above.

Therefore, compared with the prior art, the scheme provided by the application has the following advantages:

the webpage playing end sends a reverse playing request to the server end, so that the server end can obtain a slice sequence by executing reverse sequence slicing of a video stream to be played, and the server end sets a first timestamp of the slice sequence to ensure that frame sequences in the slice sequence can be transmitted sequentially; after the webpage playing end obtains the reverse-order fragments, a timestamp offset attribute value can be set for the reverse-order fragments through the media source expansion module and a source buffer thereof, and a second timestamp which is monotonically decreased according to the playing sequence can be generated according to the set timestamp offset attribute value so as to be matched with the current time value of the video element which is decreased according to the stepping value, thereby promoting the reverse-order playing of the video to be played without the need of transcoding and storing the video file. In addition, the slicing and playing are decoupled to the server side and the webpage playing side respectively, so that the slicing, the transmission and the playing can be in a pipeline form, and the time delay can be reduced in the reverse playing process. Therefore, the reverse storage of the video stream with low cost and real-time response is realized, the video stream is not required to be restored in the reverse storage process, and additional plug-in components are not required.

The server side supports the reverse order slicing of the local video files at the server side. The server side packages the reverse sequence fragments, so that the reverse sequence fragments can accord with the processing format of the media source expansion module of the webpage playing side, and the local video file does not need to be transferred and transcoded; meanwhile, the server sets a first time stamp of the slice sequence so as to ensure that the frame sequence in the slice sequence can be transmitted sequentially. In addition, the slicing and playing are decoupled to the server side and the webpage playing side respectively, so that the slicing, the transmission and the playing can be in a pipeline form, and the time delay can be reduced in the reverse playing process.

The server side supports reverse order slicing of video files of the networked system. The server side packages the reverse sequence fragments, so that the reverse sequence fragments can accord with the processing format of the media source expansion module of the webpage playing side, and video files transmitted by the networking system do not need to be transferred and transcoded; meanwhile, the server sets a first time stamp of the slice sequence so as to ensure that the frame sequence in the slice sequence can be transmitted sequentially. In addition, video stream transmission, slicing and playing are decoupled to a networking system, a server side and a webpage playing side respectively, so that the video stream transmission, slicing, transmission and playing can be in a pipeline form, and the time delay can be reduced in the rewinding process.

Drawings

The above and other features and advantages of the present application will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings.

Fig. 1 shows a flowchart of a reverse order playing method of a web page playing end according to an embodiment of the present application.

Fig. 2 shows a flow chart of a server-side reverse order slicing method according to an embodiment of the present application.

Fig. 3 shows a flow chart of a server-side reverse order slicing method according to a specific embodiment of the present application.

Fig. 4 shows a flow chart of a server-side reverse order slicing method according to another embodiment of the present application.

Fig. 5 shows a flow chart of a reverse order slicing method at a server side according to another embodiment of the present application.

FIG. 6 shows a schematic slice view of an original video file according to an embodiment of the application;

fig. 7 shows a schematic diagram of a slice sequence for acquiring an original video in reverse order according to an embodiment of the present application.

Fig. 8 shows a schematic diagram of a slice sequence with a first timestamp according to an embodiment of the application.

Fig. 9 shows a schematic diagram of a slice sequence with a second timestamp according to an embodiment of the application.

Fig. 10 shows a block diagram of a reverse playback system according to an embodiment of the application.

Fig. 11 schematically illustrates a computer-readable storage medium in an exemplary embodiment of the present disclosure.

Fig. 12 schematically illustrates a schematic diagram of an electronic device in an exemplary embodiment of the present disclosure.

Detailed Description

Example embodiments will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments may be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

Furthermore, the drawings are merely schematic illustrations of the present application and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus a repetitive description thereof will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software or in one or more hardware modules or integrated circuits or in different networks and/or processor devices and/or microcontroller devices.

The flow diagrams depicted in the figures are exemplary only and not necessarily all steps are included. For example, some steps may be decomposed, and some steps may be combined or partially combined, so that the order of actual execution may be changed according to actual situations.

Referring first to fig. 1, fig. 1 shows a flowchart of a reverse order playing method at a web page playing end according to an embodiment of the present application. The reverse order playing method of the webpage playing end provided by the application comprises the following steps:

step S111: and the webpage playing end sends a reverse playing request to the server end.

Specifically, the reverse-order slicing is performed on the video stream to be played by the server side according to the reverse-order slicing request to obtain a slice sequence, the reverse-order slicing of the slice sequence is opposite to the acquisition time arrangement, the frame sequence in each reverse-order slicing is forward to the acquisition time arrangement, the first time stamp of the frame sequence of the slice sequence is monotonically increased according to the arrangement sequence after being set, and the acquisition time refers to the time when the video is acquired and is the same as the time passing direction.

Specifically, the "reverse order" direction described in the present application is the direction opposite to the time lapse, that is, the direction opposite to the video acquisition timing.

Specifically, the reverse playback request sent by the web page playing end may include the video stream identifier to be played, the start playing position/time, and the playback direction (in this embodiment, the playback direction is in reverse order). The video stream to be played identification may be a uniform resource locator (Uniform Resource Locator, URL) of the video to be played. The video stream to be played is a video file collected in history.

Specifically, when the server side performs the reverse slicing, the slices are encapsulated so as to obtain the reverse slicing conforming to the set format and the slice sequence formed by the reverse slicing arrangement. The set format is, for example, FMP4 format conforming to the ISO BMFF specification, thereby conforming to the format requirement of the media source expansion module at the web page playing end. The ISO BMFF specification may be maximally adapted to cross-browser compatibility. A media slice (reverse order slice) in FMP4 format may be made up of moof, mdata and optionally boxes such as styrp, sidx, etc. The initialization slice of the FMP4 format may be composed of top-level boxes such as ftyp and moov.

Specifically, a reverse order slice at the server side may include one or more groups of pictures (GOP). The server side can find the corresponding initial frame according to the initial playing time of the request, and find the nearest key frame (I1 frame) in reverse order from the direction opposite to the initial playing time, the frame sequence between the I1 frame and the initial playing time forms the first original slice 1 (GOP 1), and find the key frame (I2 frame) nearest to the I1 frame from the beginning of the I1 frame, and the frame sequence between the I2 frame and the previous frame P2 of the first I1 frame forms the second original slice 2 (GOP 2), as shown in fig. 6 (the arrow direction in fig. 6 is the increasing direction of the original timestamp), so as to push, slice the original slices one by one, and the frame sequences in each slice have the original timestamp, that is, the timestamp increasing forward according to the video acquisition time.

Specifically, the server side may generate a slice sequence according to the number of original slices included in the set reverse order slices. In the preferred embodiment of the application, the reverse-order slicing can only comprise one original slicing, so that timeliness of acquiring the reverse-order slicing can be ensured. The slice sequence is shown in fig. 7 (the arrow direction in fig. 7 is the increasing direction of the original time stamp in each reverse-order slice), the reverse-order slices of the slice sequence are arranged reversely to the acquisition time reverse-order (the reverse-order slices are arranged as an original slice 3, an original slice 2 and an original slice 1 according to the positive acquisition time sequence, the reverse-order slices are arranged as an original (reverse-order) slice 1, an original (reverse-order) slice 2 and an original (reverse-order) slice 3 according to the negative acquisition time sequence), and the frame sequence in each reverse-order slice is arranged positively to the acquisition time sequence. In other words, at this time, the read slices are not yet encapsulated in fig. 7, but only the reverse-order slices are arranged according to the reverse-order reading order, so the frame sequence in each reverse-order slice still has the original time stamp, that is, the time stamp that is incremented forward according to the video capturing time. Monotonicity of the time stamps is broken due to the reverse order of reading the slice sequence. Therefore, after the server side obtains the reverse sequence fragments through encapsulation, the first time stamp can be regenerated for the reverse sequence fragments, the first time stamp monotonically increases according to the sequence of the fragments and the sequence of the frames (as shown in fig. 8, the arrow direction in fig. 8 indicates the increasing direction of the first time stamp), and therefore monotonicity of the time stamp of the frame sequence is guaranteed not to be interrupted, and transmission and frame processing are facilitated.

Step S112: and the webpage playing end sequentially acquires the reverse sequence fragments from the server end.

Specifically, the server side may send index information of the slice sequence (for example, an index list may be generated according to the slice sequence) to the web page playing side while generating the slice sequence, and the web page playing side sequentially acquires the reverse-order slices according to the index information (for example, the index list). In some embodiments, the web page playing end may acquire the reverse sequence fragments block by block in an HTTP Get manner. In other embodiments, the web page playing end may also establish a Websocket long connection with the server end, and the server end pushes the Websocket long connection from one piece to another. Thus, step S112 transmits the reverse-order slices in a reverse order compared to the acquisition time. Further, step S112 may further control the transmission rhythm according to the situation that the web page playing end plays the buffer. For example, when the play buffer of the webpage play end is idle, the transmission speed of the reverse sequence slicing is improved; when the web page playing end has cached the reverse sequence fragments, the transmission speed of the reverse sequence fragments can be reduced. The application is not limited in this regard.

Step S113: and the webpage playing end sets a timestamp offset attribute value of a source buffer of an associated media source expansion module for the obtained current reverse-order fragment and buffers the current reverse-order fragment into the source buffer, so that the timestamp offset attribute value associated with the current reverse-order fragment is the time length of subtracting the last reverse-order fragment from the timestamp offset attribute value associated with the last reverse-order fragment.

Specifically, the source cache is in a sequence mode, so that the reverse sequence fragments can be input to the source cache (SourceBuffer object) in a sequence mode.

In particular, the initial value of the timestamp offset attribute value of the source cache may be set to be greater than the total duration of the slice sequence (which may preferably be much greater than the total duration of the slice sequence) to ensure that the timestamp offset attribute value is sufficiently decremented for the slice sequence to be played in reverse order. Further, each time the web page playing end caches the reverse-order fragments into the source cache, the timestamp offset attribute value associated with the current reverse-order fragment is the timestamp offset attribute value associated with the last reverse-order fragment minus the duration of the last reverse-order fragment, so that the setting of the timestamp offset attribute value is realized.

Step S114: and the media source expansion module of the webpage playing end regenerates a second timestamp for the frame sequence in the current reverse sequence fragment according to the timestamp offset attribute value associated with the current reverse sequence fragment.

Specifically, step S114 causes the second timestamp of the reverse-order intra-slice frame sequence to monotonically increase in the arrangement order, and the second timestamp of the start frame of the previous reverse-order intra-slice frame sequence to decrease by a frame interval that is the second timestamp of the end frame of the current reverse-order intra-slice frame sequence.

In some embodiments, a second timestamp of a termination frame (e.g., a P frame) of each reverse order slice may be determined based on the timestamp offset attribute value, with the sequence of frames monotonically increasing at frame intervals in the reverse order slice in order of arrangement. In other words, after determining the second timestamp of the ending frame according to the timestamp offset attribute value, the timestamp offset attribute value is self-subtracted from the frame interval to obtain the second timestamp of the frame preceding the ending frame, and so on until the second timestamp of the starting frame of the current reverse-order slice is determined. Further, the time stamp offset attribute value of the current reverse-order slice is set to be the time stamp offset attribute value of the previous reverse-order slice minus the time length of the last reverse-order slice, and the interval frame interval between the second time stamp of the ending frame of the current reverse-order slice and the second time stamp of the starting frame of the last reverse-order slice, specifically, the second time stamp decreasing frame interval of the starting frame of the last reverse-order slice is used as the second time stamp of the ending frame of the current reverse-order slice, so that the second time stamps between adjacent reverse-order slices decrease according to the arrangement order, and the continuity of the reverse-order slice during playing is maintained.

In other embodiments, the corresponding quasi-second time stamps may be generated for the frames in the reverse-order slices according to the sequence of the frames in the reverse-order slices and the frame intervals, and then the time stamp offset attribute value associated with the reverse-order slices is added to the quasi-second time stamps corresponding to the frames in the reverse-order slices to obtain the second time stamps corresponding to the frames in the reverse-order slices. The present application may implement more variations, and will not be described in detail herein.

Specifically, through steps S113 to S114, the time stamps of adjacent reverse-order slices are decremented in the acquisition order while the time stamps of the frame sequences within each reverse-order slice are monotonically incremented in the arrangement order, as shown in fig. 9, such that the sequentially acquired second time stamps of reverse-order slice 1, reverse-order slice 2, reverse-order slice 3 are decremented in the acquisition order while the second time stamps of the frame sequences within each reverse-order slice are monotonically incremented in the arrangement order (from I1 to P1; from I2 to P2; from I3 to P3), i.e., the second time stamps are arranged in descending order in fig. 9: P1→I1→P2→I2→P3→I3, the arrangement sequence is also the reverse order play sequence; the corresponding arrow directions in fig. 9 indicate the playback direction in the reverse order slice, i.e., the reverse order slice 1 starts playing from P1 until I1.

Step S115: and the webpage playing end plays the video stream to be played in the reverse order by adopting video elements.

Specifically, the webpage playing end decrements the current time value of the video element according to the step value, so that the current time value advances the reverse order playing of the video stream to be played according to the decrementing direction of the second timestamp.

Specifically, the reverse-order clips input to the source buffer through steps S113 and S114 eventually require the video element through the browser to be played. The web page playing end can advance the playing of the video frames according to the current time value of the video elements. And the current time value is decremented in step S115, so that the reverse playback of the video stream to be played back can be advanced in the decrementing direction of the second timestamp (e.g. in fig. 9, the reverse playback is performed in the direction of p1→i1→p2→i2→p3→i3).

In some embodiments, step S115 may set a timer based on which a timed decrementing task of the current time value is performed. Since the browser will automatically align to the second timestamp according to the current time value, the step value (timing interval) of the timer may be less than or equal to the frame interval of the frame sequence. For example, the frame interval may be obtained according to a play frequency conversion of video frames. Thus, based on the timer, the current time value (currentTime attribute value) in the video element can be modified at step-by-step value timing.

Specifically, the current time value in the video element may be initialized to the timestamp of the last frame of the first reverse-order slice in the source buffer, that is, the maximum timestamp sourcebuffer. Buffered. End (0) in the source buffer, such as the second timestamp of the P1 frame of the reverse-order slice 1 in fig. 9, that is, playing from the P1 frame according to the current time value of the video element; the current time value is subtracted from the step value in each timing task, and as the step value is smaller than or equal to the frame interval and the browser can automatically position the second timestamp, in the previous example, assuming that the current time value is the second timestamp of the P1 in FIG. 9, after subtracting the step value, the browser positions the second timestamp of the next frame of the P1 frame in the direction from the P1 frame to the I1 frame for playing, and so on; and responding to the current time value equal to the minimum time stamp of the reverse sequence slicing in the source cache, wherein the webpage playing end pauses the reverse sequence playing of the video stream to be played as shown in a second time stamp of a P3 frame in fig. 9.

Specifically, when the network transmission parameter between the web page playing end and the server end is smaller than the set network transmission parameter (for example, the network transmission speed is smaller, or when the network is jittery), the web page playing end may buffer a plurality of the reverse clips in the source buffer, and then decrease the current time value of the video element according to the step value, so that the current time value advances the reverse playing of the video stream to be played according to the decreasing direction of the second timestamp, so as to avoid that the subsequent clips arrive untimely and cause unsmooth playing.

Further, the web page playing end can start to execute the playing step after receiving the first reverse sequence slicing, so as to achieve more timely response.

In the reverse order playing method of the webpage playing end, the webpage playing end sends the reverse playing request to the server end, so that the server end can obtain a slice sequence by executing the reverse order slicing of the video stream to be played, and the server end sets the first time stamp of the slice sequence to ensure that the frame sequence in the slice sequence can be transmitted in sequence; after the webpage playing end obtains the reverse-order fragments, the media source expansion module and the source cache thereof can set a timestamp offset attribute value for the reverse-order fragments, and can generate a second timestamp which is monotonically decreased according to the playing order so as to be matched with the current time value of the video element which is regularly decreased according to the stepping value, thereby promoting the reverse-order playing of the video to be played without the need of transcoding and storing the video file. In addition, the slicing and playing are decoupled to the server side and the webpage playing side respectively, so that the slicing, the transmission and the playing can be in a pipeline form, and the time delay can be reduced in the reverse playing process. Therefore, the reverse storage of the video stream with low cost and real-time response is realized, the video stream is not required to be restored in the reverse storage process, and additional plug-in components are not required.

Referring now to fig. 2, fig. 2 is a flow chart illustrating a server-side reverse order slicing method according to an embodiment of the present application. In this embodiment, the video stream to be played is a local video file at the server side. The reverse order slicing method of the server side comprises the following steps:

step S101: the server receives the reverse playing request sent by the webpage playing end,

specifically, the rewinding request includes an identifier of a video stream to be played and a start playing time, where the video stream to be played corresponding to the identifier of the video stream to be played is a local video file of the server side;

step S102: the server determines the current frame position of the video stream to be played based on the initial playing time;

step S103: the server searches the key frame closest to the current frame position in the video stream to be played in reverse order;

step S104: the server end forms the current original slicing from the current frame position to the previous frame of the searched nearest key frame, and the current original slicing is used for packaging into reversed-order slicing;

step S105: taking the key frame position of the current original slice as the current frame position to position the next original slice until the reverse-order slice is finished;

Step S106: and reversing the reverse sequence slicing to the acquisition time arrangement to generate a slice sequence, wherein the frame sequence in each reverse sequence slicing is forward to the acquisition time arrangement, the first time stamp of the frame sequence of the slice sequence monotonically increases according to the arrangement sequence, and the slice sequence is used for being sent to the webpage playing end to realize reverse sequence playing of the video stream to be played.

In the reverse-order slicing method provided by the application, the server side supports the reverse-order slicing of the local video file of the server side. The server side packages the reverse sequence fragments, so that the reverse sequence fragments can accord with the processing format of the media source expansion module of the webpage playing side, and the local video file does not need to be transferred and transcoded; meanwhile, the server sets a first time stamp of the slice sequence so as to ensure that the frame sequence in the slice sequence can be transmitted sequentially. In addition, the slicing and playing are decoupled to the server side and the webpage playing side respectively, so that the slicing, the transmission and the playing can be in a pipeline form, and the time delay can be reduced in the reverse playing process.

Referring now to fig. 3, fig. 3 is a flow chart illustrating a server-side reverse order slicing method according to an embodiment of the present application. In this embodiment, the video stream to be played is a local video file at the server side. The reverse order slicing method of the server side comprises the following steps:

Step S121: and the server receives the reverse playing request sent by the webpage playing end.

Specifically, the rewind request includes an identifier of a video stream to be played and a start playing time, where the video stream to be played corresponding to the identifier of the video stream to be played is a local video file at the server side.

Step S122: the server end determines a current frame position of the video stream to be played based on the start playing time.

Specifically, step S122 may further include a step of initializing the sharding. Further, media parameter information such as the video stream to be played SPS (Sequence Parameter Sets, sequence parameter set) and PPS (Picture Parameter Set ) may be read to initialize the slice. Further, FMP4 initial slices (Initialization Segment) may also be output based on the initial slices. The FMP4 initial slice (Initialization Segment) is the first slice of the FMP4 slice to conform to the adaptive streaming protocol. The reverse order slices generated hereinafter are the media slices containing the actual video stream after the first slice.

Step S123: and the server searches the key frame closest to the current frame position in the video stream to be played in an inverted order.

Step S124: and the server end forms the current original slice (GOP) from the current frame position to the frame before the searched latest key frame, and the current original slice is used for packaging into the reverse-order slice.

Step S125: and judging whether the number of the original fragments packaged into the current reverse sequence fragments reaches a set number threshold.

Specifically, in order to improve the instantaneity of the slice transmission, each reverse order slice may be made to include only one original slice.

If the determination in step S125 is negative, step S126 is executed: and the server takes the key frame position of the current original slice as the current frame position to position the next original slice.

After step S126, the process returns to step S123 again to locate the next original slice until the reverse order slice is completed. The reverse slicing may be based on the completion of slicing the video stream to be played or the user canceling the end of the slicing operation.

If yes in step S125, step S127 is executed: and the server side sequentially reads the frame sequence from the original intra-slice positive sequence of the set quantity threshold according to the acquisition sequence of the original slices.

Step S128: and the server packages each original fragment into a current reverse sequence fragment, generates a first timestamp which monotonically increases according to the arrangement sequence for the read frame sequence of the current reverse sequence fragment, and generates the first timestamp of the frame sequence of the current reverse sequence fragment according to the first timestamp of the frame sequence of the previous reverse sequence fragment.

Specifically, the original slices may be packaged as reverse order slices in FMP4 format. The first timestamp is monotonically increasing between and within the reverse order slices (as in fig. 8).

In particular, the first timestamp of the start frame of the frame sequence of the current reverse order slice may be determined based on the first timestamp of the end frame of the frame sequence of the previous reverse order slice, e.g. such that the first timestamp of the start frame of the frame sequence of the current reverse order slice is the first timestamp of the start frame of the frame sequence of the current reverse order slice plus the frame interval, thereby such that the first timestamps of the frame sequences in the slice sequence monotonically increase in the order of arrangement.

Step S129: and judging whether the reverse sequence slicing is finished or not.

If the determination in step S129 is no, step S1210 is executed: and the server takes the searched latest key frame as the current frame position to position the original fragment of the next reverse sequence fragment.

After step S1210, the process returns to step S123 again to locate the next original slice until the reverse order slice is completed.

If the determination in step S129 is yes, step S1211 is performed: and reversing the reverse sequence slicing to acquisition time arrangement to generate a slice sequence.

The reverse sequence slicing of the slice sequence is opposite to the acquisition time arrangement, the frame sequence in each reverse sequence slicing is forward to the acquisition time arrangement, the first time stamp of the frame sequence of the slice sequence monotonically increases according to the arrangement sequence, and the slice sequence is used for being sent to the webpage playing end to realize the reverse sequence playing of the video stream to be played.

Referring now to fig. 4, fig. 4 is a flow chart illustrating a server-side reverse order slicing method according to another embodiment of the present application. In this embodiment, the video stream to be played is stored in a networking system.

Step S141: and the server receives the reverse playing request sent by the webpage playing end.

The reverse playing request comprises an identifier of a video stream to be played and a starting playing time, and the video stream to be played corresponding to the identifier of the video stream to be played is stored in a networking system;

step S142: the server takes the initial playing time as the request time for generating the positive sequence playing request.

Specifically, the server side may generate an initial normal play request according to the initial play time. And receiving a video stream to be played, which is sent by the networking system from the initial playing time positive sequence, and taking the received first key frame as the initial key frame of the current reverse sequence slicing.

Step S143: the server side generates a positive sequence playing request and sends the positive sequence playing request to the networking system;

step S144: the server receives a video stream to be played, which is sent by the networking system from a keyframe positive sequence closest to the request time, takes the received first frame as a starting keyframe of the current reverse-sequence slicing, and packages a frame from the starting keyframe of the current reverse-sequence slicing to a frame before the starting keyframe of the previous reverse-sequence slicing into the current reverse-sequence slicing.

Specifically, when the server side generates a first positive sequence playing request, the server side receives a video stream to be played, which is sent by the networking system from a positive sequence of key frames with the closest request time, takes the received first frame as a starting key frame of a first reverse sequence slice, and encapsulates the received video frame into the first reverse sequence slice according to the set time/the set number of key frames/a termination frame of the video stream to be played.

Step S145: the server takes N seconds before the start key frame of the current reverse sequence slicing as the request time to generate the forward sequence playing request again until the reverse sequence slicing is finished;

specifically, after the positive sequence play request is generated again in step S145, steps S143 to 144 are performed until the reverse sequence slicing is completed. The reverse sequence slicing end is ended when a slicing canceling instruction is received or the video stream is ended.

Step S146: and the server-side inverts the inverted sequence slicing to the acquisition time arrangement to generate a slicing sequence.

The frame sequence in each reverse-order slice is forward arranged in the acquisition time, the first time stamp of the frame sequence of the slice sequence monotonically increases according to the arrangement order, and the slice sequence is used for being sent to the webpage playing end to realize reverse-order playing of the video stream to be played.

In the reverse-order slicing method provided by the application, the server side supports the reverse-order slicing of the video files of the networking system. The server side packages the reverse sequence fragments, so that the reverse sequence fragments can accord with the processing format of the media source expansion module of the webpage playing side, and video files transmitted by the networking system do not need to be transferred and transcoded; meanwhile, the server sets a first time stamp of the slice sequence so as to ensure that the frame sequence in the slice sequence can be transmitted sequentially. In addition, video stream transmission, slicing and playing are decoupled to a networking system, a server side and a webpage playing side respectively, so that the video stream transmission, slicing, transmission and playing can be in a pipeline form, and the time delay can be reduced in the rewinding process.

Referring now to fig. 5, fig. 5 is a flow chart illustrating a server-side reverse order slicing method according to another embodiment of the present application. In this embodiment, the video stream to be played is stored in a networking system.

Step S131: and the server receives the reverse playing request sent by the webpage playing end.

Specifically, the playback request includes an identifier of a video stream to be played and a start playing time, where the video stream to be played corresponding to the identifier of the video stream to be played is stored in a networking system.

Step S132: the server takes the initial playing time as the request time for generating the initial positive sequence playing request.

Step S133: and the server side sends an initial positive sequence playing request to the networking system.

Specifically, the initial positive sequence play request may include a request time, where the request time is a start play time of the reverse play request. The initial positive sequence play request may include a video stream identifier to be played and a termination play time.

Further, the positive sequence play request may further include a play multiplier. In some embodiments, the positive sequence play request may be a single-speed play request, so that the networking system sends the video stream to be played to the server according to the normal play speed, so that the server receives the video stream to be played and then performs reverse sequence slicing.

In other embodiments, in order to ensure that the web page playing end can obtain the reverse sequence fragments in real time and play the fragments, the positive sequence play request in step S132 may be a multiple speed positive sequence play request in consideration of video transmission and communication between the networking system and the server end. Therefore, the networking system can send the video stream to be played to the server side in the multiple-speed positive sequence based on the multiple-speed positive sequence playing request. For example, the positive sequence playing request may be 1.5 times speed, 2 times speed, or 4 times speed positive sequence playing request, which is not limited by the present application.

Step S134: the server receives a video stream to be played, which is sent by the networking system from the key frame positive sequence with the nearest initial playing time, takes the received first frame as the initial key frame of the first reverse sequence slice, and encapsulates the received video frame into the first reverse sequence slice according to the set time/the set number of key frames/the termination frame of the video stream to be played.

Specifically, the set duration of the first reverse order slice or the set number of key frames may be preset. And determining the slicing position of the first reverse sequence slice according to whether the received duration of the video stream to be played meets the set duration and whether the number of key frames contained in the received video stream to be played meets the set number of key frames. Further, the first reverse sequence slicing may also be used from the start key frame of the first reverse sequence slicing to the end frame of the video stream to be played.

Specifically, the server side continuously receives a video stream sent by the networking system based on the positive sequence playing request and packages the packets into frames.

Specifically, in step S134, the server generates a first timestamp for the frame sequence in the first reverse-order slice of the package, so that the first timestamp of the frame sequence in the first reverse-order slice monotonically increases according to the arrangement order and monotonically decreases according to the play order.

Specifically, step S134 may also initialize the step of slicing. Further, the fragmentation may be initialized by the server side and the networked system negotiating returned or parsed media parameter information in the video stream. Further, FMP4 initial slices (Initialization Segment) may also be output based on the initial slices. The FMP4 initial slice (Initialization Segment) is the first slice of the FMP4 slice to conform to the adaptive streaming protocol. The reverse order slices generated hereinafter are the media slices containing the actual video stream after the first slice.

Step S134 may further include the step of taking the first reverse order slice that has been encapsulated as the previous reverse order slice.

Step S135: the server side obtains the characteristic data of the initial key frame of the previous reverse sequence fragment, takes N seconds before the initial key frame of the previous reverse sequence fragment as the request time, and generates a positive sequence playing request according to the request time.

Specifically, the positive play request, similar to the initial positive play request, may include a request time. It may also include the video stream identification to be played, the termination play time, and the play multiplier.

Specifically, N is a constant greater than 0. N may be set according to the spacing of the key frames. N may be set to, for example, 1 second, 2 seconds, or the like, and the present application is not limited thereto. The positive sequence play request is similar to the initial positive sequence play request, and may include a request time and a play speed, which are not described herein.

Specifically, N seconds before the start key frame of the last reverse sequence slice is taken as a new request time, a positive sequence playing request is generated according to the new request time, and the packaged first reverse sequence slice is taken as the previous reverse sequence slice.

Step S136: the server side sends a positive sequence playing request to a networking system, receives a video stream to be played, which is sent by the networking system from a key frame with the closest request time according to a positive sequence, and takes the received first frame as a starting key frame of current reverse sequence slicing.

Step S137: judging whether the current frame received by the server is a key frame, if not, executing step S138: writing the current frame into the current reverse sequence slicing, continuously receiving the next frame of the current frame as the current frame, and entering step S137;

if yes, step S139 is executed: matching the characteristic data of the current key frame with the characteristic data of the starting key frame of the previous reverse sequence slicing;

if step S139 is matched, step S1310 is performed: and identifying the current key frame as a starting key frame of the previous reverse-order slice, and packaging the starting key frame of the current reverse-order slice to the previous frame of the key frame into the current reverse-order slice.

Specifically, in step S1310, the key frame refers to the current key frame identified as the starting key frame of the previous reverse-order slice; when the set position data of the coding load of the key frame is consistent with the set position data of the coding load of the starting key frame of the previous reverse-order slice, the key frame is matched with the starting key frame of the previous reverse-order slice, so that the current key frame can be used as the starting key frame of the previous reverse-order slice to position the ending frame of the current reverse-order slice. For example, the first 128 bits of the encoding load of the key frames may be used as a feature value to facilitate matching between key frames. Further, in step S133, data of a set position of the encoding load of the start key frame of the preceding reverse-order slice may be set as the feature data. Therefore, in step S135, if the current frame is a key frame, the feature data of the current frame is matched with the feature data of the starting key frame of the previous reverse-order slice, and if the matching is consistent, the current frame is matched with the starting key frame of the previous reverse-order slice, thereby improving the matching efficiency of the key frames. Other key frame matching means are also within the scope of the present application. Further, the current reverse-order sliced termination frame is determined to be the frame before the start key frame of the previous reverse-order sliced by matching the current key frame with the start key frame of the previous reverse-order sliced, namely, the starting key frame of the current reverse-order sliced to the frame before the key frame are encapsulated into the current reverse-order sliced, so that the number of key frames contained in each reverse-order sliced is not required to be limited.

If step S139 is not matched, step S138 is performed.

Specifically, step S1310 may encapsulate the currently received video frame into FMP4 reverse sequence slices, so that the media source expansion module at the web page playing end can process the video frame.

Step S1311: and the server generates a first time stamp for the frame sequence in the packaged current reverse sequence slice, and outputs the packaged current reverse sequence slice.

Specifically, the first time stamp monotonically increases in the order of arrangement of the frame sequences within and between the reverse order slices. In particular, the first timestamp of the start frame of the current reverse order slice may be determined from the first timestamp of the end frame of the frame sequence of the previous reverse order slice. For example, the first time stamp of the start frame of the frame sequence of the current reverse order slice is made to be the first time stamp of the end frame of the frame sequence of the previous reverse order slice plus the frame interval, thereby causing the first time stamp of the frame sequence in the slice sequence to monotonically increase in the order of arrangement.

Step S1312: and judging whether the reverse sequence slicing is finished or not.

If step S1312 determines that it is not, step S1313 is executed: taking the packaged current reverse sequence slice as the previous reverse sequence slice, and re-executing the step S135.

If yes in step S1312, step S1314 is performed: and reversing the reverse sequence slicing after each package to the acquisition time arrangement to generate a slicing sequence.

The above steps are described with reference to fig. 6 to 8. After receiving the reverse playing request sent by the webpage playing end, the server end takes the initial playing time in the reverse playing request as the request time, generates a reverse playing request and sends the request to the networking system. The networking system locates the nearest key frame (I1, for example) according to the initial playing time, and sends the video stream to be played to the server in positive sequence. The server side takes the received first frame (I1) as a starting key frame of a first reverse-order slice, then positions the first frame (I1) to a termination frame (P1, a non-key frame or a non-key frame and one or more key frames can be included between the I1 and the P1) of the first reverse-order slice according to a set duration/a set number of key frames/a termination frame of a video stream to be played, encapsulates the first frame (I1) to the frame sequence (original slice 1) of the I1 to the P1 into the first reverse-order slice, and sets a first time stamp for the frame sequence of the first reverse-order slice, so that the frame sequence in the first reverse-order slice monotonically increases according to the arrangement order (as shown in figure 8). After the first reverse sequence slicing is packaged by the server, data (such as data of 128 bits before encoding load) of the set position of the encoding load of the I1 frame is used as characteristic data. And taking N seconds before the I1 frame as new request time, generating an updated reverse play request, and sending the updated reverse play request to a networking system.

The networking system locates the nearest key frame (i.e. I2) according to N seconds before the I1 frame, and sends the video stream to be played to the server from I2. After the server receives the I2, comparing whether the characteristic data of the I2 are matched with the characteristic data of the I1, if the characteristic data are not matched, writing the I2 as a starting key frame of a second reverse-order slicing into the second reverse-order slicing, continuously receiving video frames of a video stream to be played, judging whether the currently received video frames are key frames or not when the received video frames are received, if the currently received video frames are not key frames, directly writing the video frames into the second reverse-order slicing, if the currently received video frames are not matched with the characteristic data of the I1, comparing whether the characteristic data of the current key frames are matched with the characteristic data of the I1, if the characteristic data are matched with the characteristic data, indicating that the currently received key frames are the starting key frame I1 of the first reverse-order slicing, continuously reading the video frames which enable the second reverse-order slicing to be repeated with the video frames of the first reverse-order slicing, therefore, the previous frame P2 of the I1 frame can be used as a termination frame of the second reverse-order slicing, packaging the video frames into the second reverse-order slicing according to a frame sequence (original slicing 2) is the second reverse-order slicing, if the characteristic data are matched with the characteristic data of the I1, and if the characteristic data are matched, and the first time sequence is arranged in a monotone sequence of the first time sequence of the I2 is arranged according to the first time sequence, and the time sequence is 8, and the time sequence is arranged in a monotone sequence of the time sequence of the first time sequence is increased. And so on until the cancel operation or no history video stream is readable.

If the networking system locates the nearest key frame as I3 according to N seconds before the I1 frame, and sends the video stream to be played to the server from the beginning of the I3. The server side continuously receives the video frame of the video stream to be played by taking the I3 as the initial key frame of the second reverse-order slicing, judges whether the currently received video frame is the key frame or not when the received video frame is the video frame, if not, directly writes the second reverse-order slicing, if yes, compares whether the characteristic data of the current key frame with the characteristic data of the I1, if yes, judges that the received key frame is I2, takes the data (such as the data of 128 bits before the encoding load) of the encoding load of the received key frame I2 as the characteristic data, matches the characteristic data of the initial key frame I1 of the first reverse-order slicing, if not, continuously receives the video frame until the received key frame is I1, and therefore, the former frame P2 of the initial key frame of the first reverse-order slicing is taken as the termination frame of the second reverse-order slicing, and the server side packages the frame sequence of the first reverse-order slicing into the second reverse-order slicing according to the I3-P2, and sets the first time sequence for the frame sequence of the second reverse-order slicing, so that the frame sequence of the first reverse-order slicing is arranged according to the sequence of the monotone reverse-order stamps. And so on until the cancel operation or no history video stream is readable.

The foregoing illustrates various embodiments of the present application, and the present application is not limited thereto, and in each embodiment, steps are added, omitted, and sequence changed are all within the scope of the present application; the embodiments may be implemented alone or in combination.

The following describes an inverted playback system 200 provided by the present application in conjunction with fig. 10. The reverse playback system 200 includes a web page playback end 210 and a server end 220. The web page playing end 210 may perform the steps shown in fig. 1, and the server end 220 may perform the steps shown in fig. 2 to 5, thereby supporting the reverse order playing of the source of the video stream to be played. Further, the reverse playback system 200 may further include a networking system 230, which can implement reverse playback of the front-end monitoring video. The modules included in the web page playing side 210, the server side 220, and the networking system 230 are schematically shown in fig. 10, and the present application is not limited thereto.

The web page playing end 210 may include a control module 211, an acquisition module 212, and a playing module 213.

Specifically, the control module 211 of the web page playing end 210 may be configured to select a video stream to be played and a start playing time/position, send a reverse playing request to the server end 220, and receive index information of a slice sequence sent by the server end 220.

Specifically, the obtaining module 212 of the web page playing end 210 may be configured to download the reverse order fragments (FMP 4 fragments) generated by the server end 220, where the downloading of the reverse order fragments and the generating of the reverse order fragments are performed simultaneously. Meanwhile, the obtaining module 212 may adjust the downloading rhythm according to the local fragment buffering condition of the player 210.

Specifically, the playing module 213 of the web page playing end 210 may input the reverse sequence fragments to the source buffer object of the media source extension module, and finally input the fragments to the video element. In addition, the playing module 213 also achieves the purpose of playing control according to the specified time sequence by modifying the current time value of the video element by decreasing the time. The media source expansion module is applied to the HTML5 technology, so that the video reverse playing can be realized based on the HTML5 technology.

The server side 220 comprises a signaling module 221, an access module 222, a sending module 223 and a slicing module 224.

Specifically, the signaling module 221 of the server 220 may be configured to receive and process a control request, such as a reverse play, sent by the web page playing end 210, and feed back notification information, such as an index information update, issued by the server 220 to the web page playing end 210. The signaling module 221 may be configured to interact with the slicing module 224, issue a specific code stream slicing task, and receive a slice index update notification fed back by the slicing module 224. The signaling module 221 may be used to forward the requested sliced code stream, etc., to the networking system 230 to slice module 224.

In particular, the access module 222 of the server-side 220 may be responsible for maintaining links, signaling adaptations and conversions of the networking system 230. The access module 222 may act as a protocol gateway. Optionally, the access module 222 may support GB/T28181 (media control related sub-protocol is SIP), one (media control related sub-protocol is RTSP) among industry specification protocols that may be supported.

Specifically, the sending module 223 of the server 220 may be configured to respond to the request for downloading the sliced data from the obtaining module 212 of the web page playing end 210, read the reverse sliced data generated by the slicing module 224 from the local cache, and send the reverse sliced data to the obtaining module 212 of the web page playing end 210.

Specifically, the slicing module 224 of the server 220 may be configured to perform reverse slicing on the video code stream to be played, and the video code stream to be played may be stored in the server local video library 225. The slicing module 224 of the server-side 220 may also support the reverse slicing of the video bitstream to be played sent by the remote networking system 230. At this point, the slicing module 224 may act as an on-demand gateway.

The networking system 230 may be deployed according to the actual scenario. In some embodiments, the present application is used to monitor a scenario, such that the networking system 230 may include a front-end monitoring device 231 and a lower monitoring platform 232, and the present application is not limited thereto, and other remote networking systems 230 are within the scope of the present application.

In the reverse order playing system provided by the application, the webpage playing end sends the reverse order playing request to the server end, so that the server end can obtain a slice sequence by executing the reverse order slicing of the video stream to be played, and the server end sets the first time stamp of the slice sequence to ensure that the frame sequence in the slice sequence can be transmitted in sequence; the server side supports the slice inversion of the local video files of the server side and/or supports the slice inversion of the video files of the networking system. The server side packages the reverse sequence fragments, so that the reverse sequence fragments can accord with the processing format of the media source expansion module of the webpage playing side, and the local video file does not need to be transferred and transcoded; after the webpage playing end obtains the reverse-order fragments, the media source expansion module and the source cache thereof can set a timestamp offset attribute value for the reverse-order fragments, and can generate a second timestamp which is monotonically decreased according to the playing order so as to be matched with the current time value of the video element which is regularly decreased according to the stepping value, thereby promoting the reverse-order playing of the video to be played without the need of transcoding and storing the video file. In addition, the slicing and playing are decoupled to the server side and the webpage playing side respectively, so that the slicing, the transmission and the playing can be in a pipeline form, and the time delay can be reduced in the reverse playing process. Therefore, the reverse storage of the video stream with low cost and real-time response is realized, the video stream is not required to be restored in the reverse storage process, and additional plug-in components are not required.

Furthermore, the server side of the application can also support the reverse sequence slicing of the local video stream and the video stream of the networking system at the same time so as to adapt to different scenes.

The application provides an HTML5 technology-based method for no-plug-in reverse playing video recording without transcoding, integral storage and real-time response, which aims to solve the problem that the video monitoring video recording and playing reverse playing function cannot be webbed or the reverse playing function with low cost and real-time response cannot be supported after webbed.

The present application may implement the reverse playback system 200 in software, hardware, firmware, and any combination thereof. Fig. 10 is a schematic illustration of the reverse playback system 200 provided by the present application, and the splitting, merging, and adding of the modules are all within the scope of the present application without departing from the concept of the present application.

In an exemplary embodiment of the present disclosure, there is also provided a computer-readable storage medium having stored thereon a computer program which, when executed by, for example, a processor, can implement the steps of the reverse order playback method or the reverse order slicing method described in any one of the above embodiments. In some possible embodiments, the aspects of the application may also be implemented in the form of a program product comprising program code for causing a terminal device to carry out the steps according to the various exemplary embodiments of the application as described in the above-mentioned reverse playback method or reverse slicing method section of this specification, when said program product is run on the terminal device.

Referring to fig. 11, a program product 800 for implementing the above-described method according to an embodiment of the present application is described, which may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present application is not limited thereto, and in this document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium would include the following: an electrical connection having one or more wires, a portable disk, a hard disk, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

The computer readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, with readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A readable storage medium may also be any readable medium that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Program code for carrying out operations of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the tenant computing device, partially on the tenant device, as a stand-alone software package, partially on the tenant computing device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the tenant computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected through the internet using an internet service provider).

In an exemplary embodiment of the present disclosure, an electronic device is also provided, which may include a processor, and a memory for storing executable instructions of the processor. Wherein the processor is configured to perform the steps of the reverse playback method or the reverse slicing method described in any of the embodiments above via execution of the executable instructions.

Those skilled in the art will appreciate that the various aspects of the application may be implemented as a system, method, or program product. Accordingly, aspects of the application may be embodied in the following forms, namely: an entirely hardware embodiment, an entirely software embodiment (including firmware, micro-code, etc.) or an embodiment combining hardware and software aspects may be referred to herein as a "circuit," module "or" system.

An electronic device 600 according to this embodiment of the application is described below with reference to fig. 12. The electronic device 600 shown in fig. 12 is merely an example, and should not be construed as limiting the functionality and scope of use of embodiments of the present application.

As shown in fig. 12, the electronic device 600 is in the form of a general purpose computing device. Components of electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one memory unit 620, a bus 630 connecting the different system components (including the memory unit 620 and the processing unit 610), a display unit 640, etc.

Wherein the storage unit stores program code that is executable by the processing unit 610 such that the processing unit 610 performs the steps according to various exemplary embodiments of the present application described in the above-described reverse order playback method or reverse order slicing method section of the present specification. For example, the processing unit 610 may perform the steps as shown in fig. 1 to 3.

The memory unit 620 may include readable media in the form of volatile memory units, such as Random Access Memory (RAM) 6201 and/or cache memory unit 6202, and may further include Read Only Memory (ROM) 6203.

The storage unit 620 may also include a program/utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment.

Bus 630 may be a local bus representing one or more of several types of bus structures including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or using any of a variety of bus architectures.

The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboard, pointing device, bluetooth device, etc.), one or more devices that enable a tenant to interact with the electronic device 600, and/or any device (e.g., router, modem, etc.) that enables the electronic device 600 to communicate with one or more other computing devices. Such communication may occur through an input/output (I/O) interface 650. Also, electronic device 600 may communicate with one or more networks such as a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the Internet, through network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 over the bus 630. It should be appreciated that although not shown, other hardware and/or software modules may be used in connection with electronic device 600, including, but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, data backup storage systems, and the like.

From the above description of embodiments, those skilled in the art will readily appreciate that the example embodiments described herein may be implemented in software, or may be implemented in software in combination with the necessary hardware. Thus, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (may be a CD-ROM, a usb disk, a mobile hard disk, etc.) or on a network, and includes several instructions to cause a computing device (may be a personal computer, a server, or a network device, etc.) to perform the above-described reverse order playback method or reverse order slicing method according to the embodiments of the present disclosure.

the webpage playing end sends a reverse playing request to the server end, so that the server end can obtain a slice sequence by executing reverse sequence slicing of a video stream to be played, and the server end sets a first timestamp of the slice sequence to ensure that frame sequences in the slice sequence can be sequentially transmitted; after the webpage playing end obtains the reverse-order fragments, the media source expansion module and the source cache thereof can set a timestamp offset attribute value for the reverse-order fragments, and can generate a second timestamp which is monotonically decreased according to the playing order so as to be matched with the current time value of the video element which is regularly decreased according to the stepping value, thereby promoting the reverse-order playing of the video to be played without the need of transcoding and storing the video file. In addition, the slicing and playing are decoupled to the server side and the webpage playing side respectively, so that the slicing, the transmission and the playing can be in a pipeline form, and the time delay can be reduced in the reverse playing process. Therefore, the reverse storage of the video stream with low cost and real-time response is realized, the video stream is not required to be restored in the reverse storage process, and additional plug-in components are not required.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any adaptations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

Claims

1. The reverse order playing method is characterized by being applied to a webpage playing end and comprising the following steps of:

2. The reverse playback method of claim 1, wherein the step-down the current time value of the video element by the web page playback end so that the current time value advances the reverse playback of the video stream to be played in the decreasing direction of the second timestamp comprises:

3. The reverse playback method of claim 1, wherein the initial value of the current time value is set to a maximum timestamp of the reverse-order slice in the source buffer, the step value is less than or equal to a frame interval of the reverse-order slice, the web page playback end decrements the current time value of the video element by the step value, so that the current time value advances the reverse-order playback of the video stream to be played in the decrementing direction of the second timestamp comprises:

4. The reverse order slicing method is characterized by being applied to a server side and comprising the following steps of:

5. The method for reverse slicing according to claim 4, wherein when the server generates the first forward play request, the server receives the video stream to be played sent in forward sequence from the key frame with the closest request time by the networking system, takes the received first frame as the start key frame of the first reverse slicing, and encapsulates the received video frame into the first reverse slicing according to the set time/the set number of key frames/the end frame of the video stream to be played.

6. The reverse slicing method of claim 4 wherein the server side receiving a video stream to be played sent by the networking system in positive sequence from a keyframe closest to the request time, taking the received first frame as a starting keyframe of a current reverse slice, and encapsulating the starting keyframe of the current reverse slice into a frame preceding the starting keyframe of a previous reverse slice into the current reverse slice comprises:

7. The reverse order slicing method of claim 4, wherein said encapsulating a frame preceding a frame of a start key frame of said current reverse order slice into said current reverse order slice comprises:

8. The reverse order slicing method of claim 4, wherein said positive order play request is a multiple speed positive order play request.

9. A reverse order play system, comprising:

a web page playing end executing the reverse order playing method according to any one of claims 1 to 3;

a server side performing the reverse order slicing method according to any one of claims 4 to 8.

10. The reverse playback system of claim 9, further comprising: