CN115150675A

CN115150675A - Reverse order playing method and system and reverse order slicing method

Info

Publication number: CN115150675A
Application number: CN202210759187.8A
Authority: CN
Inventors: 康守伟; 黄至春; 李嘉; 张剑; 姜涛
Original assignee: Suzhou Keyuan Software Technology Development Co ltd; Suzhou Keda Technology Co Ltd
Current assignee: Suzhou Keyuan Software Technology Development Co ltd; Suzhou Keda Technology Co Ltd
Priority date: 2022-06-30
Filing date: 2022-06-30
Publication date: 2022-10-04
Anticipated expiration: 2042-06-30
Also published as: CN115150675B

Abstract

The application provides a reverse playing method, a system and a reverse slicing method, wherein the reverse playing method of a webpage playing end comprises the following steps: sending a reverse playing request to a server side, and enabling the server side to perform reverse slicing on a video stream to be played according to the reverse playing request to obtain a slicing sequence; sequentially acquiring reverse sequence fragments from a server; setting a timestamp offset attribute value of a source cache of a related media source expansion module for the obtained current reverse-order fragment and caching the current reverse-order fragment into the source cache; the media source expansion module regenerates a second time stamp for the frame sequence in the current reverse sequence fragment according to the time stamp offset attribute value associated with the current reverse sequence fragment, so that the second time stamp of the frame sequence in the current reverse sequence fragment is monotonically increased according to the arrangement sequence; and decreasing the current time value of the video element according to the step value, so that the current time value advances the reverse-order playing of the video stream to be played in the decreasing direction of the second timestamp. The method and the device for playing the videos in the reverse order are achieved in the browser.

Description

Reverse order playing method and system and reverse order slicing method

Technical Field

The present application relates to the field of video playing, and in particular, to a reverse order playing method, system and reverse order slicing method.

Background

Video surveillance recording playback is a basic function in application of security solutions, and generally, a time period in which video resources can be played back is displayed in a time axis manner at a client, and a user selects a certain time point to start forward or backward playback, or switches the playback direction back and forth in the playback process, so as to flexibly and quickly find a target image of interest and observe the change process of the target image. Reverse playback (reverse playback for short) is used for positioning security events in video surveillance videos in the security field and has very practical value.

In a security solution, the application of audio and video media (including video on demand) needs to meet the requirements of image retrieval response real-time performance, function diversity, flexibility of cross-service integration, and the like, and currently, the main challenges are as follows:

1) Because a Web page (Web) browser has very limited capability support and interface opening to audio and video media, most audio and video applications are based on a client-server architecture (CS architecture), but the application is very difficult to meet the integration flexibility of cross-service and multi-version compatibility, and is not as flexible and efficient as a browser-server architecture (BS architecture);

2) Then, the audio and video media core function of the client is essentially realized in a local (Native) mode based on a pseudo BS architecture evolved based on a Web plug-in technology (such as an active x (open integrated platform), an NPAPI (Netscape plug Application Programming Interface), a PPAPI (pepper plug Application Programming Interface), and the like), and the mode has obvious defects in plug-in installation experience, version compatibility, and security, especially considering security, and the current mainstream browser no longer supports the Web plug-in technology;

3) With the development of Web browser technology, especially the emergence of HTML specification of the 5 th generation, the technologies that can be used by the browser to support Media playing currently are video (video) tags, webrtc (Web Real-Time Communication ), MSE (Media Source Extensions), WASM (Web assembly, a binary instruction set), webCodecs (allowing Web applications to encode and decode audio and video), and so on. The technologies have obvious improvement on the aspect of meeting the application of audio and video media, but no open and ready interface can support the special functions of 'reverse play' and the like at present.

In the traditional scheme, the video reverse playing is realized mainly based on a CS (circuit switched) architecture, because the decoding, rendering and other processes of the code stream can be completely autonomously controlled at the local client, and can control the forward and reverse playing frame by frame.

However, implementing the reverse play function based on the BS architecture is limited by the capability of the browser and its open interface, and thus faces a lot of difficulties. For example, in the video tag and Webrtc scheme provided by Web, the transmission, decoding and display are realized and encapsulated by a browser, an application layer cannot flexibly control the processes, and especially, code streams are cached in advance as required to ensure fluency when the video tag and the Webrtc scheme are reversely played; for the WASM scheme, although the decoding and rendering processes of the code stream can be completely and autonomously controlled, the decoding performance is poor, and a plurality of scenes cannot be satisfied; for the WebCodecs scheme, although frame data before and after decoding can be controlled, only a few models and a new version of browser in the last year support the WebCodecs scheme at present, and compatibility cannot meet requirements. For the MSE scheme, although the transmission mode can be autonomously controlled, the decoding and rendering processes are encapsulated by a browser, and the decoded frame data cannot be directly operated.

Therefore, how to implement reverse-order playing of videos in a browser is a technical problem that needs to be solved urgently by those skilled in the art.

Disclosure of Invention

In order to overcome the defects in the prior art, the application provides a reverse order playing method, a reverse order playing system and a reverse order slicing method, so that reverse order playing of videos is realized in a browser.

According to one aspect of the application, a reverse order playing method is provided, which is applied to a webpage playing end and comprises the following steps:

the webpage playing end sends a reverse playing request to a server end, the server end carries out reverse slicing on a video stream to be played according to the reverse playing request to obtain a slicing sequence, reverse slicing of the slicing sequence is arranged in a reverse direction of acquisition time, a frame sequence in each reverse slicing is arranged in a forward direction of the acquisition time, and first time stamps of the frame sequences of the slicing sequence are set and then monotonically increase in the arrangement sequence;

the webpage playing end sequentially obtains the reverse order fragments from the server end;

the webpage playing end sets a timestamp offset attribute value of a source cache of a related media source expansion module for the obtained current reverse-order fragment and caches the current reverse-order fragment into the source cache, so that the timestamp offset attribute value related to the current reverse-order fragment is the time length obtained by subtracting the last reverse-order fragment from the timestamp offset attribute value related to the last reverse-order fragment;

a media source expansion module of the webpage playing end regenerates a second timestamp for the frame sequence in the current reverse sequence fragment according to the timestamp offset attribute value associated with the current reverse sequence fragment, so that the second timestamp of the frame sequence in the current reverse sequence fragment is monotonically increased according to the arrangement sequence, and the second timestamp of the initial frame of the frame sequence in the last reverse sequence fragment is decreased by the frame interval to be the second timestamp of the ending frame of the frame sequence in the current reverse sequence fragment;

and the webpage playing end plays the video stream to be played in a reverse order by adopting video elements, wherein the webpage playing end decrements the current time value of the video elements according to a stepping value, so that the current time value advances the reverse order playing of the video stream to be played in the decrementing direction of the second timestamp.

In some embodiments of the present application, the web page playing end decrementing the current time value of the video element by a step value, so that the advancing the reverse order playing of the video stream to be played by the current time value in the decrementing direction of the second timestamp comprises:

and in response to that the network transmission parameter between the webpage playing end and the server end is smaller than a set network transmission parameter, after caching a plurality of reverse-order fragments in the source cache, the webpage playing end decrements the current time value of the video element according to a step value, so that the current time value advances the reverse-order playing of the video stream to be played according to the decrementing direction of the second timestamp.

In some embodiments of the application, an initial value of the current time value is set to a maximum timestamp of the reverse tile in the source cache, the step value is less than or equal to a frame interval of the reverse tile, the webpage playing end decrements the current time value of the video element by the step value, so that the reverse playing of the video stream to be played is advanced by the current time value in the decrementing direction of the second timestamp, including:

the webpage playing end regularly decrements the current time value of the video element according to the stepping value, so that the current time value advances the reverse order playing of the video stream to be played according to the decremental direction of the second timestamp;

and in response to the current time value being equal to the minimum time stamp of the reverse-order fragment in the source cache, the webpage playing end pauses the reverse-order playing of the video stream to be played.

According to another aspect of the present application, there is also provided a reverse slicing method applied to a server, including:

the server receives a reverse playing request sent by a webpage playing end, wherein the reverse playing request comprises an identifier of a video stream to be played and an initial playing time, and the video stream to be played corresponding to the identifier of the video stream to be played is a local video file of the server;

the server side determines the position of the current frame of the video stream to be played based on the initial playing time;

the server side searches the nearest key frame from the current frame position in the video stream to be played in a reverse order;

the server end enables the current frame position to be the previous frame of the found nearest key frame to form a current original fragment, and the current original fragment is used for being packaged into a reverse fragment;

taking the key frame position of the current original fragment as the current frame position to position the next original fragment until the reverse-order fragment is finished;

and reversely arranging the reverse-order fragments in the acquisition time to generate a fragment sequence, wherein the frame sequence in each reverse-order fragment is arranged in the forward direction of the acquisition time, the first timestamps of the frame sequences of the fragment sequence are monotonically increased according to the arrangement sequence, and the fragment sequence is used for being sent to the webpage playing end to realize the reverse-order playing of the video stream to be played.

In some embodiments of the present application, the current original fragment is used for encapsulation as a reverse-order fragment; using the key frame position of the current original fragment as the current frame position to position the next original fragment until the reverse fragment is finished, comprising the following steps: in response to that the number of original fragments packaged as the current reverse-order fragment does not reach a set number threshold, the server takes the key frame position of the current original fragment as the current frame position to position the next original fragment until the reverse-order fragment is finished;

in response to the number of original fragments for encapsulation as the current reverse order fragment reaching a set number threshold:

the server side sequentially reads a frame sequence from the original fragments with the set quantity threshold in a positive sequence according to the acquisition sequence of the original fragments;

the server side packages each original fragment into a current reverse sequence fragment, generates a first time stamp which is monotonically increased in the sequence order for the read frame sequence of the current reverse sequence fragment, and generates the first time stamp of the frame sequence of the current reverse sequence fragment according to the first time stamp of the frame sequence of the previous reverse sequence fragment;

and the server takes the key frame position of the current original fragment as the current frame position to position the original fragment of the next reverse fragment until the reverse fragment is finished.

the server side receives a reverse playing request sent by a webpage playing side, the reverse playing request comprises an identifier of a video stream to be played and initial playing time, and the video stream to be played corresponding to the identifier of the video stream to be played is stored in a networking system;

the server takes the initial playing time as the request time for generating a forward playing request;

the server side generates a positive sequence playing request and sends the positive sequence playing request to the networking system;

the server receives a video stream to be played, which is sent by the networking system in a forward sequence from a key frame closest to the request time, takes a received first frame as a starting key frame of a current reverse sequence fragment, and encapsulates the starting key frame of the current reverse sequence fragment to a frame before the starting key frame of a previous reverse sequence fragment into the current reverse sequence fragment;

the server takes N seconds before the initial key frame of the current reverse-order fragment as request time to generate the forward-order playing request again until the reverse-order fragment is finished;

the server side arranges the reverse fragments in the reverse direction of the acquisition time to generate a fragment sequence, the frame sequence in each reverse fragment is arranged in the forward direction of the acquisition time, the first timestamps of the frame sequences of the fragment sequence are monotonically increased according to the arrangement sequence, and the fragment sequence is used for being sent to the webpage playing side to realize the reverse playing of the video stream to be played.

In some embodiments of the present application, when the server generates a first forward playing request, the server receives a video stream to be played sent by the networking system in forward sequence from a key frame closest to the request time, uses the received first frame as a starting key frame of a first reverse segment, and encapsulates the received video frame into the first reverse segment according to set time/set key frame number/end frame of the video stream to be played.

In some embodiments of the present application, the receiving, by the server side, a video stream to be played, which is sent by the networking system in a forward order from the key frame closest to the request time, taking the received first frame as an initial key frame of a current reverse order segment, and encapsulating the initial key frame of the current reverse order segment to a frame before the initial key frame of a previous reverse order segment into the current reverse order segment includes:

the server generates a first time stamp for a frame sequence in a packaged current reverse sequence fragment, and the first time stamp of the frame sequence of the current reverse sequence fragment is generated according to the first time stamp of the frame sequence of a previous reverse sequence fragment, so that the first time stamps of the frame sequence in the fragment sequence are monotonically increased in the arrangement sequence.

In some embodiments of the present application, encapsulating the first key frame of the current reverse-order segment to the previous frame of the first reverse-order segment into the current reverse-order segment includes:

in response to the current video frame received by the server being a key frame, matching the feature data of the key frame with the feature data of the starting key frame of the previous reverse-order fragment, if not, writing the key frame into the current reverse-order fragment and continuously receiving the video frame; if so, identifying the key frame as the initial key frame of the previous reverse fragment, and encapsulating the initial key frame of the current reverse fragment to the previous frame of the key frame into the current reverse fragment;

when the server side packages the current reverse-order fragment, the data of the set position of the coding load of the initial key frame of the current reverse-order fragment is used as the characteristic data of the initial key frame of the current reverse-order fragment.

In some embodiments of the present application, the forward-order play request is a multiple-speed forward-order play request.

In some embodiments of the present application, the reverse order segments are FMP4 media segments.

According to another aspect of the present application, there is also provided a reverse order playing system, including:

the webpage playing end executes the reverse playing method;

and the server side executes the reverse slicing method.

In some embodiments of the present application, further comprising:

and the networking system is used for communicating with the server end so as to send the video stream to be played to the server end.

According to yet another aspect of the present application, there is also provided an electronic apparatus, including: a processor; a storage medium having stored thereon a computer program which, when executed by the processor, performs the steps as described above.

According to yet another aspect of the present application, there is also provided a storage medium having stored thereon a computer program which, when executed by a processor, performs the steps as described above.

Therefore, compared with the prior art, the scheme provided by the application has the following advantages:

the method comprises the steps that a webpage playing end sends a reverse playing request to a server end, so that the server end can obtain a slicing sequence by executing reverse slicing of a video stream to be played, and the server end sets a first time stamp of the slicing sequence to ensure that a frame sequence in the slicing sequence can be transmitted in sequence; after the webpage playing end acquires the reverse-order fragments, the timestamp offset attribute values can be set for the reverse-order fragments through the media source expansion module and the source cache of the media source expansion module, and second timestamps which are monotonically decreased in the playing sequence can be generated according to the set timestamp offset attribute values so as to be matched with the current time values of video elements which are decreased in the stepping value, the reverse-order playing of the video to be played is promoted, and the video file does not need to be transferred and transcoded. In addition, the slicing and the playing are respectively decoupled to the server side and the webpage playing side, so that the slicing, the transmission and the playing can be in a pipeline form, and the time delay can be reduced in the process of playing reversely. Therefore, the low-cost and real-time response video stream reverse playing is realized, the video stream does not need to be stored in a reverse playing process, and an additional plug-in is not needed.

And the server supports reverse slicing of the local video file of the server. The server side enables the reverse-order fragments to accord with the processing format of a media source expansion module at the webpage playing side through packaging of the reverse-order fragments, and local video files do not need to be stored and transcoded; meanwhile, the server sets the first time stamp of the slicing sequence to ensure that the frame sequences in the slicing sequence can be transmitted in sequence. In addition, the slicing and the playing are respectively decoupled to the server side and the webpage playing side, so that the slicing, the transmission and the playing can be in a pipeline form, and the time delay can be reduced in the process of playing reversely.

And the server side supports the reverse slicing of the video files of the networking system. The server side encapsulates the reverse-order fragments, so that the reverse-order fragments can accord with the processing format of a media source expansion module of the webpage playing side, and video files transmitted by the networking system do not need to be subjected to dump and transcoding; meanwhile, the server side sets the first time stamp of the slicing sequence to ensure that the frame sequences in the slicing sequence can be transmitted in sequence. In addition, the video stream transmission, the slicing and the playing are respectively decoupled to the networking system, the server side and the webpage playing side, so that the video stream transmission, the slicing, the transmission and the playing can be in a pipeline form, and the time delay can be reduced in the process of playing backwards.

Drawings

The above and other features and advantages of the present application will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings.

Fig. 1 shows a flowchart of a reverse order playing method of a webpage playing end according to an embodiment of the present application.

Fig. 2 shows a flowchart of a reverse-order fragmentation method at a server according to an embodiment of the present application.

Fig. 3 shows a flowchart of a reverse-order fragmentation method at a server according to a specific embodiment of the present application.

Fig. 4 shows a flowchart of a reverse-order fragmentation method at a server according to another embodiment of the present application.

Fig. 5 is a flowchart illustrating a reverse-order fragmentation method at a server according to another embodiment of the present application.

FIG. 6 shows a slice schematic of an original video file according to an embodiment of the present application;

fig. 7 shows a schematic diagram of acquiring a slice sequence of an original video in reverse order according to an embodiment of the present application.

Fig. 8 shows a schematic diagram of a slice sequence with a first time stamp according to an embodiment of the application.

Fig. 9 shows a schematic diagram of a slice sequence with a second timestamp according to an embodiment of the present application.

Fig. 10 is a block diagram illustrating a reverse order playback system according to an embodiment of the present application.

Fig. 11 schematically illustrates a computer-readable storage medium in an exemplary embodiment of the disclosure.

Fig. 12 schematically illustrates an electronic device in an exemplary embodiment of the disclosure.

Detailed Description

Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

Furthermore, the drawings are merely schematic illustrations of the present application and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repetitive description will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.

The flow charts shown in the drawings are merely illustrative and do not necessarily include all of the steps. For example, some steps may be decomposed, and some steps may be combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.

Referring first to fig. 1, fig. 1 is a flowchart illustrating a reverse order playing method of a webpage playing end according to an embodiment of the present application. The reverse playing method of the webpage playing end comprises the following steps:

step S111: and the webpage playing end sends a reverse playing request to the server end.

Specifically, the reverse playing request is provided for the server to perform reverse slicing on the video stream to be played according to the reverse playing request to obtain a slice sequence, the reverse slices of the slice sequence are arranged in a reverse direction to the acquisition time, the frame sequence in each reverse slice is arranged in a forward direction to the acquisition time, the first timestamps of the frame sequences of the slice sequence are set and then monotonically increase in the arrangement sequence, and the acquisition time refers to the time during video acquisition and is the same as the time lapse direction.

In particular, the "reverse" direction described in this application is the direction opposite to the passage of time, i.e. opposite to the video capture timing.

Specifically, the reverse play request sent by the webpage playing end may include the video stream identifier to be played, the starting playing position/time, and the playback direction (in this embodiment, the playback direction is in reverse order). The video stream identifier to be played may be a Uniform Resource Locator (URL) of the video to be played. The video stream to be played is a video file collected in history.

Specifically, when slicing in the reverse order, the server may package the slices to obtain reverse-order slices conforming to the set format and a slice sequence formed by arranging the reverse-order slices. The format is set to be, for example, the FMP4 format in accordance with ISO BMFF specification, so that the format requirement of the media source expansion module at the webpage playing end is met. The ISO BMFF specification may maximally accommodate cross-browser compatibility. A media fragment in FMP4 format (reverse order fragment) may be composed of moof, mdata, and optional boxes such as styp, sidx, etc. Initialization fragmentation in FMP4 format may be used composed of ftyp and moov and other top box.

Specifically, one reverse-order slice on the server side may include one or more Group of pictures (GOP). The server may find a corresponding start frame according to the requested start playing time, search for a nearest key frame (I1 frame) in reverse order from a direction opposite to the start playing time, form a first original slice 1 (GOP 1) from a frame sequence between the I1 frame and the start playing time, search for a key frame in reverse order from the I1 frame again, find a key frame (I2 frame) closest to the I1 frame, form a second original slice 2 (GOP 2) from a frame sequence between the I2 frame and a previous frame P2 of the first I1 frame, as shown in fig. 6 (the arrow direction in fig. 6 is the direction of increasing the original timestamp), and so on, slice by slice of the original slices, where the frame sequence in each slice has an original timestamp, that is, a timestamp increasing in the forward direction according to the video acquisition time.

Specifically, the server may generate the slice sequence according to the number of original slices included in the set reverse-order slice. In a preferred embodiment of the present application, the reverse-order fragment may include only one original fragment, so that timeliness of obtaining the reverse-order fragment can be ensured. The slicing sequence is shown in fig. 7 (the arrow direction in fig. 7 is the increasing direction of the original timestamp in each reverse-order slice), the reverse-order slices of the slicing sequence are arranged in reverse order of the acquisition time (the reverse-order slices are arranged into an original slice 3, an original slice 2 and an original slice 1 in forward order of the acquisition time, the reverse-order slices are arranged into an original (reverse-order) slice 1, an original (reverse-order) slice 2 and an original (reverse-order) slice 3 in reverse order of the acquisition time), and the frame sequence in each reverse-order slice is arranged in forward direction of the acquisition time. In other words, at this time, the read slices are not encapsulated in fig. 7, but the reverse order slices are arranged according to the reverse order reading order, so the frame sequence in each reverse order slice still has the original timestamp, i.e. the timestamp that is incremented in the forward direction of the video capture time. After reading the sequence of slices in reverse order, the monotonicity of the time stamps is disrupted. Therefore, the server side can regenerate the first timestamp for the reverse fragment after the reverse fragment is obtained by encapsulation, and the first timestamp monotonically increases according to the sequence of the fragments and the sequence of the frames thereof (as shown in fig. 8, the arrow direction in fig. 8 indicates the increasing direction of the first timestamp), so that the monotonicity of the timestamps of the sequences of the frames is ensured not to be interrupted, and transmission and frame processing are facilitated.

Step S112: and the webpage playing end sequentially obtains the reverse-order fragments from the server end.

Specifically, while the server generates the slice sequence, the server may send index information of the slice sequence (for example, an index list may be generated according to the slice sequence) to the web page playing side, and the web page playing side sequentially obtains the reverse order slices according to the index information (for example, the index list). In some embodiments, the webpage playing end may obtain the reverse order fragments block by block in an HTTP Get manner. In other embodiments, the web playing side may also establish a Websocket long connection with the server side, and the server side pushes the pieces one by one. Thus, step S112 transmits the reverse order slices in a reverse order compared to the acquisition time. Further, step S112 may also control the rhythm of transmission according to the condition of the playing buffer of the web page playing end. For example, when the play cache of the webpage play end is idle, the transmission speed of the reverse slicing is increased; when the reverse-order fragments are cached at the webpage playing end, the transmission speed of the reverse-order fragments can be reduced. The present application is not so limited.

Step S113: and the webpage playing end sets a timestamp offset attribute value of a source cache of a related media source expansion module for the obtained current reverse-order fragment and caches the current reverse-order fragment into the source cache, so that the timestamp offset attribute value related to the current reverse-order fragment is the time length obtained by subtracting the time length of the last reverse-order fragment from the timestamp offset attribute value related to the last reverse-order fragment.

Specifically, the source cache is in a sequence mode, so that the reverse-order fragments can be input to the source cache (SourceBuffer object) in a sequence manner.

In particular, the initial value of the timestamp offset attribute value of the source buffer may be set to be greater than (and preferably may be much greater than) the total duration of the sequence of slices to ensure that the timestamp offset attribute value is sufficient for decrementing of the reverse play of the sequence of slices. Furthermore, when the webpage playing end caches the reverse fragments into the source cache each time, the timestamp offset attribute value associated with the current reverse fragment is the time length obtained by subtracting the time length of the last reverse fragment from the timestamp offset attribute value associated with the last reverse fragment, so that the setting of the timestamp offset attribute value is realized.

Step S114: and the media source expansion module of the webpage playing end regenerates a second timestamp for the frame sequence in the current reverse fragment according to the timestamp offset attribute value associated with the current reverse fragment.

Specifically, step S114 makes the second timestamps of the frame sequences in the reverse slice monotonically increase in the arrangement order, and the second timestamp of the starting frame of the frame sequence in the last reverse slice decreases by the second timestamp of the ending frame of the frame sequence in the current reverse slice.

In some embodiments, a second timestamp of an ending frame (e.g., a P-frame) of each reverse slice may be determined from the timestamp offset attribute value, the sequence of frames within the reverse slices monotonically increasing at frame intervals in the order of arrangement. In other words, after determining the second timestamp of the ending frame based on the timestamp offset attribute value, the timestamp offset attribute value is made to be a self-decreasing frame interval to obtain the second timestamp of the frame immediately preceding the ending frame, and so on until determining the second timestamp of the starting frame of the current reverse slice. Further, the timestamp offset attribute value of the current reverse sequence fragment is set to be the timestamp offset attribute value of the previous reverse sequence fragment minus the duration of the previous reverse sequence fragment, and a frame interval is formed between the second timestamp of the ending frame of the current reverse sequence fragment and the second timestamp of the starting frame of the previous reverse sequence fragment, specifically, a frame interval decreased by the second timestamp of the starting frame of the previous reverse sequence fragment is set to be the second timestamp of the ending frame of the current reverse sequence fragment, so that the second timestamps between adjacent reverse sequence fragments are decreased in the arrangement order, and the continuity of the reverse sequence fragments in playing is maintained.

In other embodiments, the quasi-second timestamps corresponding to the frames in the reverse order slices may be generated for the frames in the reverse order slices according to the frame sequence in the reverse order slices and the frame intervals, and then the quasi-second timestamps corresponding to the frames in the reverse order slices are added to the timestamp offset attribute value associated with the reverse order slices to obtain the second timestamps corresponding to the frames in the reverse order slices. The present application can implement more variations, which are not described herein.

Specifically, through steps S113 to S114, the timestamps of the adjacent reverse slices decrease in the obtaining order, and the timestamps of the frame sequences in the reverse slices increase monotonically in the arranging order, as shown in fig. 9, so that the second timestamps of the sequentially obtained

reverse slices

1, 2, and 3 decrease in the obtaining order, and the second timestamps of the frame sequences in each reverse slice increase monotonically in the arranging order (from I1 to P1; from I2 to P2; from I3 to P3), that is, the second timestamps are arranged in the decreasing order in fig. 9: p1 → I1 → P2 → I2 → P3 → I3, the arrangement order is also the reverse order playing order; the corresponding arrow direction in fig. 9 indicates the playing direction in the reverse order slice, such as the reverse order slice 1 is played from P1 to I1.

Step S115: and the webpage playing end plays the video stream to be played in a reverse order by adopting video elements.

Specifically, the webpage playing end decrements the current time value of the video element by a step value, so that the current time value advances the reverse order playing of the video stream to be played in the decremental direction of the second timestamp.

Specifically, the reverse-order segment input to the source cache through steps S113 and S114 finally needs to be played through the video element of the browser. The webpage playing end can advance the playing of the video frame according to the current time value of the video element. And by decreasing the current time value in step S115, the reverse play of the video stream to be played can be advanced in the decreasing direction of the second timestamp (as in fig. 9, the reverse play is performed in the direction of P1 → I1 → P2 → I2 → P3 → I3).

In some embodiments, step S115 may set a timer, and perform the timed decrementing task for the current time value based on the timer. Since the browser will automatically align to the second timestamp based on the current time value, the step value (timing interval) of the timer may be less than or equal to the frame interval of the frame sequence. For example, the frame interval may be obtained by scaling the playing frequency of the video frame. Thus, based on the timer, the current time value (currentTime attribute value) in the video element may be modified at a step value timing.

Specifically, the current time value in the video element may be initialized to the time stamp of the last frame of the first reverse-order segment in the source cache, that is, the maximum time stamp source buffer is buffered in the source cache (0), as shown in the second time stamp of the P1 frame of the reverse-order segment 1 in fig. 9, that is, the video element is played from the P1 frame according to the current time value of the video element; the current time value is automatically decreased by the step value in each timing task, and since the step value is less than or equal to the frame interval and the browser can automatically position the second timestamp, in the above example, it is assumed that the current time value is the second timestamp of P1 in fig. 9, and after the step value is subtracted from the current time value, the browser positions the second timestamp of the next frame of the P1 frame in the direction from the P1 frame to the I1 frame for playing, and so on; in response to that the current time value is equal to the minimum timestamp of the reverse-order segment in the source cache, such as the second timestamp of the P3 frame in fig. 9, the webpage playing end pauses the reverse-order playing of the video stream to be played.

Specifically, when the network transmission parameter between the web page playing end and the server end is smaller than a set network transmission parameter (for example, the network transmission speed is low, or the network shakes), the web page playing end may decrease the current time value of the video element by a step value after caching a plurality of reverse-order segments in the source cache, so that the current time value advances the reverse-order playing of the video stream to be played in the decreasing direction of the second timestamp, thereby preventing the subsequent segments from arriving in time and causing unsmooth playing.

Further, the webpage playing end may also start to execute the playing step after receiving the first reverse segment, so as to achieve a more timely response.

In the reverse playing method of the webpage playing end, the webpage playing end sends a reverse playing request to the server end, so that the server end can obtain a slicing sequence by executing reverse slicing of a video stream to be played, and the server end sets a first time stamp of the slicing sequence to ensure that a frame sequence in the slicing sequence can be transmitted in sequence; after the webpage playing end acquires the reverse-order fragments, the timestamp offset attribute values can be set for the reverse-order fragments through the media source expansion module and the source cache of the media source expansion module, and second timestamps which are monotonically decreased in the playing sequence can be generated so as to be matched with the current time values of the video elements which are regularly decreased in the stepping value, the reverse-order playing of the video to be played is promoted, and the video file does not need to be stored and transcoded. In addition, the slicing and the playing are respectively decoupled to the server side and the webpage playing side, so that the slicing, the transmission and the playing can be in a pipeline form, and the time delay can be reduced in the process of playing reversely. Therefore, the low-cost and real-time response video stream reverse playing is realized, the video stream does not need to be stored in a reverse playing process, and an additional plug-in is not needed.

Referring now to fig. 2, fig. 2 is a flow chart illustrating a method for server-side reverse slicing according to an embodiment of the present application. In this embodiment, the video stream to be played is a local video file of the server. The reverse slicing method of the server side comprises the following steps:

step S101: the server receives a reverse playing request sent by the webpage playing end,

specifically, the reverse play request includes an identifier of a video stream to be played and an initial play time, and the video stream to be played corresponding to the identifier of the video stream to be played is a local video file of the server;

step S102: the server side determines the position of the current frame of the video stream to be played based on the initial playing time;

step S103: the server side searches the key frames with the nearest distance from the current frame position in the video stream to be played in a reverse order;

step S104: the server end enables the current frame position to be the previous frame of the found nearest key frame to form a current original fragment, and the current original fragment is used for being packaged into a reverse fragment;

step S105: taking the key frame position of the current original fragment as the current frame position to position the next original fragment until the reverse-order fragment is finished;

step S106: and arranging the reverse-order fragments in the reverse direction of the acquisition time to generate a fragment sequence, arranging the frame sequence in each reverse-order fragment in the forward direction of the acquisition time, monotonically increasing the first time stamp of the frame sequence of the fragment sequence according to the arrangement sequence, and sending the fragment sequence to the webpage playing end to realize the reverse-order playing of the video stream to be played.

In the reverse-order slicing method provided by the application, the server side supports reverse-order slicing on the local video file of the server side. The server side encapsulates the reverse-order fragments, so that the reverse-order fragments can accord with the processing format of a media source expansion module of the webpage playing side, and the local video file does not need to be subjected to unloading and transcoding; meanwhile, the server side sets the first time stamp of the slicing sequence to ensure that the frame sequences in the slicing sequence can be transmitted in sequence. In addition, the slicing and the playing are respectively decoupled to the server side and the webpage playing side, so that the slicing, the transmission and the playing can be in a pipeline form, and the time delay can be reduced in the process of playing reversely.

Referring now to fig. 3, fig. 3 is a flow chart illustrating a method for server-side reverse slicing according to an embodiment of the present application. In this embodiment, the video stream to be played is a local video file of the server. The reverse slicing method of the server side comprises the following steps:

step S121: the server receives the web page and a reverse playing request is sent by the playing end.

Specifically, the reverse play request includes an identifier of a video stream to be played and an initial play time, and the video stream to be played corresponding to the identifier of the video stream to be played is a local video file of the server.

Step S122: and the server determines the current frame position of the video stream to be played based on the initial playing time.

Specifically, step S122 may further include a step of initializing the slice. Further, media Parameter information such as SPS (Sequence Parameter Sets) and PPS (Picture Parameter Set) of the video stream to be played may be read to initialize the slice. Further, the FMP4 initial Segment (Initialization Segment) may also be output based on the Initialization Segment. The FMP4 initial Segment (Initialization Segment) is the first Segment of the FMP4 Segment to conform to the adaptive streaming media protocol. The reverse order slices generated hereinafter are media slices following the first slice that contain the actual video stream.

Step S123: and the server side searches the key frames with the nearest distance from the current frame position in the video stream to be played in a reverse order.

Step S124: and the server side forms a current original fragment (GOP) from the current frame position to the previous frame of the searched nearest key frame, and the current original fragment is used for being packaged into a reverse sequence fragment.

Step S125: and judging whether the number of the original fragments for packaging the current reverse sequence fragments reaches a set number threshold value.

In particular, to improve the real-time performance of the fragment transmission, each reverse fragment may only include one original fragment.

If the determination in step S125 is no, step S126 is executed: and the server takes the key frame position of the current original fragment as the current frame position so as to position the next original fragment.

After step S126, step S123 is returned to again to locate the next original segment until the segment in the reverse order is finished. The reverse slicing can be finished based on that the video stream to be played is sliced completely or the user cancels the slicing operation.

If yes in step S125, step S127 is executed: and the server side sequentially reads the frame sequences from the original fragments with the set quantity threshold in the positive sequence according to the acquisition sequence of the original fragments.

Step S128: and the server side encapsulates each original fragment into a current reverse fragment, generates a first time stamp which is monotonically increased according to the arrangement sequence for the read frame sequence of the current reverse fragment, and generates the first time stamp of the frame sequence of the current reverse fragment according to the first time stamp of the frame sequence of the previous reverse fragment.

In particular, the original slices may be packaged as reverse order slices in FMP4 format. The first timestamp is monotonically increasing both between reverse order slices and within reverse order slices (see fig. 8).

In particular, the first timestamp of the starting frame of the current reverse sliced frame sequence may be determined based on the first timestamp of the ending frame of the previous reverse sliced frame sequence, e.g., such that the first timestamp of the starting frame of the current reverse sliced frame sequence is the first timestamp of the starting frame of the current reverse sliced frame sequence plus the frame interval, thereby causing the first timestamps of the frame sequences in the slice sequence to monotonically increase in rank order.

Step S129: and judging whether the reverse-order slicing is finished or not.

If the determination in step S129 is no, step S1210 is executed: and the server side takes the found nearest key frame as the position of the current frame so as to position the original fragment of the next reverse fragment.

After step S1210, go back to step S123 again to locate the next original segment until the segment in the reverse order is finished.

If the determination in step S129 is yes, step S1211 is executed: and arranging the reverse-order slices in the reverse direction of the acquisition time to generate a slice sequence.

The reverse sequence of the slicing sequence is arranged in the reverse direction of the acquisition time, the frame sequence in each reverse sequence of the slicing sequence is arranged in the forward direction of the acquisition time, the first time stamps of the frame sequence of the slicing sequence are monotonically increased in the arrangement sequence, and the slicing sequence is used for being sent to the webpage playing end to realize the reverse sequence playing of the video stream to be played.

Referring now to fig. 4, fig. 4 is a flow chart illustrating a server-side reverse slicing method according to another embodiment of the present application. In this embodiment in (1), the video stream to be played is stored in a networking system.

Step S141: and the server receives a reverse playing request sent by the webpage playing end.

The reverse playing request comprises an identifier of a video stream to be played and an initial playing time, and the video stream to be played corresponding to the identifier of the video stream to be played is stored in the networking system;

step S142: and the server takes the initial playing time as the request time for generating the forward playing request.

Specifically, the server may generate an initial forward playing request according to the starting playing time. And receiving the video stream to be played, which is sent by the networking system in the positive sequence from the initial playing time, and taking the received first key frame as the initial key frame of the current reverse-sequence fragment.

Step S143: the server side generates a forward playing request, and sending to the networking system;

step S144: and the server receives the video stream to be played, which is sent by the networking system in the forward sequence from the key frame closest to the request time, takes the received first frame as the initial key frame of the current reverse sequence fragment, and encapsulates the initial key frame of the current reverse sequence fragment to the previous frame of the initial key frame of the previous reverse sequence fragment into the current reverse sequence fragment.

Specifically, when the server generates a first forward playing request, the server receives a video stream to be played, which is sent by the networking system from a key frame closest to the request time in a forward sequence, takes the received first frame as a starting key frame of a first reverse segment, and encapsulates the received video frame into the first reverse segment according to the set time, the set key frame number and the ending frame of the video stream to be played.

Step S145: the server takes N seconds before the initial key frame of the current reverse-order fragment as request time to generate the forward-order playing request again until the reverse-order fragment is finished;

specifically, after the forward playing request is generated again in step S145, steps S143 to 144 are executed until the reverse slicing is finished. And ending the reverse-order slicing when a slicing canceling instruction is received or the video stream is ended.

Step S146: and the server side arranges the reverse fragments in the reverse direction of the acquisition time to generate a fragment sequence.

The frame sequences in each reverse-order sub-slice are arranged in the forward direction of the acquisition time, first time stamps of the frame sequences of the slice sequences are monotonically increased according to the arrangement sequence, and the slice sequences are used for being sent to the webpage playing end to achieve reverse-order playing of the video stream to be played.

In the reverse slicing method provided by the application, the server side supports reverse slicing of the video files of the networking system. The server side encapsulates the reverse-order fragments, so that the reverse-order fragments can accord with the processing format of a media source expansion module of the webpage playing side, and video files transmitted by the networking system do not need to be subjected to dump and transcoding; meanwhile, the server side sets the first time stamp of the slicing sequence to ensure that the frame sequences in the slicing sequence can be transmitted in sequence. In addition, the video stream transmission, the slicing and the playing are respectively decoupled to the networking system, the server end and the webpage playing end, therefore, the transmission, the slicing, the transmission and the playing of the video stream can be in a pipeline form, and the reduction of time delay can be realized in the process of reverse playing.

Referring now to fig. 5, fig. 5 is a flow chart illustrating a server-side reverse slicing method according to another embodiment of the present application. In this embodiment, the video stream to be played is stored in a networked system.

Step S131: and the server receives a reverse playing request sent by the webpage playing end.

Specifically, the reverse play request includes an identifier of a video stream to be played and an initial play time, and the video stream to be played corresponding to the identifier of the video stream to be played is stored in the networking system.

Step S132: and the server takes the initial playing time as the request time for generating the initial forward playing request.

Step S133: and the server side sends an initial forward playing request to the networking system.

Specifically, the initial forward play request may include a request time, which is a start play time of the reverse play request. The initial forward-sequence play request may include an identification of the video stream to be played and an end play time.

Further, the forward play request may further include a play double speed. In some embodiments, the forward-order playing request may be a single-speed playing request, so that the networked system sends the video stream to be played to the server at a normal playing speed, so that the server receives the video stream to be played and then performs reverse-order slicing.

In other embodiments, in view of video transmission and communication between the networked system and the server, in order to ensure that the webpage playing end can obtain the reverse-order segments in real time and play the reverse-order segments, the forward-order playing request in step S132 may be a multiple-speed forward-order playing request. Therefore, the networking system can send the video stream to be played to the server side in the multi-speed positive sequence based on the multi-speed positive sequence playing request. For example, the forward-sequence play request may be a 1.5-speed, 2-speed, or 4-speed forward-sequence play request, which is not limited in this application.

Step S134: and the server receives the video stream to be played, which is sent by the networking system from the key frame closest to the initial playing time in the forward sequence, takes the received first frame as the initial key frame of the first reverse-sequence fragment, and encapsulates the received video frame into the first reverse-sequence fragment according to the set time/the set key frame number/the termination frame of the video stream to be played.

Specifically, the set duration of the first reverse-order slice or the set number of key frames may be preset. Therefore, the slicing position of the first reverse slicing is determined according to whether the time length of the received video stream to be played meets the set time length and whether the number of key frames contained in the received video stream to be played meets the set number of key frames. Further, the start key frame of the first reverse-order segment to the end frame of the video stream to be played may also be used as the first reverse-order segment.

Specifically, the server side continuously receives the video stream transmitted by the networking system based on the forward playing request and encapsulates the packets into frames.

Specifically, in step S134, the server generates first timestamps for the frame sequences in the first reverse segment, so that the first timestamps of the frame sequences in the first reverse segment monotonically increase according to the arrangement order and monotonically decrease according to the playing order.

Specifically, step S134 may also initialize the step of slicing. Further, the fragmentation can be initialized by the server side and the networking system negotiating the media parameter information returned or parsed in the video stream. Further, an FMP4 initial Segment (Initialization Segment) may also be output based on the Initialization Segment. The FMP4 initial Segment (Initialization Segment) is the first Segment of the FMP4 Segment to conform to the adaptive streaming media protocol. The reverse order slices generated hereinafter are media slices following the first slice that contain the actual video stream.

Step S134 may further include the step of taking the packaged first reverse order segment as the previous reverse order segment.

Step S135: the server side obtains the characteristic data of the initial key frame of the previous reverse sequence fragment, takes N seconds before the initial key frame of the previous reverse sequence fragment as request time, and generates a forward sequence playing request according to the request time.

Specifically, the forward play request may include a request time, similar to the initial forward play request. It may also include the identification of the video stream to be played, the end play time, and the play speed.

Specifically, N is a constant greater than 0. N may be set according to the interval of the key frame. N may be set to, for example, 1 second, 2 seconds, etc., and the present application is not limited thereto. The forward play request is similar to the initial forward play request, and may include a request time and a play speed, which are not described herein.

Specifically, the N seconds before the start key frame of the last reverse-order fragment is used as new request time, a forward-order playing request is generated according to the new request time, and the packaged first reverse-order fragment is used as a previous reverse-order fragment.

Step S136: the server side sends a forward playing request to a networking system, receives a video stream to be played sent by the networking system in a forward sequence from a key frame closest to the request time, and takes a received first frame as a starting key frame of the current reverse-order fragment.

Step S137: judging whether the current frame received by the server is a key frame, if not, executing step S138: writing the current frame into the current reverse-order fragment, continuously receiving the next frame of the current frame as the current frame, and entering step S137;

if yes, go to step S139: matching the feature data of the current key frame with the feature data of the initial key frame of the previous reverse-order fragment;

if the step S139 is matched, execute step S1310: and identifying the current key frame as the starting key frame of the previous reverse sequence fragment, and encapsulating the starting key frame of the current reverse sequence fragment to the previous frame of the key frame into the current reverse sequence fragment.

Specifically, in step S1310, the key frame is the current key frame identified as the starting key frame of the previous reverse slice; when the set position data of the coding load of the key frame is consistent with the set position data of the coding load of the starting key frame of the previous reverse fragment, the key frame is matched with the starting key frame of the previous reverse fragment, so that the current key frame can be used as the starting key frame of the previous reverse fragment to position the ending frame of the current reverse fragment. For example, the first 128 bits of the encoded payload of the key frames may be used as feature values to facilitate matching between key frames. Further, in step S133, data of the set position of the coding payload of the start key frame of the previous reverse slice may be used as the feature data. Therefore, in step S135, if the current frame is a key frame, the feature data of the current frame is matched with the feature data of the starting key frame of the previous reverse segment, and if the matching is consistent, the current frame is matched with the starting key frame of the previous reverse segment, so that the matching efficiency of the key frame can be improved. Other key frame matching approaches are within the scope of the present application. Furthermore, the ending frame of the current reverse sequence fragment is determined to be the previous frame of the starting key frame of the previous reverse sequence fragment through matching of the current key frame and the starting key frame of the previous reverse sequence fragment, that is, the starting key frame of the current reverse sequence fragment and the previous frame of the key frame are encapsulated into the current reverse sequence fragment, so that the number of the key frames contained in each reverse sequence fragment is not required to be limited.

If the step S139 is not matched, the step S138 is executed.

Specifically, step S1310 may encapsulate the currently received video frame into FMP4 reverse segment, so that the media source extension module at the web page playing end can process the video frame.

Step S1311: and the server side generates a first time stamp for the frame sequence in the packaged current reverse-order fragment and outputs the packaged current reverse-order fragment.

Specifically, the first timestamps monotonically increase in the order of arrangement of the frame sequences within the reverse tiles and between the reverse tiles. In particular, the first timestamp of the starting frame of the current reverse slice may be determined from the first timestamps of the ending frames of the sequence of frames of the previous reverse slice. For example, the first timestamp of the starting frame of the current reverse sliced frame sequence is the first timestamp of the ending frame of the previous reverse sliced frame sequence plus the frame interval, thereby making the first timestamps of the frame sequences in the slice sequence monotonically increasing in the order of arrangement.

Step S1312: and judging whether the reverse-order fragmentation is finished or not.

If the determination in step S1312 is no, step S1313 is executed: the packaged current reverse-order fragment is taken as the previous reverse-order fragment, and step S135 is executed again.

If the determination in step S1312 is yes, step S1314 is executed: and arranging all the packaged reverse-order slices in the reverse direction of the acquisition time to generate a slice sequence.

The above steps are explained with reference to fig. 6 to 8. After receiving the reverse play request sent by the webpage playing end, the server end takes the initial play time in the reverse play request as the request time to generate a reverse play request and sends the reverse play request to the networking system. The networking system locates the nearest key frame (for example, I1) according to the starting playing time, and sends the video stream to be played to the server in the forward order. The server side takes the received first frame (I1) as a starting key frame of a first reverse fragment, then positions to an ending frame of the first reverse fragment (P1, between I1 and P1 may include a non-key frame or include a non-key frame and one or more key frames) according to a set duration/a set key frame number/an ending frame of a video stream to be played, encapsulates the first reverse fragment according to a sequence of frames from I1 to P1 (original fragment 1), and sets a first timestamp for the sequence of frames of the first reverse fragment, so that the sequence of frames in the first reverse fragment monotonically increases according to the arrangement sequence (as in fig. 8). After the server side encapsulates the first reverse-order fragment, the data (for example, the data of the first 128 bits of the encoded payload) of the set position of the encoded payload of the I1 frame is used as the feature data. And taking N seconds before the I1 frame as new request time, generating an updated reverse order playing request and sending the updated reverse order playing request to the networking system.

The networking system locates the nearest key frame (such as I2) according to N seconds before the I1 frame, and sends the video stream to be played to the server side from the I2. After the server receives the I2, comparing whether the characteristic data of the I2 is matched with the characteristic data of the I1, if not, writing the I2 as the initial key frame of the second reverse fragment into the second reverse fragment, continuously receiving the video frame of the video stream to be played, judging whether the currently received video frame is the key frame or not when the received video frame is the key frame, if not, directly writing the second reverse fragment, if so, comparing whether the characteristic data of the current key frame is matched with the characteristic data of the I1, if the characteristic data of the current key frame is consistent with the characteristic data of the I1, showing that the currently received key frame is the initial key frame I1 of the first reverse fragment, then continuously reading the frame which can enable the video frame of the second reverse fragment to be repeated with the video frame of the first reverse fragment, therefore, the previous frame P2 of the I1 frame can be used as the ending frame of the second reverse fragment, packaging the server into the second reverse fragment according to the frame sequence of the I2-P2 (the original fragment 2) and setting a first time stamp for the second reverse fragment, enabling the second time stamp to be arranged in the monotonous sequence of the second fragment (the original fragment) to be increased gradually as a first time stamp sequence of the I2), and the I2 is increased gradually increased as a first time sequence of the first time stamp. And the like until the operation is cancelled or no history video stream can be read.

And if the networking system locates the nearest key frame as I3 according to N seconds before the I1 frame, and the video stream to be played is sent to the server side from the I3. The server side takes I3 as a starting key frame of a second reverse sequence fragment, continuously receives video frames of a video stream to be played, judges whether the currently received video frames are key frames or not when receiving the video frames, if not, directly writes the video frames into the second reverse sequence fragment, if yes, compares whether the feature data of the current key frames are matched with the feature data of I1, if the received key frames are I2, takes data (such as data of the first 128 bits of a coding load) of the coding load of the key frames I2 as the feature data, matches with the feature data of the starting key frame I1 of the first reverse sequence fragment, and if not, continuously receives the video frames until the received key frames are I1 frames, therefore, the previous frame P2 of the starting key frame of the first reverse sequence fragment can be taken as a termination frame of the second reverse sequence fragment, the server side is packaged into the second reverse sequence fragment according to a frame sequence from I3 to P2, and sets a first time stamp for the frame sequence of the second reverse sequence fragment, so that the second reverse sequence of the frame is gradually increased. And so on until the operation is cancelled or no history video stream can be read.

The above exemplary embodiments of the present application are shown, the present application is not limited thereto, and in each embodiment, the addition, the omission, and the sequence change of the steps are all within the protection scope of the present application; the embodiments may be implemented individually or in combination.

The reverse order playback system 200 provided by the present application is described below with reference to fig. 10. The reverse playing system 200 includes a web playing side 210 and a server side 220. The web playing end 210 may perform the steps shown in fig. 1, and the server end 220 may perform the steps shown in fig. 2 to 5, thereby supporting the reverse playing of the video stream to be played. Further, the reverse order playing system 200 may further include a networking system 230, which can implement reverse playing of the front-end surveillance video. Fig. 10 schematically illustrates the web playing side 210, the server side 220, and modules included in the networking system 230, which is not intended to limit the present application.

The web playing end 210 may include a control module 211, an obtaining module 212, and a playing module 213.

Specifically, the control module 211 of the web page playing end 210 may be configured to select a video stream to be played and a start playing time/position, send a reverse request to the server end 220, and receive index information of a slice sequence sent by the server end 220.

Specifically, the obtaining module 212 of the web page playing end 210 may be configured to download the reverse order segment (FMP 4 segment) generated by the server 220, where the downloading of the reverse order segment and the generation of the reverse order segment are performed simultaneously. Meanwhile, the obtaining module 212 may adjust the downloading rhythm according to the local fragment cache condition of the player 210.

Specifically, the playing module 213 of the web page playing end 210 may input the reverse-order segments to the source cache object of the media source extension module, and finally input the reverse-order segments to the video element. In addition, the playing module 213 further modifies the current time value of the video element by a timed decrement, so as to achieve the purpose of playing control according to the specified time sequence. The media source extension module is applied to the HTML5 technology, so that the application can realize the reverse-order playing of the videos based on the HTML5 technology.

The server side 220 includes a signaling module 221, an access module 222, a sending module 223, and a slicing module 224.

Specifically, the signaling module 221 of the server side 220 may be configured to receive and process a control request, such as a reverse play, sent by the web playing side 210, and feed back notification information, such as index information update, issued by the server side 220 to the web playing side 210. The signaling module 221 may be configured to interact with the slicing module 224, issue a specific code stream slicing task, and receive a slice index update notification fed back by the slicing module 224. The signaling module 221 may be configured to forward the request slicing code stream of the slicing module 224 to the networking system 230, and the like.

In particular, the access module 222 of the server side 220 may be used for maintaining the link, signaling adaptation and conversion of the networking system 230. The access module 222 may act as a protocol gateway. Optionally, the access module 222 may support GB/T28181 (SIP for media control related sub-protocol) and ONVIF (RTSP for media control related sub-protocol) as industry specification protocols.

Specifically, the sending module 223 of the server side 220 may be configured to respond to a request for downloading fragment data from the obtaining module 212 of the web page playing side 210, read the reverse-order fragment generated by the slicing module 224 from the local cache, and send the reverse-order fragment to the obtaining module 212 of the web page playing side 210.

Specifically, the slicing module 224 of the server side 220 may be configured to perform reverse order slicing on the video stream to be played, and the video stream to be played may be stored in the server local video library 225. The slicing module 224 of the server side 220 may also support slicing the video streams to be played sent by the remote networking system 230 in reverse order. At this point, the slicing module 224 may act as an on-demand gateway.

The networked system 230 may be deployed according to an actual scenario. In some embodiments, the present application is used for monitoring a scene, so that the networked system 230 may include a front-end monitoring device 231 and a lower-level monitoring platform 232, which is not limited to this application, and other remote networked systems 230 are also within the scope of the present application.

In the reverse playing system provided by the application, the webpage playing end sends a reverse playing request to the server end, so that the server end can obtain a slice sequence by executing reverse slicing of a video stream to be played, and the server end sets a first time stamp of the slice sequence to ensure that a frame sequence in the slice sequence can be transmitted in sequence; the server side supports the inverted arrangement of the fragments of the local video file of the server side and/or supports the inverted arrangement of the fragments of the video file of the networking system. The server side encapsulates the reverse-order fragments, so that the reverse-order fragments can accord with the processing format of a media source expansion module of the webpage playing side, and the local video file does not need to be subjected to unloading and transcoding; after the webpage playing end acquires the reverse-order fragments, the timestamp offset attribute values can be set for the reverse-order fragments through the media source expansion module and the source cache of the media source expansion module, and second timestamps which are monotonically decreased in the playing sequence can be generated so as to be matched with the current time values of the video elements which are regularly decreased in the stepping value, the reverse-order playing of the video to be played is promoted, and the video file does not need to be stored or transcoded. In addition, the slicing and the playing are respectively decoupled to the server side and the webpage playing side, so that the slicing, the transmission and the playing can be in a pipeline form, and the time delay can be reduced in the process of playing reversely. Therefore, the low-cost and real-time response video stream reverse playing is realized, the video stream does not need to be stored in a reverse playing process, and an additional plug-in is not needed.

Further, the server side can simultaneously support reverse slicing of the local video stream and the video stream of the networking system to adapt to different scenes.

The application provides a method for playing back video records without plug-in based on HTML5 technology, without transcoding, integral unloading and real-time response, so as to solve the problem that the video monitoring video record playing back function can not be web-enabled, or the playing back function with low cost and real-time response can not be supported after the video monitoring video record playing back function is web-enabled.

The reverse order playback system 200 may be implemented by software, hardware, firmware, or any combination thereof. Fig. 10 is a schematic diagram illustrating the reverse playing system 200 provided in the present application, and the splitting, merging and adding of modules are within the protection scope of the present application without departing from the concept of the present application.

In an exemplary embodiment of the present disclosure, a computer-readable storage medium is further provided, on which a computer program is stored, which when executed by, for example, a processor, may implement the steps of the reverse-order playing method or the reverse-order slicing method described in any one of the above embodiments. In some possible embodiments, the various aspects of the present application may also be implemented in the form of a program product comprising program code for causing a terminal device to perform the steps according to various exemplary embodiments of the present application described in the reverse playback method or the reverse slicing method section above of this specification, if the program product is run on the terminal device.

Referring to fig. 11, a program product 800 for implementing the above method according to an embodiment of the present application is described, which may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present application is not limited thereto, and in this document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

The computer readable storage medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable storage medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Program code for carrying out operations of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the tenant computing device, partly on the tenant device, as a stand-alone software package, partly on the tenant computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing devices may be connected to the tenant computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).

In an exemplary embodiment of the present disclosure, there is also provided an electronic device that may include a processor, and a memory for storing executable instructions of the processor. Wherein the processor is configured to perform the steps of the reverse playing method or the reverse slicing method in any of the above embodiments via execution of the executable instructions.

As will be appreciated by one skilled in the art, aspects of the present application may be embodied as a system, method or program product. Accordingly, various aspects of the present application may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" system.

An electronic device 600 according to this embodiment of the present application is described below with reference to fig. 12. The electronic device 600 shown in fig. 12 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.

As shown in fig. 12, the electronic device 600 is embodied in the form of a general purpose computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 that connects the various system components (including the storage unit 620 and the processing unit 610), a display unit 640, and the like.

Wherein the storage unit stores program code executable by the processing unit 610 to cause the processing unit 610 to perform steps according to various exemplary embodiments of the present application described in the reverse-order playing method or the reverse-order slicing method section of the present specification described above. For example, the processing unit 610 may perform the steps as shown in fig. 1 to 3.

The storage unit 620 may include readable media in the form of volatile memory units, such as a random access memory unit (RAM) 6201 and/or a cache memory unit 6202, and may further include a read-only memory unit (ROM) 6203.

The memory unit 620 may also include a program/utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.

Bus 630 may be one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboard, pointing device, bluetooth device, etc.), with one or more devices that enable a tenant to interact with the electronic device 600, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device 600 to communicate with one or more other computing devices. Such communication may occur via an input/output (I/O) interface 650. Also, the electronic device 600 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the Internet) via the network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 via the bus 630. It should be appreciated that although not shown, other hardware and/or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.

Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which may be a personal computer, a server, or a network device, etc.) to execute the above-mentioned reverse playing method or reverse slicing method according to the embodiments of the present disclosure.

the webpage playing end sends a reverse playing request to the server end, so that the server end can obtain a slicing sequence by executing reverse slicing of a video stream to be played, and the server end sets a first time stamp of the slicing sequence to ensure that a frame sequence in the slicing sequence can be transmitted in sequence; after the webpage playing end acquires the reverse-order fragments, the timestamp offset attribute values can be set for the reverse-order fragments through the media source expansion module and the source cache of the media source expansion module, and second timestamps which are monotonically decreased in the playing sequence can be generated so as to be matched with the current time values of the video elements which are regularly decreased in the stepping value, the reverse-order playing of the video to be played is promoted, and the video file does not need to be stored and transcoded. In addition, the slicing and the playing are respectively decoupled to the server side and the webpage playing side, so that the slicing, the transmission and the playing can be in a pipeline form, and the time delay can be reduced in the process of playing reversely. Therefore, the low-cost and real-time response video stream reverse playing is realized, the video stream does not need to be stored in a reverse playing process, and an additional plug-in is not needed.

And the server side supports the reverse-order slicing of the local video file of the server side. The server side encapsulates the reverse-order fragments, so that the reverse-order fragments can accord with the processing format of a media source expansion module of the webpage playing side, and the local video file does not need to be subjected to unloading and transcoding; meanwhile, the server side sets the first time stamp of the slicing sequence to ensure that the frame sequences in the slicing sequence can be transmitted in sequence. In addition, the slicing and the playing are respectively decoupled to the server side and the webpage playing side, so that the slicing, the transmission and the playing can be in a pipeline form, and the time delay can be reduced in the process of playing reversely.

And the server side supports the reverse-order slicing of the video files of the networking system. The server side encapsulates the reverse-order fragments, so that the reverse-order fragments can accord with the processing format of a media source expansion module of the webpage playing side, and video files transmitted by the networking system do not need to be subjected to dump and transcoding; meanwhile, the server side sets the first time stamp of the slicing sequence to ensure that the frame sequences in the slicing sequence can be transmitted in sequence. In addition, the video stream transmission, the slicing and the playing are decoupled to the networking system, the server side and the webpage playing side respectively, so that the video stream transmission, the slicing, the transmission and the playing can be in a pipeline form, and the time delay can be reduced in the process of playing reversely.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice in the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

Claims

1. A reverse order playing method is applied to a webpage playing end and comprises the following steps:

and the webpage playing end plays the video stream to be played in a reverse order by adopting video elements, wherein the webpage playing end decreases the current time value of the video elements according to a step value, so that the current time value advances the reverse order playing of the video stream to be played in the decreasing direction of the second timestamp.

2. The reverse-order playing method of claim 1, wherein the web playing side decrements the current time value of the video element by a step value, so that the current time value advances the reverse-order playing of the video stream to be played in the decrementing direction of the second timestamp, comprising:

3. The reverse-order playing method according to claim 1, wherein an initial value of the current time value is set to a maximum timestamp of the reverse-order segment in the source buffer, the step value is less than or equal to a frame interval of the reverse-order segment, and the web playing side decrements the current time value of the video element by a step value, so that the current time value advances reverse-order playing of the video stream to be played in a decrementing direction of the second timestamp includes:

4. A reverse slicing method is applied to a server side and comprises the following steps:

the server receives a reverse play request sent by a webpage playing end, wherein the reverse play request comprises an identifier of a video stream to be played and initial playing time, and the video stream to be played corresponding to the identifier of the video stream to be played is a local video file of the server;

the server determines the position of the current frame of the video stream to be played based on the starting playing time;

the server side searches the key frames with the nearest distance from the current frame position in the video stream to be played in a reverse order;

the server takes the key frame position of the current original fragment as the current frame position to position the next original fragment until the reverse fragment is finished;

5. The reverse slicing method as claimed in claim 4, wherein the step of using the key frame position of the current original slice as the current frame position to locate the next original slice until the reverse slice ends comprises:

in response to that the number of original fragments packaged as the current reverse-order fragment does not reach a set number threshold, the server takes the key frame position of the current original fragment as the current frame position to position the next original fragment until the reverse-order fragment is finished;

in response to the number of original slices for packaging as a current reverse order slice reaching a set number threshold:

6. A reverse slicing method is applied to a server side and comprises the following steps:

the server takes the starting playing time as the request time for generating the forward playing request;

the server side enables the reverse order fragments to be arranged reversely to the collecting time to generate a fragment sequence, the frame sequence in each reverse order fragment is arranged in the forward direction to the collecting time, the first time stamps of the frame sequence of the fragment sequence are monotonically increased according to the arrangement sequence, and the fragment sequence is used for being sent to the webpage playing side to achieve reverse order playing of the video stream to be played.

7. The reverse-order slicing method according to claim 6, wherein when the server generates a first forward-order playing request, the server receives a video stream to be played, which is sent by the networking system in forward order from the key frame closest to the request time, takes the received first frame as a start key frame of the first reverse-order slice, and encapsulates the received video frame into the first reverse-order slice according to the set time/the set key frame number/the end frame of the video stream to be played.

8. The reverse-order slicing method according to claim 6, wherein the server receives a video stream to be played, which is sent by the networking system in a forward order from the key frame closest to the request time, takes the received first frame as a starting key frame of a current reverse-order slice, and encapsulates the starting key frame of the current reverse-order slice to a frame before the starting key frame of a previous reverse-order slice into the current reverse-order slice comprises:

the server generates a first time stamp for a frame sequence in a packaged current reverse sequence fragment, and the first time stamp of the frame sequence of the current reverse sequence fragment is generated according to the first time stamp of the frame sequence of a previous reverse sequence fragment, so that the first time stamps of the frame sequences in the slice sequence are monotonically increased in the arrangement sequence.

9. The reverse slicing method as claimed in claim 6, wherein said encapsulating the starting key frame of the current reverse slice to the previous frame of the starting key frame of the previous reverse slice into the current reverse slice comprises:

responding to that the current video frame received by the server is a key frame, matching the feature data of the key frame with the feature data of the initial key frame of the previous reverse-order fragment, and if the feature data of the key frame is not matched with the feature data of the initial key frame of the previous reverse-order fragment, writing the key frame into the current reverse-order fragment and continuously receiving the video frame; if so, identifying the key frame as the initial key frame of the previous reverse fragment, and encapsulating the initial key frame of the current reverse fragment to the previous frame of the key frame into the current reverse fragment;

and when the server side packages the current reverse-order fragment, taking the data of the set position of the coding load of the initial key frame of the current reverse-order fragment as the characteristic data of the initial key frame of the current reverse-order fragment.

10. The reverse-order slicing method as claimed in claim 6, wherein said forward-order play request is a multiple-speed forward-order play request.

11. A reverse order playback system, comprising:

a webpage playing end for executing the reverse playing method according to any one of claims 1 to 3;

the server side, which executes the reverse slicing method according to any one of claims 4 to 11.

12. The reverse order playback system of claim 11, further comprising: