WO2024002264A1

WO2024002264A1 - Video processing device, progress bar time updating method, apparatus, and electronic device

Info

Publication number: WO2024002264A1
Application number: PCT/CN2023/104007
Authority: WO
Inventors: 张俊杰; 张镇
Original assignee: 杭州海康威视数字技术股份有限公司
Priority date: 2022-06-29
Filing date: 2023-06-29
Publication date: 2024-01-04
Also published as: CN117376648A

Abstract

Disclosed in the present application are a video processing device, a progress bar time updating method, an apparatus, and a storage medium. The video processing device is used for: in response to receiving a playback instruction, acquiring a program stream assigned by the playback instruction, the program stream being obtained by performing program stream packaging on an encoded video stream of shot original video data; decoding the program stream to obtain a decoded video stream, the decoded video stream comprising one or more video sequences, and each video sequence comprising a key frame and at least one target frame associated with the key frame; for any target video sequence among the one or more video sequences, determining the absolute time of the target frame on the basis of the absolute time of the key frame and the time interval between the key frame and the target frame, the absolute time being the actual recording time of the video picture corresponding to the video frame; and by means of the absolute time of the key frame and the absolute time of the target frame, determining the current playing time displayed on the playback progress bar when the decoded video stream is played back.

Description

Video processing equipment, progress bar time update method and device, electronic equipment

Technical field

The present application relates to the field of video surveillance technology, and in particular to a video processing device, a progress bar time update method and device, electronic equipment and a storage medium.

Background technique

In the field of video surveillance, existing video processing equipment (for example, digital hard disk video recorders, network hard disk video recorders, etc.) not only supports access preview and video storage functions for connected cameras, but also has the ability to play back locally stored videos. function. The video playback interface includes a progress bar, which is used to display the length information of the video file played back and the time information of the current video frame displayed on the playback interface during the playback process.

Video files are generally in PS encapsulation format. Under PS encapsulation format, only I frames can parse out the absolute time, that is, the actual recording time. During the playback of video files, the actual recording time of the I frame is usually used to update the time on the progress bar. However, for most video processing equipment, the default frame rate is 25 frames per second, and in the video stream In the data, there are 50 P frames between every two I frames. That is to say, when the video processing device plays back the video stream data, an I frame will appear every 2 seconds, so the time on the progress bar will appear every 2 seconds. It only updates once every 2 seconds, and the update frequency is slow, causing the currently played back video frame to not correspond to the time on the progress bar.

Contents of the invention

This application provides a video processing device, a progress bar time update method, a device and a storage medium, which can increase the time update frequency on the progress bar.

In a first aspect, the application provides a video processing device. The video processing device is configured to, in response to receiving a playback instruction, obtain a program flow specified by the playback instruction, wherein the program flow is obtained by processing the original shot. The encoded video stream of the video data is obtained by encapsulating the program stream; the program stream is decoded to obtain a decoded video stream, wherein the decoded video stream includes one or more video sequences, each video sequence includes a key frame and At least one target frame associated with the key frame; for any target video sequence of the one or more video sequences, based on the absolute time of the key frame and the time between the key frame and the target frame interval to determine the absolute time of the target frame, where the absolute time is the actual recording time of the video frame corresponding to the video frame; using the absolute time of the key frame and the target frame The absolute time is used to determine the current playback time displayed on the playback progress bar when the decoded video stream is played back.

In a possible implementation, the video processing device is specifically configured to: obtain the timestamp of the key frame and the timestamp of the target frame, where the timestamp is used to represent the relative time of the video frame to the reference time. time offset; determine the time interval between the key frame and the target frame according to the time stamp of the key frame and the time stamp of the target frame.

In another possible implementation, the video processing device is specifically configured to: when the timestamp of the target frame is greater than the timestamp of the key frame, subtract the timestamp of the target frame from the The difference in timestamps of key frames is obtained to determine the time interval.

In another possible implementation, the video processing device is specifically configured to: obtain the frame number of the key frame, the frame number of the target frame, and the frame interval of the target video sequence, wherein the frame interval Used to represent the time interval between two adjacent video frames in the target video sequence; calculate the frame number difference obtained by subtracting the frame number of the key frame from the frame number of the target frame; according to the frame number difference The product with the frame interval of the target video sequence determines the time interval between the key frame and the target frame.

In another possible implementation, the key frame is an I frame; and/or the target frame is an auxiliary frame.

In another possible implementation, the video processing device is specifically configured to determine the current playback time displayed on the playback progress bar based on the minimum time unit of the playback progress bar.

In another possible implementation, the playback instruction comes from a user terminal that can communicate with the video processing device; the video processing device is further configured to: send the decoded video stream to the sending user terminal to The decoded video stream is played back on the display interface of the user terminal; wherein the current playback time displayed on the playback progress bar on the display interface corresponds to the video frame currently being played back.

In another possible implementation, the functions of the video processing device and the functions of the user terminal are integrated into a single device.

In a second aspect, the application provides a progress bar time update method, including: obtaining a target video sequence, wherein the target video sequence includes a key frame and at least one target frame associated with the key frame; an absolute value based on the key frame time and the time interval between the key frame and the target frame to determine the absolute time of the target frame, where the absolute time is the actual recording time of the video picture corresponding to the video frame; using the The absolute time of the key frame and the absolute time of the target frame are used to determine the current playback time displayed on the playback progress bar when the target video sequence is played back.

In another possible implementation, the method further includes: obtaining the timestamp of the key frame and the timestamp of the target frame, where the timestamp is used to represent the time offset of the video frame relative to the reference time. Shift; determine the time interval between the key frame and the target frame according to the timestamp of the key frame and the timestamp of the target frame.

In another possible implementation, determining the time interval between the key frame and the target frame includes: when the timestamp of the target frame is greater than the timestamp of the key frame, based on The time interval is determined by subtracting the time stamp of the key frame from the time stamp of the target frame.

In another possible implementation, the method further includes: obtaining the frame number of the key frame, the frame number of the target frame, and the frame interval of the target video sequence, where the frame interval is used to represent The time interval between two adjacent video frames in the target video sequence; the frame number difference is obtained by subtracting the frame number of the key frame from the frame number of the target frame; according to the frame number difference and the The product of the frame intervals of the target video sequence determines the time interval between the key frame and the target frame.

In another possible implementation, determining the current playback time displayed on the playback progress bar includes: determining the current playback time displayed on the playback progress bar based on the minimum time unit of the playback progress bar.

In a third aspect, the application provides a progress bar time updating device, including: an acquisition module, configured to acquire a target video sequence, wherein the target video sequence includes a key frame and at least one target frame associated with the key frame; first Determining module, configured to determine the absolute time of the target frame based on the absolute time of the key frame and the time interval between the key frame and the target frame, wherein the absolute time is a video frame the actual recording time of the corresponding video picture; a second determination module, configured to use the absolute time of the key frame and the absolute time of the target frame to determine the playback progress bar when the target video sequence is played back The current playback time displayed on.

In another possible implementation, the acquisition module is also used to acquire the timestamp of the key frame and the timestamp of the target frame, where the timestamp is used to represent the time stamp of the video frame relative to the reference time. Time offset; the first determination module is further configured to determine the time interval between the key frame and the target frame based on the timestamp of the key frame and the timestamp of the target frame.

In another possible implementation, the first determination module is specifically configured to subtract the timestamp of the target frame based on the timestamp of the key frame when the timestamp of the target frame is greater than the timestamp of the key frame. The timestamp of the above key frame is obtained The difference is obtained to determine the time interval.

In another possible implementation, the acquisition module is also used to acquire the frame number of the key frame, the frame number of the target frame, and the frame interval of the target video sequence, where the frame interval is represents the time interval between two adjacent video frames in the target video sequence; the first determination module is specifically used to obtain a frame number by subtracting the frame number of the key frame from the frame number of the target frame The time interval between the key frame and the target frame is determined based on the product of the frame number difference and the frame interval of the target video sequence.

In another possible implementation, the second determination module is specifically configured to determine the current playback time displayed on the playback progress bar according to the minimum time unit of the playback progress bar.

In a fourth aspect, the present application provides an electronic device, including: at least one processor; at least one memory storing machine-executable instructions; when the at least one processor executes the machine-executable instructions, the second aspect is implemented Any progress bar time update method provided by the aspect.

In a fifth aspect, this application provides a computer-readable storage medium. The computer-readable storage medium stores computer execution instructions. When the computer execution instructions are run on a computer, the computer is caused to execute any of the progress provided in the second aspect. Article time update method.

For detailed descriptions of the third to fifth aspects and their various implementations in this application, reference may be made to the detailed descriptions of the first aspect, the second aspect and their various implementations. For the beneficial effects of the third to fifth aspects and their various implementation methods, please refer to the analysis of the beneficial effects of the first aspect, the second aspect and their various implementation methods, and will not be repeated here.

These and other aspects of the present application will be more clearly understood in the following description.

Description of drawings

Figure 1 is a schematic diagram of the implementation environment involved in a progress bar time update method provided by an embodiment of the present application;

Figure 2 is a schematic diagram of the implementation environment involved in a progress bar time update method provided by an embodiment of the present application;

Figure 3 is a schematic diagram of a video sequence provided by an embodiment of the present application;

Figure 4 is a schematic diagram of a PS package provided by an embodiment of the present application;

Figure 5 is a schematic diagram of a playback interface provided by an embodiment of the present application;

Figure 6 is a schematic diagram of the implementation environment involved in a progress bar time update method provided by an embodiment of the present application;

Figure 7 is a flow chart of a progress bar time update method provided by an embodiment of the present application;

Figure 8 is a flow chart of a progress bar time update method provided by an embodiment of the present application;

Figure 9 is a schematic diagram of a video sequence provided by an embodiment of the present application;

Figure 10 is a schematic diagram of a video sequence provided by an embodiment of the present application;

Figure 11 is a schematic diagram of a video sequence provided by an embodiment of the present application;

Figure 12 is a schematic diagram of a playback progress bar provided by an embodiment of the present application;

Figure 13 is a schematic structural diagram of a progress bar time updating device provided by an embodiment of the present application;

Figure 14 is a schematic structural diagram of a progress bar time updating device provided by an embodiment of the present application.

Detailed ways

The term "and/or" in this article is just an association relationship that describes related objects, indicating that three relationships can exist. For example, A and/or B can mean: A exists alone, A and B exist simultaneously, and they exist alone. B these three situations.

The terms “first” and “second” in the description of this application and the drawings are used to distinguish different objects, or to distinguish different processes on the same object, rather than to describe a specific order of objects.

Furthermore, references to the terms "including" and "having" and any variations thereof in the description of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes other unlisted steps or units, or optionally also includes Includes other steps or units that are inherent to such processes, methods, products, or devices.

It should be noted that in the embodiments of this application, words such as "exemplary" or "for example" are used to represent examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "such as" in the embodiments of the present application is not to be construed as preferred or advantageous over other embodiments or designs. Rather, use of the words "exemplary" or "such as" is intended to present the concept in a concrete manner.

In the description of this application, unless otherwise stated, the meaning of "plurality" means two or more.

In order to facilitate understanding of the technical solution of this application, the terminology involved in this application is briefly introduced below.

1. Digital video recorder (DVR). DVR is a type of video processing equipment used in conjunction with analog cameras. The main working mode of DVR is to access analog video and audio signals and record video and audio through the hard disk. The core of DVR lies in hard disk recording, so DVR is also called hard disk video recorder.

DVR can integrate cameras, mice, remote controls, remote terminal equipment, etc. to form a complete monitoring system, realizing the functions of long-term recording, remote monitoring, remote control, playback, and backup of audio and video signal data.

2. Network video recorder (NVR). NVR is a type of video processing equipment that is used in conjunction with a network camera or video encoder to record digital video transmitted through the network.

The main function of NVR is to receive the digital video stream transmitted by IPC (Internet Camera) equipment through the network, and store and manage it, thereby realizing the advantages of distributed architecture brought by networking. Through NVR, you can watch, browse, playback, manage, and store multiple network cameras at the same time.

3. Network camera (IP CAMERA, IPC). Network camera is a new generation of camera that combines traditional cameras and network technology. It can transmit video images to the other side of the earth through the network, and the remote viewer does not need to Its video images can be monitored using any professional software, as long as a standard web browser (such as Microsoft IE or Netscape). IPC generally consists of lens, image sensor, sound sensor, signal processor, A/D converter, encoding chip, main control chip, network and control interface, etc.

4. Program stream or program stream (PS) encapsulation. MPEG2-PS is an encapsulation container for multiplexing digital audio, video, etc. The PS stream is obtained by PS encapsulating the basic code stream output from the encoder; the PS stream is composed of PS packets.

5. Packetized elementary stream (PES), the basic code stream ES is divided into data packets of different lengths as needed, and each data packet is added with a header to form a grouped basic code stream. These data packets are called PES packets, which are a data structure used to transmit basic code streams.

6. Program stream map (PSM). PSM provides a description of the original streams in the program stream and the relationships between them. PSM is a PES packet.

7. Group of pictures (GOP). GOP is a group of continuous pictures, and the number of frames it contains can be is defined. In some examples, a GOP consists of a key frame and several target frames. As an auxiliary frame, the target frame needs to rely on the information of the key frame to be predicted. The type of the target frame is not limited and can be B frame, P frame, etc.

In some examples, the key frame may be an I frame (i.e., intra-coded frame) and the target frame may be a P frame (i.e., forward predicted frame). For example, an I frame is a complete picture, while a P frame records changes relative to the I frame. Without I frames, P frames cannot be decoded.

In the H.264 or H.265 compression standard, I frames and P frames are used to represent transmitted video images. The encoder encodes multiple images and generates corresponding code stream data. At the decoding end, the code stream data can be divided into GOPs based on key frames (I frames). For example, an I frame and P frames between the I frame and the next I frame are regarded as one GOP.

In some examples, the size of the GOP can be set according to the keyframe interval, where when the keyframe interval is set to be shorter, the size of the GOP is smaller, and when the keyframe interval is set to be longer, the size of the GOP is bigger. For example, if the key frame is an I frame and the interval between key frames is 50 frames, then a GOP includes 1 I frame and 50 P frames. When another I frame is inserted into the GOP, the new GOP generated may include one I frame and 30 P frames.

8. Video frame. A video consists of more than one independent picture and presents a continuous dynamic visual effect. Each picture can also be called a "video frame". In order to ensure continuity and smoothness, the number of frames per second of the video is fixed, called the frame rate, for example, 25 frames/s, 30 frames/s, 50 frames/s, etc.

9. Timestamp is a time value corresponding to each video frame in the code stream encapsulation. The time value can be represented by a relative value or other methods. In some examples, a relative time value may represent a time offset relative to a base time (eg, a power-on time point (starting from 0)). The main purpose of using timestamps is to determine the time change between video frames (that is, the frame interval). Therefore, the representation method of the timestamp and the reference time of the timestamp are not limited in this disclosure. Under normal circumstances, the changing trend of timestamps is uniformly increasing. The difference between the timestamps of two adjacent video frames can represent the playback duration of the previous video frame.

Furthermore, the timestamp has a maximum value, and when the timestamp reaches the maximum value, it will be accumulated evenly starting from the base time (e.g., 0) again.

10. Absolute time, that is, system time, for example, X year X month X day X hour X minute X second, is used to represent the time when each video frame in the video is generated. In this embodiment of the present application, the absolute time of a video frame is used to represent the actual recording time of the video picture corresponding to the video frame.

The above is an introduction to some concepts involved in the embodiments of this application, and they will not be described again below.

In the field of video surveillance, in order to facilitate the transmission and storage of video data, the video data is usually encoded at the sending end of the video (for example, camera equipment), and the encoded video data is sent to the receiving end of the video (for example, video processing equipment). equipment). After receiving the encoded video data, the video receiving end performs PS encapsulation on the encoded video stream to obtain the PS stream, and stores the PS stream in the hard disk. During the PS encapsulation process, the time information of I frames and P frames will also be encapsulated in the corresponding PS packets. The time information of the I frame includes the absolute time of the I frame and the timestamp of the I frame; the time information of the P frame includes the timestamp of the P frame.

During the playback process, it is necessary to obtain the PS stream corresponding to the recorded video to be played back from the hard disk, decode it, obtain the decoded video stream, and play back the decoded video stream. In the prior art, the absolute time corresponding to the currently played back video frame is usually used to update the current playback time on the playback progress bar. However, since only the I frame has an absolute time and the P frame does not have an absolute time, the I frame is usually used. The absolute time to update the time on the playback progress bar.

In this way, an I frame will only appear every fixed time (for example, 2 seconds), and the time on the playback progress bar will only be updated every corresponding fixed time (for example, 2 seconds), and the update frequency is slow. , causing the currently played back video frame to not correspond to the time on the progress bar.

To this end, this application provides a progress bar time update method. The idea is: during the playback process, first, determine the absolute time of the key frame in the target video sequence, and the time interval between the key frame and the target frame, Furthermore, the absolute time of the target frame is determined based on the absolute time of the key frame and the time interval between the key frame and the target frame; finally, the current playback time of the playback progress bar is updated based on the absolute time of the key frame and the absolute time of the target frame. . In this way, compared with the prior art method of updating the current playback time of the playback progress bar based only on the absolute time of the key frame, the embodiment of the present application simultaneously updates the playback progress bar based on the absolute time of the key frame and the absolute time of the target frame. The current playback time effectively increases the update frequency of the playback progress bar, so that the currently played back video frame corresponds to the current playback time on the progress bar.

The implementation of the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

Please refer to FIG. 1 , which shows a schematic diagram of the implementation environment involved in a progress bar time update method provided by an embodiment of the present application. As shown in FIG. 1 , the implementation environment may include: a camera device 10 , a video processing device 20 and a terminal device 30 .

The camera device 10 is used to capture video.

For example, the camera device 10 may be an analog camera or an IPC.

In some embodiments, as shown in FIG. 2 , the camera device 10 includes a video encoder 11 for receiving the original video data captured by the camera device 10 and using a specific compression standard (for example, H.264 compression standard) to encode the original video data. Encode the video data to obtain the encoded video stream.

The encoded video stream may include one or more video sequences (ie, GOP). Each video sequence includes a key frame and at least one target frame. As shown in Figure 3, the I frame is the first video frame of the video sequence, including the complete video picture, and can be encoded independently; at least one target frame is located after the key frame, which is a forward prediction encoding frame, and the target frame only includes Relative to changes in the previous video frame (ie, the previous key frame or target frame closest to the target frame), encoding needs to be completed relying on the previous video frame.

In some embodiments, the camera device 10 is also used to send the encoded video stream to the video processing device 20 .

The video processing device 20 is configured to receive and store the encoded video stream. In some embodiments, the video processing device 20 is also used to decode the encoded video stream to obtain one or more video sequences and play them back.

For example, the video processing device 20 may be a digital video recorder (DVR) device or a network hard disk recorder (NVR) device.

In some embodiments, after receiving the encoded video stream, the video processing device 20 performs PS encapsulation on the encoded video stream to obtain the PS stream, and stores the PS stream in a storage area (for example, a local hard disk or a other devices with which video processing device 20 can communicate).

During the PS encapsulation process, the video processing device 20 determines the time information corresponding to each video frame (including key frames and target frames) in the encoded video stream, and encapsulates the time information corresponding to each video frame in the corresponding PS. in the bag. For example, the time information corresponding to the I frame includes the absolute time of the I frame and the timestamp of the I frame; the time information corresponding to the P frame includes the timestamp of the P frame.

Among them, the PS stream consists of multiple PS GOPs; a PS GOP consists of one or more PS packets, and each PS packet corresponds to the information of one or more video frames in the corresponding GOP. Exemplarily, the structure of the PS packet is shown in Figure 4, where the structure of the first PS packet in the PS GOP (that is, the PS packet corresponding to the I frame) can be as shown in (a) of Figure 4, The structure of other PS packets in the PS GOP (that is, the PS packet corresponding to the P frame) can be shown in (b) of Figure 4.

Wherein, PSH is the header of the PS packet, which mainly includes the timestamp of the video frame corresponding to the PS packet. PES is the basic stream of the encoded video stream, that is, the encoded video stream is grouped to obtain one or more PES. PSM Is a special PES, PSM is used to place the absolute time corresponding to the video frame (ie, I frame). PSM only appears in I frames, that is, only I frames have absolute time, and P frames do not have absolute time.

In some embodiments, after receiving the playback instruction, the video processing device 20 is also configured to obtain the PS stream corresponding to the playback instruction from the storage area, and decode the PS stream to obtain a decoded video stream. For example, the above playback instruction may be issued by the terminal device 30 .

In some embodiments, as shown in Figure 2, the video processing device 20 includes a video decoder 21 for decoding the PS stream using a specific compression standard (for example, H.264 compression standard) to obtain a decoded video stream. .

In some embodiments, the decoded video stream includes one or more video sequences, each video sequence consisting of a key frame and at least one target frame.

In some embodiments, during the decoding process of the video decoder, the time information of the key frame and the time information of the target frame in the video sequence can be parsed, as well as the frame number of the key frame and the frame number of the target frame. Here, the frame number refers to the sequence number of each video frame. For example, the frame number of the key frame refers to the sequence number of the key frame, for example, F0 or 1; the frame number of the target frame refers to the sequence number of the target frame, for example, F1 or 2.

In some embodiments, the video processing device 20 is also used to send the decoded video stream to the terminal device 30 .

The terminal device 30 is used to play the decoded video stream.

In some embodiments, as shown in FIG. 5 , the terminal device 30 includes a display screen, the playback interface of the display screen includes a first area and a second area, and the first area is smaller than the second area; wherein, the first area is used for The video pictures (video frames) of the decoded video stream are displayed, and the second area includes a playback progress bar. The playback progress bar is used to reflect the time information corresponding to the video pictures (video frames) displayed in the first area.

In some embodiments, the terminal device 30 is also configured to obtain the time information of each video frame in the video sequence, and update the current playback time of the playback progress bar according to the time information of each video frame.

For example, the terminal device 30 may be an electronic device with a playback function, such as a mobile phone, a tablet computer, a desktop, a laptop, a handheld computer, a notebook computer, or an ultra-mobile personal computer (UMPC). , netbooks, as well as cellular phones, personal digital assistants (PDAs), augmented reality (AR)/virtual reality (VR) devices, etc. This disclosure does not place any special restrictions on the specific form of the terminal device 30 .

Optionally, the video processing device 20 and the terminal device 30 in FIG. 1 may be two separate devices, or the video processing device 20 and the terminal device 30 may be the same device, that is, the video processing device 20 or the corresponding function and the terminal. Device 30 or corresponding functions can be integrated on the same device.

The camera device 10 and the video processing device 20 can communicate, and the video processing device 20 and the terminal device 30 can communicate. For example, the video processing device 20 can receive an encoded video stream from the camera device 10, and the terminal device 30 can The decoded video stream is received from video processing device 20. In one example, one or more communication media may be included between the camera device 10 and the video processing device 20 and between the video processing device 20 and the terminal device 30 , and the one or more communication media may include a router, a switch or a base station. and other equipment.

Specifically, as shown in FIG. 6 , the video processing device 20 receives the encoded video stream sent by the camera device 10 , performs PS encapsulation on the encoded video stream to obtain the PS stream, and stores the PS stream in the storage area. When the user needs to view the video recording of a certain period of time, the user can send a playback instruction to the video processing device 20 through the terminal device 30; when the video processing device 20 receives the playback instruction, the PS corresponding to the video recording to be played back is obtained from the storage area. stream, decode the PS stream to obtain the decoded video stream; the video processing device 20 sends the decoded video stream to the terminal device 30; the terminal device 30 displays the decoded video stream, and based on the decoded video The time corresponding to the stream updates the current playback time on the playback progress bar.

The following is a detailed introduction to a progress bar time updating method provided by the embodiment of the present application.

The progress bar time updating method provided by the embodiment of the present application can be executed by the video processing device shown in Figure 1. As shown in Figure 7, the method includes the following steps:

S101. Obtain the target video sequence.

Wherein, the target video sequence includes key frames and at least one target frame associated with the key frames. A keyframe is the first frame in the target video sequence, and at least one target frame follows the keyframe. Both key frames and target frames are used to represent video frames.

In some embodiments, the target video sequence is any video sequence in the recorded videos to be played back. For example, if the recorded video to be played back includes three video sequences, namely video sequence A, video sequence B, and video sequence C, the video processing device obtains video sequence A, video sequence A, and video sequence C by decoding the PS stream corresponding to the recorded video. Sequence B and video sequence C.

It can be understood that, in order to facilitate storage, after receiving the encoded video stream output by the video encoder, the video processing device performs PS encapsulation on the encoded video stream to generate a PS stream and store it in the storage area. Furthermore, during the playback process, it is necessary to obtain the PS stream of the recorded video to be played back from the hard disk, and decode the PS stream into the form of one or more video sequences.

S102. Based on the absolute time of the key frame and the time interval between the key frame and the target frame, determine the absolute time of the target frame.

The absolute time of the target frame is the actual shooting time of the video picture corresponding to the target frame.

In some embodiments, the absolute time of the target frame is determined based on the sum of the absolute time of the key frame and the time interval between the key frame and the target frame. For example, if the absolute time of the key frame is G0 and the time interval between the key frame and the target frame is T1-T0, then the absolute time of the target frame satisfies: G0+(T1-T0). As another example, if the absolute time of the key frame is G0 and the time interval between the key frame and the target frame is (F1-F0)*D, then the absolute time of the target frame satisfies: G0+(F1-F0)*D .

For example, if the absolute time of the key frame is 12 hours, 0 minutes and 0 seconds, and the time interval between the key frame and the target frame is 1 second, then the absolute time of the target frame is: 12 hours, 0 minutes, 0 seconds + 1 second = 12 hours 0 minutes and 1 second.

S103. Based on the absolute time of the key frame and the absolute time of the target frame, determine the current playback time displayed on the playback progress bar when the target video sequence is played back.

In some embodiments, the key frame may be an I frame; and/or the target frame may be an auxiliary frame.

In some embodiments, as shown in Figure 12, the playback progress bar is used to display the length information of the video recording file where the target video sequence is located, and the time information corresponding to the currently played back video frame.

In some embodiments, the video processing device sends the video image corresponding to the key frame to the terminal device, so that the video image corresponding to the key frame is played on the display screen of the terminal device. At the same time, the video processing device uses the absolute time of the key frame to update the playback progress. The current playback time of the bar. The video processing device sends the video image corresponding to the target frame to the terminal device, so that the video image corresponding to the target frame is played on the display screen of the terminal device. At the same time, the video processing device uses the absolute time of the target frame to update the current playback time of the playback progress bar.

In some embodiments, determining the current playback time displayed on the playback progress bar includes: determining the current playback time displayed on the playback progress bar based on the minimum time unit of the playback progress bar.

In some embodiments, when the smallest time unit on the playback progress bar is seconds, the current playback time of the playback progress bar is updated to seconds based on the absolute time of the key frame and the absolute time of the target frame. For example, if the absolute time of the target frame is 12:01 minutes and 200 milliseconds, the current playback time of the updated playback progress bar is 12:01 minutes.

In some embodiments, when the minimum time unit on the playback progress bar is milliseconds, the current playback time of the playback progress bar is updated to milliseconds based on the absolute time of the key frame and the absolute time of the target frame. In this way, the frequency of time update of the progress bar can be increased and the time on the progress bar can be made more accurate. For example, if the absolute time of the target frame is 12:01 minutes and 200 milliseconds, the current playback time of the updated playback progress bar is 12:01 minutes and 200 milliseconds.

It can be understood that the technical solution provided by the embodiment of the present application first determines the absolute time of the key frame in the target video sequence and the time interval between the key frame and the target frame, and then, based on the absolute time of the key frame and the key frame The time interval between the key frame and the target frame determines the absolute time of the target frame; finally, the current playback time of the playback progress bar is updated based on the absolute time of the key frame and the absolute time of the target frame. In this way, compared with the prior art method of updating the current playback time of the playback progress bar based only on the absolute time of the key frame, the embodiment of the present application simultaneously updates the playback progress bar based on the absolute time of the key frame and the absolute time of the target frame. The current playback time effectively increases the update frequency of the playback progress bar, so that the currently played back video frame corresponds to the current playback time on the progress bar.

The progress bar time updating method provided by the embodiment of the present application can be executed by the video processing device shown in Figure 1. As shown in Figure 8, the method includes the following steps:

S201. Obtain the target video sequence.

S202. Determine the absolute time of the key frame and the time interval between the key frame and the target frame.

Among them, the absolute time of the key frame is the actual recording time of the video picture corresponding to the key frame. For example, if the video picture corresponding to the key frame was shot at 12:00:0 on May 17, 2022, then the absolute time of the key frame is 12:0:0 on May 17, 2022.

In some embodiments, the target frame is any one of the at least one target frame.

In some embodiments, the time interval between the key frame and the target frame refers to the time difference between the key frame and the target frame. The embodiment of the present application provides the following two methods for determining the time interval between the key frame and the target frame.

Method 1: Determine the time interval between the key frame and the target frame based on the timestamp

Step 1. Get the timestamp of the key frame and the timestamp of the target frame.

Among them, the timestamp of the key frame is used to represent the time offset of the key frame compared to the base time; the timestamp of the target frame is used to represent the time offset of the target frame compared to the base time. Usually, the reference time is fixed (the reference time can be the power-on time point (for example, 0), or it can be an arbitrarily set time point (for example, 5ms, 10ms, 20ms, etc.). The embodiment of the present application does not change the reference time. The value of time is not specifically limited), therefore, the time offset relative to the reference time can uniquely characterize the actual recording time of the video frame, that is, the timestamp of each video frame can uniquely characterize the actual recording time of the video frame. Recording time. For example, the timestamp of the key frame is 40ms, the first The timestamp of the target frame is 80ms, and the timestamp of the second target frame is 120ms, which are accumulated evenly until the timestamp reaches the maximum value.

In some embodiments, the video processing device obtains the timestamp of the key frame by parsing the PS packet corresponding to the key frame; the video processing device obtains the timestamp of the target frame by parsing the PS packet corresponding to the target frame.

Step 2: Determine the time interval between the key frame and the target frame based on the timestamp of the key frame and the timestamp of the target frame.

In some embodiments, the time interval between the key frame and the target frame is determined based on the difference between the timestamp of the target frame and the timestamp of the key frame. For example, if the timestamp of the target frame is T1 and the timestamp of the key frame is T0, then the time interval between the key frame and the target frame satisfies: T1-T0.

For example, as shown in Figure 9, if the timestamp of the key frame is 40ms, the timestamp of the target frame is 120ms. Then the time interval between the key frame and the target frame is: 120ms-40ms=80ms.

In some embodiments, the above step 2 can be implemented as: when the timestamp of the target frame is greater than the timestamp of the key frame, determine the distance between the key frame and the target frame based on the timestamp of the key frame and the timestamp of the target frame. time interval.

It can be understood that in a video sequence, the key frame comes first and the target frame follows, and the timestamps increase from front to back, so the timestamp of the target frame should be greater than the timestamp of the key frame. However, the timestamp has a maximum value. When the timestamp reaches the maximum value, it will continue to accumulate from 0. At this time, there may be a situation where the timestamp of the target frame is smaller than the timestamp of the key frame (that is, when the time stamp occurs Stamp overflow), in which case errors may occur when determining the time interval between the keyframe and the target frame based on the timestamp of the keyframe and the timestamp of the target frame. For example, as shown in Figure 10, if the maximum value of the timestamp is 200ms, the timestamp of the key frame is 160ms, and the timestamp of the target frame is 40ms, at this time, the timestamp of the target frame is smaller than the timestamp of the key frame, If based on the difference between the timestamp of the target frame and the timestamp of the key frame, the time interval between the key frame and the target frame is: 40ms-160ms=-120ms. Obviously this result is incorrect. Therefore, In the case of timestamp overflow, the time interval between the key frame and the target frame cannot be determined based on the timestamp.

Therefore, the method provided in the first mode is preferably applicable to the situation where the timestamp of the target frame is greater than the timestamp of the key frame.

Method 2: Determine the time interval between the key frame and the target frame based on the frame number

Step 1. Obtain the frame number of the key frame, the frame number of the target frame and the frame interval of the target video sequence.

The frame interval of the target video sequence is used to represent the time interval between two adjacent video frames in the target video sequence.

In some embodiments, the video processing device obtains the frame number of the key frame by parsing the PS packet corresponding to the key frame; the video processing device obtains the frame number of the target frame by parsing the PS packet corresponding to the target frame.

Step 2: Determine the time interval between the key frame and the target frame based on the frame number of the key frame, the frame number of the target frame, and the frame interval of the target video sequence.

In some embodiments, the time interval between the key frame and the target frame is determined based on the product of the frame number difference between the frame number of the target frame and the frame number of the key frame and the frame interval of the target video sequence. For example, if the frame number of the target frame is F1, the frame number of the key frame is F0, and the frame interval of the target video sequence is D, then the time interval between the key frame and the target frame satisfies: (F1-F0)*D .

For example, as shown in Figure 11, if the frame number of the key frame is 1, the frame number of the target frame is 6, and the frame interval of the target video sequence is 40ms, then the time interval between the key frame and the target frame is: ( 6-1)*40ms＝200ms.

It can be understood that based on the frame number difference between the frame number of the key frame and the frame number of the target frame, as well as the target video sequence The frame time interval of the column is used to determine the time interval between the key frame and the target frame. Timestamp overflow will not occur and the time interval between the key frame and the target frame can be accurately determined.

S203. Based on the absolute time of the key frame and the time interval between the key frame and the target frame, determine the absolute time of the target frame.

S204. Based on the absolute time of the key frame and the absolute time of the target frame, update the current playback time of the playback progress bar.

As shown in Figure 13, an embodiment of the present application provides a progress bar time updating device for executing the progress bar time updating method shown in Figure 7. The progress bar time updating device 300 includes: an acquisition module 301, a first determination module 302 and a second determination module 303.

The acquisition module 301 is used to acquire a target video sequence, where the target video sequence includes a key frame and at least one target frame associated with the key frame, and the time information corresponding to the key frame includes the absolute time of the key frame and the timing of the key frame. Timestamp, the time information corresponding to the target frame includes the timestamp of the target frame.

The first determination module 302 is configured to determine the absolute time of the target frame based on the absolute time of the key frame and the time interval between the key frame and the target frame, where the absolute time is the actual recording time of the video picture corresponding to the video frame.

The second determination module 303 is configured to use the absolute time of the key frame and the absolute time of the target frame to determine the current playback time displayed on the playback progress bar when the target video sequence is played back.

In a possible implementation, the acquisition module 301 is also used to obtain the timestamp of the key frame and the timestamp of the target frame, where the timestamp is used to represent the time of the video frame relative to the reference time. Offset; the first determination module 302 is further configured to determine the time interval between the key frame and the target frame based on the timestamp of the key frame and the timestamp of the target frame.

In a possible implementation, the first determination module 302 is specifically configured to subtract the time stamp based on the target frame when the timestamp of the target frame is greater than the timestamp of the key frame. The time interval is determined by obtaining the difference between the timestamps of the key frames.

In a possible implementation, the obtaining module 301 is also used to obtain the frame number of the key frame, the frame number of the target frame and the frame interval of the target video sequence, wherein the frame interval Used to represent the time interval between two adjacent video frames in the target video sequence; the first determination module 302 is specifically used to obtain the result by subtracting the frame number of the key frame from the frame number of the target frame. The frame number difference is determined based on the product of the frame number difference and the frame interval of the target video sequence to determine the time interval between the key frame and the target frame.

In a possible implementation, the key frame is an I frame; and/or the target frame is an auxiliary frame.

In a possible implementation, the second determination module 303 is specifically configured to determine the current playback time displayed on the playback progress bar according to the minimum time unit of the playback progress bar.

Embodiments of the present application also provide a video processing device, which is configured to obtain a program flow specified by the playback instruction in response to receiving a playback instruction, wherein the program flow is obtained by processing the original video captured. The encoded video stream of the data is obtained by encapsulating the program stream; the program stream is decoded to obtain a decoded video stream, wherein the decoded video stream includes one or more video sequences, and each video sequence includes a key frame and the At least one target frame associated with the key frame; for any target video sequence of the one or more video sequences, based on the absolute time of the key frame and the time interval between the key frame and the target frame , to determine the absolute time of the target frame, where the absolute time is the actual recording time of the video picture corresponding to the video frame; using the absolute time of the key frame and the absolute time of the target frame time to determine the current playback time displayed on the playback progress bar when the decoded video stream is played back.

In a possible implementation, the video processing device is specifically configured to: when the timestamp of the target frame is greater than the timestamp of the key frame, subtract the timestamp of the target frame from the The difference in timestamps of key frames is obtained to determine the time interval.

In a possible implementation, the video processing device is specifically configured to: obtain the frame number of the key frame, the frame number of the target frame, and the frame interval of the target video sequence, wherein the frame interval Used to represent the time interval between two adjacent video frames in the target video sequence; calculate the frame number difference obtained by subtracting the frame number of the key frame from the frame number of the target frame; according to the frame number difference The product of the frame interval of the target video sequence determines the time interval between the key frame and the target frame.

In a possible implementation, the video processing device is specifically configured to determine the current playback time displayed on the playback progress bar according to the minimum time unit of the playback progress bar.

In a possible implementation, the playback instruction comes from a user terminal that can communicate with the video processing device; the video processing device is also configured to: send the decoded video stream to the sending user terminal to The decoded video stream is played back on the display interface of the user terminal; wherein the current playback time displayed on the playback progress bar on the display interface corresponds to the video frame currently being played back.

In a possible implementation, the functions of the video processing device and the functions of the user terminal are integrated into a single device.

When the functions of the above-mentioned integrated modules are implemented in the form of hardware, the embodiment of the present application provides another possible structural schematic diagram of the progress bar time updating device involved in the above-mentioned embodiment. As shown in Figure 14, the progress bar time updating device 400 includes: a processor 402, a communication interface 403, and a bus 404. Optionally, the progress bar time updating device may also include a memory 401.

Processor 402 may implement or execute various exemplary logical blocks, modules and circuits described in connection with the disclosure of this application. The processor 402 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various illustrative logical blocks, modules, and circuits described in connection with this disclosure. The processor 402 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

Communication interface 403 is used to connect with other devices through a communication network. The communication network can be Ethernet, wireless access network, wireless local area networks (WLAN), etc.

Memory 401 can be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (random access memory, RAM) or other types that can store information and instructions. A type of dynamic storage device, which can also be an electrically erasable programmable read-only memory (EEPROM), a disk storage medium or other magnetic storage device, or can be used to carry or store instructions or data structures Without limitation, any other medium in the form of the desired program code and capable of being accessed by a computer.

As a possible implementation, the memory 401 may exist independently of the processor 402. The memory 401 may be connected to the processor 402 through the bus 404 for storing instructions or program codes. When the processor 402 calls and executes the instructions or program codes stored in the memory 401, it can implement the progress bar time updating method provided by the embodiment of the present application.

In another possible implementation, the memory 401 can also be integrated with the processor 402 .

Bus 404 may be an extended industry standard architecture (EISA) bus, etc. The bus 404 can be divided into an address bus, a data bus, a control bus, etc. For ease of presentation, only one thick line is used in Figure 14, but it does not mean that there is only one bus or one type of bus.

Through the above description of the embodiments, those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above functional modules is used as an example. In actual applications, the above functions can be allocated as needed. It is completed by different functional modules, that is, the internal structure of the progress bar time update device is divided into different functional modules to complete all or part of the functions described above.

An embodiment of the present application also provides a computer-readable storage medium. All or part of the processes in the above method embodiments can be completed by computer instructions to instruct relevant hardware. The program can be stored in the above computer-readable storage medium. When executed, the program can include the processes of the above method embodiments. . Computer-readable storage media may So the memory of any of the aforementioned embodiments. The computer-readable storage medium may also be an external storage device of the progress bar time update device, such as a plug-in hard disk, a smart media card (SMC), or a secure digital device equipped on the progress bar time update device. digital, SD) card, flash card, etc. Furthermore, the computer-readable storage medium may also include both an internal storage unit and an external storage device of the progress bar time updating device. The computer-readable storage medium is used to store the computer program and other programs and data required by the progress bar time updating device. The above-mentioned computer-readable storage media can also be used to temporarily store data that has been output or is to be output.

An embodiment of the present application also provides a computer program product. The computer product includes a computer program. When the computer program product is run on a computer, it causes the computer to execute any of the progress bar time updating methods provided in the above embodiments.

Although the present application has been described herein in connection with various embodiments, in practicing the claimed application, those skilled in the art can understand and implement the disclosure by reviewing the drawings, the disclosure, and the appended claims. Other variations of the embodiment. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality. A single processor or other unit may perform several of the functions recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not mean that a combination of these measures cannot be combined to advantageous effects.

Although the present application has been described in conjunction with specific features and embodiments thereof, it will be apparent that various modifications and combinations may be made without departing from the spirit and scope of the application. Accordingly, the specification and drawings are intended to be merely illustrative of the application as defined by the appended claims and are to be construed to cover any and all modifications, variations, combinations or equivalents within the scope of the application. Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. In this way, if these modifications and variations of the present application fall within the scope of the claims of the present application and equivalent technologies, the present application is also intended to include these modifications and variations.

The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be covered by the protection scope of the present application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

A video processing device, characterized in that the video processing device is used for:

In response to receiving the playback instruction, obtain the program flow specified by the playback instruction, wherein the program flow is obtained by performing program flow encapsulation on the encoded video stream of the captured original video data;

decoding the program stream to obtain a decoded video stream, wherein the decoded video stream includes one or more video sequences, each video sequence including a key frame and at least one target frame associated with the key frame;

For any target video sequence of the one or more video sequences, determining the target frame based on the absolute time of the key frame and the time interval between the key frame and the target frame. Absolute time, wherein the absolute time is the actual recording time of the video picture corresponding to the video frame;

The absolute time of the key frame and the absolute time of the target frame are used to determine the current playback time displayed on the playback progress bar when the decoded video stream is played back.
The video processing device according to claim 1, characterized in that the video processing device is specifically used for:

Obtain the timestamp of the key frame and the timestamp of the target frame, where the timestamp is used to represent the time offset of the video frame relative to the reference time;

The time interval between the key frame and the target frame is determined according to the time stamp of the key frame and the time stamp of the target frame.
The video processing device according to claim 2, characterized in that the video processing device is specifically used for:

In the case where the timestamp of the target frame is greater than the timestamp of the key frame, the time interval is determined based on the difference obtained by subtracting the timestamp of the key frame from the timestamp of the target frame.
The video processing device according to any one of claims 1 to 3, characterized in that the video processing device is specifically used for:

Obtain the frame number of the key frame, the frame number of the target frame and the frame interval of the target video sequence, where the frame interval is used to represent the interval between two adjacent video frames in the target video sequence. time interval;

Calculate the frame number difference obtained by subtracting the frame number of the key frame from the frame number of the target frame;

The time interval between the key frame and the target frame is determined according to the product of the frame number difference and the frame interval of the target video sequence.
The video processing device according to any one of claims 1 to 4, characterized in that the key frame is an I frame; and/or

The target frame is an auxiliary frame.
The video processing device according to any one of claims 1 to 5, characterized in that the video processing device is specifically configured to determine the current playback value displayed on the playback progress bar according to the minimum time unit of the playback progress bar. time.
The video processing device according to any one of claims 1 to 6, characterized in that the playback instruction comes from a user terminal that can communicate with the video processing device; the video processing device is also used to:

Send the decoded video stream to the user terminal to play back the decoded video stream on the display interface of the user terminal;

Wherein, the current playback time displayed on the playback progress bar on the display interface corresponds to the video frame currently being played back.
The video processing device according to claim 7, characterized in that the functions of the video processing device and the functions of the user terminal are integrated in a single device.
A method for updating progress bar time, which is characterized by including:

Obtaining a target video sequence, wherein the target video sequence includes a key frame and at least one target frame associated with the key frame;

Determine based on the absolute time of the key frame and the time interval between the key frame and the target frame The absolute time of the target frame, wherein the absolute time is the actual recording time of the video picture corresponding to the video frame;

The absolute time of the key frame and the absolute time of the target frame are used to determine the current playback time displayed on the playback progress bar when the target video sequence is played back.
The method of claim 9, further comprising:

Obtain the timestamp of the key frame and the timestamp of the target frame, where the timestamp is used to represent the time offset of the video frame relative to the reference time;

The time interval between the key frame and the target frame is determined according to the time stamp of the key frame and the time stamp of the target frame.
The method according to claim 10, wherein determining the time interval between the key frame and the target frame includes:

In the case where the timestamp of the target frame is greater than the timestamp of the key frame, the time interval is determined based on the difference obtained by subtracting the timestamp of the key frame from the timestamp of the target frame.
The method according to any one of claims 9 to 11, characterized in that the method further includes:

Obtain the frame number of the key frame, the frame number of the target frame and the frame interval of the target video sequence, where the frame interval is used to represent the interval between two adjacent video frames in the target video sequence. time interval;

The frame number difference is obtained by subtracting the frame number of the key frame from the frame number of the target frame;

The time interval between the key frame and the target frame is determined according to the product of the frame number difference and the frame interval of the target video sequence.
The method according to any one of claims 9 to 12, characterized in that the key frame is an I frame; and/or

The target frame is an auxiliary frame.
The method according to any one of claims 9 to 13, wherein determining the current playback time displayed on the playback progress bar includes:

The current playback time displayed on the playback progress bar is determined according to the minimum time unit of the playback progress bar.
A progress bar time updating device, characterized by including:

An acquisition module, configured to acquire a target video sequence, wherein the target video sequence includes key frames and at least one target frame associated with the key frames;

A first determination module, configured to determine the absolute time of the target frame based on the absolute time of the key frame and the time interval between the key frame and the target frame, wherein the absolute time is The actual recording time of the video frame corresponding to the video frame;

The second determination module is configured to use the absolute time of the key frame and the absolute time of the target frame to determine the current playback time displayed on the playback progress bar when the target video sequence is played back.
The device according to claim 15, characterized in that:

The acquisition module is also used to acquire the timestamp of the key frame and the timestamp of the target frame, wherein the timestamp is used to represent the time offset of the video frame relative to the reference time;

The first determining module is further configured to determine the time interval between the key frame and the target frame based on the timestamp of the key frame and the timestamp of the target frame.
The device according to claim 16, wherein the first determination module is specifically configured to determine based on the time of the target frame when the timestamp of the target frame is greater than the timestamp of the key frame. The time interval is determined by subtracting the time stamp from the key frame's timestamp.
The device according to any one of claims 15 to 17, characterized in that,

The acquisition module is also used to acquire the frame number of the key frame, the frame number of the target frame and the frame interval of the target video sequence, wherein the frame interval is used to represent the corresponding phase in the target video sequence. The time interval between two adjacent video frames;

The first determination module is specifically configured to obtain a frame number difference by subtracting the frame number of the key frame from the frame number of the target frame, and based on the product of the frame number difference and the frame interval of the target video sequence , determine the time interval between the key frame and the target frame.
The device according to any one of claims 15 to 18, characterized in that the key frame is an I frame; and/or

The target frame is an auxiliary frame.
The device according to any one of claims 15 to 19, wherein the second determination module is specifically configured to determine the current playback time displayed on the playback progress bar according to the minimum time unit of the playback progress bar. .
An electronic device, characterized by including:

at least one processor;

At least one memory storing machine-executable instructions;

Wherein, when the at least one processor executes the machine executable instructions, the method according to any one of claims 9 to 14 is implemented.
A computer program product, characterized by comprising a computer program that implements the method according to any one of claims 9 to 14 when run on a computer.