WO2022206212A1 - Video data storage method and apparatus, and electronic device and readable storage medium - Google Patents

Abstract

A video data storage method, comprising: in response to decoding processing of a video frame to be reconstructed, when a reference video frame of the video frame to be reconstructed is not present in an external memory of a video decoder, and the reference video frame is present in an external main memory of the video decoder, reading the reference video frame from the main memory; and storing the read reference video frame in an internal cache of the video decoder.

Description

Video data storage method, apparatus, electronic device and readable storage medium

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the priority of the Chinese patent application filed on April 1, 2021 with the application number 202110355145.3 and the invention titled "video data storage method, device, electronic device and readable storage medium", the entire content of which is Incorporated herein by reference.

technical field

The present application relates to the technical field of video processing, and in particular, to a video data storage method, apparatus, electronic device, and computer-readable storage medium.

Background technique

With the development of new video standards, such as MPEG-1/2/4/5, H.263/264/265/266, VP8/9, etc., the picture size of the video is getting bigger and bigger, and the frame rate of the video is getting bigger and bigger. The higher the value, the higher the power consumption of electronic devices is caused by the traditional video data storage method.

SUMMARY OF THE INVENTION

Various embodiments of the present application provide a video data storage method, apparatus, electronic device, and computer-readable storage medium.

A video data storage method, comprising:

In response to the coding process of the video frame to be reconstructed, the reference video frame of the video frame to be reconstructed does not exist in the external storage of the video coder, and the reference video exists in the main storage external to the video coder frame, reading the reference video frame from the main storage; and

The reference video frame in main storage is stored in an internal buffer of the video coder.

A method for reading video data, the method comprising:

In response to the decoding process of the acquired video code stream, read the reference video block that does not exist in the external storage of the video decoder from the external main storage of the video decoder to the internal buffer of the video decoder. Perform motion compensation or block copying;

Alternatively, motion compensation or block copying is performed after the reference video block is read from the video decoder's external storage.

A video data storage device, comprising:

A first read module for responding to the decoding process of the video frame to be reconstructed, the reference video frame of the video frame to be reconstructed does not exist in the external storage of the video decoder, and the reference video frame of the video frame to be reconstructed does not exist in the external storage of the video decoder. reading the reference video frame from the main storage if the reference video frame exists in the main storage; and

A reference video frame storage module, configured to store the reference video frame read from the main storage into an internal buffer of the video decoder.

A video data reading device, comprising:

The second reading module is configured to, in response to the decoding processing of the acquired video code stream, read the reference video blocks that do not exist in the external storage of the video decoder from the external main storage of the video decoder to Motion compensation or block copying after the video decoder's internal buffering;

The third reading module is used for performing motion compensation or block copy after reading the reference video block from the external storage of the video decoder.

An electronic device includes a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the operations of each method.

A computer-readable storage medium having a computer program stored thereon, the computer program implementing the operations of each method when executed by a processor.

The above-mentioned video data storage method, apparatus, electronic device and readable storage medium, in response to the decoding process of the video frame to be reconstructed, there is no reference video frame of the video frame to be reconstructed in the external storage of the video decoder, and in the video frame. When there is a reference video frame in the external main storage of the decoder, the reference video frame is read from the main storage, and the reference video frame read from the main storage is stored in the internal buffer of the video decoder; The video data existing in the external storage of the video decoder is not stored in the internal buffer of the video decoder, and the video data in the main storage is stored in the internal buffer of the video decoder. Compared with the traditional video data, whether it is located in the external storage or the main storage, the decoder will put it into the internal buffer of the decoder. The internal buffer of the decoder reduces the use of the internal buffer of the video decoder, increases the efficiency of the internal buffer of the decoder, and further reduces the power consumption of the data that is only left in the main storage.

Description of drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.

FIG. 1 is a schematic structural diagram of a conventional video data storage method in an embodiment.

Figure 2 is a conventional video decoder data processing architecture in one embodiment.

FIG. 3 is a schematic structural diagram of a conventional video data storage method in another embodiment.

FIG. 4 is a schematic diagram of a curve of system power consumption in a conventional manner in one embodiment.

FIG. 5 is a flowchart of a video data storage method in one embodiment.

FIG. 6 is a schematic structural diagram of video data storage in one embodiment.

FIG. 7 is a schematic diagram of an architecture using a system cache in one embodiment.

FIG. 8 is a schematic diagram of an architecture using a system buffer in one embodiment.

FIG. 9 is a schematic diagram of a decoding process in an embodiment.

FIG. 10 is a flowchart of a method for reading video data in one embodiment.

FIG. 11 is a schematic flowchart of reconstructing a video frame in one embodiment.

FIG. 12 is a schematic diagram of a storage method of internal storage of a conventional video decoder in an embodiment.

FIG. 13 is a schematic diagram of a storage method of internal storage of a video decoder in an embodiment.

FIG. 14 is a structural block diagram of a video data storage device according to an embodiment.

FIG. 15 is a structural block diagram of a video data reading apparatus according to an embodiment.

FIG. 16 is a schematic diagram of the internal structure of an electronic device in one embodiment.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application will be described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application.

It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various data, but these data are not limited by these terms. These terms are only used to distinguish the first data from another. For example, without departing from the scope of the present application, the first order condition may be referred to as the second order condition, and similarly, the second order condition may be referred to as the first order condition. Both the first degree condition and the second degree condition are degree conditions, but they are not the same degree condition.

In one embodiment, traditional electronic devices basically use DRAM as a storage space for video frames. As shown in FIG. 1 , it is a schematic structural diagram of a traditional video data storage manner in one embodiment. Figure 1 includes a central processing unit (Central Processing Unit, CPU), a video decoder (Video Decoder, VDEC), a display processor (Display Processing Unit) and a dynamic random access memory controller (DRAM Controller, DRAMC). Among them, the central processing unit includes a master port (master), the video decoder includes an internal cache (cache) and a master port (master), the display processor includes a master port (master), and the dynamic random access memory controller includes Including the slave port (slave). Arrows in FIG. 1 represent video data. When there is video data, the video decoder can obtain the video data from the dynamic random access memory controller for video decoding, reconstruct the video frame, and transmit the reconstructed video frame to the display processor for display.

Figure 2 is a conventional video decoder data processing architecture in one embodiment. Input the video bitstream (Video Bitstreams) into the video decoder for decoding processing, and generate video frame (frame buffer) and some temporary data (Temporary data) such as Temporal Motion Vector (TMV). And the video bit stream, video frame and temporary data are all stored in DRAM.

Traditional video decoders can have corresponding buffering strategies for processing various video bit streams. However, with the new video standards, such as H.265/266, AV, MPEG-5 and other standards, for video with larger and larger size and higher frame rate, the traditional technology adopts the architecture shown in Figure 3. Widen the bandwidth of DRAM or increase the frequency of DRAM to speed up data throughput. FIG. 3 is a schematic structural diagram of a conventional video data storage method in another embodiment. The figure includes a central processing unit, a video decoder, a display processor and a multi-channel dynamic random access memory controller (Multi-channel DramCtrl). The multi-channel dynamic random access memory controller can be used to control multiple dynamic random access memories ( DRAM).

FIG. 4 is a schematic diagram of a curve of system power consumption in a conventional manner in one embodiment. In Figure 4, the abscissa is the position in the image (Picture Position), and the ordinate is the system power. The dotted line represents the System Power Barrier. As shown in FIG. 4 , the system DRAM consumes relatively high system power in order to meet the operating speed requirements of the video decoder. When the video decoder is overly dependent on DRAM or other power-hungry storage, the speed of the video decoder cannot meet the demand or the system overheats due to the limited power consumption that the system can provide. The traditional approach would result in higher power consumption for the video decoder when it finishes at the expected time.

In order to solve the problem of high power consumption of electronic equipment, an embodiment of the present application provides a video data storage method. In response to the decoding process of the video frame to be reconstructed, there is no video frame to be reconstructed in the external storage of the video decoder. the reference video frame, and if there is a reference video frame in the external main storage of the video decoder, read the reference video frame from the main storage; store the reference video frame read from the main storage to the video decoder In the internal buffer of the decoder; compared with the traditional video data, whether it is located in the external storage or the main storage, the decoder will put it into the internal buffer of the decoder. The video data stored in the main storage is put into the internal buffer of the decoder, which reduces the use of the internal buffer of the video decoder and increases the efficiency of the internal buffer of the decoder. Power consumption is reduced.

The video data storage method is described below with reference to specific embodiments. In the embodiments of the present application, the main storage is DRAM as an example for description. FIG. 5 is a flowchart of a video data storage method in one embodiment. The video data storage method in this embodiment is described by taking the operation on an electronic device as an example. As shown in FIG. 5 , the video data storage method includes operations 502 to 504 .

Operation 502, in response to the decoding process of the video frame to be reconstructed, the reference video frame of the video frame to be reconstructed does not exist in the external storage of the video coder, and the reference video frame exists in the main storage external to the video coder , the reference video frame is read from main storage.

The video frame to be reconstructed refers to a video frame that has not been successfully constructed yet. After the video is compressed, if the video frame needs to be obtained, it needs to be decoded. The video coder's external storage refers to storage located in the electronic device and other than the main storage and the video coder's internal cache. For example, the external storage of the video decoder may be at least one of a system cache and a system buffer. Main memory refers to the main memory (Main memory), referred to as the main memory. The main storage is an important part of computer hardware, its function is to store instructions and data, and can be directly accessed randomly by the central processing unit (CPU). The main storage may specifically be DRAM (Dynamic Random Access Memory, dynamic random access memory).

The reference video frame can be used to reconstruct the target video frame corresponding to the video frame to be reconstructed. The reference video frame may be a video frame displayed before or after the video frame to be reconstructed. The reference video frame is specifically the video frame that has been reconstructed before the video frame to be reconstructed, or the reference video frame can also be the reconstructed part of the video frame to be reconstructed itself, such as for intra-block copying in the screen content coding technology. copy) of the block vector (block vector) to distinguish the range of the picture to be accessed. The specification of the reference picture range allowed in the screen content coding technology has been described in detail in various video standards, and will not be repeated here.

Specifically, in response to the decoding process of the to-be-reconstructed video frame, the electronic device acquires the target motion vector or block vector corresponding to the to-be-reconstructed video frame. It is detected whether a reference video frame of the video frame to be reconstructed exists in the external storage of the video coder. In the case that the reference video frame corresponding to the target motion vector or block vector does not exist in the external storage of the video decoder, and the reference video frame exists in the main storage external to the video decoder, the electronic device removes the The reference video frame is read, and the read reference video frame is stored in the internal buffer of the video decoder.

In operation 504, the reference video frame read from the main storage is stored into an internal buffer of the video decoder.

Specifically, the electronic device stores the reference video frames read from the main storage into the internal buffer of the video decoder.

In the video data storage method in this embodiment, in response to the decoding process of the video frame to be reconstructed, the reference video frame of the video frame to be reconstructed does not exist in the external storage of the video decoder, and the reference video frame of the video frame to be reconstructed does not exist in the external storage of the video decoder. When there is a reference video frame in the main storage, the reference video frame is read from the main storage, and the reference video frame read from the main storage is stored in the internal buffer of the video decoder, that is, in the video decoder's internal buffer. The video data existing in the external storage is not stored in the internal buffer of the video decoder, and the video data in the main storage will be stored in the internal buffer of the video decoder. Compared with the traditional video data, whether it is located in the external storage or the main storage, the decoder will put it into the internal buffer of the decoder. The internal buffer of the decoder reduces the use of the internal buffer of the video decoder, increases the efficiency of the internal buffer of the decoder, and further reduces the power consumption of the data that is only left in the main storage.

In one embodiment, as shown in FIG. 6 , it is a schematic diagram of the architecture of video data storage in one embodiment. The white part indicates that it is stored in DRAM, that is, this part of the data will be stored in the internal buffer of the video decoder later, and the gray part indicates that it is stored in the external storage of the video decoder. Compared with the storage method in FIG. 3 , some of the video bit stream, video frame data, and temporary data in FIG. 6 are stored in the DRAM, and some are stored in the external storage of the video decoder.

In one embodiment, each video frame is divided into multiple blocks. In the case that the reference video frame of the video frame to be reconstructed does not exist in the external storage of the video decoder, and if the reference video frame exists in the main storage external to the video decoder, the reference video frame is read from the main storage, include:

If the reference video block corresponding to the video frame to be reconstructed does not exist in the external storage of the video decoder, and if the reference video block exists in the external main storage of the video decoder, read from the main storage Refer to the video block.

The storing of the reference video frames read from the main storage into the internal buffer of the video decoder includes: storing the reference video blocks read from the main storage into the internal buffer of the video decoder.

The block refers to a part of the video frame. For example, the size of the block may be 16×16 pixels, 64×64 pixels, etc., but not limited thereto.

Specifically, in response to the decoding processing of the acquired video, the electronic device divides each video frame in the video into a plurality of blocks, and obtains motion vectors corresponding to each block. The electronic device determines, based on the motion vector of the current block in the current video frame, the reference video block in the reference video frame referenced by the video frame to be reconstructed. When the reference video block corresponding to the video frame to be reconstructed does not exist in the external storage of the video decoder, and if the reference video block exists in the main storage outside the video decoder, the electronic device retrieves the data from the main storage. Read the reference video block. The electronic device stores the reference video block read from the main storage into an internal buffer of the video decoder.

In this embodiment, in the case that the reference video block corresponding to the video frame to be reconstructed does not exist in the external storage of the video decoder, and the reference video block exists in the external main storage of the video decoder, the electronic The device reads the reference video block from the main storage and stores it in the internal buffer of the video decoder. Since the internal buffer in the video decoder is small, the storage block method is more suitable for the internal buffer in the video decoder. Cache to improve the accuracy of video data storage.

In one embodiment, the video data storage method further comprises: if the reference video frame exists in the external storage of the video decoder, reading the reference video frame from the external storage of the video decoder.

Specifically, in the case that the reference video frame exists in the external storage of the video decoder, the electronic device reads the reference video frame from the external storage of the video decoder, and based on the reading from the external storage of the video decoder The target video frame corresponding to the to-be-reconstructed video frame is obtained by reconstructing the reference video frame of .

In this embodiment, when there is a reference video frame in the external storage of the video decoder, the reference video frame is read from the external storage of the video decoder, and data access is performed through the external storage of the video decoder, The efficiency of the internal cache of the decoder can be increased, and further power consumption can be reduced specifically for the data that is still only in the main storage.

In one embodiment, in response to the decoding process of the video frame to be reconstructed, the reference video frame of the video frame to be reconstructed does not exist in the external storage of the video coder and in the main storage external to the video coder In the presence of a reference video frame, before reading the reference video frame from the main storage, the video data storage method further includes:

In operation (a1), a motion vector corresponding to each video frame in the video is obtained in response to the decoding process of the obtained video.

The types of videos are not limited. For example, it may be a downloaded video stored locally on the electronic device, or it may be a video obtained from a server. The video may be a TV series video, a movie video, a TV series video, and the like. The length of the video is not limited, for example, it can be 10 seconds, 1 minute, 10 minutes, etc. without limitation. The decoding process refers to the process of decoding and restoring the encoded video to reconstruct the video frame. The decoding process may also be referred to as a decoding process. Motion Vector (MV) represents the relative displacement between the block of the current video frame and its matching block in the reference image. A block vector (Block Vector, BV) represents the relative displacement between a block of the current video frame and its reconstructed matching block in the current image frame.

Specifically, in response to the decoding process of the acquired video, the electronic device obtains the motion vector corresponding to each video frame in the video through a video decoder. It can be understood that, in response to the decoding processing of the acquired video, the electronic device performs decoding processing on some video frames in the video to obtain the motion vector corresponding to each video frame. For example, a video includes 100 video frames, and the electronic device may perform a decoding process on the first 50 video frames of the 100 video frames.

In operation (a2), the reference times of each video frame is determined based on the motion vector corresponding to each video frame.

The number of times of being referenced refers to the number of times that the video frame is referenced by other video frames during reconstruction of the video frame. For example, the video frame A is referenced by the video frame B to be reconstructed, the video frame C to be reconstructed... The video frame Z to be reconstructed is referenced 26 times in total.

Specifically, the electronic device determines the reference video frame corresponding to each video frame based on the motion vector corresponding to each video frame in the video, and counts the number of times the reference video frame is referenced. Taking video frame X, video frame Y and video frame Z as an example, video frame X is referenced by video frame Y and video frame Z, then video frame X is used as the reference video frame of video frame Y and video frame Z, then video frame X is The number of references is 2 times.

In operation (a3), the video frames whose reference times satisfy the condition of the first number of times are stored in the external main storage of the video decoder.

Wherein, the condition of the first number of times can be adjusted as required. The number of times represented by the first degree condition is smaller than the number of times represented by the second degree condition. For example, satisfying the first degree condition is equal to 1.

Specifically, the electronic device stores the video frames whose reference times satisfy the condition of the first number of times into the main storage outside the video decoder. Subsequent video coders may read video frames that satisfy the first number of conditions from main storage and store them in an internal buffer of the video coder.

In operation (a4), the video frames whose reference times satisfy the second number condition are stored in the external storage of the video decoder; wherein the number represented by the second number condition is greater than the number represented by the first number condition.

Wherein, the second order condition can be adjusted as required. The number of times represented by the second degree condition is greater than the number of times represented by the first degree condition. For example, if the number of times to be referenced satisfies the first number of times condition, the number of times to be referenced satisfies 1, and the number of times to be referenced satisfies the second condition is that the number of times to be referenced satisfies more than 1. Alternatively, if the number of times to be referenced satisfies the condition of the first number of times, the number of times to be referenced is less than or equal to 2, and the number of times to be referenced satisfies the condition of the second number of times is that the number of times to be referenced is greater than 2, and so on.

Specifically, the electronic device stores the video frames whose reference times satisfy the condition of the second number of times into the external storage of the video decoder. External storage of the video coder refers to storage located in the electronic device but not in the video coder. The external storage of the video decoder may be an on-chip memory. For example, the external storage of the video decoder may be at least one of a system cache and a system buffer.

In the video data storage method in this embodiment, in response to the decoding processing of the acquired video, the motion vector corresponding to each video frame in the video is obtained, and the reference times of each video frame is determined based on the motion vector of each video frame, then It can be known how many times the video frame is to be used; the video frame whose number of references meets the condition of the first number of times is stored in the internal buffer of the video decoder, and the video frame whose number of times of reference meets the condition of the second number of times is stored in the video In the external storage of the decoder, the video frames that have been referenced less frequently are stored in the internal buffer of the video decoder, and the video frames that have been referenced more frequently are stored in the external storage of the video decoder. The use of the decoder's internal cache utilizes low-power external storage, thereby reducing the power consumption of electronic equipment. In addition, because the internal buffer of the video decoder is limited, the video frames that are used less frequently are buffered in the video decoder, and these video frames in the external buffer can be updated faster, and the external buffer can be fully utilized to speed up the decoding. code processing.

In one embodiment, the external storage of the video coder includes at least one of a system cache or a system buffer. Among them, the system cache (system cache) and the system cache area (system buffer) both belong to the on-chip memory (on-chip memory). And both the system cache and the system cache area belong to static random-access memory (Static Random-Access Memory, SRAM). In this embodiment, by storing the video frames that meet the conditions of the second number of times in at least one of the system cache and the system buffer, the data read from the system cache and the system buffer is faster and consumes less power. Therefore, the internal storage of the decoder does not need to store the video data from the system cache and system buffer once, and the decoding speed of the decoder is maintained and the power consumption of the electronic device is reduced.

In one embodiment, determining the reference times of each video frame based on the motion vector corresponding to each video frame includes: determining the reference video frame referenced by the current video frame based on the motion vector of the current video frame; number of references.

Specifically, the electronic device performs decoding processing on the acquired video, and firstly decodes to obtain the motion vector of the video frame. The motion vector of the video frame is used to represent the displacement between the current video frame and the reference video frame. The electronic device determines the reference video frame referenced by the current video frame based on the motion vector of the current frame, and counts the reference times of the reference video frame, then the reference times of each video frame can be obtained. For example, the current video frame is video frame C, and the reference video frames referenced by video frame C are video frame A and video frame B as an example for description. Then, the reference times of the video frame A is 1 at this time, and the reference times of the video frame B is also 1. Then, when the current video frame is video frame D, and the reference video frames referenced by video frame D are also video frame A and video frame B, then the reference times of video frame A is 2, and the reference times of video frame B is also is 2, which can obtain the reference times of each video frame in the video.

In the video data storage method in this embodiment, the reference video frame referenced by the current video frame is determined based on the motion vector of the current video frame, and the number of times the reference video frame is referenced is counted, that is, the current video frame and the motion vector are known. , the reference video frame referenced by the current video frame can be obtained, and the reference times of each video frame in the video can be obtained, and the video data will be stored based on the reference times. The storage or the main storage will be put into the internal buffer of the decoder by the decoder. In this embodiment, only the video data that does not exist in the external storage and exists in the main storage needs to be put into the internal buffer of the decoder, reducing the number of video data. The use of the internal buffer of the decoder increases the efficiency of the internal buffer of the decoder, and further reduces the power consumption especially for the data still only in the main storage.

In one embodiment, the video data storage method further includes: when the number of video frames between the current video frame and the target reference video frame reaches a preset number, stopping counting the number of times the target reference video frame is referenced, and obtaining the The number of times the target reference video frame has been referenced. The preset number can be set according to actual needs. The preset number is used to characterize the number of maximum interval frames between the video frame and the reference video frame. Since there are generally not too many frames between the current video frame and the referenced video frame, when the number of video frames between the current video frame and the target reference video frame reaches the preset number, the video frames after the current video frame The frame generally does not refer to the target reference video frame, so the reference times of the target reference video frame can be directly obtained.

In one embodiment, each frame of video frame is divided into multiple blocks; determining the reference video frame referenced by the current video frame based on the motion vector of the current video frame, including: based on the motion vector of the current block in the current video frame , to determine the reference video block in the reference video frame referenced by the current video frame.

Counting the reference times of the reference video frame includes: counting the reference times of the reference video block in the reference video frame.

The video frame that satisfies the condition of the first number of times is stored in the external main storage of the video decoder, including: the reference video block that satisfies the condition of the first number of times is stored in the video decoder. in external primary storage.

Storing the video frames that meet the condition of the second number of times to be referenced in the external storage of the video decoder includes: storing the reference video blocks that meet the condition of the second number of times to be referenced in the external storage of the video decoder.

The block refers to a part of the video frame. For example, the size of the block may be 16×16 pixels, 64×64 pixels, etc., but not limited thereto.

Specifically, in response to the decoding processing of the acquired video, the electronic device divides each video frame in the video into a plurality of blocks, and obtains motion vectors corresponding to each block. The electronic device determines, based on the motion vector of the current block in the current video frame, the reference video block in the reference video frame referenced by the current video frame. The electronic device counts the number of times the reference video block is referenced. The electronic device stores the reference video block whose number of references meets the condition of the first number of times in the internal buffer of the video decoder. The electronic device stores the reference video blocks whose number of references meets the condition of the second number of times in the external storage of the video decoder.

In this embodiment, the reference video block to be referenced is determined based on the motion vector of the current block, the reference times of the reference video block are counted, and the reference video blocks whose reference times satisfy the condition of the first times are stored in the In the main storage, the reference video block that satisfies the condition of the second number of times by reference is stored in the external storage of the video decoder, then the reference video block referenced can be determined based on the current block, and the stored video data is more accurate , since the internal buffer in the video decoder is small, the storage block method is more suitable for the internal buffer in the video decoder.

In this embodiment, as shown in FIG. 7 , it is a schematic diagram of the architecture of using a system cache in one embodiment. These include the central processing unit, video decoder, display processor, dynamic random access memory controller and system cache. The system cache can obtain data from DRAMC. The system cache size in Figure 7 is 4MB (megabytes). And the system cache is a kind of SRAM (Static Random-Access Memory). Then, in response to the decoding processing of the acquired video, the motion vector corresponding to each video frame in the video is obtained by the video decoder, and the reference times of each video frame is determined based on the motion vector corresponding to each video frame; The video frames whose reference times meet the conditions of the first times are read from DRAM and stored in the internal buffer of the video decoder; some video frames whose reference times meet the conditions of the second times are read from DRAM and stored in the system cache. It can be understood that the system cache may not be directly connected to the DRAMC, and FIG. 6 is just an example.

In this embodiment, as shown in FIG. 8 , it is a schematic diagram of the architecture of using the system buffer area in one embodiment. These include the central processing unit, video decoder, display processor, dynamic random access memory controller, and system buffer. The system cache size in Figure 7 is 4MB (megabytes). And the system buffer is a type of SRAM (Static Random-Access Memory). Then, in response to the decoding processing of the acquired video, the motion vector corresponding to each video frame in the video is obtained by the video decoder, and the reference times of each video frame is determined based on the motion vector corresponding to each video frame; The video frames whose reference times meet the conditions of the first times are read from DRAM and stored in the internal buffer of the video decoder; the video frames whose reference times meet the conditions of the second times are directly stored in the system cache.

In one embodiment, the video data storage method further includes: in the case that the internal buffer in the video decoder is full and a new video frame is stored, deleting the least important video frame in the internal buffer.

Specifically, when the internal buffer in the video decoder is full and new video frames need to be stored, the electronic device deletes the least important video frame in the internal buffer, so as to ensure that the video decoder can always Store data to ensure the operation of the video decoder. Least important can be the oldest video frame, or it can be judged in other ways.

In one embodiment, as shown in FIG. 9 , it is a schematic diagram of a decoding process in one embodiment. The figure includes I-frames, B-frames and P-frames. The I frame belongs to intra-frame compression, and only the information of the I frame itself needs to be used during decoding. The P frame is a forward predictive coded frame, that is, when the P frame is decoded, it is necessary to refer to the information of the previous relevant frame before decoding. The B frame is a bidirectional predictive coding frame, and it is necessary to refer to both the previous frame and the frame to be decoded later when decoding. It will be appreciated that some video standards allow both reference objects of a B-frame to be previous frames. Among them, the display order (Display Order) is the order from left to right as shown in FIG. 9 . The decoding order (Decoding Order) is the order of 0, 3, 2, 4, 1, 7, 6, 8, and 5 in FIG. 9 . The decoding order in FIG. 9 is not the same as the display order. The arrows indicate references, such as the B frame of display order 2, which is reconstructed by referring to the I frame of display order 0 and the P frame of display order 1, and the B frame of display order 2 can be used as display order 1 and display order. 3 frames of reference video frames. From this, video frames of the video can be reconstructed. The blank part in the figure indicates that it is stored in DRAM, that is, it will be stored in the internal buffer of the video decoder later. Parts in gray indicate storage in the video decoder's external storage. Then the I frame of display order 0, the B frame of display order 2, the B frame of display order 3, the P frame of display order 4, the B frame of display order 6 and the B frame of display order 8 all have partial blocks stored in the video. In the external storage of the decoder, another part is stored in the DRAM or the internal buffer of the video decoder. The B frames of display order 1, the B frames of display order 5, and the B frames of display order 7 are all stored in the DRAM. In this embodiment, after performing some analysis on the video, it is known which frames or which blocks are not suitable for storing in DRAM, or in other words, not suitable for storing in the internal cache of the video decoder, then it is suitable for storing in the video decoder. In the external storage outside the device, such as at least one of the system cache and the system buffer. In the case of such a storage distribution, the decoder will make proper use of the internal buffer of the decoder, so that the data that only exists in the external main storage of the video decoder can survive for a longer time in the internal buffer of the decoder, so as to reduce the cost of System bandwidth to achieve the ultimate power saving purpose.

The basic concept of the embodiments of the present application is to make proper use of each available on-chip memory (on-chip memory, that is, the external storage and internal storage of the video decoder) to reduce the amount of DRAM access. The internal buffering strategy of the video decoder is Changes to be made when assisted by other on-chip memory. Store the entire reference video frame in the video or a partial block of a reference video frame, and store the data that will be reused relatively many times in the low-power storage space such as the system cache or system buffer, and the rest only needs to be The data that cannot be put into the cache or the system buffer will be transferred to the internal buffer of the video decoder for processing.

A video data storage method, comprising:

In operation (b1), in response to the decoding process of the acquired video, a motion vector corresponding to each video frame in the video is obtained.

In operation (b2), the reference video block in the reference video frame referenced by the current video frame is determined based on the motion vector of the current block in the current video frame.

In operation (b3), the number of times the reference video block in the reference video frame is referenced is counted.

In operation (b4), the reference video blocks whose reference times satisfy the condition of the first times are stored in the external main storage of the video decoder.

In operation (b5), the reference video blocks whose reference times satisfy the condition of the second times are stored in the external storage of the video decoder. The number of times represented by the second order condition is greater than the number of times represented by the first order condition.

Operation (b6), in response to the decoding process of the video frame to be reconstructed, the reference video block corresponding to the video frame to be reconstructed does not exist in the external storage of the video decoder, and the main storage outside the video decoder does not exist If there is a reference video block in the , the reference video block is read from the main storage. The external storage of the video decoder includes at least one of a system cache or a system buffer.

In operation (b7), the read reference video block is stored in the internal buffer of the video decoder.

Operation (b8), if the reference video frame exists in the external storage of the video coder, the reference video frame is read from the external storage of the video coder.

In operation (b9), when the internal buffer in the video decoder is full and a new video frame is stored, delete the video frame that is least needed in the internal buffer. (The least needed can be the oldest or selected by other methods of judgment).

In the video data storage method in this embodiment, in response to the decoding processing of the acquired video, the motion vector corresponding to each video frame in the video is obtained, and the reference times of each video frame is determined based on the motion vector of each video frame, then It can be known that the video frame is to be used several times; the video frame that meets the condition of the first number of times is stored in the internal buffer of the video decoder, and the video frame that meets the condition of the second number of times is stored in the decoder. In the external storage of the decoder, the video frames that have been referenced less frequently are stored in the internal buffer of the video decoder, and the video frames that have been referenced more frequently are stored in the external storage of the decoder. Whether the video data is located in the external storage or the main storage, the decoder will put it into the internal buffer of the decoder. In this embodiment, it is only necessary to put the video data that does not exist in the external storage and exists in the main storage into the decoder. The cache reduces the use of the internal cache of the video decoder, increases the efficiency of the internal cache of the decoder, and further reduces power consumption for data that is still only in the main storage. In addition, since the internal buffer of the video decoder is limited and the processing speed of the decoding processing through the internal buffer is fast, the video frames that are used less frequently are buffered in the video decoder, and the speed at which these buffered video frames are updated can be improved. Faster, can fully utilize the buffer of the video coder to speed up the decoding process.

FIG. 10 is a flowchart of a method for reading video data in one embodiment. The video data storage method in this embodiment is described by taking the operation on an electronic device as an example. As shown in FIG. 10 , the video data storage method includes operations 1002 to 1004 .

Operation 1002, in response to a decoding process on the acquired video code stream, read a reference video block that does not exist in the external storage of the video decoder from the main storage outside the video decoder to the video decoder's external storage. Motion compensation or block copying after internal caching.

Among them, the main storage refers to the main storage of the electronic device. The main storage may be, for example, DRAM. Motion Compensation, also known as motion compensation, is a method to describe the difference between nearby frames (nearby means nearby in the coding relationship, and two frames may not be nearby in the playback order). Specifically, it describes the previous one. How each small block of the frame is moved to a certain position in the video frame to be reconstructed.

Specifically, in response to the decoding processing of the acquired video code stream, the electronic device acquires the target motion vector of the video frame to be reconstructed; the electronic device reads from the external main storage of the video decoder according to the target motion vector, which does not exist in the video decoder. The reference video block stored outside the video decoder is stored in the internal buffer of the video decoder to perform motion compensation; the reference video block is the block of the video frame referenced by the video frame to be reconstructed. Alternatively, the electronic device obtains the block vector of the video frame to be reconstructed; according to the block vector, the reference video block that does not exist in the external storage of the video decoder is read from the main storage to the internal buffer of the video decoder and then performs block copying. The video frame to be reconstructed refers to a video frame that has not been successfully constructed yet. The target motion vector of the video frame to be reconstructed represents the displacement between the current video frame and the reference video block. The video data stored in the video decoder's internal buffer and the video decoder's external storage may or may not be the same. For example, the video contains 100 video frames, which are video frame 1, video frame 2, video frame 3...video frame 100 respectively. The internal buffer of the video decoder includes video frame 1...video frame 50, and the external storage of the video decoder includes video frame 51, video frame 52...video frame 100.

Operation 1004, or perform motion compensation or block copy after reading the reference video block from the external storage of the video decoder.

Specifically, the electronic device performs motion compensation after reading the reference video block from the external storage of the video decoder. The electronic device reads the reference video block from the external storage of the video decoder, and reconstructs the target video frame based on the read reference video block. For example, the electronic device performs weighted fusion reconstruction on the read reference video block to obtain the target video frame. Alternatively, the electronic device performs block copy after reading the reference video block from the external storage of the video decoder.

In the video data reading method in this embodiment, by responding to the decoding processing of the acquired video code stream, the reference video block that does not exist in the external storage of the video decoder is read from the main storage to the video decoding. Motion compensation or block copy is performed after the internal buffer of the video decoder, or motion compensation or block copy is performed after the reference video block is read from the external storage of the video decoder, that is, part of the video can be read from the outside of the video decoder. Data, using low-power external storage, also reduces the use of the video decoder's internal buffer to reduce the power consumption of electronic equipment.

In one embodiment, in response to the decoding process on the acquired video stream, a reference video block that does not exist in the external storage of the video coder is read from the main storage external to the video coder to the video coder The motion compensation is performed after the internal buffer of the video decoder; or the motion compensation or block copy is performed after reading the reference video block from the external storage of the video decoder, including:

In response to the decoding process to the video code stream obtained, look up from the external storage of the video decoder, find the reference video block in the external storage, then read the reference video block from the external storage. Perform motion compensation or block copying;

In the case where the reference video block is not found in the external storage, the reference video block is found from the external main storage of the video decoder, and the reference video block is read from the main storage to the interior of the video decoder Motion compensation or block copying after caching.

Specifically, the electronic device searches for the corresponding reference video block from the external storage of the video decoder according to the target motion vector of the video frame to be reconstructed. When the reference video block corresponding to the video frame to be reconstructed is found in the external storage, motion compensation or block copying is performed after reading the reference video block corresponding to the video frame to be reconstructed from the external storage. If the reference video block corresponding to the video frame to be reconstructed is not found in the external storage, the electronic device finds the reference video block corresponding to the video frame to be reconstructed from the main storage, and reads the reference video from the main storage Motion compensation or block copying is performed after the block is stored in the video decoder's internal buffer. The reference video block is a part of the video area in the reference video frame. The number of reference video blocks corresponding to the video frame to be reconstructed is not limited. For example, it may be 1, 2, 3... but not limited thereto.

Taking the external storage as the system cache, the main storage as the DRAM, and the internal cache of the video decoder as the VDEC cache as an example, the electronic device searches for the corresponding reference video block from the system cache according to the target motion vector of the video frame to be reconstructed. When the system cache finds the corresponding reference video block, the reference video block corresponding to the video frame to be reconstructed is read from the system cache. If the reference video block corresponding to the video frame to be reconstructed is not found in the system cache, the reference video block corresponding to the video frame to be reconstructed is found from the DRAM, and the reference video is read from the main storage Motion compensation is performed after the block is sent to the VDEC cache.

In this embodiment, when all reference video blocks corresponding to the video frames to be reconstructed are not found in the external storage, the reference video blocks corresponding to the remaining video frames to be reconstructed are searched from the main storage, and the reference video blocks corresponding to the remaining video frames to be reconstructed are searched from the main storage. The reference video block is read from the storage to the internal buffer of the video decoder for motion compensation or block copying.

In this embodiment, when the reference video block is found from the external storage of the video decoder, motion compensation or block copy is performed after reading the reference video block from the external storage; the reference video block is not found in the external storage. In the case of the reference video block, find the reference video block from the external main storage of the video decoder, find the reference video block from the main storage, read the reference video block from the main storage to the video The internal buffer of the decoder performs motion compensation or block copy; the internal buffer of the video decoder reads only the data from the main storage, compared with the data stored in the internal buffer of the traditional video decoder and the external storage. The data consistency method can save unnecessary buffering operations inside the video decoder, and because the amount of video data buffered in the video decoder is small, the bandwidth of the main storage can be further reduced, thereby reducing the power consumption of electronic equipment.

In one embodiment, the video data reading method further includes: in response to decoding processing of the acquired video, obtaining a motion vector corresponding to each video frame in the video; determining each video frame based on the motion vector corresponding to each video frame The number of times the frame is referenced; the video frame whose number of times to be referenced meets the condition of the first number of times is read from the external main storage of the video decoder and stored in the internal buffer of the video decoder; The video frame of the number of times is stored in the external storage of the video decoder; the number of times represented by the second-order condition is greater than the number of times represented by the first-time condition.

In this embodiment, in response to the decoding processing of the acquired video, the motion vector corresponding to each video frame in the video is obtained, and the number of times each video frame is referenced is determined based on the motion vector of each video frame, then the video frame can be known. To be used several times; store the video frames that meet the condition of the first number of times from the main storage to the internal buffer of the video decoder, and store the video frames that meet the condition of the second number of times to the decoder In the external storage of the video decoder, the video frames that have been referenced less frequently are stored in the internal buffer of the video decoder, and the video frames that have been referenced many times are stored in the external storage of the decoder; the internal buffer of the video decoder is reduced. The use of low-power external storage reduces the power consumption of electronic devices. In addition, because the internal buffer of the video decoder is limited, the video frames that are used less frequently are buffered in the video decoder, and these video frames in the external buffer can be updated faster, and the external buffer can be fully utilized to speed up the decoding. code processing.

In one embodiment, the external storage of the video decoder includes at least one of a system cache, a system buffer. Among them, the system cache (system cache) and the system cache area (system buffer) both belong to the on-chip memory (on-chip memory). And both the system cache and the system cache area belong to static random-access memory (Static Random-Access Memory, SRAM). In this embodiment, by storing the video frames that meet the conditions of the second number of times in at least one of the system cache and the system buffer, the data read from the system cache and the system buffer is faster and consumes less power. Therefore, the decoding speed of the video decoder can be maintained, and the data stored in these buffers can not be put into the internal buffer of the decoder.

In one embodiment, as shown in FIG. 11 , it is a schematic flowchart of reconstructing a video frame in one embodiment. It includes A video frame 1102, B video frame 1104, C video frame 1106, and to-be-reconstructed video frame 1108, and A video frame 1102, A video frame 1104, and A video frame 1106 are all video frames that have been reconstructed, and are to be reconstructed. Video frame 1108 has not yet been reconstructed. Among them, the white parts in the A video frame 1102, the A video frame 1104 and the A video frame 1106 are stored in the DRAM, that is, stored in the internal buffer of the video decoder; the blocks in the gray shaded part are stored in the video decoder. in the external storage of the encoder. And the A video frame 1102 , the B video frame 1104 and the C video frame 1106 are all reference video frames of the video frame 1108 to be reconstructed. As can be seen from the figure, different blocks of a video frame can be stored in different stores. In FIG. 11 , the block of 16×16 and the video frame 1108 to be reconstructed as a B frame are taken as an example for description.

A small square in the video frame to be reconstructed represents a current block. The reference blocks in Figure 11 are not reused. The electronic device obtains the 1st, 2nd, 3rd, 4th, 5th, 6th and 7th sub-target motion vectors of the video frame to be reconstructed, and can determine the reference based on the sub-target motion vectors block. For example, the first block of the video frame 1108 to be reconstructed can be reconstructed from the 0 block in the A video frame 1102 and the 1 block in the A video frame 1104 . The second block of the video frame 1108 to be reconstructed can be reconstructed from 2 blocks in the A video frame 1102 and 3 blocks in the A video frame 1106 . The third block of the video frame 1108 to be reconstructed can be reconstructed from 4 blocks in the A video frame 1104 and 5 blocks in the A video frame 1104 . The fourth block of the video frame 1108 to be reconstructed can be reconstructed from 6 blocks in the A video frame 1104 and 7 blocks in the A video frame 1106 . The fifth block of the video frame 1108 to be reconstructed can be reconstructed from 8 blocks in the A video frame 1102 and 9 blocks in the A video frame 1104 . The sixth block of the video frame 1108 to be reconstructed can be reconstructed from the A block in the A video frame 1104 and the B block in the A video frame 1106 . The seventh block of the to-be-reconstructed video frame 1108 can be reconstructed from the C block of the A video frame 1102 and the D block of the A video frame 1104 .

In one embodiment, as shown in FIG. 12 , it is a schematic diagram of a storage manner of internal storage of a conventional video decoder in one embodiment. Taking the block in FIG. 11 as an example, the storage method of the internal storage of the video decoder in FIG. 12 stores the blocks in sequence. For example, the first step stores block 0, and the second step stores blocks 1, 2, 3, 4, 5, 6, and 7. When a new block is stored, the third step stores blocks 6, 7. , 8, 9, A, B, C and D. That is to say, the internal buffer of the video decoder in Figure 12 will temporarily store the blocks read from any area, whether it is read from DRAM (white part) or from outside the video decoder Stored (gray section) read. In this way, the video data that does not need the internal buffer of the video decoder at all to save the throughput of the DRAM is also stored in the internal buffer of the video decoder, which wastes the read and write energy consumption of the internal buffer of the video decoder.

In one embodiment, as shown in FIG. 13 , it is a schematic diagram of a storage manner of internal storage of the video decoder in one embodiment. Wherein, the white part in the internal buffer of the video decoder refers to the block originally stored in the DRAM, that is, the block that will be subsequently stored in the internal buffer of the video decoder. Compared with the method of FIG. 12 , the internal buffer of the video decoder of FIG. 13 only stores blocks from DRAM. Therefore, the blocks stored in the external storage of the video decoder and the blocks stored in the internal buffer of the video decoder will not be stored repeatedly, and the blocks stored in the DRAM are distinguished and stored in the DRAM The block is read into the internal buffer of the video decoder, and another part of the block is stored in the external storage of the video decoder, so that the reading of the DRAM is reduced as a whole during the decoding process, and the bandwidth of the DRAM is reduced. Reduce power consumption of electronic equipment. In the system buffer/cache, the data itself is power-saving, and the access speed is fast and controllable. If such data is cached again by using the VDEC internal cache, it will not help the decoding speed and consume more energy for accessing the SRAM. In addition, if there is already a power-saving system cache/buffer to store part of the frame buffer data, the remaining data that is not stored in the system cache/buffer will enter the VDEC internal cache to save DRAM bandwidth.

In one embodiment, Table 1 is a schematic diagram of the difference in energy consumption between SRAM and DRAM in one embodiment. The power consumption of SRAM (Static Random-Access Memory, static random access memory) is about 5PJ (picojoule), while the power consumption of DRAM is about 640PJ (picojoule), the difference between the two is two orders of magnitude. In terms of relative energy consumption, the relative energy consumption of SRAM is 10 ² , while the relative energy consumption of DRAM is 10 ⁴ , the difference between the two is 2 orders of magnitude. Although it is described in Table 1 that accessing DRAM consumes more than 100 times the power compared to accessing SRAM, even if there is a system buffer/cache (system cache/cache) in an electronic device, the amount is limited, and it is impossible to unrestrictedly expand the memory outside VDEC. on-chip SRAM. Therefore, making good use of the internal cache of the original video decoder can reduce the required DRAM bandwidth even when the system cache/buffer is used, and achieve more extreme power saving and low power consumption.

Table 1

storage name	Energy consumption (energy pJ)	Relative Energy Cost
static random access memory	5	10 2
dynamic random access memory	640	10 4

It should be understood that, although the respective operations in the flowcharts of FIG. 5 and FIG. 10 are displayed in sequence according to the arrows, these operations are not necessarily executed sequentially in the order indicated by the arrows. Unless otherwise specified herein, the execution of these operations is not strictly limited in order, and these operations may be performed in other orders. Moreover, at least a part of the operations in FIG. 5 and FIG. 10 may include multiple sub-operations or multiple stages. These sub-operations or stages are not necessarily executed and completed at the same time, but may be executed at different times. These sub-operations or The order of execution of the stages is also not necessarily sequential, but may be performed alternately or alternately with other operations or sub-operations of other operations or at least a portion of a stage.

FIG. 14 is a structural block diagram of a video data storage device according to an embodiment. As shown in Figure 14, a video data storage device includes a first reading module 1402 and a reference video frame storage module 1404, wherein:

The first reading module 1402 is configured to respond to the decoding process of the video frame to be reconstructed, the reference video frame of the video frame to be reconstructed does not exist in the external storage of the video decoder, and the main external storage of the video decoder does not exist. Reading the reference video frame from main storage if the reference video frame exists in storage; and

The reference video frame storage module 1404 is used for storing the reference video frame read from the main storage into the internal buffer of the video decoder.

In the video data storage device in this embodiment, in response to the decoding process of the video frame to be reconstructed, the reference video frame of the video frame to be reconstructed does not exist in the external storage of the video decoder, and the reference video frame of the video frame to be reconstructed does not exist in the external storage of the video decoder. When there is a reference video frame in the main storage, the reference video frame is read from the main storage, and the reference video frame read from the main storage is stored in the internal buffer of the video decoder, that is, in the video decoder's internal buffer. The video data in the external storage is not stored in the internal buffer of the video decoder, and the video data in the main storage will be stored in the internal buffer of the video decoder. The main storage will be put into the internal buffer of the decoder by the decoder. In this embodiment, only the video data that does not exist in the external storage and exists in the main storage needs to be put into the internal buffer of the decoder, reducing the number of video decoders. The use of the internal cache increases the efficiency of the internal cache of the decoder, and further reduces power consumption specifically for the data that remains only in the main storage.

In one embodiment, each video frame is divided into multiple video blocks. The first reading module 1402 is used for the situation that the reference video block corresponding to the video frame to be reconstructed does not exist in the external storage of the video decoder, and the reference video block exists in the external main storage of the video decoder Next, the reference video block is read from the main storage. The reference video frame storage module 1404 is used for storing the reference video blocks read from the main storage into the internal buffer of the video decoder.

In this embodiment, the reference video block corresponding to the video frame to be reconstructed does not exist in the external storage of the video decoder, and if the reference video block exists in the main storage, the electronic device reads from the main storage Refer to the video block and store it in the internal cache of the video decoder. Since the internal cache in the video decoder is small, the storage block is more suitable for the internal cache in the video decoder, improving the storage capacity of video data. precision.

In one embodiment, the first reading module 1402 is further configured to read the reference video frame from the external storage of the video coder if the reference video frame exists in the external storage of the video coder.

In this embodiment, when there is a reference video frame in the external storage of the video decoder, the reference video frame is read from the external storage of the video decoder, and data access is performed through the external storage of the video decoder, The efficiency of the internal cache of the decoder can be increased, and further power consumption can be reduced specifically for the data that is still only in the main storage.

In one embodiment, the video data storage device further includes a motion vector acquisition module, a number of times determination module, a first storage module and a second storage module, wherein:

a motion vector acquisition module, used for obtaining the motion vector corresponding to each video frame in the video in response to the decoding process of the acquired video;

A number of times determination module, used for determining the number of times each video frame is referenced based on the motion vector corresponding to each video frame;

The first storage module is used to store the video frame whose number of references meets the condition of the first number of times in the main storage of the video decoder;

The second storage module is used to store the video frames whose reference times satisfy the second time condition in the external storage of the video decoder; wherein, the times represented by the second time condition is greater than the number represented by the first time condition .

In this embodiment, in response to the decoding processing of the acquired video, the motion vector corresponding to each video frame in the video is obtained, and the number of times each video frame is referenced is determined based on the motion vector of each video frame, then the video frame can be known. To be used several times; store the video frames that meet the condition of the first number of times to the internal buffer of the video decoder, and store the video frames that meet the condition of the second number of times to the external storage of the decoder , the video frames that will be referenced less frequently will be stored in the internal buffer of the video decoder, and the video frames that will be referenced more frequently will be stored in the external storage of the decoder. Use, take advantage of low-power external storage, thereby reducing the power consumption of electronic equipment. In addition, because the internal buffer of the video decoder is limited, the video frames that are used less frequently are buffered in the video decoder, and these video frames in the external buffer can be updated faster, and the external buffer can be fully utilized to speed up the decoding. code processing.

In one embodiment, the number of times determination module is configured to determine the reference video frame referenced by the current video frame based on the motion vector of the current video frame; and count the number of times the reference video frame is referenced.

The video data storage device in this embodiment determines the reference video frame referenced by the current video frame based on the motion vector of the current video frame, and counts the number of times the reference video frame is referenced, that is, to know which current video frame is and what is the motion vector , the reference video frame referenced by the current video frame can be obtained, thereby obtaining the reference times of each video frame in the video, and storing video data based on the reference times to reduce the power consumption of the electronic device.

In one embodiment, the number of times determination module is further configured to stop counting the number of times the target reference video frame is referenced when the number of video frames between the current video frame and the target reference video frame reaches a preset number, and obtain the target reference video frame. The number of times the video frame has been referenced. Since there are generally not too many frames between the current video frame and the referenced video frame, when the number of video frames between the current video frame and the target reference video frame reaches the preset number, the video frames after the current video frame The frame generally does not refer to the target reference video frame, so the reference times of the target reference video frame can be directly obtained.

In one embodiment, each video frame is divided into multiple blocks. The number of times determination module is used to determine the reference video block in the reference video frame referenced by the current video frame based on the motion vector of the current block in the current video frame; count the referenced times of the reference video block in the reference video frame; The reference video blocks that meet the condition of the first number of times are stored in the internal buffer of the video decoder; the reference video blocks that meet the condition of the second number of times are stored in the external storage of the video decoder.

In this embodiment, the reference video block to be referenced is determined based on the motion vector of the current block, the reference times of the reference video block are counted, and the reference video blocks whose reference times satisfy the condition of the first times are stored in the In the internal storage of the video decoder, the reference video block whose number of references meets the condition of the second number of times is stored in the external storage of the video decoder, then the referenced reference video block can be determined based on the current block and stored. The video data is more accurate. Since the internal buffer in the video decoder is small, the storage block method is more suitable for the internal buffer in the video decoder.

In one embodiment, the external storage of the video coder includes at least one of a system cache or a system buffer. In this embodiment, by storing the video frames that meet the conditions of the second number of times in at least one of the system cache and the system buffer, the data read from the system cache and the system buffer is faster and consumes less power. Therefore, the decoding speed of the video decoder can be maintained, and the power consumption of the electronic equipment can be reduced.

In one embodiment, the main storage external to the video coder includes dynamic random access memory.

In one embodiment, the first storage module is further configured to delete the most unnecessary video frames in the internal buffer when the internal buffer in the video decoder is full and new video frames are stored. In this embodiment, when the internal buffer in the video decoder is full and new video frames need to be stored, the electronic device deletes the least-needed video frames in the internal buffer to ensure that the video decoder Data can always be stored to ensure the operation of the video decoder. For example, it is possible to determine which video frames are least needed by looking at old and new. The least needed video frames may be, for example, the oldest. In other embodiments, other judgment methods may also be used to select the least needed video frames.

FIG. 15 is a structural block diagram of a video data reading apparatus according to an embodiment. As shown in FIG. 15, a video data reading device includes a second reading module 1502 and a third reading module 1504, wherein:

The second reading module 1502 is configured to, in response to the decoding processing of the acquired video code stream, read the reference video blocks that do not exist in the external storage of the video decoder from the external main storage of the video decoder to the internal buffer of the video decoder for motion compensation or block copying;

or,

The third reading module 1504 is configured to perform motion compensation or block copy after reading the reference video block from the external storage of the video decoder.

The device for reading video data in this embodiment reads the reference video blocks that do not exist in the external storage of the video decoder from the main storage in response to the decoding processing of the acquired video code stream to the video decoding Motion compensation or block copy is performed after the internal buffer of the video decoder, or motion compensation or block copy is performed after the reference video block is read from the external storage of the video decoder, that is, part of the video data is stored outside the video decoder, The use of low-power external storage reduces the use of the internal buffer of the video decoder, and also reduces the bandwidth of the main storage, thereby reducing the power consumption of electronic equipment.

In one embodiment, the second reading module 1502 is configured to, in response to the decoding processing of the acquired video code stream, search from the external storage of the video decoder, and find the reference video block in the external storage. In this case, motion compensation or block copying is performed after reading the reference video block from the external storage;

The third reading module 1504 is used to find the reference video block from the external main storage of the video decoder when the reference video block is not found in the external storage, and read the reference video from the main storage Motion compensation or block copying is performed after the block is stored in the video decoder's internal buffer.

In this embodiment, when the reference video block is found from the external storage of the video decoder, the reference video block is read from the external storage; when the reference video block is not found in the external storage , find the reference video block from the main storage again, find the reference video block from the main storage, read the reference video block from the main storage to the internal buffer of the video decoder and perform motion compensation; then the video decoding The internal buffer of the decoder only reads the data from the main storage. Compared with the traditional video decoder, the internal buffer is consistent with the data stored in the external storage, which can save unnecessary buffering operations inside the video decoder. , thereby reducing the power consumption of electronic equipment.

In one embodiment, the video data reading device further includes a storage module, and the storage module is configured to obtain motion vectors corresponding to each video frame in the video in response to the decoding processing of the acquired video; The motion vector determines the reference times of each video frame; the video frames whose reference times meet the condition of the first times are stored in the external main storage of the video decoder; the video frames whose reference times meet the conditions of the second times are stored to the external storage of the video decoder; wherein, the number of times represented by the second order condition is greater than the number of times represented by the first order condition.

In this embodiment, in response to the decoding processing of the acquired video, the motion vector corresponding to each video frame in the video is obtained, and the number of times each video frame is referenced is determined based on the motion vector of each video frame, then the video frame can be known. To be used several times; store the video frames that meet the condition of the first number of times to the main storage, and store the video frames that meet the condition of the second number of times to the external storage of the decoder, and are about to be referenced The video frames with a few times are stored in the main storage, and the video frames with a large number of references are stored in the external storage of the decoder; thereby reducing the use of the internal buffer of the video decoder, using the low-power external storage, Thereby reducing the power consumption of the electronic device. In addition, because the internal buffer of the video decoder is limited, the video frames that are used less frequently are buffered in the video decoder, and these video frames in the external buffer can be updated faster, and the external buffer can be fully utilized to speed up the decoding. code processing. In one embodiment, the external storage of the video decoder includes at least one of a system cache, a system buffer. In this embodiment, by storing the video frames that meet the conditions of the second number of times in at least one of the system cache and the system buffer, the data read from the system cache and the system buffer is faster and consumes less power. Therefore, the internal storage of the decoder does not need to store the video data from the system cache and system buffer again, which reduces the power consumption of electronic equipment.

The division of each module in the above-mentioned video data storage device and video data reading device is only used for illustration. In other embodiments, the video data storage device and the video data reading device can be divided into different modules as required to complete the All or part of the functions of the above video data storage device and video data reading device.

For specific limitations on the video data storage device and the video data reading device, reference may be made to the limitations on the methods above, and details are not repeated here. Each module in the above-mentioned video data storage device and video data reading device may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

FIG. 16 is a schematic diagram of the internal structure of an electronic device in one embodiment. As shown in Figure 16, the electronic device includes a processor and memory connected by a system bus. Among them, the processor is used to provide computing and control capabilities to support the operation of the entire electronic device. The memory may include non-volatile storage media and internal memory. The nonvolatile storage medium stores an operating system and a computer program. The computer program can be executed by the processor to implement the video data storage method and the video data read method provided by the following embodiments. Internal memory provides a cached execution environment for operating system computer programs in non-volatile storage media. The electronic device may be any terminal device such as a mobile phone, a tablet computer, a PDA (Personal Digital Assistant), a POS (Point of Sales, a sales terminal), a vehicle-mounted computer, a wearable device, and the like.

The implementation of each module in the video data storage device and the video data reading device provided in the embodiments of the present application may be in the form of computer programs. The computer program can be run on a terminal or server. The program modules constituted by the computer program can be stored on the memory of the electronic device. When the computer program is executed by the processor, the operations of the methods described in the embodiments of the present application are implemented.

Embodiments of the present application also provide a computer-readable storage medium. One or more non-volatile computer-readable storage media containing computer-executable instructions, when executed by one or more processors, cause the processors to perform the operations of the video data storage method.

Embodiments of the present application also provide a computer-readable storage medium. One or more non-volatile computer-readable storage media containing computer-executable instructions that, when executed by one or more processors, cause the processors to perform the operations of the video data reading method .

A computer program product containing instructions, when executed on a computer, causes the computer to perform a video data storage method.

A computer program product containing instructions, when executed on a computer, causes the computer to perform a video data reading method.

Any reference to a memory, storage, database, or other medium as used herein may include non-volatile and/or volatile memory. Nonvolatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which acts as external cache memory. By way of illustration and not limitation, RAM is available in various forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous Link (Synchlink) DRAM (SLDRAM), Memory Bus (Rambus) Direct RAM (RDRAM), Direct Memory Bus Dynamic RAM (DRDRAM), and Memory Bus Dynamic RAM (RDRAM).

The above-mentioned embodiments only represent several embodiments of the present application, and the descriptions thereof are relatively specific and detailed, but should not be construed as a limitation on the scope of the patent of the present application. It should be pointed out that for those skilled in the art, without departing from the concept of the present application, several modifications and improvements can be made, which all belong to the protection scope of the present application. Therefore, the scope of protection of the patent of the present application shall be subject to the appended claims.

Claims (20)

1. A video data storage method, comprising:

   In response to the decoding process of the video frame to be reconstructed, the reference video frame of the video frame to be reconstructed does not exist in the external storage of the video coder, and the reference video frame of the reference video frame exists in the main storage external to the video coder. case, read the reference video frame from the main storage; and

   The reference video frames read from the main storage are stored in an internal buffer of the video coder.
2. The method according to claim 1, each frame of video frame is divided into a plurality of blocks;

   The reference video frame of the video frame to be reconstructed does not exist in the external storage of the video coder, and if the reference video frame exists in the main storage external to the video coder, from the main storage Read the reference video frame, including:

   In the case that the reference video block corresponding to the video frame to be reconstructed does not exist in the external storage of the video coder, and the reference video block exists in the main storage external to the video coder, from the main storage reading the reference video block in storage; and

   The storing of the reference video frame read from the main storage into the internal buffer of the video decoder includes:

   The reference video block read from the main storage is stored in an internal buffer of the video coder.
3. The method of claim 1, further comprising:

   The reference video frame is read from the video coder's external storage if the reference video frame exists in the video coder's external storage.
4. The method of claim 1, further comprising:

   In response to the decoding process of the acquired video, obtain the motion vector corresponding to each video frame in the video;

   Determine the reference times of each video frame based on the motion vector corresponding to each video frame;

   storing the video frame whose referenced number of times satisfies the condition of the first number of times in a main storage outside the video decoder; and

   storing the video frame whose referenced number of times meets the second number of times condition in the external storage of the video decoder, wherein the number of times represented by the second number of times condition is greater than the number of times represented by the first number of times condition .
5. 4. The method of any of claims 1 to 4, the external storage of the video coder comprising at least one of a system cache or a system buffer.
6. 4. The method of any of claims 1 to 4, the main storage external to the video coder comprising dynamic random access memory.
7. A method for reading video data, the method comprising:

   In response to the decoding process of the acquired video code stream, read the reference video block that does not exist in the external storage of the video decoder from the external main storage of the video decoder to the internal buffer of the video decoder. Perform motion compensation or block copying;

   Alternatively, motion compensation or block copying is performed after the reference video block is read from the external storage of the video decoder.
8. The method according to claim 7, wherein in response to the decoding process of the acquired video code stream, a reference video that does not exist in the external storage of the video decoder is read from a main storage outside the video decoder Motion compensation or block copy is performed after the block is stored in the internal buffer of the video decoder; or motion compensation or block copy is performed after the reference video block is read from the external storage of the video decoder, including:

   In response to the decoding processing of the acquired video code stream, look up from the external storage of the video decoder, and in the case of finding the reference video block in the external storage, read from the external storage performing motion compensation or block copying after taking the reference video block; and

   In the case where the reference video block is not found in the external storage, the reference video block is found from the external main storage of the video decoder, and the reference video block is read from the main storage to The video coder's internal buffering is followed by motion compensation or block copying.
9. The method of claim 7, further comprising:

   In response to the decoding process of the acquired video, obtain the motion vector corresponding to each video frame in the video;

   Determine the reference times of each video frame based on the motion vector corresponding to each video frame;

   storing the video frame whose referenced number of times satisfies the condition of the first number of times in a main storage outside the video decoder; and

   storing the video frame whose referenced number of times meets the second number of times condition in the external storage of the video decoder, wherein the number of times represented by the second number of times condition is greater than the number of times represented by the first number of times condition .
10. 9. The method of any one of claims 7 to 9, the external storage of the video coder comprising at least one of a system cache or a system buffer.
11. 9. The method of any of claims 7 to 9, the main storage external to the video coder comprising dynamic random access memory.
12. A video data storage device, comprising:

    A first reading module, for responding to the decoding process of the video frame to be reconstructed, the reference video frame of the video frame to be reconstructed does not exist in the external storage of the video decoder, and the main storage outside the video decoder does not exist read the reference video frame from the main storage if the reference video frame exists in the ; and

    A reference video frame storage module, configured to store the reference video frame read from the main storage into an internal buffer of the video decoder.
13. The device according to claim 12, each frame of video frame is divided into a plurality of blocks;

    The first reading module is also used for the reference video block corresponding to the video frame to be reconstructed does not exist in the external storage of the video decoder, and the reference video exists in the main storage outside the video decoder In the case of a block, the reference video block is read from the main storage; and

    The reference video frame storage module is further configured to store the reference video block read from the main storage into an internal buffer of the video decoder.
14. 13. The apparatus of claim 12, wherein the first read module is further configured to read from the external storage of the video coder if the reference video frame exists in the external storage of the video coder Take the reference video frame.
15. The apparatus according to claim 12, further comprising a motion vector acquisition module, a number of times determination module, a first storage module and a second storage module, wherein:

    The motion vector obtaining module is configured to obtain the motion vector corresponding to each video frame in the video in response to the decoding processing of the obtained video;

    The times determining module is configured to determine the number of times that each video frame is referenced based on the motion vector corresponding to each video frame;

    The first storage module is configured to store the video frames whose referenced times meet the condition of the first number of times in the external main storage of the video decoder; and

    The second storage module is configured to store the video frames whose referenced times satisfy the condition of the second number of times in the external storage of the video decoder, wherein the number of times represented by the condition of the second number of times is greater than that of the first The number of times represented by the one-count condition.
16. A video data reading device, comprising:

    The second reading module is configured to, in response to the decoding processing of the acquired video code stream, read the reference video blocks that do not exist in the external storage of the video decoder from the external main storage of the video decoder to Motion compensation or block copying after the video decoder's internal buffering;

    or,

    The third reading module is used to perform motion compensation or block copy after reading the reference video block from the external storage of the video decoder.
17. The apparatus according to claim 16, wherein the second reading module is further configured to search from an external storage of the video decoder in response to the decoding processing of the acquired video code stream, and store the data in the external storage. Under the situation that the reference video block is found in the middle, then motion compensation or block copying is performed after reading the reference video block from the external storage;

    The third reading module is also used to find the reference video block from the external main storage of the video decoder when the reference video block is not found in the external storage, and from the main storage Motion compensation or block copy is performed after reading the reference video block into the internal buffer of the video decoder.
18. The device according to claim 16, further comprising a storage module; the storage module is configured to, in response to decoding processing of the acquired video, obtain a motion vector corresponding to each video frame in the video; The motion vector corresponding to the video frame determines the number of times that each video frame is referenced; the video frame whose number of times to be referenced satisfies the condition of the first number of times is stored in the external main storage of the video decoder; The video frames whose number of times satisfies the second number of times condition are stored in the external storage of the video decoder, wherein the number of times represented by the second number of times condition is greater than the number of times represented by the first number of times condition.
19. An electronic device, comprising a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor is made to execute the process according to any one of claims 1 to 11 method of operation.
20. A computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the operation of the method of any one of claims 1 to 11.

Images