WO2022206212A1 - Procédé et appareil de stockage de données vidéo, dispositif électronique et support de stockage lisible - Google Patents

Procédé et appareil de stockage de données vidéo, dispositif électronique et support de stockage lisible Download PDF

Info

Publication number
WO2022206212A1
WO2022206212A1 PCT/CN2022/077080 CN2022077080W WO2022206212A1 WO 2022206212 A1 WO2022206212 A1 WO 2022206212A1 CN 2022077080 W CN2022077080 W CN 2022077080W WO 2022206212 A1 WO2022206212 A1 WO 2022206212A1
Authority
WO
WIPO (PCT)
Prior art keywords
video
video frame
storage
block
decoder
Prior art date
Application number
PCT/CN2022/077080
Other languages
English (en)
Chinese (zh)
Inventor
赵娟萍
Original Assignee
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oppo广东移动通信有限公司 filed Critical Oppo广东移动通信有限公司
Publication of WO2022206212A1 publication Critical patent/WO2022206212A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H04N19/137—Motion inside a coding unit, e.g. average field, frame or block difference
    • H04N19/139—Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/156—Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H04N19/423—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation characterised by memory arrangements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H04N19/43—Hardware specially adapted for motion estimation or compensation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H04N19/43—Hardware specially adapted for motion estimation or compensation
    • H04N19/433—Hardware specially adapted for motion estimation or compensation characterised by techniques for memory access

Definitions

  • the present application relates to the technical field of video processing, and in particular, to a video data storage method, apparatus, electronic device, and computer-readable storage medium.
  • Various embodiments of the present application provide a video data storage method, apparatus, electronic device, and computer-readable storage medium.
  • a video data storage method comprising:
  • the reference video frame of the video frame to be reconstructed does not exist in the external storage of the video coder, and the reference video exists in the main storage external to the video coder frame, reading the reference video frame from the main storage;
  • the reference video frame in main storage is stored in an internal buffer of the video coder.
  • a method for reading video data comprising:
  • motion compensation or block copying is performed after the reference video block is read from the video decoder's external storage.
  • a video data storage device comprising:
  • a reference video frame storage module configured to store the reference video frame read from the main storage into an internal buffer of the video decoder.
  • a video data reading device comprising:
  • the second reading module is configured to, in response to the decoding processing of the acquired video code stream, read the reference video blocks that do not exist in the external storage of the video decoder from the external main storage of the video decoder to Motion compensation or block copying after the video decoder's internal buffering;
  • the third reading module is used for performing motion compensation or block copy after reading the reference video block from the external storage of the video decoder.
  • An electronic device includes a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the operations of each method.
  • a computer-readable storage medium having a computer program stored thereon, the computer program implementing the operations of each method when executed by a processor.
  • the above-mentioned video data storage method, apparatus, electronic device and readable storage medium in response to the decoding process of the video frame to be reconstructed, there is no reference video frame of the video frame to be reconstructed in the external storage of the video decoder, and in the video frame.
  • the reference video frame is read from the main storage, and the reference video frame read from the main storage is stored in the internal buffer of the video decoder;
  • the video data existing in the external storage of the video decoder is not stored in the internal buffer of the video decoder, and the video data in the main storage is stored in the internal buffer of the video decoder.
  • the decoder Compared with the traditional video data, whether it is located in the external storage or the main storage, the decoder will put it into the internal buffer of the decoder.
  • the internal buffer of the decoder reduces the use of the internal buffer of the video decoder, increases the efficiency of the internal buffer of the decoder, and further reduces the power consumption of the data that is only left in the main storage.
  • FIG. 1 is a schematic structural diagram of a conventional video data storage method in an embodiment.
  • Figure 2 is a conventional video decoder data processing architecture in one embodiment.
  • FIG. 3 is a schematic structural diagram of a conventional video data storage method in another embodiment.
  • FIG. 4 is a schematic diagram of a curve of system power consumption in a conventional manner in one embodiment.
  • FIG. 5 is a flowchart of a video data storage method in one embodiment.
  • FIG. 6 is a schematic structural diagram of video data storage in one embodiment.
  • FIG. 7 is a schematic diagram of an architecture using a system cache in one embodiment.
  • FIG. 8 is a schematic diagram of an architecture using a system buffer in one embodiment.
  • FIG. 9 is a schematic diagram of a decoding process in an embodiment.
  • FIG. 10 is a flowchart of a method for reading video data in one embodiment.
  • FIG. 11 is a schematic flowchart of reconstructing a video frame in one embodiment.
  • FIG. 12 is a schematic diagram of a storage method of internal storage of a conventional video decoder in an embodiment.
  • FIG. 13 is a schematic diagram of a storage method of internal storage of a video decoder in an embodiment.
  • FIG. 14 is a structural block diagram of a video data storage device according to an embodiment.
  • FIG. 15 is a structural block diagram of a video data reading apparatus according to an embodiment.
  • FIG. 16 is a schematic diagram of the internal structure of an electronic device in one embodiment.
  • first, second, etc. used in this application may be used herein to describe various data, but these data are not limited by these terms. These terms are only used to distinguish the first data from another.
  • first order condition may be referred to as the second order condition
  • second order condition may be referred to as the first order condition.
  • Both the first degree condition and the second degree condition are degree conditions, but they are not the same degree condition.
  • FIG. 1 it is a schematic structural diagram of a traditional video data storage manner in one embodiment.
  • Figure 1 includes a central processing unit (Central Processing Unit, CPU), a video decoder (Video Decoder, VDEC), a display processor (Display Processing Unit) and a dynamic random access memory controller (DRAM Controller, DRAMC).
  • the central processing unit includes a master port (master)
  • the video decoder includes an internal cache (cache) and a master port (master)
  • the display processor includes a master port (master)
  • the dynamic random access memory controller includes Including the slave port (slave).
  • Arrows in FIG. 1 represent video data.
  • the video decoder can obtain the video data from the dynamic random access memory controller for video decoding, reconstruct the video frame, and transmit the reconstructed video frame to the display processor for display.
  • FIG. 2 is a conventional video decoder data processing architecture in one embodiment.
  • Temporal data such as Temporal Motion Vector (TMV).
  • TMV Temporal Motion Vector
  • FIG. 3 is a schematic structural diagram of a conventional video data storage method in another embodiment.
  • the figure includes a central processing unit, a video decoder, a display processor and a multi-channel dynamic random access memory controller (Multi-channel DramCtrl).
  • the multi-channel dynamic random access memory controller can be used to control multiple dynamic random access memories ( DRAM).
  • FIG. 4 is a schematic diagram of a curve of system power consumption in a conventional manner in one embodiment.
  • the abscissa is the position in the image (Picture Position), and the ordinate is the system power.
  • the dotted line represents the System Power Barrier.
  • the system DRAM consumes relatively high system power in order to meet the operating speed requirements of the video decoder.
  • the speed of the video decoder cannot meet the demand or the system overheats due to the limited power consumption that the system can provide.
  • the traditional approach would result in higher power consumption for the video decoder when it finishes at the expected time.
  • an embodiment of the present application provides a video data storage method.
  • the reference video frame and if there is a reference video frame in the external main storage of the video decoder, read the reference video frame from the main storage; store the reference video frame read from the main storage to the video decoder In the internal buffer of the decoder; compared with the traditional video data, whether it is located in the external storage or the main storage, the decoder will put it into the internal buffer of the decoder.
  • the video data stored in the main storage is put into the internal buffer of the decoder, which reduces the use of the internal buffer of the video decoder and increases the efficiency of the internal buffer of the decoder. Power consumption is reduced.
  • FIG. 5 is a flowchart of a video data storage method in one embodiment.
  • the video data storage method in this embodiment is described by taking the operation on an electronic device as an example. As shown in FIG. 5 , the video data storage method includes operations 502 to 504 .
  • Operation 502 in response to the decoding process of the video frame to be reconstructed, the reference video frame of the video frame to be reconstructed does not exist in the external storage of the video coder, and the reference video frame exists in the main storage external to the video coder , the reference video frame is read from main storage.
  • the video frame to be reconstructed refers to a video frame that has not been successfully constructed yet. After the video is compressed, if the video frame needs to be obtained, it needs to be decoded.
  • the video coder's external storage refers to storage located in the electronic device and other than the main storage and the video coder's internal cache.
  • the external storage of the video decoder may be at least one of a system cache and a system buffer.
  • Main memory refers to the main memory (Main memory), referred to as the main memory.
  • the main storage is an important part of computer hardware, its function is to store instructions and data, and can be directly accessed randomly by the central processing unit (CPU).
  • the main storage may specifically be DRAM (Dynamic Random Access Memory, dynamic random access memory).
  • the reference video frame can be used to reconstruct the target video frame corresponding to the video frame to be reconstructed.
  • the reference video frame may be a video frame displayed before or after the video frame to be reconstructed.
  • the reference video frame is specifically the video frame that has been reconstructed before the video frame to be reconstructed, or the reference video frame can also be the reconstructed part of the video frame to be reconstructed itself, such as for intra-block copying in the screen content coding technology. copy) of the block vector (block vector) to distinguish the range of the picture to be accessed.
  • the specification of the reference picture range allowed in the screen content coding technology has been described in detail in various video standards, and will not be repeated here.
  • the electronic device acquires the target motion vector or block vector corresponding to the to-be-reconstructed video frame. It is detected whether a reference video frame of the video frame to be reconstructed exists in the external storage of the video coder. In the case that the reference video frame corresponding to the target motion vector or block vector does not exist in the external storage of the video decoder, and the reference video frame exists in the main storage external to the video decoder, the electronic device removes the The reference video frame is read, and the read reference video frame is stored in the internal buffer of the video decoder.
  • the reference video frame read from the main storage is stored into an internal buffer of the video decoder.
  • the electronic device stores the reference video frames read from the main storage into the internal buffer of the video decoder.
  • the reference video frame of the video frame to be reconstructed in response to the decoding process of the video frame to be reconstructed, does not exist in the external storage of the video decoder, and the reference video frame of the video frame to be reconstructed does not exist in the external storage of the video decoder.
  • the reference video frame is read from the main storage, and the reference video frame read from the main storage is stored in the internal buffer of the video decoder, that is, in the video decoder's internal buffer.
  • the video data existing in the external storage is not stored in the internal buffer of the video decoder, and the video data in the main storage will be stored in the internal buffer of the video decoder.
  • the decoder Compared with the traditional video data, whether it is located in the external storage or the main storage, the decoder will put it into the internal buffer of the decoder.
  • the internal buffer of the decoder reduces the use of the internal buffer of the video decoder, increases the efficiency of the internal buffer of the decoder, and further reduces the power consumption of the data that is only left in the main storage.
  • FIG. 6 it is a schematic diagram of the architecture of video data storage in one embodiment.
  • the white part indicates that it is stored in DRAM, that is, this part of the data will be stored in the internal buffer of the video decoder later, and the gray part indicates that it is stored in the external storage of the video decoder.
  • some of the video bit stream, video frame data, and temporary data in FIG. 6 are stored in the DRAM, and some are stored in the external storage of the video decoder.
  • each video frame is divided into multiple blocks.
  • the reference video frame of the video frame to be reconstructed does not exist in the external storage of the video decoder, and if the reference video frame exists in the main storage external to the video decoder, the reference video frame is read from the main storage, include:
  • the reference video block corresponding to the video frame to be reconstructed does not exist in the external storage of the video decoder, and if the reference video block exists in the external main storage of the video decoder, read from the main storage Refer to the video block.
  • the storing of the reference video frames read from the main storage into the internal buffer of the video decoder includes: storing the reference video blocks read from the main storage into the internal buffer of the video decoder.
  • the block refers to a part of the video frame.
  • the size of the block may be 16 ⁇ 16 pixels, 64 ⁇ 64 pixels, etc., but not limited thereto.
  • the electronic device divides each video frame in the video into a plurality of blocks, and obtains motion vectors corresponding to each block.
  • the electronic device determines, based on the motion vector of the current block in the current video frame, the reference video block in the reference video frame referenced by the video frame to be reconstructed.
  • the electronic device retrieves the data from the main storage. Read the reference video block.
  • the electronic device stores the reference video block read from the main storage into an internal buffer of the video decoder.
  • the electronic The device reads the reference video block from the main storage and stores it in the internal buffer of the video decoder. Since the internal buffer in the video decoder is small, the storage block method is more suitable for the internal buffer in the video decoder. Cache to improve the accuracy of video data storage.
  • the video data storage method further comprises: if the reference video frame exists in the external storage of the video decoder, reading the reference video frame from the external storage of the video decoder.
  • the electronic device reads the reference video frame from the external storage of the video decoder, and based on the reading from the external storage of the video decoder
  • the target video frame corresponding to the to-be-reconstructed video frame is obtained by reconstructing the reference video frame of .
  • the reference video frame when there is a reference video frame in the external storage of the video decoder, the reference video frame is read from the external storage of the video decoder, and data access is performed through the external storage of the video decoder,
  • the efficiency of the internal cache of the decoder can be increased, and further power consumption can be reduced specifically for the data that is still only in the main storage.
  • the reference video frame of the video frame to be reconstructed in response to the decoding process of the video frame to be reconstructed, does not exist in the external storage of the video coder and in the main storage external to the video coder In the presence of a reference video frame, before reading the reference video frame from the main storage, the video data storage method further includes:
  • a motion vector corresponding to each video frame in the video is obtained in response to the decoding process of the obtained video.
  • the types of videos are not limited. For example, it may be a downloaded video stored locally on the electronic device, or it may be a video obtained from a server.
  • the video may be a TV series video, a movie video, a TV series video, and the like.
  • the length of the video is not limited, for example, it can be 10 seconds, 1 minute, 10 minutes, etc. without limitation.
  • the decoding process refers to the process of decoding and restoring the encoded video to reconstruct the video frame.
  • the decoding process may also be referred to as a decoding process.
  • Motion Vector (MV) represents the relative displacement between the block of the current video frame and its matching block in the reference image.
  • a block vector (Block Vector, BV) represents the relative displacement between a block of the current video frame and its reconstructed matching block in the current image frame.
  • the electronic device in response to the decoding process of the acquired video, obtains the motion vector corresponding to each video frame in the video through a video decoder. It can be understood that, in response to the decoding processing of the acquired video, the electronic device performs decoding processing on some video frames in the video to obtain the motion vector corresponding to each video frame. For example, a video includes 100 video frames, and the electronic device may perform a decoding process on the first 50 video frames of the 100 video frames.
  • the reference times of each video frame is determined based on the motion vector corresponding to each video frame.
  • the number of times of being referenced refers to the number of times that the video frame is referenced by other video frames during reconstruction of the video frame.
  • the video frame A is referenced by the video frame B to be reconstructed, the video frame C to be reconstructed...
  • the video frame Z to be reconstructed is referenced 26 times in total.
  • the electronic device determines the reference video frame corresponding to each video frame based on the motion vector corresponding to each video frame in the video, and counts the number of times the reference video frame is referenced.
  • video frame X is referenced by video frame Y and video frame Z
  • video frame X is used as the reference video frame of video frame Y and video frame Z
  • video frame X is The number of references is 2 times.
  • the video frames whose reference times satisfy the condition of the first number of times are stored in the external main storage of the video decoder.
  • condition of the first number of times can be adjusted as required.
  • the number of times represented by the first degree condition is smaller than the number of times represented by the second degree condition. For example, satisfying the first degree condition is equal to 1.
  • the electronic device stores the video frames whose reference times satisfy the condition of the first number of times into the main storage outside the video decoder.
  • Subsequent video coders may read video frames that satisfy the first number of conditions from main storage and store them in an internal buffer of the video coder.
  • the video frames whose reference times satisfy the second number condition are stored in the external storage of the video decoder; wherein the number represented by the second number condition is greater than the number represented by the first number condition.
  • the second order condition can be adjusted as required.
  • the number of times represented by the second degree condition is greater than the number of times represented by the first degree condition. For example, if the number of times to be referenced satisfies the first number of times condition, the number of times to be referenced satisfies 1, and the number of times to be referenced satisfies the second condition is that the number of times to be referenced satisfies more than 1.
  • the number of times to be referenced satisfies the condition of the first number of times, the number of times to be referenced is less than or equal to 2, and the number of times to be referenced satisfies the condition of the second number of times is that the number of times to be referenced is greater than 2, and so on.
  • the electronic device stores the video frames whose reference times satisfy the condition of the second number of times into the external storage of the video decoder.
  • External storage of the video coder refers to storage located in the electronic device but not in the video coder.
  • the external storage of the video decoder may be an on-chip memory.
  • the external storage of the video decoder may be at least one of a system cache and a system buffer.
  • the motion vector corresponding to each video frame in the video is obtained, and the reference times of each video frame is determined based on the motion vector of each video frame, then It can be known how many times the video frame is to be used; the video frame whose number of references meets the condition of the first number of times is stored in the internal buffer of the video decoder, and the video frame whose number of times of reference meets the condition of the second number of times is stored in the video
  • the video frames that have been referenced less frequently are stored in the internal buffer of the video decoder, and the video frames that have been referenced more frequently are stored in the external storage of the video decoder.
  • the use of the decoder's internal cache utilizes low-power external storage, thereby reducing the power consumption of electronic equipment.
  • the internal buffer of the video decoder is limited, the video frames that are used less frequently are buffered in the video decoder, and these video frames in the external buffer can be updated faster, and the external buffer can be fully utilized to speed up the decoding. code processing.
  • the external storage of the video coder includes at least one of a system cache or a system buffer.
  • the system cache (system cache) and the system cache area (system buffer) both belong to the on-chip memory (on-chip memory). And both the system cache and the system cache area belong to static random-access memory (Static Random-Access Memory, SRAM).
  • SRAM static random-access memory
  • determining the reference times of each video frame based on the motion vector corresponding to each video frame includes: determining the reference video frame referenced by the current video frame based on the motion vector of the current video frame; number of references.
  • the electronic device performs decoding processing on the acquired video, and firstly decodes to obtain the motion vector of the video frame.
  • the motion vector of the video frame is used to represent the displacement between the current video frame and the reference video frame.
  • the electronic device determines the reference video frame referenced by the current video frame based on the motion vector of the current frame, and counts the reference times of the reference video frame, then the reference times of each video frame can be obtained.
  • the current video frame is video frame C
  • the reference video frames referenced by video frame C are video frame A and video frame B as an example for description.
  • the reference times of the video frame A is 1 at this time
  • the reference times of the video frame B is also 1.
  • the reference times of video frame A is 2, and the reference times of video frame B is also is 2, which can obtain the reference times of each video frame in the video.
  • the reference video frame referenced by the current video frame is determined based on the motion vector of the current video frame, and the number of times the reference video frame is referenced is counted, that is, the current video frame and the motion vector are known. , the reference video frame referenced by the current video frame can be obtained, and the reference times of each video frame in the video can be obtained, and the video data will be stored based on the reference times.
  • the storage or the main storage will be put into the internal buffer of the decoder by the decoder. In this embodiment, only the video data that does not exist in the external storage and exists in the main storage needs to be put into the internal buffer of the decoder, reducing the number of video data.
  • the use of the internal buffer of the decoder increases the efficiency of the internal buffer of the decoder, and further reduces the power consumption especially for the data still only in the main storage.
  • the video data storage method further includes: when the number of video frames between the current video frame and the target reference video frame reaches a preset number, stopping counting the number of times the target reference video frame is referenced, and obtaining the The number of times the target reference video frame has been referenced.
  • the preset number can be set according to actual needs.
  • the preset number is used to characterize the number of maximum interval frames between the video frame and the reference video frame. Since there are generally not too many frames between the current video frame and the referenced video frame, when the number of video frames between the current video frame and the target reference video frame reaches the preset number, the video frames after the current video frame The frame generally does not refer to the target reference video frame, so the reference times of the target reference video frame can be directly obtained.
  • each frame of video frame is divided into multiple blocks; determining the reference video frame referenced by the current video frame based on the motion vector of the current video frame, including: based on the motion vector of the current block in the current video frame , to determine the reference video block in the reference video frame referenced by the current video frame.
  • Counting the reference times of the reference video frame includes: counting the reference times of the reference video block in the reference video frame.
  • the video frame that satisfies the condition of the first number of times is stored in the external main storage of the video decoder, including: the reference video block that satisfies the condition of the first number of times is stored in the video decoder. in external primary storage.
  • Storing the video frames that meet the condition of the second number of times to be referenced in the external storage of the video decoder includes: storing the reference video blocks that meet the condition of the second number of times to be referenced in the external storage of the video decoder.
  • the block refers to a part of the video frame.
  • the size of the block may be 16 ⁇ 16 pixels, 64 ⁇ 64 pixels, etc., but not limited thereto.
  • the electronic device divides each video frame in the video into a plurality of blocks, and obtains motion vectors corresponding to each block.
  • the electronic device determines, based on the motion vector of the current block in the current video frame, the reference video block in the reference video frame referenced by the current video frame.
  • the electronic device counts the number of times the reference video block is referenced.
  • the electronic device stores the reference video block whose number of references meets the condition of the first number of times in the internal buffer of the video decoder.
  • the electronic device stores the reference video blocks whose number of references meets the condition of the second number of times in the external storage of the video decoder.
  • the reference video block to be referenced is determined based on the motion vector of the current block, the reference times of the reference video block are counted, and the reference video blocks whose reference times satisfy the condition of the first times are stored in the In the main storage, the reference video block that satisfies the condition of the second number of times by reference is stored in the external storage of the video decoder, then the reference video block referenced can be determined based on the current block, and the stored video data is more accurate , since the internal buffer in the video decoder is small, the storage block method is more suitable for the internal buffer in the video decoder.
  • FIG. 7 it is a schematic diagram of the architecture of using a system cache in one embodiment.
  • These include the central processing unit, video decoder, display processor, dynamic random access memory controller and system cache.
  • the system cache can obtain data from DRAMC.
  • the system cache size in Figure 7 is 4MB (megabytes).
  • the system cache is a kind of SRAM (Static Random-Access Memory).
  • the motion vector corresponding to each video frame in the video is obtained by the video decoder, and the reference times of each video frame is determined based on the motion vector corresponding to each video frame;
  • the video frames whose reference times meet the conditions of the first times are read from DRAM and stored in the internal buffer of the video decoder; some video frames whose reference times meet the conditions of the second times are read from DRAM and stored in the system cache.
  • the system cache may not be directly connected to the DRAMC, and FIG. 6 is just an example.
  • FIG. 8 it is a schematic diagram of the architecture of using the system buffer area in one embodiment.
  • These include the central processing unit, video decoder, display processor, dynamic random access memory controller, and system buffer.
  • the system cache size in Figure 7 is 4MB (megabytes).
  • the system buffer is a type of SRAM (Static Random-Access Memory).
  • the motion vector corresponding to each video frame in the video is obtained by the video decoder, and the reference times of each video frame is determined based on the motion vector corresponding to each video frame;
  • the video frames whose reference times meet the conditions of the first times are read from DRAM and stored in the internal buffer of the video decoder; the video frames whose reference times meet the conditions of the second times are directly stored in the system cache.
  • the video data storage method further includes: in the case that the internal buffer in the video decoder is full and a new video frame is stored, deleting the least important video frame in the internal buffer.
  • the electronic device deletes the least important video frame in the internal buffer, so as to ensure that the video decoder can always Store data to ensure the operation of the video decoder.
  • Least important can be the oldest video frame, or it can be judged in other ways.
  • FIG. 9 it is a schematic diagram of a decoding process in one embodiment.
  • the figure includes I-frames, B-frames and P-frames.
  • the I frame belongs to intra-frame compression, and only the information of the I frame itself needs to be used during decoding.
  • the P frame is a forward predictive coded frame, that is, when the P frame is decoded, it is necessary to refer to the information of the previous relevant frame before decoding.
  • the B frame is a bidirectional predictive coding frame, and it is necessary to refer to both the previous frame and the frame to be decoded later when decoding. It will be appreciated that some video standards allow both reference objects of a B-frame to be previous frames.
  • the display order is the order from left to right as shown in FIG. 9 .
  • the decoding order (Decoding Order) is the order of 0, 3, 2, 4, 1, 7, 6, 8, and 5 in FIG. 9 .
  • the decoding order in FIG. 9 is not the same as the display order.
  • the arrows indicate references, such as the B frame of display order 2, which is reconstructed by referring to the I frame of display order 0 and the P frame of display order 1, and the B frame of display order 2 can be used as display order 1 and display order. 3 frames of reference video frames. From this, video frames of the video can be reconstructed.
  • the blank part in the figure indicates that it is stored in DRAM, that is, it will be stored in the internal buffer of the video decoder later.
  • Parts in gray indicate storage in the video decoder's external storage. Then the I frame of display order 0, the B frame of display order 2, the B frame of display order 3, the P frame of display order 4, the B frame of display order 6 and the B frame of display order 8 all have partial blocks stored in the video. In the external storage of the decoder, another part is stored in the DRAM or the internal buffer of the video decoder. The B frames of display order 1, the B frames of display order 5, and the B frames of display order 7 are all stored in the DRAM.
  • the decoder After performing some analysis on the video, it is known which frames or which blocks are not suitable for storing in DRAM, or in other words, not suitable for storing in the internal cache of the video decoder, then it is suitable for storing in the video decoder.
  • the external storage outside the device such as at least one of the system cache and the system buffer.
  • the decoder will make proper use of the internal buffer of the decoder, so that the data that only exists in the external main storage of the video decoder can survive for a longer time in the internal buffer of the decoder, so as to reduce the cost of System bandwidth to achieve the ultimate power saving purpose.
  • the basic concept of the embodiments of the present application is to make proper use of each available on-chip memory (on-chip memory, that is, the external storage and internal storage of the video decoder) to reduce the amount of DRAM access.
  • the internal buffering strategy of the video decoder is Changes to be made when assisted by other on-chip memory. Store the entire reference video frame in the video or a partial block of a reference video frame, and store the data that will be reused relatively many times in the low-power storage space such as the system cache or system buffer, and the rest only needs to be The data that cannot be put into the cache or the system buffer will be transferred to the internal buffer of the video decoder for processing.
  • a video data storage method comprising:
  • the reference video block in the reference video frame referenced by the current video frame is determined based on the motion vector of the current block in the current video frame.
  • the reference video blocks whose reference times satisfy the condition of the first times are stored in the external main storage of the video decoder.
  • the reference video blocks whose reference times satisfy the condition of the second times are stored in the external storage of the video decoder.
  • the number of times represented by the second order condition is greater than the number of times represented by the first order condition.
  • the reference video block corresponding to the video frame to be reconstructed does not exist in the external storage of the video decoder, and the main storage outside the video decoder does not exist If there is a reference video block in the , the reference video block is read from the main storage.
  • the external storage of the video decoder includes at least one of a system cache or a system buffer.
  • the read reference video block is stored in the internal buffer of the video decoder.
  • the motion vector corresponding to each video frame in the video is obtained, and the reference times of each video frame is determined based on the motion vector of each video frame, then It can be known that the video frame is to be used several times; the video frame that meets the condition of the first number of times is stored in the internal buffer of the video decoder, and the video frame that meets the condition of the second number of times is stored in the decoder.
  • the video frames that have been referenced less frequently are stored in the internal buffer of the video decoder, and the video frames that have been referenced more frequently are stored in the external storage of the decoder.
  • the decoder will put it into the internal buffer of the decoder. In this embodiment, it is only necessary to put the video data that does not exist in the external storage and exists in the main storage into the decoder.
  • the cache reduces the use of the internal cache of the video decoder, increases the efficiency of the internal cache of the decoder, and further reduces power consumption for data that is still only in the main storage.
  • the internal buffer of the video decoder is limited and the processing speed of the decoding processing through the internal buffer is fast, the video frames that are used less frequently are buffered in the video decoder, and the speed at which these buffered video frames are updated can be improved. Faster, can fully utilize the buffer of the video coder to speed up the decoding process.
  • FIG. 10 is a flowchart of a method for reading video data in one embodiment.
  • the video data storage method in this embodiment is described by taking the operation on an electronic device as an example. As shown in FIG. 10 , the video data storage method includes operations 1002 to 1004 .
  • Operation 1002 in response to a decoding process on the acquired video code stream, read a reference video block that does not exist in the external storage of the video decoder from the main storage outside the video decoder to the video decoder's external storage. Motion compensation or block copying after internal caching.
  • the main storage refers to the main storage of the electronic device.
  • the main storage may be, for example, DRAM.
  • Motion Compensation also known as motion compensation, is a method to describe the difference between nearby frames (nearby means nearby in the coding relationship, and two frames may not be nearby in the playback order). Specifically, it describes the previous one. How each small block of the frame is moved to a certain position in the video frame to be reconstructed.
  • the electronic device acquires the target motion vector of the video frame to be reconstructed; the electronic device reads from the external main storage of the video decoder according to the target motion vector, which does not exist in the video decoder.
  • the reference video block stored outside the video decoder is stored in the internal buffer of the video decoder to perform motion compensation; the reference video block is the block of the video frame referenced by the video frame to be reconstructed.
  • the electronic device obtains the block vector of the video frame to be reconstructed; according to the block vector, the reference video block that does not exist in the external storage of the video decoder is read from the main storage to the internal buffer of the video decoder and then performs block copying.
  • the video frame to be reconstructed refers to a video frame that has not been successfully constructed yet.
  • the target motion vector of the video frame to be reconstructed represents the displacement between the current video frame and the reference video block.
  • the video data stored in the video decoder's internal buffer and the video decoder's external storage may or may not be the same.
  • the video contains 100 video frames, which are video frame 1, video frame 2, video frame 3...video frame 100 respectively.
  • the internal buffer of the video decoder includes video frame 1...video frame 50
  • the external storage of the video decoder includes video frame 51, video frame 52...video frame 100.
  • Operation 1004 or perform motion compensation or block copy after reading the reference video block from the external storage of the video decoder.
  • the electronic device performs motion compensation after reading the reference video block from the external storage of the video decoder.
  • the electronic device reads the reference video block from the external storage of the video decoder, and reconstructs the target video frame based on the read reference video block. For example, the electronic device performs weighted fusion reconstruction on the read reference video block to obtain the target video frame.
  • the electronic device performs block copy after reading the reference video block from the external storage of the video decoder.
  • the reference video block that does not exist in the external storage of the video decoder is read from the main storage to the video decoding.
  • Motion compensation or block copy is performed after the internal buffer of the video decoder, or motion compensation or block copy is performed after the reference video block is read from the external storage of the video decoder, that is, part of the video can be read from the outside of the video decoder.
  • Data, using low-power external storage also reduces the use of the video decoder's internal buffer to reduce the power consumption of electronic equipment.
  • a reference video block that does not exist in the external storage of the video coder is read from the main storage external to the video coder to the video coder.
  • the motion compensation is performed after the internal buffer of the video decoder; or the motion compensation or block copy is performed after reading the reference video block from the external storage of the video decoder, including:
  • the reference video block is found from the external main storage of the video decoder, and the reference video block is read from the main storage to the interior of the video decoder Motion compensation or block copying after caching.
  • the electronic device searches for the corresponding reference video block from the external storage of the video decoder according to the target motion vector of the video frame to be reconstructed.
  • motion compensation or block copying is performed after reading the reference video block corresponding to the video frame to be reconstructed from the external storage.
  • the electronic device finds the reference video block corresponding to the video frame to be reconstructed from the main storage, and reads the reference video from the main storage Motion compensation or block copying is performed after the block is stored in the video decoder's internal buffer.
  • the reference video block is a part of the video area in the reference video frame.
  • the number of reference video blocks corresponding to the video frame to be reconstructed is not limited. For example, it may be 1, 2, 3... but not limited thereto.
  • the electronic device searches for the corresponding reference video block from the system cache according to the target motion vector of the video frame to be reconstructed.
  • the system cache finds the corresponding reference video block
  • the reference video block corresponding to the video frame to be reconstructed is read from the system cache. If the reference video block corresponding to the video frame to be reconstructed is not found in the system cache, the reference video block corresponding to the video frame to be reconstructed is found from the DRAM, and the reference video is read from the main storage Motion compensation is performed after the block is sent to the VDEC cache.
  • the reference video blocks corresponding to the remaining video frames to be reconstructed are searched from the main storage, and the reference video blocks corresponding to the remaining video frames to be reconstructed are searched from the main storage.
  • the reference video block is read from the storage to the internal buffer of the video decoder for motion compensation or block copying.
  • the reference video block when the reference video block is found from the external storage of the video decoder, motion compensation or block copy is performed after reading the reference video block from the external storage; the reference video block is not found in the external storage.
  • find the reference video block from the external main storage of the video decoder find the reference video block from the main storage, read the reference video block from the main storage to the video
  • the internal buffer of the decoder performs motion compensation or block copy; the internal buffer of the video decoder reads only the data from the main storage, compared with the data stored in the internal buffer of the traditional video decoder and the external storage.
  • the data consistency method can save unnecessary buffering operations inside the video decoder, and because the amount of video data buffered in the video decoder is small, the bandwidth of the main storage can be further reduced, thereby reducing the power consumption of electronic equipment.
  • the video data reading method further includes: in response to decoding processing of the acquired video, obtaining a motion vector corresponding to each video frame in the video; determining each video frame based on the motion vector corresponding to each video frame The number of times the frame is referenced; the video frame whose number of times to be referenced meets the condition of the first number of times is read from the external main storage of the video decoder and stored in the internal buffer of the video decoder; The video frame of the number of times is stored in the external storage of the video decoder; the number of times represented by the second-order condition is greater than the number of times represented by the first-time condition.
  • the motion vector corresponding to each video frame in the video is obtained, and the number of times each video frame is referenced is determined based on the motion vector of each video frame, then the video frame can be known.
  • the video frame can be known. To be used several times; store the video frames that meet the condition of the first number of times from the main storage to the internal buffer of the video decoder, and store the video frames that meet the condition of the second number of times to the decoder
  • the video frames that have been referenced less frequently are stored in the internal buffer of the video decoder, and the video frames that have been referenced many times are stored in the external storage of the decoder; the internal buffer of the video decoder is reduced.
  • the use of low-power external storage reduces the power consumption of electronic devices.
  • the internal buffer of the video decoder is limited, the video frames that are used less frequently are buffered in the video decoder, and these video frames in the external buffer can be updated faster, and the external buffer can be fully utilized to speed up the decoding. code processing.
  • the external storage of the video decoder includes at least one of a system cache, a system buffer.
  • the system cache (system cache) and the system cache area (system buffer) both belong to the on-chip memory (on-chip memory).
  • both the system cache and the system cache area belong to static random-access memory (Static Random-Access Memory, SRAM).
  • SRAM static random-access memory
  • FIG. 11 it is a schematic flowchart of reconstructing a video frame in one embodiment. It includes A video frame 1102, B video frame 1104, C video frame 1106, and to-be-reconstructed video frame 1108, and A video frame 1102, A video frame 1104, and A video frame 1106 are all video frames that have been reconstructed, and are to be reconstructed. Video frame 1108 has not yet been reconstructed. Among them, the white parts in the A video frame 1102, the A video frame 1104 and the A video frame 1106 are stored in the DRAM, that is, stored in the internal buffer of the video decoder; the blocks in the gray shaded part are stored in the video decoder. in the external storage of the encoder.
  • the A video frame 1102 , the B video frame 1104 and the C video frame 1106 are all reference video frames of the video frame 1108 to be reconstructed.
  • different blocks of a video frame can be stored in different stores.
  • the block of 16 ⁇ 16 and the video frame 1108 to be reconstructed as a B frame are taken as an example for description.
  • a small square in the video frame to be reconstructed represents a current block.
  • the reference blocks in Figure 11 are not reused.
  • the electronic device obtains the 1st, 2nd, 3rd, 4th, 5th, 6th and 7th sub-target motion vectors of the video frame to be reconstructed, and can determine the reference based on the sub-target motion vectors block.
  • the first block of the video frame 1108 to be reconstructed can be reconstructed from the 0 block in the A video frame 1102 and the 1 block in the A video frame 1104 .
  • the second block of the video frame 1108 to be reconstructed can be reconstructed from 2 blocks in the A video frame 1102 and 3 blocks in the A video frame 1106 .
  • the third block of the video frame 1108 to be reconstructed can be reconstructed from 4 blocks in the A video frame 1104 and 5 blocks in the A video frame 1104 .
  • the fourth block of the video frame 1108 to be reconstructed can be reconstructed from 6 blocks in the A video frame 1104 and 7 blocks in the A video frame 1106 .
  • the fifth block of the video frame 1108 to be reconstructed can be reconstructed from 8 blocks in the A video frame 1102 and 9 blocks in the A video frame 1104 .
  • the sixth block of the video frame 1108 to be reconstructed can be reconstructed from the A block in the A video frame 1104 and the B block in the A video frame 1106 .
  • the seventh block of the to-be-reconstructed video frame 1108 can be reconstructed from the C block of the A video frame 1102 and the D block of the A video frame 1104 .
  • FIG. 12 it is a schematic diagram of a storage manner of internal storage of a conventional video decoder in one embodiment.
  • the storage method of the internal storage of the video decoder in FIG. 12 stores the blocks in sequence.
  • the first step stores block 0, and the second step stores blocks 1, 2, 3, 4, 5, 6, and 7.
  • the third step stores blocks 6, 7. , 8, 9, A, B, C and D. That is to say, the internal buffer of the video decoder in Figure 12 will temporarily store the blocks read from any area, whether it is read from DRAM (white part) or from outside the video decoder Stored (gray section) read.
  • the video data that does not need the internal buffer of the video decoder at all to save the throughput of the DRAM is also stored in the internal buffer of the video decoder, which wastes the read and write energy consumption of the internal buffer of the video decoder.
  • FIG. 13 it is a schematic diagram of a storage manner of internal storage of the video decoder in one embodiment.
  • the white part in the internal buffer of the video decoder refers to the block originally stored in the DRAM, that is, the block that will be subsequently stored in the internal buffer of the video decoder.
  • the internal buffer of the video decoder of FIG. 13 only stores blocks from DRAM.
  • the blocks stored in the external storage of the video decoder and the blocks stored in the internal buffer of the video decoder will not be stored repeatedly, and the blocks stored in the DRAM are distinguished and stored in the DRAM
  • the block is read into the internal buffer of the video decoder, and another part of the block is stored in the external storage of the video decoder, so that the reading of the DRAM is reduced as a whole during the decoding process, and the bandwidth of the DRAM is reduced.
  • Reduce power consumption of electronic equipment In the system buffer/cache, the data itself is power-saving, and the access speed is fast and controllable. If such data is cached again by using the VDEC internal cache, it will not help the decoding speed and consume more energy for accessing the SRAM.
  • the remaining data that is not stored in the system cache/buffer will enter the VDEC internal cache to save DRAM bandwidth.
  • Table 1 is a schematic diagram of the difference in energy consumption between SRAM and DRAM in one embodiment.
  • the power consumption of SRAM (Static Random-Access Memory, static random access memory) is about 5PJ (picojoule), while the power consumption of DRAM is about 640PJ (picojoule), the difference between the two is two orders of magnitude.
  • the relative energy consumption of SRAM is 10 2
  • the relative energy consumption of DRAM is 10 4
  • the difference between the two is 2 orders of magnitude.
  • FIG. 14 is a structural block diagram of a video data storage device according to an embodiment.
  • a video data storage device includes a first reading module 1402 and a reference video frame storage module 1404, wherein:
  • the first reading module 1402 is configured to respond to the decoding process of the video frame to be reconstructed, the reference video frame of the video frame to be reconstructed does not exist in the external storage of the video decoder, and the main external storage of the video decoder does not exist. Reading the reference video frame from main storage if the reference video frame exists in storage; and
  • the reference video frame storage module 1404 is used for storing the reference video frame read from the main storage into the internal buffer of the video decoder.
  • the reference video frame of the video frame to be reconstructed in response to the decoding process of the video frame to be reconstructed, does not exist in the external storage of the video decoder, and the reference video frame of the video frame to be reconstructed does not exist in the external storage of the video decoder.
  • the reference video frame is read from the main storage, and the reference video frame read from the main storage is stored in the internal buffer of the video decoder, that is, in the video decoder's internal buffer.
  • the video data in the external storage is not stored in the internal buffer of the video decoder, and the video data in the main storage will be stored in the internal buffer of the video decoder.
  • the main storage will be put into the internal buffer of the decoder by the decoder.
  • only the video data that does not exist in the external storage and exists in the main storage needs to be put into the internal buffer of the decoder, reducing the number of video decoders.
  • the use of the internal cache increases the efficiency of the internal cache of the decoder, and further reduces power consumption specifically for the data that remains only in the main storage.
  • each video frame is divided into multiple video blocks.
  • the first reading module 1402 is used for the situation that the reference video block corresponding to the video frame to be reconstructed does not exist in the external storage of the video decoder, and the reference video block exists in the external main storage of the video decoder Next, the reference video block is read from the main storage.
  • the reference video frame storage module 1404 is used for storing the reference video blocks read from the main storage into the internal buffer of the video decoder.
  • the reference video block corresponding to the video frame to be reconstructed does not exist in the external storage of the video decoder, and if the reference video block exists in the main storage, the electronic device reads from the main storage Refer to the video block and store it in the internal cache of the video decoder. Since the internal cache in the video decoder is small, the storage block is more suitable for the internal cache in the video decoder, improving the storage capacity of video data. precision.
  • the first reading module 1402 is further configured to read the reference video frame from the external storage of the video coder if the reference video frame exists in the external storage of the video coder.
  • the reference video frame when there is a reference video frame in the external storage of the video decoder, the reference video frame is read from the external storage of the video decoder, and data access is performed through the external storage of the video decoder,
  • the efficiency of the internal cache of the decoder can be increased, and further power consumption can be reduced specifically for the data that is still only in the main storage.
  • the video data storage device further includes a motion vector acquisition module, a number of times determination module, a first storage module and a second storage module, wherein:
  • a motion vector acquisition module used for obtaining the motion vector corresponding to each video frame in the video in response to the decoding process of the acquired video
  • a number of times determination module used for determining the number of times each video frame is referenced based on the motion vector corresponding to each video frame;
  • the first storage module is used to store the video frame whose number of references meets the condition of the first number of times in the main storage of the video decoder;
  • the second storage module is used to store the video frames whose reference times satisfy the second time condition in the external storage of the video decoder; wherein, the times represented by the second time condition is greater than the number represented by the first time condition .
  • the motion vector corresponding to each video frame in the video is obtained, and the number of times each video frame is referenced is determined based on the motion vector of each video frame, then the video frame can be known.
  • the video frames that meet the condition of the first number of times to the internal buffer of the video decoder and store the video frames that meet the condition of the second number of times to the external storage of the decoder , the video frames that will be referenced less frequently will be stored in the internal buffer of the video decoder, and the video frames that will be referenced more frequently will be stored in the external storage of the decoder.
  • Use take advantage of low-power external storage, thereby reducing the power consumption of electronic equipment.
  • the internal buffer of the video decoder is limited, the video frames that are used less frequently are buffered in the video decoder, and these video frames in the external buffer can be updated faster, and the external buffer can be fully utilized to speed up the decoding. code processing.
  • the number of times determination module is configured to determine the reference video frame referenced by the current video frame based on the motion vector of the current video frame; and count the number of times the reference video frame is referenced.
  • the video data storage device in this embodiment determines the reference video frame referenced by the current video frame based on the motion vector of the current video frame, and counts the number of times the reference video frame is referenced, that is, to know which current video frame is and what is the motion vector , the reference video frame referenced by the current video frame can be obtained, thereby obtaining the reference times of each video frame in the video, and storing video data based on the reference times to reduce the power consumption of the electronic device.
  • the number of times determination module is further configured to stop counting the number of times the target reference video frame is referenced when the number of video frames between the current video frame and the target reference video frame reaches a preset number, and obtain the target reference video frame.
  • the number of times the video frame has been referenced. Since there are generally not too many frames between the current video frame and the referenced video frame, when the number of video frames between the current video frame and the target reference video frame reaches the preset number, the video frames after the current video frame The frame generally does not refer to the target reference video frame, so the reference times of the target reference video frame can be directly obtained.
  • each video frame is divided into multiple blocks.
  • the number of times determination module is used to determine the reference video block in the reference video frame referenced by the current video frame based on the motion vector of the current block in the current video frame; count the referenced times of the reference video block in the reference video frame;
  • the reference video blocks that meet the condition of the first number of times are stored in the internal buffer of the video decoder; the reference video blocks that meet the condition of the second number of times are stored in the external storage of the video decoder.
  • the reference video block to be referenced is determined based on the motion vector of the current block, the reference times of the reference video block are counted, and the reference video blocks whose reference times satisfy the condition of the first times are stored in the In the internal storage of the video decoder, the reference video block whose number of references meets the condition of the second number of times is stored in the external storage of the video decoder, then the referenced reference video block can be determined based on the current block and stored.
  • the video data is more accurate. Since the internal buffer in the video decoder is small, the storage block method is more suitable for the internal buffer in the video decoder.
  • the external storage of the video coder includes at least one of a system cache or a system buffer.
  • the data read from the system cache and the system buffer is faster and consumes less power. Therefore, the decoding speed of the video decoder can be maintained, and the power consumption of the electronic equipment can be reduced.
  • the main storage external to the video coder includes dynamic random access memory.
  • the first storage module is further configured to delete the most unnecessary video frames in the internal buffer when the internal buffer in the video decoder is full and new video frames are stored.
  • the electronic device deletes the least-needed video frames in the internal buffer to ensure that the video decoder Data can always be stored to ensure the operation of the video decoder. For example, it is possible to determine which video frames are least needed by looking at old and new. The least needed video frames may be, for example, the oldest. In other embodiments, other judgment methods may also be used to select the least needed video frames.
  • FIG. 15 is a structural block diagram of a video data reading apparatus according to an embodiment. As shown in FIG. 15, a video data reading device includes a second reading module 1502 and a third reading module 1504, wherein:
  • the second reading module 1502 is configured to, in response to the decoding processing of the acquired video code stream, read the reference video blocks that do not exist in the external storage of the video decoder from the external main storage of the video decoder to the internal buffer of the video decoder for motion compensation or block copying;
  • the third reading module 1504 is configured to perform motion compensation or block copy after reading the reference video block from the external storage of the video decoder.
  • the device for reading video data in this embodiment reads the reference video blocks that do not exist in the external storage of the video decoder from the main storage in response to the decoding processing of the acquired video code stream to the video decoding
  • Motion compensation or block copy is performed after the internal buffer of the video decoder, or motion compensation or block copy is performed after the reference video block is read from the external storage of the video decoder, that is, part of the video data is stored outside the video decoder.
  • the use of low-power external storage reduces the use of the internal buffer of the video decoder, and also reduces the bandwidth of the main storage, thereby reducing the power consumption of electronic equipment.
  • the second reading module 1502 is configured to, in response to the decoding processing of the acquired video code stream, search from the external storage of the video decoder, and find the reference video block in the external storage. In this case, motion compensation or block copying is performed after reading the reference video block from the external storage;
  • the third reading module 1504 is used to find the reference video block from the external main storage of the video decoder when the reference video block is not found in the external storage, and read the reference video from the main storage Motion compensation or block copying is performed after the block is stored in the video decoder's internal buffer.
  • the reference video block when the reference video block is found from the external storage of the video decoder, the reference video block is read from the external storage; when the reference video block is not found in the external storage , find the reference video block from the main storage again, find the reference video block from the main storage, read the reference video block from the main storage to the internal buffer of the video decoder and perform motion compensation; then the video decoding
  • the internal buffer of the decoder only reads the data from the main storage.
  • the internal buffer is consistent with the data stored in the external storage, which can save unnecessary buffering operations inside the video decoder. , thereby reducing the power consumption of electronic equipment.
  • the video data reading device further includes a storage module, and the storage module is configured to obtain motion vectors corresponding to each video frame in the video in response to the decoding processing of the acquired video;
  • the motion vector determines the reference times of each video frame; the video frames whose reference times meet the condition of the first times are stored in the external main storage of the video decoder; the video frames whose reference times meet the conditions of the second times are stored to the external storage of the video decoder; wherein, the number of times represented by the second order condition is greater than the number of times represented by the first order condition.
  • the motion vector corresponding to each video frame in the video is obtained, and the number of times each video frame is referenced is determined based on the motion vector of each video frame, then the video frame can be known.
  • To be used several times store the video frames that meet the condition of the first number of times to the main storage, and store the video frames that meet the condition of the second number of times to the external storage of the decoder, and are about to be referenced
  • the video frames with a few times are stored in the main storage, and the video frames with a large number of references are stored in the external storage of the decoder; thereby reducing the use of the internal buffer of the video decoder, using the low-power external storage, Thereby reducing the power consumption of the electronic device.
  • the external storage of the video decoder includes at least one of a system cache, a system buffer.
  • the data read from the system cache and the system buffer is faster and consumes less power. Therefore, the internal storage of the decoder does not need to store the video data from the system cache and system buffer again, which reduces the power consumption of electronic equipment.
  • each module in the above-mentioned video data storage device and video data reading device is only used for illustration. In other embodiments, the video data storage device and the video data reading device can be divided into different modules as required to complete the All or part of the functions of the above video data storage device and video data reading device.
  • Each module in the above-mentioned video data storage device and video data reading device may be implemented in whole or in part by software, hardware, and combinations thereof.
  • the above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
  • FIG. 16 is a schematic diagram of the internal structure of an electronic device in one embodiment.
  • the electronic device includes a processor and memory connected by a system bus.
  • the processor is used to provide computing and control capabilities to support the operation of the entire electronic device.
  • the memory may include non-volatile storage media and internal memory.
  • the nonvolatile storage medium stores an operating system and a computer program.
  • the computer program can be executed by the processor to implement the video data storage method and the video data read method provided by the following embodiments.
  • Internal memory provides a cached execution environment for operating system computer programs in non-volatile storage media.
  • the electronic device may be any terminal device such as a mobile phone, a tablet computer, a PDA (Personal Digital Assistant), a POS (Point of Sales, a sales terminal), a vehicle-mounted computer, a wearable device, and the like.
  • each module in the video data storage device and the video data reading device provided in the embodiments of the present application may be in the form of computer programs.
  • the computer program can be run on a terminal or server.
  • the program modules constituted by the computer program can be stored on the memory of the electronic device.
  • the computer program is executed by the processor, the operations of the methods described in the embodiments of the present application are implemented.
  • Embodiments of the present application also provide a computer-readable storage medium.
  • One or more non-volatile computer-readable storage media containing computer-executable instructions, when executed by one or more processors, cause the processors to perform the operations of the video data storage method.
  • Embodiments of the present application also provide a computer-readable storage medium.
  • One or more non-volatile computer-readable storage media containing computer-executable instructions that, when executed by one or more processors, cause the processors to perform the operations of the video data reading method .
  • Nonvolatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory.
  • Volatile memory may include random access memory (RAM), which acts as external cache memory.
  • RAM is available in various forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous Link (Synchlink) DRAM (SLDRAM), Memory Bus (Rambus) Direct RAM (RDRAM), Direct Memory Bus Dynamic RAM (DRDRAM), and Memory Bus Dynamic RAM (RDRAM).
  • SRAM static RAM
  • DRAM dynamic RAM
  • SDRAM synchronous DRAM
  • DDR SDRAM double data rate SDRAM
  • ESDRAM enhanced SDRAM
  • SLDRAM synchronous Link (Synchlink) DRAM
  • SLDRAM synchronous Link (Synchlink) DRAM
  • Memory Bus Radbus
  • RDRAM Direct RAM
  • DRAM Direct Memory Bus Dynamic RAM
  • RDRAM Memory Bus Dynamic RAM

Abstract

Procédé de stockage de données vidéo, consistant, en réponse à un traitement de décodage d'une trame vidéo à reconstruire, quand une trame vidéo de référence de la trame vidéo à reconstruire n'est pas présente dans une mémoire externe d'un décodeur vidéo et que la trame vidéo de référence est présente dans une mémoire principale externe du décodeur vidéo, à lire la trame vidéo de référence depuis la mémoire principale ; et à stocker la trame vidéo de référence lue dans une mémoire cache interne du décodeur vidéo.
PCT/CN2022/077080 2021-04-01 2022-02-21 Procédé et appareil de stockage de données vidéo, dispositif électronique et support de stockage lisible WO2022206212A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110355145.3 2021-04-01
CN202110355145.3A CN115174916A (zh) 2021-04-01 2021-04-01 视频数据存储方法、装置、电子设备和可读存储介质

Publications (1)

Publication Number Publication Date
WO2022206212A1 true WO2022206212A1 (fr) 2022-10-06

Family

ID=83455179

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/077080 WO2022206212A1 (fr) 2021-04-01 2022-02-21 Procédé et appareil de stockage de données vidéo, dispositif électronique et support de stockage lisible

Country Status (2)

Country Link
CN (1) CN115174916A (fr)
WO (1) WO2022206212A1 (fr)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103729449A (zh) * 2013-12-31 2014-04-16 上海富瀚微电子有限公司 一种参考数据访问管理方法及装置
US20170150159A1 (en) * 2015-11-20 2017-05-25 Intel Corporation Method and system of reference frame caching for video coding
CN107018418A (zh) * 2015-12-23 2017-08-04 联发科技股份有限公司 参考数据重用方法、带宽估计方法及相关视频解码器
US20210127125A1 (en) * 2019-10-23 2021-04-29 Facebook Technologies, Llc Reducing size and power consumption for frame buffers using lossy compression

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103729449A (zh) * 2013-12-31 2014-04-16 上海富瀚微电子有限公司 一种参考数据访问管理方法及装置
US20170150159A1 (en) * 2015-11-20 2017-05-25 Intel Corporation Method and system of reference frame caching for video coding
CN107018418A (zh) * 2015-12-23 2017-08-04 联发科技股份有限公司 参考数据重用方法、带宽估计方法及相关视频解码器
US20210127125A1 (en) * 2019-10-23 2021-04-29 Facebook Technologies, Llc Reducing size and power consumption for frame buffers using lossy compression

Also Published As

Publication number Publication date
CN115174916A (zh) 2022-10-11

Similar Documents

Publication Publication Date Title
US9224187B2 (en) Wavefront order to scan order synchronization
US9612962B2 (en) Performing cache bank operations in offset sequences from first bank
US9336558B2 (en) Wavefront encoding with parallel bit stream encoding
US8203648B2 (en) Motion vector detecting apparatus and motion vector detecting method
US20090238282A1 (en) Method and device for generating an image data stream, method and device for reconstructing a current image from an image data stream, image data stream and storage medium carrying an image data stream
US20070071099A1 (en) External memory device, method of storing image data for the same, and image processor using the method
US20200374534A1 (en) Ai-assisted programmable hardware video codec
US10070134B2 (en) Analytics assisted encoding
US20220279224A1 (en) Systems and methods for video processing
US10757430B2 (en) Method of operating decoder using multiple channels to reduce memory usage and method of operating application processor including the decoder
TW201637444A (zh) 包括用於影像處理的編碼器之方法、系統及裝置
CN103109535A (zh) 图像再生方法、图像再生装置、图像再生程序、摄像系统及再生系统
WO2022206212A1 (fr) Procédé et appareil de stockage de données vidéo, dispositif électronique et support de stockage lisible
TWI601075B (zh) 移動補償影像處理裝置與影像處理方法
WO2022115999A1 (fr) Procédé de traitement de données et dispositif de traitement de données
US20130278775A1 (en) Multiple Stream Processing for Video Analytics and Encoding
CN114374848A (zh) 一种视频编码优化方法和系统
KR100891116B1 (ko) 대역폭 인식 움직임 보상 장치 및 그 방법
Jubran et al. Sequence-level reference frames in video coding
CN110945872A (zh) 视频处理设备和方法
Lee et al. Memory bandwidth reduction using frame pipeline in video codec chips
Isquierdo et al. VVC decoder intra prediction using approximate storage: an error resilience evaluation
Liu et al. Design of an H. 264/AVC decoder with memory hierarchy and line-pixel-lookahead
Lin et al. Reduction of Frame Memory Accesses and Motion Estimation Computations in MPEG Video Encoder

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22778397

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22778397

Country of ref document: EP

Kind code of ref document: A1