WO2024000463A1

WO2024000463A1 - Video processing method and apparatus and computer readable storage medium

Info

Publication number: WO2024000463A1
Application number: PCT/CN2022/102965
Authority: WO
Inventors: 雷涛; 向前; 石洁珂; 李宗岩
Original assignee: 华为技术有限公司
Priority date: 2022-06-30
Filing date: 2022-06-30
Publication date: 2024-01-04

Abstract

The present disclosure relates to a video processing method and apparatus and a computer readable storage medium. The method comprises: parsing a code stream of a video so as to determine information indicating whether at least one image frame is referred to by a subsequent image frame (301); when the information indicates that the at least one image frame is not referred to by the subsequent image frame, decoding the code stream (304) to obtain the at least one image frame; and when the at least one image frame does not need to be stored to an off-chip memory, transmitting the at least one image frame to a display module for displaying the at least one image frame. According to the solution, the power consumption of a processing device in the video processing process can be effectively reduced.

Description

Method, device and computer-readable storage medium for processing video

Technical field

The present disclosure relates to the technical field of video processing, and more specifically to a method, device, and computer-readable storage medium for processing video.

Background technique

As the performance of smart terminal devices such as mobile phones and tablets gradually improves, people can watch local videos or watch online videos through the Internet through these smart terminal devices anytime and anywhere. As video quality becomes increasingly sophisticated, high-resolution, high frame rate and high bit rate videos continue to emerge, which not only provides users with a better video viewing experience, but also puts forward higher requirements for the performance of terminal devices.

Currently, due to limitations in terminal device size and battery technology, terminal devices have very strict requirements on power consumption and battery life. Watching videos for a long time will cause the smart terminal device to consume too much power and shorten the battery life, which will cause great anxiety to users. Therefore, how to reduce the power consumption of video playback services and improve the battery life of terminal equipment is a key technical issue to improve the user experience of using smart terminal equipment.

Contents of the invention

In view of the above problems, at least in order to reduce the impact of video processing on the battery life of the terminal device, embodiments of the present disclosure provide a method, device, and computer-readable storage medium for processing video.

In a first aspect of the present disclosure, a method of processing video is provided. The method includes: parsing the code stream of the video to determine information indicating whether at least one image frame is referenced by a subsequent image frame; when the information indicates that at least one image frame is not referenced by a subsequent image frame, decoding the code stream to obtain at least one an image frame; and transmitting at least one image frame to a display module to display at least one image frame without storing the at least one image frame to an off-chip memory.

According to embodiments of the present disclosure, image frames are processed differently by determining whether the image frame being processed will be referenced by subsequent image frames. Image frames that will not be referenced by subsequent image frames can be directly processed online. Transmitted to the display module without having to pass through an off-chip memory that has slower reading and writing speeds and higher power consumption, thus reducing the power consumption of the terminal device.

In an implementation of the first aspect, the method further includes: when the information indicates that at least one image frame is referenced by a subsequent image frame, decoding the code stream to obtain at least one image frame; transmitting at least one image frame to an off-chip memory; and reading at least one image frame from the off-chip memory to the display module to display at least one image frame. In this way, for image frames that will be referenced by subsequent image frames, stable output of the video can be ensured through an off-chip memory with a larger storage capacity.

In an implementation of the first aspect, when the information indicates that the at least one image frame is not referenced by the subsequent image frame, the method further includes: pre-processing the image frame and audio before decoding the code stream. Synchronize. In this way, poor user experience due to out-of-sync audio and video can be avoided.

In an implementation of the first aspect, at least one image frame is stored in on-chip memory without the need to store at least one image frame in off-chip memory. In this way, through on-chip memory, diameter transmission of video data can be achieved.

In an implementation of the first aspect, the image frames are stored in the on-chip memory in the form of blocks or strips. Since the space of on-chip memory is relatively limited, in this way, the space requirements for on-chip memory can be reduced.

In an implementation of the first aspect, the method further includes: after storing the image frame to the off-chip memory, synchronizing the image frame and the audio. In this way, the user's viewing experience can be improved.

In a second aspect of the present disclosure, an apparatus for processing video is provided. The device includes a video decoding module, a display module and a central processing unit, wherein the central processing unit is configured to implement the following operations: parse the video code stream and determine information indicating whether at least one image frame is referenced by a subsequent image frame; when the information Indicates that at least one image frame is not referenced by subsequent image frames, decoding the code stream to obtain at least one image frame; and transmitting at least one image frame to the display without storing the at least one image frame to an off-chip memory. module to display at least one image frame.

In an implementation manner of the second aspect, the device further includes an off-chip memory, wherein the central processing unit is configured to implement the following operations: when the information indicates that at least one image frame is referenced by a subsequent image frame, decode the code stream to obtain at least one image frame; transmitting at least one image frame to an off-chip memory; and reading at least one image frame from the off-chip memory to the display module to display at least one image frame.

In an implementation manner of the second aspect, the central processing unit is configured to implement the following operations: when the information indicates that the at least one image frame is not referenced by the subsequent image frame, before decoding the code stream, Image frames and audio are pre-synchronized.

In an implementation of the second aspect, the apparatus further includes an on-chip memory, wherein the central processing unit is configured to perform the following operations: at least one image frame is stored without the need to store the at least one image frame to off-chip memory. on-chip memory.

In an implementation of the second aspect, the image frames are stored in the on-chip memory in the form of blocks or strips.

In an implementation of the second aspect, the central processing unit is configured to synchronize the image frames and the audio after storing the image frames to the off-chip memory.

In a third aspect of the present disclosure, there is provided a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the method of the first aspect of the present disclosure is implemented.

It should be understood that what is described in this summary is not intended to identify key or important features of the implementations of the disclosure, nor to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the description below.

Description of drawings

The above and other features, advantages and aspects of various implementations of the present disclosure will become more apparent with reference to the following detailed description taken in conjunction with the accompanying drawings. In the drawings, the same or similar reference numbers represent the same or similar elements, where:

Figure 1 shows a schematic usage environment in which methods according to embodiments of the present disclosure may be applied;

2 shows a block diagram of an electronic device according to an embodiment of the present disclosure;

Figure 3 shows a schematic flowchart of a method for processing video according to an embodiment of the present disclosure;

Figure 4 shows a data flow diagram according to an embodiment of the present disclosure; and

5A and 5B respectively illustrate schematic diagrams of image frame blocking according to embodiments of the present disclosure.

Detailed ways

Implementations of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain implementations of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure may be implemented in various forms and should not be construed as limited to the implementations set forth herein, but rather these implementations are provided for A more thorough and complete understanding of this disclosure. It should be understood that the drawings and implementations of the present disclosure are for illustrative purposes only and are not intended to limit the scope of the present disclosure.

In describing implementations of the present disclosure, the term "including" and similar expressions should be understood as an open-ended inclusion, ie, "including but not limited to." The term "based on" should be understood to mean "based at least in part on." The term "one implementation" or "the implementation" shall be understood to mean "at least one implementation". The terms "first," "second," etc. may refer to different or the same object. The term "and/or" means at least one of the two items associated with it. For example, "A and/or B" means A, B, or A and B. Other explicit and implicit definitions may be included below.

It should be understood that for the technical solutions provided by the implementation methods of this application, in the following introduction of specific implementation methods, some repetitive parts may not be described again, but it should be regarded that these specific implementation methods have mutual references and can be combined with each other.

Some illustrative implementations of the present disclosure are described below with reference to the accompanying drawings.

FIG. 1 shows a schematic usage environment in which methods according to embodiments of the present disclosure may be applied. As shown in the figure, the method of the embodiment of the present disclosure may be used in a terminal device 100 such as a mobile phone. The user can play the video on the terminal device 100. Such videos may be videos stored locally on the terminal device 100 or videos played online using various types of networks. The embodiments of the present disclosure are not particularly limited in this regard. As mentioned above, as video resolution, frame rate, and bit rate gradually increase, video playback and processing require a large amount of power from the terminal device 100 , which results in a shorter battery life of the terminal device 100 . At least to solve the above problems, embodiments of the present disclosure provide a video processing solution. This solution can effectively reduce the power consumption of the terminal device 100 due to video processing and improve the user experience. It should be understood that although the terminal device 100 is shown in the form of a mobile phone in FIG. 1 , this is only illustrative and not restrictive. The terminal device 100 may also take other forms, such as a tablet computer, a notebook computer, a desktop computer, a vehicle-mounted smart device, etc., and its specific form is not limited by the embodiments of the present disclosure.

In the context of this disclosure, a video may be composed of multiple images arranged in a time sequence and audio associated with the images. These images are sometimes referred to as "frames" or "image frames". A video can contain a sequence of code streams (also called code stream segments), where each code stream can correspond to an image frame. Taking a video with a frame rate of 60 frames per second as an example, 1 second includes 60 frames of images arranged in a specific order. It should be noted that the methods of the embodiments of the present disclosure are suitable for processing videos encoded based on various frame rates and different protocols, and there are no special restrictions on the specific frame rate and encoding protocol of the video to be processed.

A block diagram of an apparatus 200 according to an embodiment of the present disclosure is described below with reference to FIG. 2 . The apparatus 200 may be the terminal device 100 integrated into FIG. 1 . As shown in FIG. 2 , the device 200 may include a chip 210 and an off-chip memory 206 located outside the chip 210 . As shown, the chip 210 includes a central processing unit (CPU) 201. Program code for executing methods according to embodiments of the present disclosure may be stored on a non-volatile memory 208 external to the chip 210. The central processing unit 201 may Communicates with the non-volatile memory 208 so that the program code is read from the non-volatile memory 208 to execute each method step.

Continuing to refer to FIG. 2 , the central processing unit 201 may be coupled with the video decoding module 202 , the display module 203 and the on-chip memory 205 integrated on the chip 210 through the bus 204 . It should be noted that in some embodiments, the central processing unit 201, the video decoding module 202, the display module 203 and the on-chip memory 205 can be integrated on the same chip 210 and coupled to each other via the bus 204. In other embodiments, the central processing unit 201, the video decoding module 202, the display module 203 and the on-chip memory 205 may be located on different chips. That is to say, the chip 210 may be in the form of a system on a chip (SoC), or may be in the form of other multi-chip combinations. The embodiments of the present disclosure do not specifically limit the specific form of the chip 210 , as long as it can realize mutual transmission and communication between different modules on the device 200 .

Each module on the device 200 is briefly introduced below. The central processing unit 201 can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory or loaded into a random access memory. The video decoding module 202 may be configured to decode a standard compressed code stream of the video. The display module 203 may be configured to overlay image frames of each layer that need to be displayed on the screen of the terminal device 100, thereby displaying a video containing each image frame on the screen. The on-chip memory 205 may be cache or static random access memory (Static Random Access Memory, SRAM). The off-chip memory 206 can be a dynamic random access memory (Dynamic Random Access Memory, DRAM). In contrast, although the on-chip memory 205 has a smaller storage space than the off-chip memory 206, its read and write speed is much faster than the off-chip memory 206. , the read and write power consumption is also much lower than that of the off-chip memory 206. As shown in FIG. 2 , the on-chip memory 205 and the off-chip memory 206 can be coupled to the video decoding module 202 and the display module 203 through the bus 204 .

The following describes a method 300 for processing video according to an embodiment of the present disclosure with reference to FIG. 3 . As shown in Figure 3, in block 301, one or more code streams included in the video are parsed to determine information. Such information indicates whether the image frame corresponding to the code stream segment to be processed is replaced by the subsequent image frame. referenced. It should be noted that for many image frames contained in a code stream, such information is not required to be determined for all image frames. Information can be determined for the image frame corresponding to a code stream segment in the code stream. For a continuous video, there is often a strong correlation between the content reflected in the previous and later image frames that make up the video, especially when the scene changes little, this correlation will be more prominent. Therefore, among the set of image frames that constitute the video, some image frames can be used as references for subsequent image frames in the decoding process. Of course, there are also some image frames that will not be used as references for subsequent image frames. The video code stream includes information that reflects whether the image frame corresponding to the code stream segment to be processed in the video will be referenced by subsequent image frames. The corresponding information of the code stream segments in these videos can be obtained based on the corresponding software syntax analysis. It should be understood that in some embodiments, the image frame corresponding to the code stream segment to be processed may be referenced by a subsequent image frame, while in other embodiments, it may be referenced by multiple subsequent image frames. These image frames may be adjacent or separated by other image frames. That is to say, the image frame to be processed may be referenced in the decoding process of the image frame located behind it. The specific reference form is not particularly limited.

In block 302, for the video code stream, different processing methods are determined based on the information determined in block 301 indicating whether the image frame needs to be referenced by subsequent image frames. Therefore, corresponding operation modes can be adopted for the code stream according to different determination results. If it is determined at block 302 that the image corresponding to the code stream segment being processed does not need to be referenced by subsequent image frames, the method 300 will proceed to block 303. In block 303, the video may be pre-synchronized before decoding the code stream. In some embodiments, the current frame number and timestamp information can be reported to the audio and video synchronization module of the device 200 for pre-synchronization of audio and image frames, and the method 300 is entered only when the audio and video synchronization time point is reached. In block 304, the video decoding module 202 is triggered to perform the decoding process. At block 304, the code stream can be decoded using various decoding methods to obtain image frames. Subsequently, the decoded image frame is directly transmitted to the display module 203 for display through the on-chip memory 205 inside the chip 210, as shown in block 309. Thus, the display of the image frame on the device 200 is completed.

Figure 4 illustrates a data flow diagram 400 in accordance with an embodiment of the present disclosure. As can be seen in conjunction with Figure 4, according to the embodiment of the present disclosure, if it is determined that the image corresponding to the code stream segment being processed does not need to be referenced by subsequent image frames, there is no need for the participation of the off-chip memory 206, and only the chip 210 is used The on-chip memory 205 can transmit the data decoded by the video decoding module 202 to the on-chip memory 205, and finally the display module 203 reads the data from the on-chip memory 205 to display the image frame. This method can be understood as online transmission. In this way, the on-chip memory 205 with faster read and write speed and lower power consumption can be fully utilized, thereby avoiding excessive power consumption of the terminal device 100 during the video processing process.

In some embodiments, if it is determined at block 302 that the image frame corresponding to the code stream segment being processed will be referenced by subsequent image frames in the decoding process, the method 300 enters another branch to reach block 306. At block 306, the code stream can be decoded using various decoding methods to obtain image frames. Subsequently, at block 307, the obtained image frames are transferred to off-chip memory 206 located outside the chip 210. Furthermore, through the transfer of the off-chip memory 206, the image frames are transmitted to the display module 203 for display. In some embodiments, at block 308, before transmitting the image frames to the display module 203 for display, the video and audio may be synchronized to enhance the user's viewing experience.

According to an embodiment of the present disclosure, with reference to FIG. 4 , if the image frame corresponding to the code stream segment being processed will be referenced by subsequent image frames in the parsing process, the video decoding module 202 decodes the code stream being processed. Finally, the obtained image frame data is written to the off-chip memory 206, and then the display module 303 reads the data from the off-chip memory 306 for display. This method can be understood as offline transmission. In this case, the off-chip memory 306 with larger storage space is used to ensure normal display of the video.

As a result, the subsequent hardware decoding method can be determined based on the information of the image frame corresponding to the code stream segment being processed. By reasonably distinguishing the actual situation of the code stream being processed, this can achieve low power consumption and video processing. The stability shows a balance between the two.

In some embodiments, at block 304, when decoding the code stream, the image frame may be output in a certain manner. 5A and 5B respectively illustrate schematic diagrams of dividing an output image frame according to an embodiment of the present disclosure. In some embodiments, as shown in Figure 5A, the image frame 500 may be segmented in the horizontal direction, thereby forming a plurality of strips 502-1, 502-2, ..., 502-N. In other embodiments, as shown in FIG. 5B , the image frame 410 may be segmented both horizontally and vertically, thereby forming a plurality of strips 512-1, 512-2, ... 512-M. The decoded data of a strip 502-1, 502-2,..., 502-N or a block 512-1, 512-2,...512-M is directly written to the on-chip memory 205 without will be written to the off-chip memory 206, thereby reducing the bandwidth of the off-chip memory 206. In this way, for image frames that will not be referenced by subsequent image frames, since they are written into the on-chip memory 205 with a small storage space, they are divided into relatively coarse granularity, so that The space requirement for on-chip memory 205 can be reduced. It should be understood that although 4 strips are shown in FIG. 5A , this is only illustrative and not limiting. The image frame 500 can be divided into more or fewer strips according to the actual usage scenario. destination strip. Similarly, the number of bars 512-1, 512-2, ... 512-M shown in FIG. 5B is only illustrative, and in other embodiments, other numbers of bars may be formed. In addition, in some embodiments, the image frame 500 may also be divided only in the vertical direction. It should also be understood that such divisions may be equally spaced or unequally spaced. That is to say, the strips 502-1, 502-2, ..., 502-N in Figure 5A may have the same size or different sizes. Similarly, the sizes of the bars 512-1, 512-2, ... 512-M in Figure 5B may be the same or different. This application does not place a limit on this, as long as it can segment the image frame, thereby reducing the video processing time.

In some embodiments, since the data of the strips or strips decoded by the video decoding module 202 has undergone audio and video pre-synchronization in block 303, it can be directly transmitted to the display module 203 at this time for display on the terminal device. 100 on the monitor. In some embodiments, when a predetermined number of blocks or strips are transmitted to the on-chip memory 205, the central processing unit 201 may notify the display module 203, so that the image frames in the on-chip memory 205 can be transmitted to the display module 203 in time. Referring back to FIG. 2 , timely communication between the video decoding module 202 and the display module 203 can be achieved through the communication module 207 provided between the two, thereby effectively reducing the communication delay. In some embodiments, the video decoding module 202 completes writing a predetermined number of blocks 512-1, 512-2, ... 512-M or strips 502-1, 502-2, ..., 502-N of data to After the on-chip memory 205, the display module 203 can be notified of a predetermined number of strips 512-1, 512-2,... 512-M or strips 502-1, 502-2,..., 502-N via the communication module 207. The data has been decoded, and the display module 203 is informed of the storage address of these data. In this way, the display module 203 reads the stripes 512-1, 512-2, ..., 512-M or strips 502-1, 502-2, ..., 502-N in the on-chip memory 205 from the address. The data is then superimposed with the image frames of other layers and transmitted to the display of the terminal device 100 for display. After the above reading step is completed, the video decoding module 202 can also be notified through the communication module 207, so that the video decoding module 202 can recycle the address to release the space on the on-chip memory 205 for subsequent decoding. Since the space of the on-chip memory 205 is usually very limited, excessive data cannot be stored. Using the method in the embodiment of the present disclosure, through direct communication between the video decoding module 202 and the display module 203, the space of the on-chip memory 205 can be released in time to ensure efficient data processing. As a result, the power consumption of the terminal device 100 can be further reduced.

In some embodiments, the predetermined number of blocks or strips may be one. This means that as long as the video decoding module 302 writes the data of a block or strip to the on-chip memory 205, it can notify the display module 203 via the communication module 207 that the data of the block or strip has been decoded, and then store The address reads the data in the block or strip for display, and once the reading of the block or strip is completed, the space on the on-chip memory 205 is released for subsequent decoding. This makes data processing more efficient. It should be understood that, according to the requirements for the endurance performance of the terminal device 100, the predetermined number can also be other numbers, such as two, three, or more.

In some embodiments, the communication module 207 may be a direct hardware connection or a hardware scheduling accelerator disposed between the video decoding module 202 and the display module 203 .

In some embodiments, in order to ensure that the decoded image frames can be accurately and efficiently transmitted to the display module 203, the display module 203 can control the working rhythm of the video decoding module 202 through a synchronization signal, thereby adjusting the working rhythm of the video decoding module 202. Need to be consistent with display module 203. For example, if the refresh rate of the display screen of the device 200 is 60 frames per second, the display module 203 notifies the video decoding module 202 via the communication module 207 every 16.6 milliseconds (1000/60). At this time, if the video decoded by the video decoding module 202 is 30 frames per second, a decoding process is triggered every time two synchronization signals are received; if the video decoded by the video decoding module 202 is 60 frames per second, then every time Receiving a synchronization signal triggers a decoding process. In this manner, synchronous operations between the video decoding module 202 and the display module 203 can be ensured, thereby ensuring that the video can be correctly displayed on the device 200 . It should be understood that the numerical values here are only exemplary and not restrictive.

Various processes and processes described herein, such as method 300, may be performed by central processing unit 201. For example, in some embodiments, method 300 may be implemented as a computer software program tangibly embodied in a machine-readable medium. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 300 via a read-only memory and/or communication unit. When a computer program is loaded into random access memory and executed by central processing unit 301, one or more actions of method 300 described above may be performed.

The disclosure may be a method, apparatus, system and/or computer program product. The computer program product may include a computer-readable storage medium, which may be the non-volatile memory 208 described above, having thereon computer-readable program instructions for performing various aspects of the disclosure.

Computer-readable storage media may be tangible devices that can retain and store instructions for use by an instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or Flash memory), Static Random Access Memory (SRAM), Compact Disk Read Only Memory (CD-ROM), Digital Versatile Disk (DVD), Memory Stick, Floppy Disk, Mechanical Coding Device, such as a printer with instructions stored on it. Protruding structures in hole cards or grooves, and any suitable combination of the above. As used herein, computer-readable storage media are not to be construed as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or through electrical wires. transmitted electrical signals.

Computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to various computing/processing devices, or to an external computer or external storage device over a network, such as the Internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers. A network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage on a computer-readable storage medium in the respective computing/processing device .

Computer program instructions for performing operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or instructions in one or more programming languages. Source code or object code written in any combination of object-oriented programming languages - such as Smalltalk, C++, etc., and conventional procedural programming languages - such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server implement. In situations involving remote computers, the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (such as an Internet service provider through the Internet). connect). In some embodiments, by utilizing state information of computer-readable program instructions to personalize an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit can Computer readable program instructions are executed to implement various aspects of the disclosure.

Aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.

These computer-readable program instructions may be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, thereby producing a machine such that the instructions, when executed by a processing unit of the computer or other programmable data processing apparatus, , resulting in an apparatus that implements the functions/actions specified in one or more blocks in the flowchart and/or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium. These instructions cause the computer, programmable data processing device and/or other equipment to work in a specific manner. Therefore, the computer-readable medium storing the instructions includes An article of manufacture that includes instructions that implement aspects of the functions/acts specified in one or more blocks of the flowcharts and/or block diagrams.

Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other equipment, causing a series of operating steps to be performed on the computer, other programmable data processing apparatus, or other equipment to produce a computer-implemented process , thereby causing instructions executed on a computer, other programmable data processing apparatus, or other equipment to implement the functions/actions specified in one or more blocks in the flowcharts and/or block diagrams.

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions that embody one or more elements for implementing the specified logical function(s). Executable instructions. In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two consecutive blocks may actually execute substantially in parallel, or they may sometimes execute in the reverse order, depending on the functionality involved. It will also be noted that each block of the block diagram and/or flowchart illustration, and combinations of blocks in the block diagram and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts. , or can be implemented using a combination of specialized hardware and computer instructions.

Compared with existing solutions, according to the implementation of the present disclosure, image frames that do not need to be referenced after decoding are transmitted online to the display module 203 without being transferred through the off-chip memory 206, which can reduce the power consumption of the terminal device 100. the goal of. For most videos, due to the need to save network transmission bandwidth or video storage space, a large number of frames that are not referenced are usually used. Therefore, the embodiments of the present disclosure have good usage effects. As more frames are not referenced, the power consumption benefits of embodiments of the present disclosure are greater.

It should be understood that the solutions according to the embodiments of the present disclosure can also be applied to scenarios that require multiple addresses to collaboratively process video data, such as online data transmission solutions in camera recording scenarios, thereby reducing camera equipment in recording scenarios. Power consumption.

It should also be understood that steps can be reordered, added, or removed using various forms of the process shown above. For example, each step described in the present disclosure can be executed in parallel, sequentially, or in a different order. As long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, there is no limitation here.

Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

A method of processing video, the method includes:

Analyze the code stream of the video to determine information indicating whether at least one image frame is referenced by a subsequent image frame;

When the information indicates that the at least one image frame is not referenced by the subsequent image frame,

Decode the code stream to obtain the at least one image frame; and

Without storing the at least one image frame to an off-chip memory, the at least one image frame is transmitted to the display module to display the at least one image frame.
The method of claim 1, further comprising:

when said information represents that at least one image frame is referenced by said subsequent image frame,

Decode the code stream to obtain the at least one image frame;

transmitting the at least one image frame to off-chip memory; and

The at least one image frame is read from the off-chip memory to the display module to display the at least one image frame.
The method according to claim 1 or 2, when the information indicates that the at least one image frame is not referenced by the subsequent image frame, the method further includes:

Before decoding the code stream, the image frames and audio are pre-synchronized.
A method according to any one of claims 1 to 3,

The at least one image frame is stored in the on-chip memory without storing the at least one image frame to the off-chip memory.
The method of claim 4, wherein the image frames are stored in the on-chip memory in the form of blocks or strips.
The method of claim 2, further comprising:

After the image frame is stored in the off-chip memory, the image frame and audio are synchronized.
A device for processing video, including a video decoding module, a display module and a central processing unit, wherein the central processing unit is configured to implement the following operations:

Analyze the code stream of the video to determine information indicating whether at least one image frame is referenced by a subsequent image frame;

When the information indicates that the at least one image frame is not referenced by the subsequent image frame,

Decode the code stream to obtain the at least one image frame; and

Without storing the at least one image frame to an off-chip memory, the at least one image frame is transmitted to the display module to display the at least one image frame.
The apparatus of claim 7, further comprising off-chip memory, wherein the central processing unit is configured to:

when said information represents that at least one image frame is referenced by said subsequent image frame,

Decode the code stream to obtain the at least one image frame;

transmitting the at least one image frame to the off-chip memory; and

The at least one image frame is read from the off-chip memory to the display module to display the at least one image frame.
The device according to claim 7 or 8, wherein the central processing unit is configured to implement the following operation: when the information indicates that the at least one image frame is not referenced by the subsequent image frame, in the code stream Before decoding, the image frames and audio are pre-synchronized.
The apparatus of any one of claims 7 to 9, further comprising on-chip memory, wherein the central processing unit is configured to:

The at least one image frame is stored in the on-chip memory without storing the at least one image frame to the off-chip memory.
The apparatus of claim 10, wherein the image frames are stored in the on-chip memory in the form of blocks or strips.
The apparatus of claim 8, wherein the central processing unit is configured to:

After the image frame is stored in the off-chip memory, the image frame and audio are synchronized.
A computer-readable storage medium having a computer program stored thereon, which implements the method according to any one of claims 1 to 6 when executed by a processor.