CN113055742B - Video display method, device, terminal and storage medium - Google Patents
Video display method, device, terminal and storage medium Download PDFInfo
- Publication number
- CN113055742B CN113055742B CN202110246014.1A CN202110246014A CN113055742B CN 113055742 B CN113055742 B CN 113055742B CN 202110246014 A CN202110246014 A CN 202110246014A CN 113055742 B CN113055742 B CN 113055742B
- Authority
- CN
- China
- Prior art keywords
- resolution
- video frame
- video
- target
- current
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440263—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the spatial resolution, e.g. for displaying on a connected PDA
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Abstract
The application discloses a video display method, a video display device, a video display terminal and a video storage medium, and belongs to the technical field of computer vision. The method is applied to the terminal, and comprises the following steps: decoding the video stream through a decoder to obtain a current video frame and a first resolution corresponding to the current video frame; responding to the first resolution being lower than the target resolution, performing super-resolution processing on the current video frame to obtain a target video frame, wherein the target video frame corresponds to the second resolution, and the second resolution is higher than the first resolution; video display is performed based on the target video frame. In the scene of reducing the video resolution due to the flow bandwidth requirement, the video resolution can be improved through the terminal side, so that the saving requirement on the flow bandwidth is met, the requirement on the video definition of a user can be met, and the video watching quality of the user under the condition of poor network environment can be improved.
Description
Technical Field
The embodiment of the application relates to the technical field of computer vision, in particular to a video display method, a video display device, a terminal and a storage medium.
Background
The image resolution indicates the number of pixels contained in a unit inch, and in general, the higher the image resolution, the more pixels contained and the sharper the image.
In the field of video display, the higher the resolution of a video frame, the clearer the corresponding video, but higher resolution video often requires faster networking and more bandwidth support. Taking online video playing as an example, when a user is watching a video, if the current network state is poor, in order to ensure smooth video playing, a video playing application program may reduce the requirement on network bandwidth by automatically reducing the resolution of the video, and meanwhile achieve the purpose of improving the playing smoothness.
Disclosure of Invention
The embodiment of the application provides a video display method, a video display device, a terminal and a storage medium. The technical scheme is as follows:
in one aspect, an embodiment of the present application provides a video display method, where the method includes:
decoding a video stream through a decoder to obtain a current video frame and a first resolution corresponding to the current video frame;
Responding to the first resolution being lower than a target resolution, performing super-resolution processing on the current video frame to obtain a target video frame, wherein the target video frame corresponds to a second resolution which is higher than the first resolution;
and displaying the video based on the target video frame.
In another aspect, an embodiment of the present application provides a video display apparatus, including:
the video decoding module is used for decoding the video stream through a decoder to obtain a current video frame and a first resolution corresponding to the current video frame;
the first processing module is used for responding to the fact that the first resolution is lower than the target resolution, performing super-resolution processing on the current video frame to obtain a target video frame, wherein the target video frame corresponds to a second resolution which is higher than the first resolution;
and the video display module is used for displaying the video based on the target video frame.
In another aspect, embodiments of the present application provide a terminal, where the terminal includes a processor and a memory, where at least one program code is stored in the memory, and the program code is loaded and executed by the processor to implement the video display method according to the above aspect.
In another aspect, embodiments of the present application provide a computer readable storage medium having at least one program code stored therein, the program code loaded and executed by a processor to implement a video display method as described in the above aspects.
In another aspect, embodiments of the present application provide a computer program product or computer program comprising computer instructions stored in a computer-readable storage medium. The processor of the terminal reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the terminal performs the video display method provided in various optional implementations of the above aspect.
The technical scheme provided by the embodiment of the application can bring the following beneficial effects:
in an online video display scene, when the terminal detects that the resolution corresponding to the current video frame is lower or the resolution is reduced, super-resolution processing can be performed on the video frame before video rendering display is performed, so that the video resolution is improved, and the video definition is improved. In the scene of reducing the video resolution due to the flow bandwidth requirement, the video resolution can be improved through the terminal side, so that the saving requirement on the flow bandwidth is met, the requirement on the video definition of a user can be met, and the video watching quality of the user under the condition of poor network environment can be improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 illustrates a flow chart of a video display method provided by an exemplary embodiment of the present application;
FIG. 2 illustrates a super resolution process schematic diagram shown in an exemplary embodiment of the present application;
FIG. 3 illustrates a flowchart of a video display method provided in another exemplary embodiment of the present application;
FIG. 4 is a schematic diagram of resolution conversion during video display;
FIG. 5 illustrates a flowchart of a video display method provided by another exemplary embodiment of the present application;
FIG. 6 illustrates a flowchart of a video display method provided by another exemplary embodiment of the present application;
FIG. 7 illustrates a flowchart of a video display method provided by another exemplary embodiment of the present application;
FIG. 8 illustrates a flowchart of a full video display method shown in an exemplary embodiment of the present application;
Fig. 9 is a block diagram showing a structure of a video display apparatus according to an embodiment of the present application;
fig. 10 is a block diagram illustrating a structure of a terminal according to an exemplary embodiment of the present application.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
Referring to fig. 1, a flowchart of a video display method according to an exemplary embodiment of the present application is shown, where the embodiment of the present application is described by taking application of the method to a terminal as an example, and the method includes:
The video display method in the embodiment of the application is applied to an online video process, and is schematically, when a user uses a video application program to watch online video, or a user uses a live broadcast application program to watch online live broadcast, or a user uses an instant messaging application program to chat online video, the video display method in the embodiment of the application can be adopted.
In a possible implementation manner, when the terminal performs video display, two processes, namely video decoding and video rendering display, are generally required, and the video decoding is used for decoding a received video stream through a decoder, so as to obtain video frames, and then performing video rendering display based on the video frames.
Based on the video display method in the related art, the definition of the video watched by the user is completely dependent on the received video stream, that is, if the terminal receives the video stream with lower resolution, the terminal displays the video with lower resolution correspondingly, and for some scenes needing to reduce resolution, for example, because the network state is poor, the video application program may automatically adjust the resolution downwards to maintain stable playing of the video (corresponding to the video stream with lower resolution received by the terminal), which greatly affects the viewing definition of the user.
In this embodiment, a corresponding video display method is provided for the above-mentioned scene with lower resolution (reduced), so as to ensure the viewing quality of the user. Therefore, before video display, the terminal first needs to identify a scene with lower resolution (or reduced resolution), and in one possible implementation, the terminal first decodes the received video stream through the decoder to obtain the required resolution information and the current video frame, so as to execute the video display method shown in the embodiment of the present application when it is determined that the resolution is detected to be lower (or reduced) based on the resolution information.
Illustratively, the video stream may be decoded into single frame images (video frames) in yuv format, thereby obtaining resolution information corresponding to each single frame image.
And step 102, responding to the fact that the first resolution is lower than the target resolution, performing super-resolution processing on the current video frame to obtain the target video frame, wherein the target video frame corresponds to the second resolution, and the second resolution is higher than the first resolution.
The target resolution may be preset resolution, which may be set by a user, or may be set by an application developer for different application programs, or may be set dynamically based on a user's historical operating habit; or the target resolution may be a historical resolution corresponding to a previous preset time period.
In a possible implementation manner, when the terminal determines that the first resolution is lower than the target resolution, it indicates that there is a scene with lower resolution or reduced resolution currently, a resolution enhancing operation is required to enhance the video viewing quality of the user, and correspondingly, super-resolution processing is required to be performed on the current video frame, so that the current video frame is processed into a target video frame with higher video resolution, for example, a second resolution higher than the first resolution.
Optionally, the super-resolution processing can perform a super-resolution method of a preset multiple on the current video frame, and then intelligently fill pixels through an artificial intelligence (Artificial Intelligence, AI) deep learning model, so that the resolution of the video frame is improved, and correspondingly, the video definition is improved. Illustratively, the current video frame can be input into a super-resolution algorithm model, and super-resolution processing is performed on the current video frame through the super-resolution algorithm model, so that a target video frame with improved resolution is obtained.
As shown in fig. 2, a schematic diagram of a super resolution processing procedure according to an exemplary embodiment of the present application is shown. The first resolution corresponding to the current video frame 201 is "X Y", the current video frame 201 is input into the super-division processing model 202, the super-division processing model 202 performs super-division amplification on the video image at two times of the length and width, and performs pixel filling, so as to obtain a target video frame 203 output by the super-division processing model 202, and the second resolution corresponding to the target video frame 203 is "2X 2Y".
Alternatively, in order to improve the timing accuracy of determining the super-resolution process, before the super-resolution process is performed, a prompt message may be displayed on the current video interface, for example, "whether the current resolution is low or not to perform the super-resolution process", and when a confirmation operation of the user is received, the step of performing the super-resolution process may be performed.
In one possible implementation manner, after the terminal obtains the target video frame with the improved resolution, the target video frame may be displayed on a terminal screen through a display module in the terminal.
In summary, in the embodiment of the present application, in an online video display scene, when the terminal detects that the resolution corresponding to the current video frame is low or the resolution is reduced, the super-resolution processing may be performed on the video frame before the video rendering display is performed, so as to improve the video resolution, thereby improving the video definition. In the scene of reducing the video resolution due to the flow bandwidth requirement, the video resolution can be improved through the terminal side, so that the saving requirement on the flow bandwidth is met, the requirement on the video definition of a user can be met, and the video watching quality of the user under the condition of poor network environment can be improved.
In the super-resolution processing process of the current video frame, larger power consumption is required, super-resolution processing is not required in any scene with lower resolution or lower resolution, for example, in a scene with a better current network state, video watching definition can be directly improved by receiving a video stream with higher resolution, so that in a possible implementation mode, a preset super-resolution condition is set so as to perform super-resolution processing in a proper scene, and terminal power consumption is reduced as much as possible.
Referring to fig. 3, a flowchart of a video display method according to another exemplary embodiment of the present application is shown, where the embodiment of the present application is described by taking application of the method to a terminal as an example, and the method includes:
in step 301, the video stream is decoded by a decoder to obtain a current video frame and a first resolution corresponding to the current video frame.
The implementation of step 301 may refer to step 101, and this embodiment is not described herein.
In step 302, in response to the first resolution being lower than the target resolution and meeting the preset super-resolution condition, super-resolution processing is performed on the current video frame to obtain the target video frame.
The reason why the super-resolution processing operation is performed to ensure the viewing quality of the user in the scene with lower resolution may include: the application program automatically down-regulates the video resolution (video definition) according to the current network state and the user automatically down-regulates the video resolution according to the self-requirement, wherein if the user automatically down-regulates the video resolution, super-resolution processing may not be needed, so that in order to avoid the influence of super-resolution processing on the power consumption of the terminal when any condition that the resolution is lower is monitored, in one possible implementation, super-resolution conditions are set, that is, the super-resolution processing is needed for the video frame only when the preset super-resolution conditions are met.
The preset super-resolution condition may include: the current network type corresponding to the current network is at least one of a mobile network, a current network speed corresponding to the current network is lower than a preset network speed threshold value and a user operation habit.
Optionally, based on the purpose of maintaining smooth video playing, before the terminal performs the super-resolution processing, the current network state may be obtained, if the current network state is worse, for example, the current network speed corresponding to the current network is lower than a preset network speed threshold, in order to maintain video flow display or smooth video playing, the super-resolution processing may be performed on the current video frame, so that when the network state is worse, smooth video playing is considered (by adjusting the resolution downwards), and meanwhile, the viewing quality of the video is ensured (by super-resolution processing).
Please refer to fig. 4, which is a schematic diagram illustrating resolution conversion during video display. When the network state is detected to be poor, the video application program automatically adjusts the resolution of the reduced video to adapt to the bandwidth requirement, for example, the first resolution '480×640' corresponding to the first video 401 is reduced to the second resolution '240×320' corresponding to the second video 402, so that the network bandwidth is reduced, the data transmission is reduced, the continuity of the video stream is ensured, and video clamping is avoided. However, adapting the network status by reducing the resolution may result in a reduced resolution perceived by the user on the terminal screen, and in order to improve the viewing quality of the user, the second video 402 may be subjected to super-resolution processing to increase the resolution, so as to obtain the third video 403, for example, to be re-promoted from the second resolution "240×320" to the third resolution "480×640".
Optionally, based on the purpose of saving traffic bandwidth, before the terminal performs super-resolution processing, the current network type may be acquired, if the current network type is a mobile network, in order to save traffic, the video resolution may be adjusted down, so that the purpose of saving traffic bandwidth may be achieved by performing super-resolution processing on the video frame after the resolution is adjusted down, and meanwhile, viewing definition of the video may be ensured.
Optionally, for the case that the preset super-resolution condition is the operation habit of the user, when the video resolution down-regulation operation received by the terminal is actively triggered by the user, the user may indicate that the user wants to watch the video under the definition currently, and correspondingly, in order to reduce the power consumption of the terminal, the super-resolution processing on the current video frame may not be needed; if the video resolution down-regulation operation received by the terminal is triggered by the application program autonomously, it may indicate that the user does not want to down-regulate the definition, but due to the influence of external factors (such as poor network environment), smooth playing of the video can be ensured by performing super-resolution processing on the video frame.
Optionally, the preset resolution condition may further include: if the first resolution is higher than the preset resolution, no super-resolution processing is required, that is, the terminal detects the video resolution decrease, but the influence of the video resolution decrease on the video definition is small, for example, the super-resolution mode is changed to the high-resolution mode, and in consideration of the power consumption of the terminal, no super-resolution processing is required on the current video frame.
Optionally, the manner of determining the target resolution may include the steps of:
1. and determining the target resolution based on the historical resolution, wherein the historical resolution is the resolution corresponding to the video frame in a preset time period before receiving the video resolution reduction operation.
In one possible embodiment, when the terminal receives the resolution reduction operation, the history resolution indicated by the resolution reduction operation may be acquired, thereby determining the target resolution based on the history resolution.
Illustratively, if the video resolution reduction operation indicates: the video resolution is turned down from "800×480" to "400×240", with a corresponding historical resolution of "800×480".
Alternatively, in order to improve accuracy of the acquired historical resolution, a resolution corresponding to the video frame in a preset period of time before the video resolution operation may be acquired, which may be, illustratively, 10min.
2. Obtaining candidate resolutions indicated by a historical resolution updating operation; and determining the candidate resolution with the highest candidate resolution selection frequency as the target resolution.
Taking video watching as an example, a user usually selects the most suitable video definition by himself, so in order to avoid that the resolution corresponding to the video definition commonly used by the user is reduced in resolution, the target resolution can be determined by acquiring the historical resolution updating operation of the user, thereby avoiding that the video watching quality of the user is influenced by external factors.
In one possible implementation manner, the candidate resolution indicated by the historical resolution updating operation may be obtained, and the common video resolution of the user may be abstracted from the resolution selection frequency, so that the common resolution may be determined as the target resolution, so that when the resolution drops to the common resolution, the super-resolution processing may be performed on the video frame, so as to ensure the video viewing quality of the user.
Alternatively, the candidate resolution with the highest resolution selection frequency may be determined as the target resolution. Illustratively, taking video viewing as an example, a user typically manually adjusts the video definition, and a resolution corresponding to the definition commonly used by the user may be determined as the target resolution.
The implementation of this step may refer to the above embodiments, which are not described herein.
In this embodiment, whether the super-resolution processing is required for the video frame is determined by presetting a super-resolution condition, so that the super-resolution processing is stopped to avoid the influence on the power consumption of the terminal under the condition that the super-resolution processing is not required; further, by determining the history resolution as the target resolution, the resolution increasing operation can be performed in time when the resolution decreasing operation is detected; in addition, by determining the common resolution as the target resolution, the resolution improvement operation can be timely performed under the condition that the detected resolution is lower than the common resolution of the user, so as to ensure the viewing quality of the user.
In a possible application scenario, when the current network state is poor, the application program may automatically adjust the resolution to ensure smooth video playing, and correspondingly, the terminal may monitor that the first resolution is lower than the historical resolution before the resolution adjustment operation.
Referring to fig. 5, a flowchart of a video display method according to another exemplary embodiment of the present application is shown, where the embodiment of the present application is described by taking application of the method to a terminal as an example, and the method includes:
in step 501, in response to a video resolution reduction operation, a history resolution before the video resolution reduction operation is acquired.
In one possible implementation manner, when the application program detects that the current network state is poor, for example, the current network speed is lower than a preset network threshold, and when the current network state is a mobile network, in order to reduce the requirement of video viewing on bandwidth traffic, the video definition may be automatically adjusted down (i.e. the video resolution is correspondingly reduced), and correspondingly, when the terminal receives the video resolution reducing operation, the terminal may acquire the historical resolution before the video resolution reducing operation, so as to perform the super-resolution processing operation based on the historical resolution later.
Optionally, in order to avoid the influence of abrupt network condition changes on the acquired historical resolution, the corresponding historical resolution in a preset period of time before the video resolution reduction operation may be acquired. Illustratively, the preset time period may be within a preset 10 min.
Alternatively, the historical resolution may be obtained from resolution information obtained by decoding the historical video stream, or may be obtained from resolution indicated by a video resolution reduction operation, which indicates that, illustratively, if the resolution reduction operation indicates that the current video is switched from the high-definition mode to the standard-definition mode, the corresponding historical resolution may be obtained from the image resolution corresponding to the high-definition mode.
In the present embodiment, the history resolution from which it is determined whether or not to perform the super resolution processing operation may be determined as the target resolution.
In step 503, the video stream is decoded by a decoder, so as to obtain the current video frame and the first resolution corresponding to the current video frame.
The implementation of this step may refer to the above embodiments, which are not described herein.
In step 504, in response to the first resolution being lower than the target resolution, the current video frame is processed from the first resolution to a historical resolution, resulting in a target video frame.
Wherein the target resolution is a historical resolution.
In a possible implementation manner, the terminal detects resolution information corresponding to the video in real time, and when the terminal detects that the first resolution corresponding to the current video frame is lower than the historical resolution, the terminal indicates that the video resolution reducing operation is completed, and correspondingly, the terminal needs to perform super-resolution processing on the video resolution. In order to ensure the viewing quality of the user, when the super-resolution processing is performed, the current video frame can be directly processed from the first resolution to the historical resolution, so that the requirement of saving flow bandwidth is met in the video display process, and meanwhile, the viewing definition of the user is ensured.
Since the historical resolution indicated by the resolution reduction operation is dynamically determined by the application program based on the current network state, if the current network state is extremely poor, the historical resolution may be greatly different from the first resolution after the resolution reduction operation, and if the resolution is greatly different, the current video frame may need to be filled with more pixels if the current video frame is further processed as the historical resolution, and instead, the purpose of improving the definition cannot be achieved.
In one illustrative example, step 504 may include step 504A and step 504B, as shown in fig. 6, on the basis of fig. 5.
In step 504A, in response to the first resolution being lower than the target resolution, a resolution difference between the first resolution and the historical resolution is obtained.
In one possible implementation, after the terminal determines the first resolution and the historical resolution, it may determine whether the current video frame needs to be adjusted from the first resolution to the historical resolution by comparing the degree of difference between the first resolution and the historical resolution.
Wherein the difference program may be determined by the resolution difference. Illustratively, since the resolution is generally represented by a long-side pixel value and a wide-side pixel value, the resolution difference may be represented as a long-side pixel value difference or a wide-side pixel value difference.
In step 504B, in response to the resolution difference being less than the preset resolution threshold, the current video frame is processed from the first resolution to the historical resolution, and the target video frame is obtained.
The preset resolution threshold may be set by a developer. The preset resolution threshold may also be represented by a preset long-side pixel threshold and/or a preset wide-side pixel threshold. Illustratively, the preset resolution threshold may be denoted as "400×240".
In a possible implementation manner, after determining that the resolution difference is smaller than the preset resolution threshold, the difference degree between the first resolution and the historical resolution is smaller, the purpose of improving the definition can be achieved through super resolution processing, and correspondingly, the current video frame can be processed from the first resolution to the historical resolution, so that the target video frame with the historical resolution is obtained.
Optionally, when the terminal determines that the resolution difference is greater than the preset resolution threshold, it indicates that the current video frame cannot be processed from the first resolution to the historical resolution, or that the current video frame cannot be processed from the first resolution to the historical resolution, so that in order to increase the video definition as much as possible, a third resolution can be determined between the first resolution and the historical resolution, and thus the current video frame is processed from the first resolution to the third resolution.
Illustratively, when the terminal determines that the history resolution is "960×1280" and the first resolution is "240×320", it is possible to determine that the history resolution and the first resolution are different, the resolution improvement operation cannot be performed, and it is possible to determine that the third resolution is "480×640" based on the history resolution and the first resolution, and process the current video frame from the first resolution to the third resolution.
The implementation of this step may refer to the above embodiments, which are not described herein.
In this embodiment, the current video frame is processed from the first resolution to the historical resolution before the resolution reduction operation is received, so that the user can not only meet the purpose of saving the traffic bandwidth but also ensure the video watching quality of the user under the condition of no perception; in addition, whether the video frame is processed into the historical resolution from the first resolution is determined by comparing the difference degree between the first resolution and the historical resolution, so that super-resolution processing can be avoided when the difference degree is large, the video watching quality is influenced, and the effect of the super-resolution processing is ensured.
Because the super-resolution processing needs to process each video frame, correspondingly, larger power consumption is needed, in order to avoid the influence of the super-resolution processing on the terminal endurance, in a possible implementation manner, before the super-resolution processing, power consumption judgment needs to be performed to determine whether the terminal endurance is affected.
On the basis of fig. 1, step 102 may be replaced with step 701 and step 702 as shown in fig. 7.
In step 701, in response to the first resolution being lower than the target resolution and the current residual power being higher than the preset power threshold, super-resolution processing is performed on the current video frame to obtain the target video frame.
In order to avoid the influence of the super-resolution processing on the terminal endurance, in one possible implementation manner, when the first resolution is determined to be lower than the target resolution, the current residual electric quantity of the terminal is obtained, if the current residual electric quantity of the terminal is higher than the preset electric quantity threshold value, the current electric quantity of the terminal is sufficient, the influence of the super-resolution processing on the terminal endurance is small, and the super-resolution processing can be performed on the current video frame to obtain the target video frame; otherwise, if the current residual electric quantity of the terminal is lower than the preset electric quantity threshold value, the current electric quantity of the terminal is smaller, and in order to maintain the continuous voyage requirement of the terminal, the super-resolution processing of the current video frame is stopped.
Optionally, when the terminal performs the super-resolution processing, if the current residual electric quantity is detected to be lower, the terminal may stop performing the super-resolution processing on the current video frame first, and preferably maintain the cruising duration of the terminal.
Optionally, before the terminal performs the super-resolution processing, whether the terminal is in a power saving mode or not may also be determined, if the terminal is in the power saving mode, it indicates that the user has a continuous voyage requirement on the terminal, and the super-resolution processing is stopped; if the terminal is not in the power saving mode, the super resolution processing operation may be performed.
Alternatively, the preset power threshold may be set by a developer, or may be set by the user, or may be automatically adjusted based on the environment in which the terminal is located, which is schematically indicated as 40%.
In step 702, in response to the first resolution being lower than the target resolution and the memory occupancy being lower than the preset occupancy threshold, super-resolution processing is performed on the current video frame to obtain the target video frame.
Because the terminal power consumption is also related to the current terminal running state, for example, the current memory occupancy rate is higher, the current power consumption of the terminal is correspondingly higher, or the load of a current central processing unit (Central Processing Unit, CPU) is increased, and the current power consumption of the terminal is correspondingly higher, in one possible implementation manner, when the terminal determines that the first resolution is lower than the target resolution, the current memory occupancy rate can be obtained, if the current memory occupancy rate is higher and higher than the preset occupancy rate threshold value, the current power consumption of the terminal is higher, and if the super-resolution processing is performed, the terminal endurance is obviously affected, and the corresponding super-resolution processing needs to be stopped; otherwise, if the terminal determines that the current memory occupancy rate is lower and is lower than the preset occupancy rate threshold, the terminal indicates that no operation task with larger power consumption exists currently, and super-resolution processing can be performed on the current video frame.
Optionally, the preset occupancy rate threshold may be 70%, which is schematically indicated that if the current memory occupancy rate of the terminal is 80%, the current memory occupancy rate is higher, and the super-resolution processing is stopped; if the current memory occupancy rate is 30%, the current memory occupancy rate is lower, and super-resolution processing can be performed.
In the embodiment, the current power consumption state of the terminal is judged so as to stop the super-resolution processing to reduce the power consumption of the terminal when the terminal has a continuous voyage requirement, thereby improving the continuous voyage capability of the terminal; and super-resolution processing is performed under the condition of lower power consumption or sufficient electric quantity of the terminal, so that the video watching quality of a user is improved.
Referring to fig. 8, a flowchart of a complete video display method according to an exemplary embodiment of the present application is shown, the method comprising:
step 801 receives a video stream.
The video stream may be a live video stream, an instant messaging video stream, or an online video stream.
Step 802, decoding the video stream by a system decoder to obtain a video frame and resolution information corresponding to the video frame.
The system decoder is used for decoding the video stream to obtain video frames contained in the video stream and resolution information corresponding to each frame of video frame.
Step 803, performing real-time resolution monitoring on resolution information corresponding to the video frame.
Step 804, whether the resolution is reduced.
When it is determined that the resolution is reduced, step 805 may be performed to perform super-resolution processing on the video frame, otherwise, step 803 may be performed to continue with resolution monitoring without performing super-resolution processing.
In step 805, real-time super-resolution processing is performed on the video frames.
Step 806, video display is performed based on the video frames.
When the super-resolution processing is performed on the video frames, video rendering display can be performed based on the processed video frames.
The following are device embodiments of the present application, which may be used to perform method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.
Referring to fig. 9, a block diagram of a video display apparatus according to an embodiment of the present application is shown. The device has the function of realizing the execution of the terminal in the embodiment of the method, and the function can be realized by hardware or can be realized by executing corresponding software by hardware. As shown in fig. 9, the apparatus may include:
the video decoding module 901 is configured to decode a video stream by using a decoder to obtain a current video frame and a first resolution corresponding to the current video frame;
A first processing module 902, configured to perform super-resolution processing on the current video frame in response to the first resolution being lower than a target resolution, to obtain a target video frame, where the target video frame corresponds to a second resolution, and the second resolution is higher than the first resolution;
the video display module 903 is configured to perform video display based on the target video frame.
Optionally, the first processing module 902 includes:
and the first processing unit is used for responding to the fact that the first resolution is lower than the target resolution and meets the preset super-resolution condition, and performing super-resolution processing on the current video frame to obtain the target video frame.
Optionally, the preset super-resolution condition includes: the current network type corresponding to the current network is at least one of a mobile network, a current network speed corresponding to the current network is lower than a preset network speed threshold value and a user operation habit.
Optionally, the apparatus further includes:
an acquisition module configured to acquire the historical resolution before a video resolution reduction operation in response to the video resolution reduction operation;
a first determining module configured to determine the historical resolution as the target resolution;
The first processing module 902 includes:
and the second processing unit is used for processing the current video frame from the first resolution to the historical resolution to obtain the target video frame.
Optionally, the second processing unit is further configured to:
acquiring a resolution difference between the first resolution and the historical resolution;
and responding to the resolution difference value being smaller than a preset resolution threshold value, processing the current video frame from the first resolution to the historical resolution, and obtaining the target video frame.
Optionally, the apparatus further includes:
and the second processing module is used for responding to the fact that the resolution difference value is larger than the preset resolution threshold value, processing the current video frame from the first resolution to a third resolution, and obtaining the target video frame, wherein the third resolution is larger than the first resolution and smaller than the historical resolution.
Optionally, the apparatus further includes:
the second determining module is used for determining the target resolution based on the historical resolution, wherein the historical resolution is the resolution corresponding to the video frame in a preset time period before the video resolution reducing operation is received;
Or alternatively, the first and second heat exchangers may be,
a third determining module, configured to obtain a candidate resolution indicated by the historical resolution updating operation; and determining the candidate resolution with the highest candidate resolution selection frequency as the target resolution.
Optionally, the first processing module 902 includes:
the third processing unit is used for responding to the fact that the first resolution is lower than the target resolution and the current residual electricity is higher than a preset electricity threshold value, and performing super-resolution processing on the current video frame to obtain the target video frame;
or alternatively, the first and second heat exchangers may be,
and the fourth processing unit is used for responding to the fact that the first resolution is lower than the target resolution and the memory occupancy rate is lower than a preset occupancy rate threshold value, and performing super-resolution processing on the current video frame to obtain the target video frame.
In summary, in the embodiment of the present application, in an online video display scene, when the terminal detects that the resolution corresponding to the current video frame is low or the resolution is reduced, the super-resolution processing may be performed on the video frame before the video rendering display is performed, so as to improve the video resolution, thereby improving the video definition. In the scene of reducing the video resolution due to the flow bandwidth requirement, the video resolution can be improved through the terminal side, so that the saving requirement on the flow bandwidth is met, the requirement on the video definition of a user can be met, and the video watching quality of the user under the condition of poor network environment can be improved.
It should be noted that: in the device provided in the above embodiment, when implementing the functions thereof, only the division of the above functional modules is used as an example, in practical application, the above functional allocation may be implemented by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules, so as to implement all or part of the functions described above. In addition, the video display device provided in the above embodiment and the video display method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which is not described herein again.
Referring to fig. 10, a block diagram illustrating a structure of a terminal according to an exemplary embodiment of the present application is shown. Terminal 1000 can include one or more of the following components: a processor 1010, a memory 1020, and a touch display 1030.
The Memory 1020 may include a random access Memory (Random Access Memory, RAM) or a Read-Only Memory (ROM). Optionally, the memory 1020 includes a non-transitory computer readable medium (non-transitory computer-readable storage medium). Memory 1020 may be used to store instructions, programs, code, sets of codes, or instruction sets. The memory 1020 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, which may be an Android (Android) system (including a system developed based on an Android system), an IOS system developed by apple corporation (including a system developed based on an IOS system depth), or other systems, instructions for implementing at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the various method embodiments described above, and the like. The storage data area may also store data created by the terminal 1400 in use (e.g., phonebook, audio-video data, chat-record data), etc.
Optionally, terminal 1000 can also include a touch display screen, which can be a capacitive touch display screen for receiving touch operations by a user on or near any suitable object, such as a finger, stylus, etc., and displaying a user interface for each application. The touch display is typically provided on the front panel of terminal 1000. The touch display screen may be designed as a full screen, a curved screen, or a contoured screen. The touch display screen can also be designed to be a combination of a full screen and a curved screen, and a combination of a special-shaped screen and a curved screen, which is not limited in the embodiment of the application.
In addition, those skilled in the art will appreciate that the configuration of terminal 1000 shown in the above-described figures is not limiting of terminal 1000 and that a terminal may include more or fewer components than shown, or may combine certain components, or a different arrangement of components. For example, terminal 1000 can further include radio frequency circuitry, a camera module, an audio circuit, a wireless fidelity (Wireless Fidelity, wiFi) module, a power supply, a bluetooth module, etc., which are not described herein.
Embodiments of the present application also provide a computer readable storage medium storing at least one program code loaded and executed by a processor to implement the video display method according to the above embodiments.
According to one aspect of the present application, there is provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the terminal reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the terminal performs the video display method provided in various optional implementations of the above aspect.
It should be understood that references herein to "a plurality" are to two or more. "and/or", describes an association relationship of an association object, and indicates that there may be three relationships, for example, a and/or B, and may indicate: a exists alone, A and B exist together, and B exists alone. The character "/" generally indicates that the context-dependent object is an "or" relationship. In addition, the step numbers described herein are merely exemplary of one possible execution sequence among steps, and in some other embodiments, the steps may be executed out of the order of numbers, such as two differently numbered steps being executed simultaneously, or two differently numbered steps being executed in an order opposite to that shown, which is not limited by the embodiments of the present application.
The foregoing description of the preferred embodiments is merely exemplary in nature and is in no way intended to limit the invention, since it is intended that all modifications, equivalents, improvements, etc. that fall within the spirit and scope of the invention.
Claims (7)
1. A method of video display, the method comprising:
in response to a video resolution reduction operation, obtaining a historical resolution prior to the video resolution reduction operation;
determining the historical resolution as a target resolution;
decoding a video stream through a decoder to obtain a current video frame and a first resolution corresponding to the current video frame;
responding to the first resolution being lower than the target resolution, performing super-resolution processing on the current video frame to obtain a target video frame;
performing video display based on the target video frame;
the super-resolution processing is performed on the current video frame to obtain a target video frame, and the super-resolution processing includes:
acquiring a resolution difference between the first resolution and the historical resolution;
responding to the resolution difference value being smaller than a preset resolution threshold value, processing the current video frame from the first resolution to the historical resolution, and obtaining the target video frame;
After obtaining the resolution difference between the first resolution and the historical resolution, the method further comprises:
and in response to the resolution difference value being greater than the preset resolution threshold, processing the current video frame from the first resolution to a third resolution, and obtaining the target video frame, wherein the third resolution is greater than the first resolution and less than the historical resolution.
2. The method of claim 1, wherein the super-resolution processing the current video frame in response to the first resolution being below a target resolution comprises:
and responding to the first resolution being lower than the target resolution and meeting a preset super-resolution condition, performing super-resolution processing on the current video frame to obtain the target video frame.
3. The method of claim 2, wherein the preset super-resolution condition comprises: the current network type corresponding to the current network is at least one of a mobile network, a current network speed corresponding to the current network is lower than a preset network speed threshold value and a user operation habit.
4. A method according to any one of claims 1 to 3, wherein said performing super-resolution processing on said current video frame in response to said first resolution being lower than a target resolution to obtain a target video frame comprises:
Responding to the condition that the first resolution is lower than the target resolution and the current residual electricity is higher than a preset electricity threshold, and performing super-resolution processing on the current video frame to obtain the target video frame;
or alternatively, the first and second heat exchangers may be,
and responding to the condition that the first resolution is lower than the target resolution and the memory occupancy rate is lower than a preset occupancy rate threshold, performing super-resolution processing on the current video frame to obtain the target video frame.
5. A video display apparatus, the apparatus comprising:
an acquisition module for responding to a video resolution reduction operation and acquiring a history resolution before the video resolution reduction operation;
a first determining module configured to determine the historical resolution as a target resolution;
the video decoding module is used for decoding the video stream through a decoder to obtain a current video frame and a first resolution corresponding to the current video frame;
the first processing module is used for responding to the fact that the first resolution is lower than the target resolution, performing super-resolution processing on the current video frame, and obtaining a target video frame;
the video display module is used for displaying video based on the target video frame;
Wherein the first processing module comprises:
a second processing unit for acquiring a resolution difference between the first resolution and the historical resolution;
the second processing unit is further configured to process the current video frame from the first resolution to the historical resolution in response to the resolution difference value being smaller than a preset resolution threshold, so as to obtain the target video frame;
the apparatus further comprises:
and the second processing module is used for responding to the fact that the resolution difference value is larger than the preset resolution threshold value, processing the current video frame from the first resolution to a third resolution, and obtaining the target video frame, wherein the third resolution is larger than the first resolution and smaller than the historical resolution.
6. A terminal comprising a processor and a memory, wherein the memory has stored therein at least one program code that is loaded and executed by the processor to implement the video display method of any of claims 1 to 4.
7. A computer readable storage medium having stored therein at least one program code loaded and executed by a processor to implement the video display method of any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110246014.1A CN113055742B (en) | 2021-03-05 | 2021-03-05 | Video display method, device, terminal and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110246014.1A CN113055742B (en) | 2021-03-05 | 2021-03-05 | Video display method, device, terminal and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113055742A CN113055742A (en) | 2021-06-29 |
| CN113055742B true CN113055742B (en) | 2023-06-09 |
Family
ID=76510269
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110246014.1A Active CN113055742B (en) | 2021-03-05 | 2021-03-05 | Video display method, device, terminal and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113055742B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115883853B (en) * | 2021-09-26 | 2024-04-05 | 腾讯科技（深圳）有限公司 | Video frame playing method, device, equipment and storage medium |
| CN114339412B (en) * | 2021-12-30 | 2024-02-27 | 咪咕文化科技有限公司 | Video quality enhancement method, mobile terminal, storage medium and device |
| CN114466220A (en) * | 2022-01-29 | 2022-05-10 | 维沃移动通信有限公司 | Video downloading method and electronic equipment |
| CN116055778B (en) * | 2022-05-30 | 2023-11-21 | 荣耀终端有限公司 | Video data processing method, electronic device and readable storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108496198A (en) * | 2017-10-09 | 2018-09-04 | 华为技术有限公司 | A kind of image processing method and equipment |
| CN111340711A (en) * | 2020-05-21 | 2020-06-26 | 腾讯科技（深圳）有限公司 | Super-resolution reconstruction method, device, equipment and storage medium |
| CN112153465A (en) * | 2019-06-28 | 2020-12-29 | 北京京东尚科信息技术有限公司 | Image loading method and device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9036707B2 (en) * | 2012-05-07 | 2015-05-19 | Cellco Partnership | Method and apparatus for finding a motion vector |
-
2021
- 2021-03-05 CN CN202110246014.1A patent/CN113055742B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108496198A (en) * | 2017-10-09 | 2018-09-04 | 华为技术有限公司 | A kind of image processing method and equipment |
| CN112153465A (en) * | 2019-06-28 | 2020-12-29 | 北京京东尚科信息技术有限公司 | Image loading method and device |
| CN111340711A (en) * | 2020-05-21 | 2020-06-26 | 腾讯科技（深圳）有限公司 | Super-resolution reconstruction method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113055742A (en) | 2021-06-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113055742B (en) | Video display method, device, terminal and storage medium | |
| US9892716B2 (en) | Image display program, image display method, and image display system | |
| CN109729405B (en) | Video processing method and device, electronic equipment and storage medium | |
| CN109379624B (en) | Video processing method and device, electronic equipment and storage medium | |
| WO2021244440A1 (en) | Method, apparatus, and system for adjusting image quality of television, and television set | |
| CN109640167B (en) | Video processing method and device, electronic equipment and storage medium | |
| CN108156533B (en) | Smart television backlight adjusting method, smart television and storage medium | |
| CN113608713B (en) | Variable frequency display method, DDIC, display screen module and terminal | |
| CN112203034B (en) | Frame rate control method and device and electronic equipment | |
| JP7295950B2 (en) | Video enhancement control method, device, electronic device and storage medium | |
| CN109120988B (en) | Decoding method, decoding device, electronic device and storage medium | |
| CN109165103B (en) | Frame rate control method, device, terminal and storage medium | |
| CN109445918B (en) | Task scheduling method, device, terminal and storage medium | |
| EP3886447A1 (en) | Video enhancement control method and apparatus, electronic device, and storage medium | |
| CN109618228B (en) | Video enhancement control method and device and electronic equipment | |
| CN113781949B (en) | Image display method, display driving chip, display screen module and terminal | |
| US11412311B2 (en) | Methods and systems for saving data while streaming video | |
| CN112399123A (en) | Video definition adjusting method and device, electronic equipment and storage medium | |
| CN110858388B (en) | Method and device for enhancing video image quality | |
| CN109587555B (en) | Video processing method and device, electronic equipment and storage medium | |
| CN112804527B (en) | Image output method, image output device and computer-readable storage medium | |
| US20150179146A1 (en) | Graphics voltage reduction for load line optimization | |
| EP3855754A1 (en) | Video processing method and apparatus, electronic device, and storage medium | |
| WO2020038071A1 (en) | Video enhancement control method, device, electronic apparatus, and storage medium | |
| CN116132644A (en) | Automatic color saturation adjustment method and device and related equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |