CN113055742A - Video display method, device, terminal and storage medium - Google Patents
Video display method, device, terminal and storage medium Download PDFInfo
- Publication number
- CN113055742A CN113055742A CN202110246014.1A CN202110246014A CN113055742A CN 113055742 A CN113055742 A CN 113055742A CN 202110246014 A CN202110246014 A CN 202110246014A CN 113055742 A CN113055742 A CN 113055742A
- Authority
- CN
- China
- Prior art keywords
- resolution
- video frame
- target
- video
- current
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 76
- 238000012545 processing Methods 0.000 claims abstract description 108
- 230000009467 reduction Effects 0.000 claims description 24
- 230000004044 response Effects 0.000 claims description 20
- 230000008569 process Effects 0.000 description 18
- 230000006870 function Effects 0.000 description 10
- 238000010586 diagram Methods 0.000 description 8
- 238000009877 rendering Methods 0.000 description 7
- 238000004590 computer program Methods 0.000 description 4
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 230000003321 amplification Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 230000003828 downregulation Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440263—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the spatial resolution, e.g. for displaying on a connected PDA
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The application discloses a video display method, a video display device, a video display terminal and a storage medium, and belongs to the technical field of computer vision. The method is applied to the terminal and comprises the following steps: decoding the video stream through a decoder to obtain a current video frame and a first resolution corresponding to the current video frame; responding to the first resolution ratio lower than the target resolution ratio, performing super-resolution processing on the current video frame to obtain a target video frame, wherein the target video frame corresponds to a second resolution ratio which is higher than the first resolution ratio; and performing video display based on the target video frame. Under the scene that the video resolution is reduced due to the requirement of the traffic bandwidth, the video resolution can be improved through the terminal side, the requirement for saving the traffic bandwidth is met, the requirement of a user for the video definition can be considered, and the video watching quality of the user under the condition of poor network environment can be improved.
Description
Technical Field
The embodiment of the application relates to the technical field of computer vision, in particular to a video display method, a video display device, a video display terminal and a storage medium.
Background
The image resolution indicates the number of pixels contained in a unit inch, and in general, the higher the image resolution, the more pixels are contained, and the sharper the image is.
In the field of video display, if the resolution of a video frame is higher, the corresponding video is clearer, but the higher resolution video often needs faster network speed and more bandwidth support. Taking online video playing as an example, when a user is watching a video, if the current network state is poor, in order to guarantee smooth playing of the video, a video playing application program may reduce the requirement on the network bandwidth in a form of automatically reducing the video resolution, and at the same time, achieve the purpose of improving the playing smoothness, but obviously, by adopting the video playing method, the video resolution is low, thereby affecting the definition of video playing.
Disclosure of Invention
The embodiment of the application provides a video display method, a video display device, a video display terminal and a storage medium. The technical scheme is as follows:
in one aspect, an embodiment of the present application provides a video display method, where the method includes:
decoding a video stream through a decoder to obtain a current video frame and a first resolution corresponding to the current video frame;
responding to the first resolution ratio lower than a target resolution ratio, performing super-resolution processing on the current video frame to obtain a target video frame, wherein the target video frame corresponds to a second resolution ratio which is higher than the first resolution ratio;
and performing video display based on the target video frame.
In another aspect, an embodiment of the present application provides a video display apparatus, where the apparatus includes:
the video decoding module is used for decoding the video stream through a decoder to obtain a current video frame and a first resolution corresponding to the current video frame;
the first processing module is used for responding to the situation that the first resolution is lower than a target resolution, performing super-resolution processing on the current video frame to obtain a target video frame, wherein the target video frame corresponds to a second resolution, and the second resolution is higher than the first resolution;
and the video display module is used for carrying out video display based on the target video frame.
In another aspect, an embodiment of the present application provides a terminal, where the terminal includes a processor and a memory, where the memory stores at least one program code, and the program code is loaded and executed by the processor to implement the video display method according to the above aspect.
In another aspect, an embodiment of the present application provides a computer-readable storage medium, in which at least one program code is stored, and the program code is loaded and executed by a processor to implement the video display method according to the above aspect.
In another aspect, embodiments of the present application provide a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the terminal reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the terminal performs the video display method provided in the various alternative implementations of the above aspects.
The technical scheme provided by the embodiment of the application can bring the following beneficial effects:
when the terminal detects that the resolution corresponding to the current video frame is low or the resolution is reduced in an online video display scene, the terminal can perform super-resolution processing on the video frame before performing video rendering display so as to improve the video resolution and improve the video definition. Under the scene that the video resolution is reduced due to the requirement of the traffic bandwidth, the video resolution can be improved through the terminal side, the requirement for saving the traffic bandwidth is met, the requirement of a user for the video definition can be considered, and the video watching quality of the user under the condition of poor network environment can be improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 illustrates a flow chart of a video display method provided by an exemplary embodiment of the present application;
FIG. 2 is a diagram illustrating a super-resolution processing process according to an exemplary embodiment of the present application;
FIG. 3 illustrates a flow chart of a video display method provided by another exemplary embodiment of the present application;
FIG. 4 is a schematic diagram of resolution conversion during video display;
FIG. 5 illustrates a flow chart of a video display method provided by another exemplary embodiment of the present application;
FIG. 6 illustrates a flow chart of a video display method provided by another exemplary embodiment of the present application;
FIG. 7 illustrates a flow chart of a video display method provided by another exemplary embodiment of the present application;
FIG. 8 illustrates a flowchart of a complete video display method shown in an exemplary embodiment of the present application;
fig. 9 is a block diagram illustrating a configuration of a video display apparatus according to an embodiment of the present application;
fig. 10 is a block diagram illustrating a structure of a terminal according to an exemplary embodiment of the present application.
Detailed Description
To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
Referring to fig. 1, a flowchart of a video display method provided in an exemplary embodiment of the present application is shown, where the method is applied to a terminal as an example, and the method includes:
The video display method in the embodiment of the application is applied to an online video process, and illustratively, the video display method in the embodiment of the application can be applied to a process that a user watches the online video by using a video application program, a process that the user watches an online live broadcast by using a live broadcast application program, or a process that the user carries out online video chat by using an instant messaging application program.
In a possible implementation manner, when a terminal performs video display, two processes, namely video decoding and video rendering display, are generally required, where the video decoding is used to decode a received video stream through a decoder to obtain a video frame, and then perform video rendering display based on the video frame.
Based on the video display method in the related art, the definition of the video watched by the user completely depends on the received video stream, that is, if the terminal receives a video stream with a lower resolution, the corresponding video displayed by the terminal is also the video with the lower resolution, and for some scenes requiring resolution reduction, for example, when the network state is poor, the video application program may automatically adjust the resolution down to maintain stable playing of the video (corresponding to the video stream with the lower resolution received by the terminal), which may cause a large influence on the watching definition of the user.
In this embodiment, for the scene with lower resolution (reduction), a corresponding video display method is provided to ensure the viewing quality of the user. Therefore, before displaying the video, the terminal first needs to identify the scene with lower resolution (or reduced resolution), and in a possible implementation, the terminal first decodes the received video stream through a decoder to obtain the required resolution information and the current video frame, so as to execute the video display method shown in the embodiment of the present application when it is determined that the lower resolution (or reduced resolution) is detected based on the resolution information.
Illustratively, the video stream may be decoded into single-frame images (video frames) in the yuv format, so as to obtain resolution information corresponding to each single-frame image.
And 102, responding to the situation that the first resolution is lower than the target resolution, performing super-resolution processing on the current video frame to obtain a target video frame, wherein the target video frame corresponds to the second resolution, and the second resolution is higher than the first resolution.
The target resolution can be a preset resolution which can be set by a user, or can be set by an application developer aiming at different application programs or dynamically set based on historical operating habits of the user; or the target resolution may also be a historical resolution corresponding to a previous preset time period.
In a possible implementation manner, when the terminal determines that the first resolution is lower than the target resolution, it indicates that a scene with a lower resolution or a lower resolution exists currently, a resolution enhancement operation needs to be performed to improve the video viewing quality of the user, and correspondingly, a super-resolution processing needs to be performed on the current video frame, so that the current video frame is processed into a target video frame with a higher video resolution, for example, a second resolution higher than the first resolution.
Optionally, the super-resolution processing may perform a multiple-preset super-resolution method on the current video frame, and then intelligently fill pixels through an Artificial Intelligence (AI) deep learning model, so as to improve the resolution of the video frame and correspondingly improve the video definition. Illustratively, the current video frame may be input into a super-resolution algorithm model, and the super-resolution algorithm model performs super-resolution processing on the current video frame, so as to obtain a target video frame with improved resolution.
As shown in fig. 2, a schematic diagram of a super-resolution processing process according to an exemplary embodiment of the present application is shown. The first resolution corresponding to the current video frame 201 is "X × Y", the current video frame 201 is input into the super-resolution processing model 202, the super-resolution processing model 202 performs super-resolution amplification on the video image by twice the length and width, and pixel padding is performed, so that a target video frame 203 output by the super-resolution processing model 202 is obtained, and the second resolution corresponding to the target video frame 203 is "2X × 2Y".
Alternatively, in order to improve the accuracy of determining the timing of the super-resolution processing, before the super-resolution processing is performed, prompt information, such as "whether the current resolution is low and the super-resolution processing is performed" may be displayed on the current video interface, and when a confirmation operation by the user is received, the step of the super-resolution processing may be performed.
And 103, displaying the video based on the target video frame.
In a possible implementation manner, after the terminal obtains the target video frame with the improved resolution, the target video frame can be displayed on a terminal screen through a display module in the terminal.
In summary, in the embodiment of the present application, in an online video display scene, when a terminal detects that a resolution corresponding to a current video frame is low or the resolution is reduced, super-resolution processing may be performed on the video frame before video rendering and displaying is performed, so as to improve the resolution of the video, and thus improve the definition of the video. Under the scene that the video resolution is reduced due to the requirement of the traffic bandwidth, the video resolution can be improved through the terminal side, the requirement for saving the traffic bandwidth is met, the requirement of a user for the video definition can be considered, and the video watching quality of the user under the condition of poor network environment can be improved.
In the process of performing super-resolution processing on a current video frame, high power consumption is required, super-resolution processing is not required in any scenes with low resolution or reduced resolution, for example, in scenes with good current network state, the video watching definition can be directly improved by receiving a video stream with higher resolution, so that in a possible implementation mode, a preset super-resolution condition is set so as to perform super-resolution processing in a proper scene, and the power consumption of a terminal is reduced as much as possible.
Referring to fig. 3, a flowchart of a video display method provided by another exemplary embodiment of the present application is shown, where the method is applied to a terminal as an example, and the method includes:
And 302, in response to that the first resolution is lower than the target resolution and a preset super-resolution condition is met, performing super-resolution processing on the current video frame to obtain a target video frame.
The reason why the super-resolution processing operation is performed to ensure the viewing quality of the user in the low-resolution scene may include: the application program automatically adjusts the video resolution (video definition) according to the current network state, and automatically adjusts the video resolution according to the user's own requirements, wherein if the user automatically adjusts the video resolution, super-resolution processing may not be required, so that in order to avoid the influence of the super-resolution processing on the power consumption of the terminal when the resolution is arbitrarily monitored to be low, a super-resolution condition is set, that is, the super-resolution processing needs to be performed on the video frame only when the preset super-resolution condition is met.
Wherein the preset super-resolution condition may include: the current network type corresponding to the current network is at least one of a mobile network, a current network speed corresponding to the current network is lower than a preset network speed threshold value, and a user operation habit.
Optionally, based on the purpose of maintaining smooth video playing, before the terminal performs super-resolution processing, the current network state may be obtained, and if the current network state is poor, for example, the current network speed corresponding to the current network is lower than a preset network speed threshold, the super-resolution processing may be performed on the current video frame in order to maintain video flow display or smooth video playing, so that when the network state is poor, the smooth video playing is considered (the resolution is adjusted downward), and meanwhile, the viewing quality of the video is ensured (the video is viewed by the super-resolution processing).
Please refer to fig. 4, which is a schematic diagram illustrating resolution conversion in a video display process. When the network state is detected to be degraded, the video application automatically adjusts and reduces the video resolution to meet the bandwidth requirement, for example, the first resolution "480 × 640" corresponding to the first video 401 is reduced to the second resolution "240 × 320" corresponding to the second video 402, so as to reduce the network bandwidth and reduce the data transmission, thereby ensuring the continuity of the video stream and avoiding video blockage. However, adapting to the network status by reducing the resolution may result in the resolution perceived by the user on the terminal screen being reduced, and in order to improve the viewing quality of the user, the second video 402 may be super-resolution processed to improve the resolution before the video is displayed, so as to obtain the third video 403, for example, the third resolution is increased from the second resolution "240 × 320" to the third resolution "480 × 640".
Optionally, based on the purpose of saving the traffic bandwidth, before the terminal performs super-resolution processing, the current network type may be acquired, and if the current network type is a mobile network, the video resolution may be adjusted downward in order to save traffic.
Optionally, when the preset super-resolution condition is a user operation habit, when the resolution reduction operation of the video received by the terminal is actively triggered by the user, it may indicate that the user wants to watch the video at the definition currently, and correspondingly, in order to reduce the power consumption of the terminal, super-resolution processing may not be performed on the current video frame; if the resolution down-regulation operation of the video received by the terminal is triggered by the application program, it may indicate that the user does not want to down-regulate the resolution, but due to the influence of external factors (such as poor network environment), the video frame may be super-resolution processed to ensure smooth playing of the video.
Optionally, the preset resolution condition may further include: if the first resolution is higher than the preset resolution, super-resolution processing is not required, that is, although the terminal detects that the video resolution is reduced, the influence of the reduced video resolution on the video definition is small, for example, the super-definition mode is changed into the high-definition mode, and the super-resolution processing on the current video frame is not required in consideration of the power consumption of the terminal.
Optionally, the determining the target resolution may include the following steps:
the method comprises the steps of firstly, determining a target resolution based on a historical resolution, wherein the historical resolution is the resolution corresponding to a video frame in a preset time period before a video resolution reduction operation is received.
In one possible implementation, when the terminal receives a resolution reduction operation, a historical resolution indicated by the resolution reduction operation may be acquired, so that the target resolution is determined based on the historical resolution.
Illustratively, if the video resolution reduction operation indicates: the video resolution is adjusted from "800 × 480" to "400 × 240", and the corresponding historical resolution is "800 × 480".
Optionally, in order to improve the accuracy of the acquired historical resolution, the resolution corresponding to the video frame in a preset time period before the video resolution operation may be acquired, and illustratively, the preset time period may be 10 min.
Secondly, acquiring a candidate resolution indicated by the historical resolution updating operation; and determining the candidate resolution with the highest candidate resolution selection frequency as the target resolution.
Taking video watching as an example, a user usually selects the most appropriate video definition by himself, so that in order to avoid the resolution from being lowered to the resolution corresponding to the video definition commonly used by the user, the target resolution can be determined by obtaining the historical resolution updating operation of the user, thereby avoiding the influence on the video watching quality of the user due to external factors.
In one possible implementation manner, the candidate resolution indicated by the historical resolution updating operation may be acquired, and the commonly used video resolution of the user may be abstracted from the resolution selection frequency, so as to determine the commonly used resolution as the target resolution, so that when the resolution is reduced to the commonly used resolution, the super-resolution processing may be performed on the video frame to ensure the video viewing quality of the user.
Alternatively, the candidate resolution with the highest resolution selection frequency may be determined as the target resolution. Illustratively, taking video watching as an example, a user may generally manually adjust the video definition, and may determine the resolution corresponding to the definition commonly used by the user as the target resolution.
And step 303, performing video display based on the target video frame.
The implementation manner of this step may refer to the above embodiments, which are not described herein.
In the embodiment, whether the video frame needs to be subjected to super-resolution processing is determined by presetting a super-resolution condition, so that the super-resolution processing is stopped to avoid the influence on the power consumption of the terminal under the condition that the super-resolution processing is not needed; in addition, by determining the historical resolution as the target resolution, resolution raising operation can be performed in time when resolution lowering operation is detected; in addition, the common resolution is determined as the target resolution, so that the resolution can be timely improved under the condition that the detected resolution is lower than the common resolution of the user, and the watching quality of the user is ensured.
In a possible application scenario, when the current network state is relatively poor, the resolution ratio of the application program may be automatically adjusted to ensure smooth playing of the video, and correspondingly, the terminal may monitor that the first resolution ratio is lower than the historical resolution ratio before the resolution ratio adjustment operation.
Referring to fig. 5, a flowchart of a video display method provided by another exemplary embodiment of the present application is shown, where the method is applied to a terminal as an example, and the method includes:
In a possible implementation manner, when the application detects that the current network state is poor, for example, the current network speed is lower than a preset network threshold, and the current network state is a mobile network, in order to reduce a demand for bandwidth traffic in video viewing, the video definition may be automatically adjusted downward (that is, the video resolution is correspondingly reduced), and correspondingly, when the terminal receives a video resolution reduction operation, the historical resolution before the video resolution reduction operation may be obtained, so that a super-resolution processing operation is subsequently performed based on the historical resolution.
Optionally, in order to avoid an influence on the acquired historical resolution due to a sudden change of the network condition, the corresponding historical resolution within a preset time period before the video resolution reduction operation may be acquired. Illustratively, the predetermined time period may be within a predetermined 10 min.
Optionally, the historical resolution may be obtained from resolution information decoded from the historical video stream, or may be obtained from resolution indicated by the video resolution reduction operation, and illustratively, if the resolution reduction operation indicates that the current video is switched from the high-definition mode to the standard-definition mode, the historical resolution may be obtained from image resolution corresponding to the high-definition model.
In the present embodiment, the history resolution by which whether or not the super-resolution processing operation is performed can be determined as the target resolution.
The implementation manner of this step may refer to the above embodiments, which are not described herein.
Wherein the target resolution is a historical resolution.
In a possible implementation manner, the terminal detects resolution information corresponding to the video in real time, and when the terminal detects that the first resolution corresponding to the current video frame is lower than the historical resolution, it indicates that the video resolution reduction operation is completed, and correspondingly, the super-resolution processing needs to be performed on the video resolution. In order to ensure the watching quality of the user, when the super-resolution processing is carried out, the current video frame can be directly processed into the historical resolution from the first resolution, so that the requirement for saving the flow bandwidth is met in the video display process, and the watching definition of the user is ensured.
Since the historical resolution indicated by the resolution reduction operation is dynamically determined by the application program based on the current network state, if the current network state is very poor, correspondingly, the historical resolution may have a large difference from the first resolution after the resolution reduction operation, and if the resolution is large, if the current video frame is further processed into the historical resolution, more pixels may need to be filled, and instead, the purpose of improving the definition cannot be achieved.
In an illustrative example, based on fig. 5, as shown in fig. 6, step 504 may include step 504A and step 504B.
In response to the first resolution being lower than the target resolution, a resolution difference between the first resolution and the historical resolution is obtained, step 504A.
In one possible implementation, after the terminal determines the first resolution and the historical resolution, whether the current video frame needs to be adjusted from the first resolution to the historical resolution may be determined by comparing the degree of difference between the first resolution and the historical resolution.
Wherein the difference program can be determined by the resolution difference. Illustratively, since the resolution is generally represented by a long-side pixel value and a wide-side pixel value, the resolution difference can be represented by a long-side pixel value difference or a wide-side pixel value difference.
Step 504B, in response to the resolution difference being smaller than the preset resolution threshold, processing the current video frame from the first resolution to a historical resolution to obtain a target video frame.
Wherein the preset resolution threshold can be set by a developer. The preset resolution threshold may also be represented by a preset long-side pixel threshold and/or a preset wide-side pixel threshold. Illustratively, the preset resolution threshold may be expressed as "400 × 240".
In a possible implementation manner, after it is determined that the resolution difference is smaller than the preset resolution threshold, it indicates that the difference between the first resolution and the historical resolution is small, and the purpose of improving the definition can be achieved through super-resolution processing.
Optionally, when the terminal determines that the resolution difference is greater than the preset resolution threshold, it indicates that the current video frame cannot be processed from the first resolution to the historical resolution, or the current video frame is processed from the first resolution to the historical resolution so as not to achieve the purpose of improving the definition.
Illustratively, when the terminal determines that the historical resolution is "960 × 1280" and the first resolution is "240 × 320", it is seen that the historical resolution and the first resolution have a large difference and cannot perform the resolution enhancement operation, it may determine that the third resolution is "480 × 640" based on the historical resolution and the first resolution, and process the current video frame from the first resolution to the third resolution.
And 505, displaying the video based on the target video frame.
The implementation manner of this step may refer to the above embodiments, which are not described herein.
In the embodiment, the current video frame is processed into the historical resolution before the resolution reduction operation is received from the first resolution, so that the purpose of saving the traffic bandwidth is met and the video watching quality of the user is ensured under the condition of no perception of the user; in addition, whether the video frame is processed from the first resolution to the historical resolution or not is determined by comparing the difference degree between the first resolution and the historical resolution, the super-resolution processing can be avoided when the difference degree is large, the video watching quality is prevented from being influenced, and the super-resolution processing effect is ensured.
Since the super-resolution processing needs to process each frame of video frame, correspondingly, a large power consumption is needed, and in order to avoid the influence of the super-resolution processing on the endurance of the terminal, in one possible implementation, before the super-resolution processing, a power consumption judgment needs to be performed to determine whether the endurance of the terminal is influenced.
On the basis of fig. 1, as shown in fig. 7, step 102 may be replaced with step 701 and step 702.
And 701, in response to the fact that the first resolution is lower than the target resolution and the current residual electric quantity is higher than a preset electric quantity threshold, performing super-resolution processing on the current video frame to obtain a target video frame.
In order to avoid the influence of super-resolution processing on the endurance of the terminal, in one possible implementation manner, when the first resolution is determined to be lower than the target resolution, the current residual capacity of the terminal is obtained, if the current residual capacity of the terminal is higher than a preset capacity threshold, the current terminal is sufficient in capacity, the influence of super-resolution processing on the endurance of the terminal is small, and super-resolution processing can be performed on the current video frame to obtain a target video frame; otherwise, if the current residual electric quantity of the terminal is lower than the preset electric quantity threshold value, the current electric quantity of the terminal is less, and in order to maintain the endurance requirement of the terminal, the super-resolution processing on the current video frame is stopped.
Optionally, in the process of performing super-resolution processing on the terminal, if it is detected that the current remaining power is low, the super-resolution processing on the current video frame may be stopped first, and the terminal is maintained to continue the journey preferentially.
Optionally, before performing the super-resolution processing on the terminal, it may also be determined whether the terminal is in a power saving mode, and if the terminal is in the power saving mode, it indicates that a user has a cruising demand on the terminal, and the super-resolution processing is stopped; if the terminal is not in the power saving mode, the super-resolution processing operation may be performed.
Optionally, the preset electric quantity threshold may be set by a developer, or may be set by a user, or may be automatically adjusted based on an environment where the terminal is located, for example, the preset electric quantity threshold may be 40%.
Because the power consumption of the terminal is also related to the current terminal operating state, for example, the current memory occupancy rate is high, the current power consumption of the terminal is correspondingly high, or the current Central Processing Unit (CPU) load is increased, and the current power consumption of the terminal is correspondingly high, in a possible implementation manner, when the terminal determines that the first resolution is lower than the target resolution, the current memory occupancy rate can be obtained, if the current memory occupancy rate is high and higher than a preset occupancy rate threshold value, the current power consumption of the terminal is high, if the super-resolution Processing is performed, the endurance of the terminal is obviously affected, and the super-resolution Processing is correspondingly required to be stopped; on the contrary, if the terminal determines that the current memory occupancy rate is lower and is lower than the preset occupancy rate threshold value, it indicates that the terminal does not have a running task with larger power consumption currently, and the super-resolution processing can be performed on the current video frame.
Optionally, the preset occupancy rate threshold may be 70%, schematically, if the current memory occupancy rate of the terminal is 80%, which indicates that the current memory occupancy rate is high, the super-resolution processing is stopped; if the current memory occupancy rate is 30%, the current memory occupancy rate is low, and super-resolution processing can be performed.
In the embodiment, the current power consumption state of the terminal is judged, so that when the terminal has a cruising demand, super-resolution processing is stopped to reduce the power consumption of the terminal, and the cruising ability of the terminal is improved; and the super-resolution processing is carried out under the condition that the power consumption of the terminal is low or the electric quantity is sufficient, so that the video watching quality of a user is improved.
Referring to fig. 8, a flowchart illustrating a complete video display method according to an exemplary embodiment of the present application is shown, the method including:
step 801, a video stream is received.
The video stream may be a live video stream, an instant messaging video stream, and an online video stream.
Step 802, decoding the video stream through a system decoder to obtain a video frame and resolution information corresponding to the video frame.
The system decoder is used for decoding the video stream to obtain video frames contained in the video stream and resolution information corresponding to each frame of video frame.
Step 803, real-time resolution monitoring is performed on the resolution information corresponding to the video frame.
Step 804, whether the resolution is reduced.
When the resolution is determined to be reduced, the process may proceed to step 805 to perform super-resolution processing on the video frame, otherwise, the process may proceed to step 803 without performing super-resolution processing, and continue to perform resolution monitoring.
And step 805, performing real-time super-resolution processing on the video frame.
And 806, displaying the video based on the video frame.
After the super-resolution processing is performed on the video frame, video rendering display can be performed based on the processed video frame.
The following are embodiments of the apparatus of the present application that may be used to perform embodiments of the method of the present application. For details which are not disclosed in the embodiments of the apparatus of the present application, reference is made to the embodiments of the method of the present application.
Referring to fig. 9, a block diagram of a video display apparatus according to an embodiment of the present application is shown. The device has the function executed by the terminal in the method embodiment, and the function can be realized by hardware or by hardware executing corresponding software. As shown in fig. 9, the apparatus may include:
a video decoding module 901, configured to decode a video stream through a decoder to obtain a current video frame and a first resolution corresponding to the current video frame;
a first processing module 902, configured to perform super-resolution processing on the current video frame in response to that the first resolution is lower than a target resolution, so as to obtain a target video frame, where the target video frame corresponds to a second resolution, and the second resolution is higher than the first resolution;
and a video display module 903, configured to perform video display based on the target video frame.
Optionally, the first processing module 902 includes:
and the first processing unit is used for responding to the situation that the first resolution is lower than the target resolution and meets a preset super-resolution condition, and performing super-resolution processing on the current video frame to obtain the target video frame.
Optionally, the preset super-resolution condition includes: the current network type corresponding to the current network is at least one of a mobile network, a current network speed corresponding to the current network is lower than a preset network speed threshold value, and a user operation habit.
Optionally, the apparatus further comprises:
an obtaining module, configured to obtain, in response to a video resolution reduction operation, the historical resolution before the video resolution reduction operation;
a first determining module for determining the historical resolution as the target resolution;
the first processing module 902 includes:
and the second processing unit is used for processing the current video frame from the first resolution to the historical resolution to obtain the target video frame.
Optionally, the second processing unit is further configured to:
acquiring a resolution difference between the first resolution and the historical resolution;
and in response to the fact that the resolution difference value is smaller than a preset resolution threshold value, processing the current video frame from the first resolution to the historical resolution to obtain the target video frame.
Optionally, the apparatus further comprises:
and the second processing module is used for processing the current video frame from the first resolution to a third resolution in response to the resolution difference being greater than the preset resolution threshold, so as to obtain the target video frame, wherein the third resolution is greater than the first resolution and smaller than the historical resolution.
Optionally, the apparatus further comprises:
a second determining module, configured to determine the target resolution based on a historical resolution, where the historical resolution is a resolution corresponding to a video frame in a preset time period before a video resolution reduction operation is received;
or the like, or, alternatively,
a third determining module, configured to obtain a candidate resolution indicated by the historical resolution updating operation; and determining the candidate resolution with the highest candidate resolution selection frequency as the target resolution.
Optionally, the first processing module 902 includes:
the third processing unit is used for performing super-resolution processing on the current video frame to obtain the target video frame in response to the fact that the first resolution is lower than the target resolution and the current residual electric quantity is higher than a preset electric quantity threshold;
or the like, or, alternatively,
and the fourth processing unit is used for performing super-resolution processing on the current video frame to obtain the target video frame in response to that the first resolution is lower than the target resolution and the memory occupancy rate is lower than a preset occupancy rate threshold.
In summary, in the embodiment of the present application, in an online video display scene, when a terminal detects that a resolution corresponding to a current video frame is low or the resolution is reduced, super-resolution processing may be performed on the video frame before video rendering and displaying is performed, so as to improve the resolution of the video, and thus improve the definition of the video. Under the scene that the video resolution is reduced due to the requirement of the traffic bandwidth, the video resolution can be improved through the terminal side, the requirement for saving the traffic bandwidth is met, the requirement of a user for the video definition can be considered, and the video watching quality of the user under the condition of poor network environment can be improved.
It should be noted that: in the above embodiment, when the device implements the functions thereof, only the division of the functional modules is illustrated, and in practical applications, the functions may be distributed by different functional modules according to needs, that is, the internal structure of the device may be divided into different functional modules to implement all or part of the functions described above. In addition, the video display apparatus and the video display method provided by the above embodiments belong to the same concept, and specific implementation processes thereof are described in detail in the method embodiments and are not described herein again.
Referring to fig. 10, a block diagram of a terminal according to an exemplary embodiment of the present application is shown. Terminal 1000 in the present application can include one or more of the following: a processor 1010, a memory 1020, and a touch display 1030.
The Memory 1020 may include a Random Access Memory (RAM) or a Read-Only Memory (ROM). Optionally, the memory 1020 includes a non-transitory computer-readable medium. The memory 1020 may be used to store instructions, programs, code, sets of codes, or sets of instructions. The memory 1020 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playing function, an image playing function, and the like), instructions for implementing the above-described method embodiments, and the like, and the operating system may be an Android (Android) system (including a system based on Android system depth development), an IOS system developed by apple inc (including a system based on IOS system depth development), or other systems. The stored data area may also store data created by terminal 1400 in use (e.g., phone book, audio-video data, chat log data), and the like.
The touch display screen 1030 is used for receiving a touch operation of a user on or near the touch screen using a finger, a touch pen, or any other suitable object, and displaying user interfaces of various applications. The touch display screen is typically provided on the front panel of terminal 1000. The touch display screen may be designed as a full-face screen, a curved screen, or a profiled screen. The touch display screen can also be designed to be a combination of a full-face screen and a curved-face screen, and a combination of a special-shaped screen and a curved-face screen, which is not limited in the embodiment of the present application.
Optionally, terminal 1000 can also include a touch display screen, which can be a capacitive touch display screen for receiving touch operations by a user on or near the touch display screen using any suitable object such as a finger, a stylus, etc., and displaying user interfaces of various applications. The touch display screen is typically provided on the front panel of terminal 1000. The touch display screen may be designed as a full-face screen, a curved screen, or a profiled screen. The touch display screen can also be designed to be a combination of a full-face screen and a curved-face screen, and a combination of a special-shaped screen and a curved-face screen, which is not limited in the embodiment of the present application.
In addition, those skilled in the art will appreciate that the configuration of terminal 1000 illustrated in the above-described figures is not intended to be limiting, and that terminal 1000 can include more or less components than those illustrated, or some components can be combined, or a different arrangement of components. For example, the terminal 1000 further includes a radio frequency circuit, a shooting component, an audio circuit, a Wireless Fidelity (WiFi) component, a power supply, a bluetooth component, and other components, which are not described herein again.
The embodiment of the present application also provides a computer-readable storage medium, which stores at least one program code, and the program code is loaded and executed by a processor to implement the video display method according to the above embodiments.
According to an aspect of the application, a computer program product or computer program is provided, comprising computer instructions, the computer instructions being stored in a computer readable storage medium. The processor of the terminal reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the terminal performs the video display method provided in the various alternative implementations of the above aspects.
It should be understood that reference to "a plurality" herein means two or more. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. In addition, the step numbers described herein only exemplarily show one possible execution sequence among the steps, and in some other embodiments, the steps may also be executed out of the numbering sequence, for example, two steps with different numbers are executed simultaneously, or two steps with different numbers are executed in a reverse order to the order shown in the figure, which is not limited by the embodiment of the present application.
The above description is only exemplary of the present application and should not be taken as limiting, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.
Claims (11)
1. A method for video display, the method comprising:
decoding a video stream through a decoder to obtain a current video frame and a first resolution corresponding to the current video frame;
responding to the first resolution ratio lower than a target resolution ratio, performing super-resolution processing on the current video frame to obtain a target video frame, wherein the target video frame corresponds to a second resolution ratio which is higher than the first resolution ratio;
and performing video display based on the target video frame.
2. The method of claim 1, wherein said super-resolution processing the current video frame in response to the first resolution being lower than a target resolution comprises:
and responding to the fact that the first resolution is lower than the target resolution and meets a preset super-resolution condition, and performing super-resolution processing on the current video frame to obtain the target video frame.
3. The method of claim 2, wherein the preset super-resolution condition comprises: the current network type corresponding to the current network is at least one of a mobile network, a current network speed corresponding to the current network is lower than a preset network speed threshold value, and a user operation habit.
4. The method according to any of claims 1 to 3, wherein before the decoding of the video stream by the decoder to obtain the current video frame and the first resolution corresponding to the current video frame, the method further comprises:
in response to a video resolution reduction operation, acquiring the historical resolution before the video resolution reduction operation;
determining the historical resolution as the target resolution;
the super-resolution processing of the current video frame to obtain a target video frame includes:
and processing the current video frame from the first resolution to the historical resolution to obtain the target video frame.
5. The method of claim 4, wherein said processing said current video frame from said first resolution to said historical resolution to obtain said target video frame comprises:
acquiring a resolution difference between the first resolution and the historical resolution;
and in response to the fact that the resolution difference value is smaller than a preset resolution threshold value, processing the current video frame from the first resolution to the historical resolution to obtain the target video frame.
6. The method of claim 5, wherein after obtaining the resolution difference between the first resolution and the historical resolution, the method further comprises:
and in response to the fact that the resolution difference value is larger than the preset resolution threshold value, processing the current video frame from the first resolution to a third resolution to obtain the target video frame, wherein the third resolution is larger than the first resolution and smaller than the historical resolution.
7. The method of any of claims 1 to 3, wherein before performing super-resolution processing on the current video frame in response to the first resolution being lower than the target resolution to obtain the target video frame, the method further comprises:
determining the target resolution based on historical resolution, wherein the historical resolution is the resolution corresponding to a video frame in a preset time period before the video resolution reduction operation is received;
or the like, or, alternatively,
acquiring a candidate resolution indicated by the historical resolution updating operation; and determining the candidate resolution with the highest candidate resolution selection frequency as the target resolution.
8. The method of any of claims 1 to 3, wherein said performing super-resolution processing on the current video frame to obtain the target video frame in response to the first resolution being lower than the target resolution comprises:
in response to the fact that the first resolution is lower than the target resolution and the current residual electric quantity is higher than a preset electric quantity threshold value, performing super-resolution processing on the current video frame to obtain the target video frame;
or the like, or, alternatively,
and in response to the fact that the first resolution is lower than the target resolution and the memory occupancy rate is lower than a preset occupancy rate threshold, performing super-resolution processing on the current video frame to obtain the target video frame.
9. A video display apparatus, characterized in that the apparatus comprises:
the video decoding module is used for decoding the video stream through a decoder to obtain a current video frame and a first resolution corresponding to the current video frame;
the first processing module is used for responding to the situation that the first resolution is lower than a target resolution, performing super-resolution processing on the current video frame to obtain a target video frame, wherein the target video frame corresponds to a second resolution, and the second resolution is higher than the first resolution;
and the video display module is used for carrying out video display based on the target video frame.
10. A terminal characterized in that it comprises a processor and a memory, in which at least one program code is stored, which is loaded and executed by the processor to implement the video display method according to any one of claims 1 to 8.
11. A computer-readable storage medium having stored therein at least one program code, the program code being loaded and executed by a processor to implement the video display method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110246014.1A CN113055742B (en) | 2021-03-05 | 2021-03-05 | Video display method, device, terminal and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110246014.1A CN113055742B (en) | 2021-03-05 | 2021-03-05 | Video display method, device, terminal and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113055742A true CN113055742A (en) | 2021-06-29 |
| CN113055742B CN113055742B (en) | 2023-06-09 |
Family
ID=76510269
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110246014.1A Active CN113055742B (en) | 2021-03-05 | 2021-03-05 | Video display method, device, terminal and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113055742B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114339412A (en) * | 2021-12-30 | 2022-04-12 | 咪咕文化科技有限公司 | Video quality enhancement method, mobile terminal, storage medium and device |
| CN114466220A (en) * | 2022-01-29 | 2022-05-10 | 维沃移动通信有限公司 | Video downloading method and electronic equipment |
| CN115883853A (en) * | 2021-09-26 | 2023-03-31 | 腾讯科技（深圳）有限公司 | Video frame playing method, device, equipment and storage medium |
| WO2023231630A1 (en) * | 2022-05-30 | 2023-12-07 | 荣耀终端有限公司 | Video data processing method, electronic device, and readable storage medium |
| CN117896552A (en) * | 2024-03-14 | 2024-04-16 | 浙江华创视讯科技有限公司 | Video conference processing method, video conference system and related device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130294516A1 (en) * | 2012-05-07 | 2013-11-07 | Cellco Partnership D/B/A Verizon Wireless | Method and apparatus for finding a motion vector |
| CN108496198A (en) * | 2017-10-09 | 2018-09-04 | 华为技术有限公司 | A kind of image processing method and equipment |
| CN111340711A (en) * | 2020-05-21 | 2020-06-26 | 腾讯科技（深圳）有限公司 | Super-resolution reconstruction method, device, equipment and storage medium |
| CN112153465A (en) * | 2019-06-28 | 2020-12-29 | 北京京东尚科信息技术有限公司 | Image loading method and device |
-
2021
- 2021-03-05 CN CN202110246014.1A patent/CN113055742B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130294516A1 (en) * | 2012-05-07 | 2013-11-07 | Cellco Partnership D/B/A Verizon Wireless | Method and apparatus for finding a motion vector |
| CN108496198A (en) * | 2017-10-09 | 2018-09-04 | 华为技术有限公司 | A kind of image processing method and equipment |
| CN112153465A (en) * | 2019-06-28 | 2020-12-29 | 北京京东尚科信息技术有限公司 | Image loading method and device |
| CN111340711A (en) * | 2020-05-21 | 2020-06-26 | 腾讯科技（深圳）有限公司 | Super-resolution reconstruction method, device, equipment and storage medium |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115883853A (en) * | 2021-09-26 | 2023-03-31 | 腾讯科技（深圳）有限公司 | Video frame playing method, device, equipment and storage medium |
| CN115883853B (en) * | 2021-09-26 | 2024-04-05 | 腾讯科技（深圳）有限公司 | Video frame playing method, device, equipment and storage medium |
| CN114339412A (en) * | 2021-12-30 | 2022-04-12 | 咪咕文化科技有限公司 | Video quality enhancement method, mobile terminal, storage medium and device |
| CN114339412B (en) * | 2021-12-30 | 2024-02-27 | 咪咕文化科技有限公司 | Video quality enhancement method, mobile terminal, storage medium and device |
| CN114466220A (en) * | 2022-01-29 | 2022-05-10 | 维沃移动通信有限公司 | Video downloading method and electronic equipment |
| WO2023231630A1 (en) * | 2022-05-30 | 2023-12-07 | 荣耀终端有限公司 | Video data processing method, electronic device, and readable storage medium |
| CN117896552A (en) * | 2024-03-14 | 2024-04-16 | 浙江华创视讯科技有限公司 | Video conference processing method, video conference system and related device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113055742B (en) | 2023-06-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113055742B (en) | Video display method, device, terminal and storage medium | |
| CN112331145B (en) | Display screen frequency conversion method, DDIC chip, display screen module and terminal | |
| CN109729405B (en) | Video processing method and device, electronic equipment and storage medium | |
| US9892716B2 (en) | Image display program, image display method, and image display system | |
| CN109379627B (en) | Video processing method, video processing device, electronic equipment and storage medium | |
| CN109379624B (en) | Video processing method and device, electronic equipment and storage medium | |
| CN109640167B (en) | Video processing method and device, electronic equipment and storage medium | |
| CN113608713B (en) | Variable frequency display method, DDIC, display screen module and terminal | |
| EP4002281A1 (en) | Layer composition method and apparatus, electronic device, and storage medium | |
| JP7295950B2 (en) | Video enhancement control method, device, electronic device and storage medium | |
| US11627369B2 (en) | Video enhancement control method, device, electronic device, and storage medium | |
| CN109120988B (en) | Decoding method, decoding device, electronic device and storage medium | |
| CN112203034B (en) | Frame rate control method and device and electronic equipment | |
| CN109165103B (en) | Frame rate control method, device, terminal and storage medium | |
| CN113781949B (en) | Image display method, display driving chip, display screen module and terminal | |
| CN109361950B (en) | Video processing method and device, electronic equipment and storage medium | |
| EP3660823A1 (en) | Method and device for video enhancement, and electronic device using the same | |
| CN115100993A (en) | Display frame rate adjusting method and device, application processor and electronic equipment | |
| CN110858388B (en) | Method and device for enhancing video image quality | |
| CN109587555B (en) | Video processing method and device, electronic equipment and storage medium | |
| CN109167946B (en) | Video processing method, video processing device, electronic equipment and storage medium | |
| CN110572867A (en) | method and device for reducing power consumption of electronic equipment | |
| CN110858389B (en) | Method, device, terminal and transcoding equipment for enhancing video image quality | |
| CN113038276A (en) | Video playing method and device, electronic equipment and storage medium | |
| CN112675535A (en) | Display interface adjusting method and device, network equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |