WO2022161310A1

WO2022161310A1 - Display method and apparatus, and electronic device

Info

Publication number: WO2022161310A1
Application number: PCT/CN2022/073465
Authority: WO
Inventors: 张俪馨
Original assignee: 维沃移动通信有限公司
Priority date: 2021-01-27
Filing date: 2022-01-24
Publication date: 2022-08-04
Also published as: CN112910875A; CN112910875B

Abstract

The present application belongs to the technical field of video processing. Disclosed are a display method and apparatus, and an electronic device. The method comprises: when it is detected that a network state of an electronic device is abnormal, and that a bitrate at which the electronic device plays a target video is reduced to a first bitrate, determining, from among N preset vertical super-resolution models in the electronic device, a target vertical super-resolution model of the target video, wherein N is an integer greater than 1; and when an image frame of the target video is transmitted at the first bitrate, displaying a target image frame, wherein the target image frame is an image frame obtained after the image frame of the target video is processed on the basis of the target vertical super-resolution model.

Description

Display method, device and electronic device

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to Chinese Patent Application No. 202110113902.6 filed in China on Jan. 27, 2021, the entire contents of which are hereby incorporated by reference.

technical field

The present application belongs to the technical field of video processing, and in particular relates to a display method, apparatus and electronic device.

Background technique

With the development of communication technology, electronic devices such as mobile phones and tablet computers have become indispensable tools in people's daily life. Through the video application program installed in the electronic device, the user can use the electronic device to play videos, such as movies and short videos, and so on.

However, in the process of playing the video on the electronic device, the network condition connected to the electronic device may be abnormal. For example, the electronic device enters a tunnel or moves at a high speed, etc., resulting in poor network speed. At this time, in order to ensure that the video can continue to be played, the electronic device usually reduces the bit rate of the currently playing video, for example, from a high-definition bit rate (such as 1080p) to a low-definition bit rate (such as 480p), etc., while the video playback bit rate Lowering will result in lower image quality for video playback.

It can be seen that, when an abnormal network condition occurs in the current video playback process of the electronic device, there is a problem that the image quality of the played video is low due to the reduction of the video playback bit rate.

SUMMARY OF THE INVENTION

The purpose of the embodiments of the present application is to provide a display method, device and electronic device, which can solve the problem that when the current electronic device has abnormal network conditions during the video playback process, there is an image of the video played due to the reduction of the video playback bit rate. low quality issues.

In a first aspect, an embodiment of the present application provides a display method, including:

When it is detected that the network state of the electronic device is abnormal, and the bit rate of the target video played by the electronic device is reduced to the first bit rate, in the N preset vertical super-score models in the electronic device, determine the The target vertical super-score model of the target video, wherein the N is an integer greater than 1;

In the case of transmitting the image frame of the target video at the first bit rate, displaying the target image frame, where the target image frame is an image frame of the target video processed based on the target vertical super-resolution model image frame.

In a second aspect, an embodiment of the present application provides a display device, including:

The super-score model determination module is configured to, when it is detected that the network state of the electronic device is abnormal, and the bit rate of the target video played by the electronic device is reduced to the first bit rate, N presets in the electronic device In the vertical super-resolution model, the target vertical super-resolution model of the target video is determined, wherein the N is an integer greater than 1;

A display module, configured to display the target image frame in the case of transmitting the image frame of the target video at the first bit rate.

In a third aspect, embodiments of the present application provide an electronic device, the electronic device includes a processor, a memory, and a program or instruction stored on the memory and executable on the processor, the program or instruction being The processor implements the steps of the method according to the first aspect when executed.

In a fourth aspect, an embodiment of the present application provides a readable storage medium, where a program or an instruction is stored on the readable storage medium, and when the program or instruction is executed by a processor, the steps of the method according to the first aspect are implemented .

In a fifth aspect, an embodiment of the present application provides a chip, the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is configured to run a program or an instruction, and implement the first aspect the method described.

In a sixth aspect, embodiments of the present application provide a computer program product, including a computer program tangibly embodied on a computer-readable medium, the computer program including program code for executing the method according to the first aspect.

In a seventh aspect, an embodiment of the present application provides an electronic device configured to execute the steps of the method described in the first aspect.

In this embodiment of the present application, when the electronic device reduces the bit rate of the playing video (ie, the target video) due to an abnormal network state, the electronic device can select the corresponding target video from a plurality of preset vertical super-resolution models. The vertical super-resolution model (namely the target vertical super-resolution model), and through the selected vertical super-resolution model, the image frames of the video to be played are processed, and the video after the image frame processing is played. In this way, through the embodiments of the present application, the electronic device can process the image frames of the video played under the low bit rate through the selected vertical super-resolution model, so as to improve the image quality of the video played under the low bit rate, and further improve the electronic device's performance in the video. The image quality when the video is played at a low bit rate due to abnormal network conditions; in addition, since the selected vertical super-resolution model corresponds to the played video, the selected vertical super-resolution model is more suitable for the processing of the played video, Thereby, the processing efficiency can be improved and the energy consumption during the processing can be reduced.

Description of drawings

1 is a schematic flowchart of an embodiment of a display method provided by the present application;

2 is a schematic structural diagram of an embodiment of a display device provided by the present application;

3 is a schematic diagram of a hardware structure of an embodiment of an electronic device provided by the present application;

FIG. 4 is a schematic diagram of a hardware structure of an embodiment of another electronic device provided by the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

The terms "first", "second" and the like in the description and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances so that the embodiments of the present application can be practiced in sequences other than those illustrated or described herein, and distinguish between "first", "second", etc. The objects are usually of one type, and the number of objects is not limited. For example, the first photographing object may be one or more than one. In addition, "and/or" in the description and claims indicates at least one of the connected objects, and the character "/" generally indicates that the associated objects are in an "or" relationship.

The display method provided by the embodiments of the present application will be described in detail below with reference to the accompanying drawings through specific embodiments and application scenarios thereof.

Referring to FIG. 1 , an embodiment of the present application provides a display method, and the display method is applied to an electronic device. As shown in FIG. 1 , the method includes the

following steps

101 and 102 .

Step 101, in the case of detecting that the network state of the electronic device is abnormal, and the code rate of the target video played by the electronic device is reduced to the first code rate, in the preset N vertical super-resolution models in the electronic device, determine the target video. The target vertical super-resolution model, wherein the target vertical super-resolution model is a vertical super-resolution model among N preset vertical super-resolution models in the electronic device, and N is an integer greater than 1.

Step 102 , in the case of transmitting the image frame of the target video at the first bit rate, displaying the target image frame, wherein the target image frame is an image frame after processing the image frame of the target video based on the target vertical super-resolution model.

Here, in the case where the electronic device reduces the bit rate of the playing video (that is, the target video) due to an abnormal network state, the electronic device can select a vertical super-resolution model corresponding to the target video from the preset multiple vertical super-resolution models (that is, the target vertical super-resolution model), and process the image frames of the played video through the selected vertical super-resolution model, and play the video after image frame processing. In this way, through the embodiments of the present application, the electronic device can process the image frames of the video played under the low bit rate through the selected vertical super-resolution model, so as to improve the image quality of the video played under the low bit rate, and further improve the electronic device's performance in the video. The image quality when the video is played at a low bit rate due to abnormal network conditions; in addition, since the selected vertical super-resolution model corresponds to the played video, the selected vertical super-resolution model is more suitable for the processing of the played video, Thereby, the processing efficiency can be improved and the energy consumption during the processing can be reduced.

In the above step 101, during the process of the electronic device running the video playback application and playing the target video, the server can detect whether the current network state of the electronic device is abnormal, and when detecting that the network state of the electronic device is abnormal, it can The current bit rate (for ease of description, referred to as "second bit rate" in this embodiment of the present application) currently used by the electronic device to play the video (that is, the server transmits the image frame of the played video to the electronic device) is reduced to the first bit rate.

Wherein, the above-mentioned server detects whether the current network state of the electronic device is abnormal, which may be that the electronic device reports the network speed or network signal strength of the network it is currently connected to to the server, and the server reports whether the network speed or network signal strength is lower than Corresponding preset thresholds, and in the case of being lower than the corresponding preset thresholds, it is determined that the network state of the electronic device is abnormal, and so on.

In addition, the above-mentioned reduction of the code rate of the target video played by the electronic device to the first code rate may be that the server determines the corresponding first code rate according to the network state after the abnormality of the electronic device occurs, and adjusts the code of the image frame of the transmission target video. The first bit rate is lower than the second bit rate at which the electronic device plays the target video before the abnormal network state occurs.

For example, when the electronic device plays a video (ie, the target video) at a bit rate of 5Mb/s (ie, the second bit rate), if the electronic device detects that its network speed is reduced, and the reduced network speed can only meet the requirements of To play a video at a bit rate of 2Mb/s, the electronic device reduces the bit rate of the video to be played from 5Mb/s to 2Mb/s.

In this embodiment of the present application, after the bit rate of the target video played by the electronic device is reduced to the first bit rate, the electronic device may determine a vertical super-resolution model corresponding to the target video among the preset N vertical super-resolution models As the above target vertical superscore model.

Among them, each vertical super-score model in the above-mentioned N vertical super-score models is an arbitrary deep learning model that can improve image quality. Different vertical super-resolution models in the sub-model are different in at least one of network parameters, network structure, processing accuracy and processing speed, so that different vertical super-resolution models are suitable for processing videos with different video information.

In addition, the above-mentioned N vertical super-score models may be learned in the server and sent to the above-mentioned electronic device for storage by the server, or may be directly learned in the electronic device, and the learning process of the vertical super-score model is as follows: Those skilled in the art are well-known, and will not be described in detail here.

In the embodiment of the present application, the above-mentioned target vertical super-score model based on determining the target video may be a target vertical super-score model corresponding to the target video determined from N vertical super-score models according to a preset rule.

Wherein, the above-mentioned preset rule may be to determine the target vertical super-resolution model based on the corresponding relationship between the video information and the vertical super-resolution model, that is, the above-mentioned determination of the target vertical super-resolution model of the target video may include: In the model, the vertical super-score model that has a corresponding relationship with the video information of the target video is determined as the target vertical super-score model.

It should be noted that the video information of the above-mentioned target video can be any information used as an indicator for selecting a vertical super-score model, and it can be the image quality (including the resolution) of the target video played under the above-mentioned first bit rate. etc.), the bit rate (ie, the first bit rate) of the target video to be played after the network state is abnormal, and the video type information, and so on.

Then, in the case that the above-mentioned video information includes the image quality of the target video played at the first bit rate, the above-mentioned determining the target vertical super-score model may include: in the N vertical super-score models, determining the The vertical superdivision model corresponding to the image quality of the target video played at the high rate is the target vertical superdivision waveform.

For example, it is assumed that the electronic device is preset with a vertical super-resolution model 1 and a vertical super-resolution model 2, and the vertical super-resolution model 1 is suitable for improving the image quality of the video frame of a video with a resolution of FULL D1 (ie, a resolution of 720×576). , the vertical super-resolution model 2 is suitable for improving the image quality of the video frame of the video whose definition is HALF D1 (ie, the resolution is 720×288). If the definition of the target video played by the electronic device at the first bit rate is FULL D1, then the electronic device determines that the vertical super-resolution model 1 is the target vertical super-resolution model; if the target video played by the electronic device at the first bit rate has a If the definition is HALF D1, the electronic device determines the vertical super-resolution model 2 as the target vertical super-resolution model.

Or, in the case that the above-mentioned video information includes the above-mentioned first code rate, the above-mentioned determining the target vertical super-score model may include: among the N vertical super-score models, determining a vertical super-score model that has a corresponding relationship with the first code rate is the vertical superresolution model for the target.

Alternatively, in the case that the above-mentioned video information includes the video type of the target video, the above-mentioned step 102 may include:

Get the video type of the target video;

Among the N vertical super-resolution models, the vertical super-score model corresponding to the video type of the target video is determined as the target vertical super-score model.

Here, the electronic device can determine the corresponding target vertical super-score model among the preset N vertical super-score models according to the video type of the target video, so that the selected vertical super-score model is more suitable for playing at the first bit rate The processing of the video frame of the target video further improves the image quality of the video played at a low bit rate.

In the embodiment of the present application, the video types of the above-mentioned target videos may include any one of characters, animals, plants, buildings, landscapes, animations, etc., and the electronic device may preset a corresponding vertical superscore for each video type The model, that is, the above-mentioned target vertical super-score model is a vertical super-score model that has a preset corresponding relationship with the video type of the above-mentioned target video among the above-mentioned N vertical super-score models.

For example, a vertical super-score model 3 and a vertical hyper-score model 4 are preset in the electronic device, and the vertical hyper-score model 3 has a corresponding relationship with the video type being a character, and the vertical hyper-score model 4 has a corresponding relationship with the video type landscape. Then, if the above-mentioned target video is a video of shooting a person (that is, the video type of the target video is a person), the electronic device determines that the vertical super-score model 3 is the above-mentioned target vertical super-score model.

It should be noted that the above video information of the target video may be video information obtained by the electronic device detecting the target video, or may be video information received by the electronic device and sent by the server.

Specifically, in the case that the above-mentioned video information is the above-mentioned video type, the above-mentioned acquisition of the video type of the target video may include:

In the case of receiving the video type information of the target video, obtain the video type indicated by the video type information, and the video type information is the information sent by the server when the video data of the target video is transmitted; or,

The video type of the target video is determined based on the image of the target image frame, where the target image frame is an image frame of the target video played by the electronic device within a preset time period before the bit rate of the target video is reduced to the first bit rate.

Here, the electronic device can determine the video type of the target video according to the video type information transmitted by the server, thereby improving the efficiency of video processing; it can also determine the video type of the target video according to the video frame of the target video, so that the video of the target video can be obtained. Types are flexible and can meet different needs.

Wherein, in the process of playing the target video, the electronic device can receive the video type information sent by the server in the process of transmitting the video data of the target video, and the electronic device can obtain the video type indicated by the video type information.

In addition, the electronic device may also acquire some image frames of the target video, and determine the video type of the target video according to the image information in the partial image frames.

For example, in the process of playing the target video, the electronic device may acquire image frames within a preset time period (such as 1s, etc.) before reducing the bit rate of the target video to be played back to the first bit rate, and determine the target according to the acquired image frames. The video type of the video. Specifically, if there are more than 60 image frames including animals in the acquired 100 image frames, it is determined that the video type of the target video is animals, and so on.

Of course, the above-mentioned electronic equipment determines the target vertical super-resolution model corresponding to the target video in the N vertical super-resolution models, and is not limited to determining the corresponding target vertical super-resolution model based on the video information of the target video. It can also be based on other The rules determine the target vertical superscore model for the target video.

Specifically, in the above-mentioned N vertical super-resolution models preset in the electronic device, before determining the target vertical super-resolution model of the target video, the method further includes:

The first video clip sent by the server is received, where the first video clip is a partial video clip of the target video.

Among the N preset vertical super-resolution models in the electronic device, determining the target vertical super-resolution model of the target video may include:

In the case where the electronic device plays the target video at the first bit rate, a second video clip is acquired, the second video clip is a video clip with the same content as the first video clip, and the image quality of the first video clip is higher than that of the first video clip The image quality of the two video clips;

Based on the first video segment and the second video segment, a target vertical super-score model is determined among the N vertical super-score models.

Here, the electronic device may determine the above target in the N vertical super-score models according to the first video clip of the target video sent by the server, and the second video clip played at the first bit rate with the same content as the first video clip The vertical super-score model makes the determined target vertical super-score model for processing the target video played at the first bit rate more suitable, and further improves the image quality of the target video played at the first bit rate.

In the embodiment of the present application, the above-mentioned first video clip is a partial video clip of the target video sent by the server before the electronic device reduces the bit rate of playing the target video to the first bit rate; and the above-mentioned second video clip is the electronic device. A video clip having the same content as the first video clip is played at the first bit rate, and the image quality (including resolution, etc.) of the first video clip is higher than that of the second video clip.

Wherein, the above-mentioned acquisition of the first video clip may be that when the server receives a request from the electronic device to play the target video, the server may transmit the video data of the target video to the electronic device, and also acquire at least one video clip of the target video for sending. The data is sent to the electronic device, and the image quality of the at least one video segment is the first image quality. In the case where the electronic device reduces the bit rate of the target video to be played to the first bit rate, the electronic device can obtain the at least one video segment to play The video segment whose time period is after the current time and closest to the current time is regarded as the first video segment.

For example, in the process of transmitting the video data of the target video, the server may send m video clips of the target video to the electronic device, and two adjacent video clips in the m video clips are separated by a first preset duration (eg, 5s). Or 10s, etc.), and the duration of each video clip is the second preset duration (such as 0.5s or 1s, etc.) and the image quality is the first image quality (such as 1080P, etc.), if the electronic device is playing the target video for 20s. If the bit rate is lowered, the electronic device determines a video segment whose playback period is after 20 s among the m video segments as the above-mentioned first video segment, where m is an integer greater than 1.

It should be noted that when the bit rate of the played video is reduced, the reduced first bit rate is no longer compatible with the image quality of the original video. To ensure the smoothness of video playback, the electronic device will The image quality is reduced to an image quality suitable for the first bit rate.

For example, in the case where an electronic device plays a video with a resolution of 1080P (that is, a resolution of 1920×1080) at a bit rate of 5Mb/s, if the electronic device reduces the bit rate of the played video to 2Mb/s, the electronic device will When the video is played at 2Mb/s, the image quality of the played video will be reduced to the definition FUUL D1 (that is, the resolution is 720×576).

The first image quality of the first video clip is higher than the image quality of the second video clip, that is, the first image quality may be the image quality of the target video before the bit rate of the target video is reduced to the first bit rate; Alternatively, it may also be a preset image quality, and the preset image quality is at least higher than the image quality of the target video played at the first bit rate.

For example, in the process of playing the target video at a bit rate of 5Mb/s, the electronic device can receive m video clips with a video resolution of 1080P sent by the server, and the electronic device reduces the bit rate of the target video to 2Mb/s. In the case of s, the electronic device determines the first video clip among the m video clips, that is, the resolution of the first video clip is 1080P, and acquires a second video clip with the same content as the first video clip, that is, the second video clip The resolution is FUUL D1.

In this embodiment of the present application, after the electronic device acquires the first video clip and the second video clip, the electronic device can compare the first video clip and the second video clip, and according to the comparison results, make a comparison between the N vertical super-score models. Determine the target vertical superresolution model.

Specifically, the above-mentioned determination of the target vertical super-score model among the N vertical super-score models based on the first video clip and the second video clip may include:

The image frames of the second video segment are respectively processed based on the N vertical super-resolution models, and N video segments to be compared corresponding to the N vertical super-resolution models are obtained;

Among the N video clips to be compared, determine the target video clip to be compared whose image quality is the closest to the image quality of the first video clip;

The vertical super-score model corresponding to the target video segment to be compared is determined as the target vertical super-score model.

Here, the electronic device may process the second video segment by using the preset N vertical super-score models, and compare the second video segment (ie, the video segment to be compared) processed by each vertical super-score model with the first video segment. Fragments are compared, and the vertical super-score model whose image quality of the processed second video clip is closest to that of the first video clip is determined as the above-mentioned target vertical The image quality processing effect of the video is the best, and the vertical super-score model is used as the target vertical super-score model to further improve the image quality of the target video played at a low bit rate.

For example, assuming that the vertical super-resolution model 5, the vertical super-resolution model 6 and the vertical super-resolution model 7 are preset in the electronic device, the image resolution of the first video clip is 1080P, and the image resolution of the second video clip is Hafl D1 Under the situation, if the image definition of the video clip processed by the vertical super-division model 5 to the second video clip is Full D1, and the image definition of the video clip processed by the vertical super-division model 6 to the second video clip is 960P, The image definition of the video clip processed by the vertical super-resolution model 7 on the second video clip is 720P, then the electronic device determines that 960P is the closest to 1080P, and determines the vertical super-resolution model 6 as the above-mentioned target vertical super-resolution model.

It should be noted that, since the process of processing the video frame of the video by the vertical super-segmentation model is well known to those skilled in the art, it will not be repeated here.

Alternatively, the above-mentioned determination of the target vertical super-score model among the N vertical super-score models based on the first video clip and the second video clip may include:

obtaining the degree of difference between the image quality of the second video clip and the image quality of the first video clip;

Among the N vertical super-resolution models, the vertical super-resolution model corresponding to the obtained degree of difference is determined as the target vertical super-resolution model.

Here, the electronic device may also determine the target vertical super-score model corresponding to the degree of difference among the N vertical super-score models according to the degree of difference between the image quality of the second video segment and the image quality of the first video segment, so as to determine the The method of the target vertical super-resolution model is simpler, the efficiency of determining the target vertical super-resolution model is improved, and the efficiency of video processing is further improved.

It should be noted that, in the above-mentioned N vertical super-resolution models, the vertical super-resolution model corresponding to the obtained degree of difference is determined as the target vertical super-resolution model, which may be preset in the electronic device. The corresponding relationship of the models, the electronic device may determine the vertical super-resolution model that has a corresponding relationship with the acquired difference degree as the above-mentioned target vertical super-resolution model.

For example, it is assumed that a vertical super-score model 8 and a vertical super-score model 9 are preset in the electronic device, and the preset vertical super-score model 8 corresponds to 1 to 2 preset levels (including 1080P, 960P, 720P, Full D1, which are decreased in sequence. , Half D1, etc.), and the vertical super-score model 9 corresponds to the difference in image quality of 3 to 5 preset levels, if the resolution of the first video clip is 1080P, and the second video clip has If the resolution is 720P, the image quality of the second video clip differs from the image quality of the first video clip by two preset levels, and the electronic device determines that the vertical super-score model 8 is the above-mentioned target vertical super-score model.

In the above-mentioned step 102, after the electronic device determines the above-mentioned target vertical super-resolution model, the electronic device can process the video frame of the target video played under the first code rate through the target vertical super-resolution model, thereby realizing the improvement of the first code rate. The image quality of the target video played below.

It should be noted that during the process of the electronic device playing the target video at the first bit rate, the network state may recover, for example, the electronic device moves from an area with poor signal to an area with good signal. Then, the electronic device can adjust its code rate to a second code rate, where the second code rate is adapted to the restored network state.

At this time, after the above step 102, it may further include:

When it is detected that the network state of the electronic device returns to normal, the bit rate of the target video played by the electronic device is restored to the second bit rate, and the target vertical super-score model is stopped from processing the image frames of the target video.

Here, when the network status of the electronic device is restored to normal and the bit rate of the target video played by the electronic device is restored to the second bit rate, the image quality of the target video played by the electronic device can be recovered along with the recovery of the bit rate. The electronic device stops the processing of the image frame of the target video played at the second bit rate by the target vertical super-resolution model in time, which can save the power consumption caused by the unnecessary image processing of the target vertical super-resolution model, and realize the saving of electronic equipment. The power of the device.

Referring to FIG. 2 , an embodiment of the present application provides a display device. As shown in FIG. 2, the display device 200 includes:

The super-score model determination module 201 is configured to, when it is detected that the network state of the electronic device is abnormal, and the bit rate of the target video played by the electronic device is reduced to the first bit rate, the preset N vertical super-scores in the electronic device In the model, the target vertical super-score model of the target video is determined, wherein N is an integer greater than 1;

The display module 202 is configured to display the target image frame when the image frame of the target video is transmitted at the first bit rate.

Here, the display apparatus 200 can process the image frame of the video played under the low bit rate by using the selected vertical super-resolution model, so as to improve the image quality of the video played under the low bit rate, and further improve the electronic equipment when abnormal network conditions occur. The image quality when the video is played at a low bit rate; in addition, since the selected vertical super-resolution model corresponds to the played video, the selected vertical super-resolution model is more suitable for the processing of the played video, which can improve the processing efficiency As well as reducing energy consumption during processing.

Optionally, the super-score model determination module 201 includes:

a video type acquisition unit, used to acquire the video type of the target video;

The first super-score model determination unit is configured to determine, among the N vertical super-score models, the vertical super-score model corresponding to the video type of the target video as the target vertical super-score model.

Here, the display device 200 may determine a corresponding target vertical super-score model among the preset N vertical super-score models according to the video type of the target video, so that the selected vertical super-score model is more suitable for the first bit rate The processing of the video frame of the target video to be played further improves the image quality of the video played at a low bit rate.

Optionally, the video type acquisition unit, which is specifically used for:

The video type of the target video is determined based on the image of the target image frame, and the target image frame is: a partial image frame of the target video.

Here, the display device 200 can determine the video type of the target video according to the video type information transmitted by the server, thereby improving the efficiency of video processing; it can also determine the video type of the target video according to the video frame of the target video, so as to obtain the video type of the target video. The video type is flexible and can meet different needs.

Optionally, the apparatus 200 further includes:

The video clip receiving module is configured to receive the first video clip sent by the server, wherein the first video clip is a partial video clip of the target video.

The super-score model determination module 201 may include:

A video clip obtaining unit, configured to obtain a second video clip when the electronic device plays the target video at the first bit rate, where the second video clip is a video clip with the same content as the first video clip, and the first video clip has the same content as the first video clip. The image quality of the clip is higher than the image quality of the second video clip;

The second super-score model determining unit is configured to determine a target vertical super-score model among the N vertical super-score models based on the first video segment and the second video segment.

Here, the display apparatus 200 may determine the above-mentioned among the N vertical superscore models according to the first video segment of the target video sent by the server and the second video segment played at the first bit rate with the same content as the first video segment The target vertical super-score model makes the determined target vertical super-score model for processing the target video played at the first bit rate more suitable, and further improves the image quality of the target video played at the first bit rate.

Optionally, the second super-score model determination unit includes:

a video segment processing subunit, configured to process the image frames of the second video segment based on the N vertical super-resolution models, respectively, to obtain N video segments to be compared corresponding to the N vertical super-resolution models;

A comparison subunit, used to determine, among the N video clips to be compared, a target video clip to be compared whose image quality is the closest to the image quality of the first video clip;

The first super-resolution model determination subunit is configured to determine the vertical super-resolution model corresponding to the target video segment to be compared as the target vertical super-resolution model.

Here, the display device 200 may process the second video segment by using the preset N vertical super-score models, and compare the second video segment processed by each vertical super-score model (ie, the video segment to be compared) with the second video segment. The video clips are compared, and the vertical super-score model whose image quality of the processed second video clip is the closest to the image quality of the first video clip is determined as the above-mentioned target vertical super-score model, so that it is possible to choose to play at a low bit rate. The image quality processing effect of the video is the best, and the vertical super-score model is used as the target vertical super-score model, which further improves the image quality of the target video played at a low bit rate.

Optionally, the second super-score model determination unit includes:

a degree of difference obtaining subunit, used to obtain the degree of difference between the image quality of the second video clip and the image quality of the first video clip;

The second super-resolution model determination subunit is configured to determine, among the N vertical super-resolution models, the vertical super-resolution model corresponding to the acquired degree of difference as the target vertical super-resolution model.

Here, the display apparatus 200 may, according to the degree of difference between the image quality of the second video segment and the image quality of the first video segment, determine the target vertical super-score model corresponding to the degree of difference among the N vertical super-score models, so that the determination The method of the target vertical super-resolution model is simpler, the efficiency of determining the target vertical super-resolution model is improved, and the efficiency of video processing is further improved.

Optionally, the apparatus 200 further includes:

The bit rate recovery module is used to restore the bit rate of the target video played by the electronic device to the second bit rate when it is detected that the network state of the electronic device returns to normal, and stop the target vertical super-resolution model from performing the image frame of the target video. to be processed.

Here, the display device 200 can stop the processing of the image frame of the target video played at the second bit rate by the target vertical super-resolution model in time, which can save the power consumption caused by unnecessary image processing by the target vertical super-resolution model. , to save the power of electronic equipment.

The display device in this embodiment of the present application may be a device, or may be a component, an integrated circuit, or a chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device may be a mobile phone, a tablet computer, a notebook computer, a palmtop computer, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (personal digital assistant). assistant, PDA), etc., non-mobile electronic devices can be servers, network attached storage (Network Attached Storage, NAS), personal computer (personal computer, PC), television (television, TV), teller machine or self-service machine, etc., this application Examples are not specifically limited.

The display device in this embodiment of the present application may be a device having an operating system. The operating system may be an Android (Android) operating system, an ios operating system, or other possible operating systems, which are not specifically limited in the embodiments of the present application.

The display device provided by the embodiment of the present application can implement each process implemented by the method embodiment of FIG. 1 , and to avoid repetition, details are not described here.

Optionally, as shown in FIG. 3 , an embodiment of the present application further provides an electronic device 300 , including a processor 301 , a memory 302 , a program or instruction stored in the memory 302 and executable on the processor 301 , the program Or, when the instruction is executed by the processor 301, each process of the above-mentioned display method embodiment can be implemented, and the same technical effect can be achieved. In order to avoid repetition, details are not repeated here.

It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

The electronic device 400 includes but is not limited to: a radio frequency unit 401, a network module 402, an audio output unit 403, an input unit 404, a sensor 405, a display unit 406, a user input unit 407, an interface unit 408, a memory 409, and a processor 410, etc. part.

Those skilled in the art can understand that the electronic device 400 may also include a power source (such as a battery) for supplying power to various components, and the power source may be logically connected to the processor 410 through a power management system, so that the power management system can manage charging, discharging, and power management. consumption management and other functions. The structure of the electronic device shown in FIG. 4 does not constitute a limitation to the electronic device. The electronic device may include more or less components than the one shown, or combine some components, or arrange different components, which will not be repeated here. .

The processor 410 is used for:

When it is detected that the network status of the electronic device is abnormal, and the bit rate of the target video played by the electronic device is reduced to the first bit rate, determine the target vertical rate of the target video in the N vertical super-score models preset in the electronic device A super-score model, where N is an integer greater than 1;

Display unit 406 for:

In the case where the image frame of the target video is transmitted at the first bit rate, the target image frame is displayed, and the target image frame is an image frame obtained by processing the image frame of the target video based on the target vertical superdivision model.

Here, the electronic device 400 can process the image frame of the video played under the low bit rate by using the selected vertical super-resolution model, so as to improve the image quality of the video played under the low bit rate, thereby improving the electronic device when abnormal network conditions occur. The image quality when the video is played at a low bit rate; in addition, since the selected vertical super-resolution model corresponds to the played video, the selected vertical super-resolution model is more suitable for the processing of the played video, which can improve the processing efficiency As well as reducing energy consumption during processing.

Optionally, the processor 410 is further configured to:

Get the video type of the target video;

Here, the electronic device 400 can determine a corresponding target vertical super-score model among the preset N vertical super-score models according to the video type of the target video, so that the selected vertical super-score model is more suitable for the first bit rate The processing of the video frame of the target video to be played further improves the image quality of the video played at a low bit rate.

Optionally, the processor 410 is further configured to:

In the case of receiving the video type information of the target video, obtain the video type indicated by the video type information, wherein the video type information is the information sent by the server when the video data of the target video is transmitted; or,

The video type of the target video is determined based on the image of the target image frame, where the target image frame is: a partial image frame of the target video.

Here, the electronic device 400 can determine the video type of the target video according to the video type information transmitted by the server, so that the efficiency of video processing can be improved; it can also determine the video type of the target video according to the video frame of the target video, so as to obtain the video type of the target video. The video type is flexible and can meet different needs.

Optionally, the processor 410 is further configured to:

receiving the first video clip sent by the server, where the first video clip is a partial video clip of the target video;

Here, the electronic device 400 may determine the above-mentioned among the N vertical super-score models according to the first video segment of the target video sent by the server and the second video segment played at the first bit rate with the same content as the first video segment The target vertical super-score model makes the determined target vertical super-score model for processing the target video played at the first bit rate more suitable, and further improves the image quality of the target video played at the first bit rate.

Optionally, the processor 410 is further configured to:

Here, the electronic device 400 may process the second video segment by using the preset N vertical super-score models, and compare the second video segment processed by each vertical super-score model (ie, the video segment to be compared) with the second video segment. The video clips are compared, and the vertical super-score model whose image quality of the processed second video clip is the closest to the image quality of the first video clip is determined as the above-mentioned target vertical super-score model, so that it is possible to choose to play at a low bit rate. The image quality processing effect of the video is the best, and the vertical super-score model is used as the target vertical super-score model, which further improves the image quality of the target video played at a low bit rate.

Optionally, the processor 410 is further configured to:

Here, the electronic device 400 may, according to the degree of difference between the image quality of the second video segment and the image quality of the first video segment, determine the target vertical super-score model corresponding to the degree of difference among the N vertical super-score models, so that the determination The method of the target vertical super-resolution model is simpler, the efficiency of determining the target vertical super-resolution model is improved, and the efficiency of video processing is further improved.

Optionally, the processor 410 is further configured to:

Here, the electronic device 400 can stop the processing of the image frame of the target video played at the second bit rate by the target vertical super-resolution model in time, which can save the power consumption caused by unnecessary image processing by the target vertical super-resolution model. , to save the power of electronic equipment.

It should be understood that, in this embodiment of the present application, the input unit 404 may include a graphics processor (Graphics Processing Unit, GPU) 4041 and a microphone 4042. Such as camera) to obtain still pictures or video image data for processing. The display unit 406 may include a display panel 4061, which may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like. The user input unit 407 includes a touch panel 4071 and other input devices 4072 . The touch panel 4071 is also called a touch screen. The touch panel 4071 may include two parts, a touch detection device and a touch controller. Other input devices 4072 may include, but are not limited to, physical keyboards, function keys (such as volume control keys, switch keys, etc.), trackballs, mice, and joysticks, which are not described herein again. Memory 409 may be used to store software programs as well as various data including, but not limited to, application programs and operating systems. The processor 410 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and application programs, and the like, and the modem processor mainly handles wireless communication. It can be understood that, the above-mentioned modulation and demodulation processor may not be integrated into the processor 410.

The embodiments of the present application further provide a readable storage medium, where a program or an instruction is stored on the readable storage medium, and when the program or instruction is executed by a processor, each process of the above-mentioned display method embodiment can be realized, and the same technical effect can be achieved , in order to avoid repetition, it will not be repeated here.

The processor is the processor in the electronic device in the above embodiment. Readable storage media, including computer-readable storage media, such as computer read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disks or non-transitory computer-readable storage such as optical disks medium.

An embodiment of the present application further provides a chip, where the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run a program or an instruction, implement each process of the above display method embodiment, and can achieve the same technology The effect, in order to avoid repetition, is not repeated here.

It should be understood that the chip mentioned in the embodiments of the present application may also be referred to as a system-on-chip, a system-on-chip, a system-on-a-chip, or a system-on-a-chip, or the like.

It should be noted that, herein, the terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements, It also includes other elements not expressly listed or inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in a process, method, article or apparatus that includes the element. Furthermore, it should be noted that the scope of the methods and apparatus in the embodiments of the present application is not limited to performing the functions in the order shown or discussed, but may also include performing the functions in a substantially simultaneous manner or in the reverse order depending on the functions involved. To perform functions, for example, the described methods may be performed in an order different from that described, and various steps may also be added, omitted, or combined. Additionally, features described with reference to some examples may be combined in other examples.

From the description of the above embodiments, those skilled in the art can clearly understand that the method of the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is better implementation. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product in essence or in a part that contributes to the prior art, and the computer software product is stored in a storage medium (such as ROM/RAM, magnetic disk, CD-ROM), including several instructions to make a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) execute the methods described in the various embodiments of this application.

The embodiments of the present application have been described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific embodiments, which are merely illustrative rather than restrictive. Under the inspiration of this application, without departing from the scope of protection of the purpose of this application and the claims, many forms can be made, which all fall within the protection of this application.

Claims

A display method including:

When it is detected that the network state of the electronic device is abnormal, and the bit rate of the target video played by the electronic device is reduced to the first bit rate, in the N preset vertical super-score models in the electronic device, determine the The target vertical super-score model of the target video, wherein the N is an integer greater than 1;

In the case of transmitting the image frame of the target video at the first bit rate, displaying the target image frame, where the target image frame is an image frame of the target video processed based on the target vertical super-resolution model image frame.
The method according to claim 1, wherein, in the N preset vertical super-resolution models in the electronic device, determining the target vertical super-resolution model of the target video comprises:

Obtain the video type of the target video;

Among the N vertical super-resolution models, the vertical super-score model corresponding to the video type of the target video is determined as the target vertical super-score model.
The method according to claim 2, wherein the acquiring the video type of the target video comprises:

In the case of receiving the video type information of the target video, obtain the video type indicated by the video type information, where the video type information is the information sent by the server when the video data of the target video is transmitted; or,

A video type of the target video is determined based on images of partial image frames of the target video.
The method according to claim 1, wherein, before determining the target vertical super-score model of the target video among the N preset vertical super-score models in the electronic device, the method further comprises:

receiving a first video clip sent by a server, where the first video clip is a partial video clip of the target video;

Described in the preset N vertical super-resolution models in the electronic device, determining the target vertical super-resolution model of the target video, including:

In the case where the electronic device plays the target video at a first bit rate, acquire a second video clip, where the second video clip is a video clip having the same content as the first video clip, and The image quality of the first video clip is higher than the image quality of the second video clip;

Based on the first video segment and the second video segment, a target vertical super-score model is determined among the N vertical super-score models.
The method according to claim 4, wherein the determining a target vertical super-score model among the N vertical super-score models based on the first video segment and the second video segment comprises:

respectively process the image frames of the second video segment based on the N vertical super-resolution models, to obtain N video segments to be compared corresponding to the N vertical super-resolution models;

Among the N video clips to be compared, determining a target video clip to be compared with the image quality closest to the image quality of the first video clip;

The vertical super-score model corresponding to the target to-be-compared video segment is determined as the target vertical super-score model.
The method according to claim 4, wherein the determining a target vertical super-score model among the N vertical super-score models based on the first video segment and the second video segment comprises:

obtaining the degree of difference between the image quality of the second video clip and the image quality of the first video clip;

Among the N vertical super-score models, the vertical super-score model corresponding to the acquired degree of difference is determined as the target vertical over-score model.
The method according to claim 1, wherein, in the case of transmitting the image frame of the target video at the first bit rate, after displaying the target image frame, the method further comprises:

In the case of detecting that the network state of the electronic device returns to normal, restore the bit rate of the target video played by the electronic device to the second bit rate, and stop the target vertical super-score model to image the target video frame for processing.
A display device, comprising:

The super-score model determination module is configured to, when it is detected that the network state of the electronic device is abnormal, and the bit rate of the target video played by the electronic device is reduced to the first bit rate, N presets in the electronic device In the vertical super-resolution model, the target vertical super-resolution model of the target video is determined, wherein the N is an integer greater than 1;

A display module, configured to display the target image frame in the case of transmitting the image frame of the target video at the first bit rate.
The apparatus according to claim 8, wherein the super-resolution model determination module comprises:

a video type acquisition unit, used for acquiring the video type of the target video;

The first super-score model determining unit is configured to determine, among the N vertical super-score models, the vertical super-score model corresponding to the video type of the target video as the target vertical super-score model.
The device according to claim 9, wherein the video type acquiring unit is specifically configured to:

In the case of receiving the video type information of the target video, obtain the video type indicated by the video type information, where the video type information is sent by the server when the video data of the target video is transmitted information; or,

The video type of the target video is determined based on the image of the target image frame, wherein the target image frame is: a partial image frame of the target video.
The apparatus of claim 8, further comprising:

a video clip receiving module, configured to receive a first video clip sent by a server, wherein the first video clip is a partial video clip of the target video;

The super-score model determination module includes:

a video clip obtaining unit, configured to obtain a second video clip when the electronic device plays the target video at a first bit rate, wherein the second video clip is the same as the first video clip video clips of the same content, and the image quality of the first video clip is higher than that of the second video clip;

A second super-score model determining unit, configured to determine a target vertical super-score model among the N vertical super-score models based on the first video segment and the second video segment.
The apparatus according to claim 11, wherein the second super-score model determining unit comprises:

a video segment processing subunit, configured to process image frames of the second video segment based on the N vertical super-resolution models, respectively, to obtain N video segments to be compared corresponding to the N vertical super-resolution models;

a comparison subunit, configured to determine, among the N video clips to be compared, a target video clip to be compared whose image quality is closest to that of the first video clip;

The first super-resolution model determination subunit is configured to determine the vertical super-resolution model corresponding to the target video segment to be compared as the target vertical super-resolution model.
The apparatus according to claim 11, wherein the second super-score model determining unit comprises:

a degree of difference acquiring subunit, configured to acquire the degree of difference between the image quality of the second video clip and the image quality of the first video clip;

The second super-score model determination subunit is configured to, among the N vertical super-score models, determine the vertical over-score model corresponding to the acquired degree of difference as the target vertical over-score model.
The apparatus of claim 8, further comprising:

A bit rate recovery module, configured to restore the bit rate of the target video played by the electronic device to a second bit rate when it is detected that the network state of the electronic device returns to normal, and stop the target vertical super-score model The image frames of the target video are processed.
An electronic device comprising a processor, a memory, and a program or instruction stored on the memory and executable on the processor, the program or instruction being executed by the processor to achieve the The steps of the display method described in any one of claims 1 to 7.
A readable storage medium on which programs or instructions are stored, and when the programs or instructions are executed by a processor, the steps of the display method according to any one of claims 1 to 7 are implemented.
A chip, the chip includes a processor and a communication interface, the communication interface is coupled with the processor, and the processor is used for running a program or an instruction to realize the display according to any one of claims 1 to 7 steps of the method.
A computer program product comprising a computer program tangibly embodied on a computer readable medium, the computer program containing program code for performing the method of any one of claims 1 to 7.
An electronic device configured to perform the steps of the method of any one of claims 1 to 7.