WO2022161310A1 - Procédé et appareil d'affichage et dispositif électronique - Google Patents

Procédé et appareil d'affichage et dispositif électronique Download PDF

Info

Publication number
WO2022161310A1
WO2022161310A1 PCT/CN2022/073465 CN2022073465W WO2022161310A1 WO 2022161310 A1 WO2022161310 A1 WO 2022161310A1 CN 2022073465 W CN2022073465 W CN 2022073465W WO 2022161310 A1 WO2022161310 A1 WO 2022161310A1
Authority
WO
WIPO (PCT)
Prior art keywords
video
target
vertical super
electronic device
super
Prior art date
Application number
PCT/CN2022/073465
Other languages
English (en)
Chinese (zh)
Inventor
张俪馨
Original Assignee
维沃移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 维沃移动通信有限公司 filed Critical 维沃移动通信有限公司
Publication of WO2022161310A1 publication Critical patent/WO2022161310A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/80—Responding to QoS
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440281—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the temporal resolution, e.g. by frame skipping
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/647—Control signaling between network components and server or clients; Network processes for video distribution between server and clients, e.g. controlling the quality of the video stream, by dropping packets, protecting content from unauthorised alteration within the network, monitoring of network load, bridging between two different networks, e.g. between IP and wireless
    • H04N21/64723—Monitoring of network processes or resources, e.g. monitoring of network load
    • H04N21/64738—Monitoring network characteristics, e.g. bandwidth, congestion level
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/647—Control signaling between network components and server or clients; Network processes for video distribution between server and clients, e.g. controlling the quality of the video stream, by dropping packets, protecting content from unauthorised alteration within the network, monitoring of network load, bridging between two different networks, e.g. between IP and wireless
    • H04N21/64746—Control signals issued by the network directed to the server or the client
    • H04N21/64761—Control signals issued by the network directed to the server or the client directed to the server
    • H04N21/64769—Control signals issued by the network directed to the server or the client directed to the server for rate control
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Definitions

  • the present application belongs to the technical field of video processing, and in particular relates to a display method, apparatus and electronic device.
  • the network condition connected to the electronic device may be abnormal.
  • the electronic device enters a tunnel or moves at a high speed, etc., resulting in poor network speed.
  • the electronic device in order to ensure that the video can continue to be played, the electronic device usually reduces the bit rate of the currently playing video, for example, from a high-definition bit rate (such as 1080p) to a low-definition bit rate (such as 480p), etc., while the video playback bit rate Lowering will result in lower image quality for video playback.
  • a high-definition bit rate such as 1080p
  • a low-definition bit rate such as 480p
  • the purpose of the embodiments of the present application is to provide a display method, device and electronic device, which can solve the problem that when the current electronic device has abnormal network conditions during the video playback process, there is an image of the video played due to the reduction of the video playback bit rate. low quality issues.
  • an embodiment of the present application provides a display method, including:
  • the bit rate of the target video played by the electronic device is reduced to the first bit rate, in the N preset vertical super-score models in the electronic device, determine the The target vertical super-score model of the target video, wherein the N is an integer greater than 1;
  • the target image frame is an image frame of the target video processed based on the target vertical super-resolution model image frame.
  • an embodiment of the present application provides a display device, including:
  • the super-score model determination module is configured to, when it is detected that the network state of the electronic device is abnormal, and the bit rate of the target video played by the electronic device is reduced to the first bit rate, N presets in the electronic device In the vertical super-resolution model, the target vertical super-resolution model of the target video is determined, wherein the N is an integer greater than 1;
  • a display module configured to display the target image frame in the case of transmitting the image frame of the target video at the first bit rate.
  • embodiments of the present application provide an electronic device, the electronic device includes a processor, a memory, and a program or instruction stored on the memory and executable on the processor, the program or instruction being The processor implements the steps of the method according to the first aspect when executed.
  • an embodiment of the present application provides a readable storage medium, where a program or an instruction is stored on the readable storage medium, and when the program or instruction is executed by a processor, the steps of the method according to the first aspect are implemented .
  • an embodiment of the present application provides a chip, the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is configured to run a program or an instruction, and implement the first aspect the method described.
  • embodiments of the present application provide a computer program product, including a computer program tangibly embodied on a computer-readable medium, the computer program including program code for executing the method according to the first aspect.
  • an embodiment of the present application provides an electronic device configured to execute the steps of the method described in the first aspect.
  • the electronic device when the electronic device reduces the bit rate of the playing video (ie, the target video) due to an abnormal network state, the electronic device can select the corresponding target video from a plurality of preset vertical super-resolution models.
  • the vertical super-resolution model namely the target vertical super-resolution model
  • the selected vertical super-resolution model the image frames of the video to be played are processed, and the video after the image frame processing is played.
  • the electronic device can process the image frames of the video played under the low bit rate through the selected vertical super-resolution model, so as to improve the image quality of the video played under the low bit rate, and further improve the electronic device's performance in the video.
  • the image quality when the video is played at a low bit rate due to abnormal network conditions since the selected vertical super-resolution model corresponds to the played video, the selected vertical super-resolution model is more suitable for the processing of the played video, Thereby, the processing efficiency can be improved and the energy consumption during the processing can be reduced.
  • FIG. 1 is a schematic flowchart of an embodiment of a display method provided by the present application.
  • FIG. 2 is a schematic structural diagram of an embodiment of a display device provided by the present application.
  • FIG. 3 is a schematic diagram of a hardware structure of an embodiment of an electronic device provided by the present application.
  • FIG. 4 is a schematic diagram of a hardware structure of an embodiment of another electronic device provided by the present application.
  • first, second and the like in the description and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances so that the embodiments of the present application can be practiced in sequences other than those illustrated or described herein, and distinguish between “first”, “second”, etc.
  • the objects are usually of one type, and the number of objects is not limited.
  • the first photographing object may be one or more than one.
  • “and/or” in the description and claims indicates at least one of the connected objects, and the character “/" generally indicates that the associated objects are in an "or” relationship.
  • an embodiment of the present application provides a display method, and the display method is applied to an electronic device. As shown in FIG. 1 , the method includes the following steps 101 and 102 .
  • Step 101 in the case of detecting that the network state of the electronic device is abnormal, and the code rate of the target video played by the electronic device is reduced to the first code rate, in the preset N vertical super-resolution models in the electronic device, determine the target video.
  • the target vertical super-resolution model wherein the target vertical super-resolution model is a vertical super-resolution model among N preset vertical super-resolution models in the electronic device, and N is an integer greater than 1.
  • Step 102 in the case of transmitting the image frame of the target video at the first bit rate, displaying the target image frame, wherein the target image frame is an image frame after processing the image frame of the target video based on the target vertical super-resolution model.
  • the electronic device can select a vertical super-resolution model corresponding to the target video from the preset multiple vertical super-resolution models (that is, the target vertical super-resolution model), and process the image frames of the played video through the selected vertical super-resolution model, and play the video after image frame processing.
  • the electronic device can process the image frames of the video played under the low bit rate through the selected vertical super-resolution model, so as to improve the image quality of the video played under the low bit rate, and further improve the electronic device's performance in the video.
  • the image quality when the video is played at a low bit rate due to abnormal network conditions since the selected vertical super-resolution model corresponds to the played video, the selected vertical super-resolution model is more suitable for the processing of the played video, Thereby, the processing efficiency can be improved and the energy consumption during the processing can be reduced.
  • the server can detect whether the current network state of the electronic device is abnormal, and when detecting that the network state of the electronic device is abnormal, it can The current bit rate (for ease of description, referred to as "second bit rate" in this embodiment of the present application) currently used by the electronic device to play the video (that is, the server transmits the image frame of the played video to the electronic device) is reduced to the first bit rate.
  • the current bit rate for ease of description, referred to as "second bit rate" in this embodiment of the present application
  • the above-mentioned server detects whether the current network state of the electronic device is abnormal, which may be that the electronic device reports the network speed or network signal strength of the network it is currently connected to to the server, and the server reports whether the network speed or network signal strength is lower than Corresponding preset thresholds, and in the case of being lower than the corresponding preset thresholds, it is determined that the network state of the electronic device is abnormal, and so on.
  • the above-mentioned reduction of the code rate of the target video played by the electronic device to the first code rate may be that the server determines the corresponding first code rate according to the network state after the abnormality of the electronic device occurs, and adjusts the code of the image frame of the transmission target video.
  • the first bit rate is lower than the second bit rate at which the electronic device plays the target video before the abnormal network state occurs.
  • the electronic device plays a video (ie, the target video) at a bit rate of 5Mb/s (ie, the second bit rate)
  • a video ie, the target video
  • 5Mb/s ie, the second bit rate
  • the electronic device detects that its network speed is reduced, and the reduced network speed can only meet the requirements of To play a video at a bit rate of 2Mb/s
  • the electronic device reduces the bit rate of the video to be played from 5Mb/s to 2Mb/s.
  • the electronic device may determine a vertical super-resolution model corresponding to the target video among the preset N vertical super-resolution models As the above target vertical superscore model.
  • each vertical super-score model in the above-mentioned N vertical super-score models is an arbitrary deep learning model that can improve image quality.
  • Different vertical super-resolution models in the sub-model are different in at least one of network parameters, network structure, processing accuracy and processing speed, so that different vertical super-resolution models are suitable for processing videos with different video information.
  • the above-mentioned N vertical super-score models may be learned in the server and sent to the above-mentioned electronic device for storage by the server, or may be directly learned in the electronic device, and the learning process of the vertical super-score model is as follows: Those skilled in the art are well-known, and will not be described in detail here.
  • the above-mentioned target vertical super-score model based on determining the target video may be a target vertical super-score model corresponding to the target video determined from N vertical super-score models according to a preset rule.
  • the above-mentioned preset rule may be to determine the target vertical super-resolution model based on the corresponding relationship between the video information and the vertical super-resolution model, that is, the above-mentioned determination of the target vertical super-resolution model of the target video may include: In the model, the vertical super-score model that has a corresponding relationship with the video information of the target video is determined as the target vertical super-score model.
  • the video information of the above-mentioned target video can be any information used as an indicator for selecting a vertical super-score model, and it can be the image quality (including the resolution) of the target video played under the above-mentioned first bit rate. etc.), the bit rate (ie, the first bit rate) of the target video to be played after the network state is abnormal, and the video type information, and so on.
  • the above-mentioned determining the target vertical super-score model may include: in the N vertical super-score models, determining the The vertical superdivision model corresponding to the image quality of the target video played at the high rate is the target vertical superdivision waveform.
  • the electronic device is preset with a vertical super-resolution model 1 and a vertical super-resolution model 2, and the vertical super-resolution model 1 is suitable for improving the image quality of the video frame of a video with a resolution of FULL D1 (ie, a resolution of 720 ⁇ 576).
  • the vertical super-resolution model 2 is suitable for improving the image quality of the video frame of the video whose definition is HALF D1 (ie, the resolution is 720 ⁇ 288).
  • the electronic device determines that the vertical super-resolution model 1 is the target vertical super-resolution model; if the target video played by the electronic device at the first bit rate has a If the definition is HALF D1, the electronic device determines the vertical super-resolution model 2 as the target vertical super-resolution model.
  • the above-mentioned determining the target vertical super-score model may include: among the N vertical super-score models, determining a vertical super-score model that has a corresponding relationship with the first code rate is the vertical superresolution model for the target.
  • the above-mentioned step 102 may include:
  • the vertical super-score model corresponding to the video type of the target video is determined as the target vertical super-score model.
  • the electronic device can determine the corresponding target vertical super-score model among the preset N vertical super-score models according to the video type of the target video, so that the selected vertical super-score model is more suitable for playing at the first bit rate
  • the processing of the video frame of the target video further improves the image quality of the video played at a low bit rate.
  • the video types of the above-mentioned target videos may include any one of characters, animals, plants, buildings, landscapes, animations, etc.
  • the electronic device may preset a corresponding vertical superscore for each video type
  • the model that is, the above-mentioned target vertical super-score model is a vertical super-score model that has a preset corresponding relationship with the video type of the above-mentioned target video among the above-mentioned N vertical super-score models.
  • a vertical super-score model 3 and a vertical hyper-score model 4 are preset in the electronic device, and the vertical hyper-score model 3 has a corresponding relationship with the video type being a character, and the vertical hyper-score model 4 has a corresponding relationship with the video type landscape. Then, if the above-mentioned target video is a video of shooting a person (that is, the video type of the target video is a person), the electronic device determines that the vertical super-score model 3 is the above-mentioned target vertical super-score model.
  • the above video information of the target video may be video information obtained by the electronic device detecting the target video, or may be video information received by the electronic device and sent by the server.
  • the above-mentioned acquisition of the video type of the target video may include:
  • the video type information is the information sent by the server when the video data of the target video is transmitted;
  • the video type of the target video is determined based on the image of the target image frame, where the target image frame is an image frame of the target video played by the electronic device within a preset time period before the bit rate of the target video is reduced to the first bit rate.
  • the electronic device can determine the video type of the target video according to the video type information transmitted by the server, thereby improving the efficiency of video processing; it can also determine the video type of the target video according to the video frame of the target video, so that the video of the target video can be obtained.
  • Types are flexible and can meet different needs.
  • the electronic device in the process of playing the target video, can receive the video type information sent by the server in the process of transmitting the video data of the target video, and the electronic device can obtain the video type indicated by the video type information.
  • the electronic device may also acquire some image frames of the target video, and determine the video type of the target video according to the image information in the partial image frames.
  • the electronic device may acquire image frames within a preset time period (such as 1s, etc.) before reducing the bit rate of the target video to be played back to the first bit rate, and determine the target according to the acquired image frames.
  • the video type of the video Specifically, if there are more than 60 image frames including animals in the acquired 100 image frames, it is determined that the video type of the target video is animals, and so on.
  • the above-mentioned electronic equipment determines the target vertical super-resolution model corresponding to the target video in the N vertical super-resolution models, and is not limited to determining the corresponding target vertical super-resolution model based on the video information of the target video. It can also be based on other The rules determine the target vertical superscore model for the target video.
  • the method further includes:
  • the first video clip sent by the server is received, where the first video clip is a partial video clip of the target video.
  • determining the target vertical super-resolution model of the target video may include:
  • a second video clip is acquired, the second video clip is a video clip with the same content as the first video clip, and the image quality of the first video clip is higher than that of the first video clip The image quality of the two video clips;
  • a target vertical super-score model is determined among the N vertical super-score models.
  • the electronic device may determine the above target in the N vertical super-score models according to the first video clip of the target video sent by the server, and the second video clip played at the first bit rate with the same content as the first video clip
  • the vertical super-score model makes the determined target vertical super-score model for processing the target video played at the first bit rate more suitable, and further improves the image quality of the target video played at the first bit rate.
  • the above-mentioned first video clip is a partial video clip of the target video sent by the server before the electronic device reduces the bit rate of playing the target video to the first bit rate; and the above-mentioned second video clip is the electronic device.
  • a video clip having the same content as the first video clip is played at the first bit rate, and the image quality (including resolution, etc.) of the first video clip is higher than that of the second video clip.
  • the above-mentioned acquisition of the first video clip may be that when the server receives a request from the electronic device to play the target video, the server may transmit the video data of the target video to the electronic device, and also acquire at least one video clip of the target video for sending.
  • the data is sent to the electronic device, and the image quality of the at least one video segment is the first image quality.
  • the electronic device reduces the bit rate of the target video to be played to the first bit rate
  • the electronic device can obtain the at least one video segment to play The video segment whose time period is after the current time and closest to the current time is regarded as the first video segment.
  • the server may send m video clips of the target video to the electronic device, and two adjacent video clips in the m video clips are separated by a first preset duration (eg, 5s). Or 10s, etc.), and the duration of each video clip is the second preset duration (such as 0.5s or 1s, etc.) and the image quality is the first image quality (such as 1080P, etc.), if the electronic device is playing the target video for 20s. If the bit rate is lowered, the electronic device determines a video segment whose playback period is after 20 s among the m video segments as the above-mentioned first video segment, where m is an integer greater than 1.
  • a first preset duration eg, 5s). Or 10s, etc.
  • the duration of each video clip is the second preset duration (such as 0.5s or 1s, etc.) and the image quality is the first image quality (such as 1080P, etc.)
  • the electronic device determines a video segment whose playback period is after 20 s among the
  • the bit rate of the played video is reduced, the reduced first bit rate is no longer compatible with the image quality of the original video.
  • the electronic device will The image quality is reduced to an image quality suitable for the first bit rate.
  • an electronic device plays a video with a resolution of 1080P (that is, a resolution of 1920 ⁇ 1080) at a bit rate of 5Mb/s
  • the electronic device if the electronic device reduces the bit rate of the played video to 2Mb/s, the electronic device will When the video is played at 2Mb/s, the image quality of the played video will be reduced to the definition FUUL D1 (that is, the resolution is 720 ⁇ 576).
  • the first image quality of the first video clip is higher than the image quality of the second video clip, that is, the first image quality may be the image quality of the target video before the bit rate of the target video is reduced to the first bit rate; Alternatively, it may also be a preset image quality, and the preset image quality is at least higher than the image quality of the target video played at the first bit rate.
  • the electronic device can receive m video clips with a video resolution of 1080P sent by the server, and the electronic device reduces the bit rate of the target video to 2Mb/s.
  • the electronic device determines the first video clip among the m video clips, that is, the resolution of the first video clip is 1080P, and acquires a second video clip with the same content as the first video clip, that is, the second video clip
  • the resolution is FUUL D1.
  • the electronic device can compare the first video clip and the second video clip, and according to the comparison results, make a comparison between the N vertical super-score models. Determine the target vertical superresolution model.
  • the above-mentioned determination of the target vertical super-score model among the N vertical super-score models based on the first video clip and the second video clip may include:
  • the image frames of the second video segment are respectively processed based on the N vertical super-resolution models, and N video segments to be compared corresponding to the N vertical super-resolution models are obtained;
  • the target video clip to be compared determines the target video clip to be compared whose image quality is the closest to the image quality of the first video clip
  • the vertical super-score model corresponding to the target video segment to be compared is determined as the target vertical super-score model.
  • the electronic device may process the second video segment by using the preset N vertical super-score models, and compare the second video segment (ie, the video segment to be compared) processed by each vertical super-score model with the first video segment. Fragments are compared, and the vertical super-score model whose image quality of the processed second video clip is closest to that of the first video clip is determined as the above-mentioned target vertical The image quality processing effect of the video is the best, and the vertical super-score model is used as the target vertical super-score model to further improve the image quality of the target video played at a low bit rate.
  • the image resolution of the first video clip is 1080P
  • the image resolution of the second video clip is Hafl D1
  • the electronic device determines that 960P is the closest to 1080P, and determines the vertical super-resolution model 6 as the above-mentioned target vertical super-resolution model.
  • the above-mentioned determination of the target vertical super-score model among the N vertical super-score models based on the first video clip and the second video clip may include:
  • the vertical super-resolution model corresponding to the obtained degree of difference is determined as the target vertical super-resolution model.
  • the electronic device may also determine the target vertical super-score model corresponding to the degree of difference among the N vertical super-score models according to the degree of difference between the image quality of the second video segment and the image quality of the first video segment, so as to determine the
  • the method of the target vertical super-resolution model is simpler, the efficiency of determining the target vertical super-resolution model is improved, and the efficiency of video processing is further improved.
  • the vertical super-resolution model corresponding to the obtained degree of difference is determined as the target vertical super-resolution model, which may be preset in the electronic device.
  • the electronic device may determine the vertical super-resolution model that has a corresponding relationship with the acquired difference degree as the above-mentioned target vertical super-resolution model.
  • a vertical super-score model 8 and a vertical super-score model 9 are preset in the electronic device, and the preset vertical super-score model 8 corresponds to 1 to 2 preset levels (including 1080P, 960P, 720P, Full D1, which are decreased in sequence.
  • the vertical super-score model 9 corresponds to the difference in image quality of 3 to 5 preset levels, if the resolution of the first video clip is 1080P, and the second video clip has If the resolution is 720P, the image quality of the second video clip differs from the image quality of the first video clip by two preset levels, and the electronic device determines that the vertical super-score model 8 is the above-mentioned target vertical super-score model.
  • the electronic device can process the video frame of the target video played under the first code rate through the target vertical super-resolution model, thereby realizing the improvement of the first code rate.
  • the network state may recover, for example, the electronic device moves from an area with poor signal to an area with good signal. Then, the electronic device can adjust its code rate to a second code rate, where the second code rate is adapted to the restored network state.
  • step 102 it may further include:
  • the bit rate of the target video played by the electronic device is restored to the second bit rate, and the target vertical super-score model is stopped from processing the image frames of the target video.
  • the image quality of the target video played by the electronic device can be recovered along with the recovery of the bit rate.
  • the electronic device stops the processing of the image frame of the target video played at the second bit rate by the target vertical super-resolution model in time, which can save the power consumption caused by the unnecessary image processing of the target vertical super-resolution model, and realize the saving of electronic equipment. The power of the device.
  • the display device 200 includes:
  • the super-score model determination module 201 is configured to, when it is detected that the network state of the electronic device is abnormal, and the bit rate of the target video played by the electronic device is reduced to the first bit rate, the preset N vertical super-scores in the electronic device In the model, the target vertical super-score model of the target video is determined, wherein N is an integer greater than 1;
  • the display module 202 is configured to display the target image frame when the image frame of the target video is transmitted at the first bit rate.
  • the display apparatus 200 can process the image frame of the video played under the low bit rate by using the selected vertical super-resolution model, so as to improve the image quality of the video played under the low bit rate, and further improve the electronic equipment when abnormal network conditions occur.
  • the super-score model determination module 201 includes:
  • a video type acquisition unit used to acquire the video type of the target video
  • the first super-score model determination unit is configured to determine, among the N vertical super-score models, the vertical super-score model corresponding to the video type of the target video as the target vertical super-score model.
  • the display device 200 may determine a corresponding target vertical super-score model among the preset N vertical super-score models according to the video type of the target video, so that the selected vertical super-score model is more suitable for the first bit rate
  • the processing of the video frame of the target video to be played further improves the image quality of the video played at a low bit rate.
  • the video type acquisition unit which is specifically used for:
  • the video type information is the information sent by the server when the video data of the target video is transmitted;
  • the video type of the target video is determined based on the image of the target image frame, and the target image frame is: a partial image frame of the target video.
  • the display device 200 can determine the video type of the target video according to the video type information transmitted by the server, thereby improving the efficiency of video processing; it can also determine the video type of the target video according to the video frame of the target video, so as to obtain the video type of the target video.
  • the video type is flexible and can meet different needs.
  • the apparatus 200 further includes:
  • the video clip receiving module is configured to receive the first video clip sent by the server, wherein the first video clip is a partial video clip of the target video.
  • the super-score model determination module 201 may include:
  • a video clip obtaining unit configured to obtain a second video clip when the electronic device plays the target video at the first bit rate, where the second video clip is a video clip with the same content as the first video clip, and the first video clip has the same content as the first video clip.
  • the image quality of the clip is higher than the image quality of the second video clip;
  • the second super-score model determining unit is configured to determine a target vertical super-score model among the N vertical super-score models based on the first video segment and the second video segment.
  • the display apparatus 200 may determine the above-mentioned among the N vertical superscore models according to the first video segment of the target video sent by the server and the second video segment played at the first bit rate with the same content as the first video segment
  • the target vertical super-score model makes the determined target vertical super-score model for processing the target video played at the first bit rate more suitable, and further improves the image quality of the target video played at the first bit rate.
  • the second super-score model determination unit includes:
  • a video segment processing subunit configured to process the image frames of the second video segment based on the N vertical super-resolution models, respectively, to obtain N video segments to be compared corresponding to the N vertical super-resolution models;
  • a comparison subunit used to determine, among the N video clips to be compared, a target video clip to be compared whose image quality is the closest to the image quality of the first video clip;
  • the first super-resolution model determination subunit is configured to determine the vertical super-resolution model corresponding to the target video segment to be compared as the target vertical super-resolution model.
  • the display device 200 may process the second video segment by using the preset N vertical super-score models, and compare the second video segment processed by each vertical super-score model (ie, the video segment to be compared) with the second video segment.
  • the video clips are compared, and the vertical super-score model whose image quality of the processed second video clip is the closest to the image quality of the first video clip is determined as the above-mentioned target vertical super-score model, so that it is possible to choose to play at a low bit rate.
  • the image quality processing effect of the video is the best, and the vertical super-score model is used as the target vertical super-score model, which further improves the image quality of the target video played at a low bit rate.
  • the second super-score model determination unit includes:
  • a degree of difference obtaining subunit used to obtain the degree of difference between the image quality of the second video clip and the image quality of the first video clip
  • the second super-resolution model determination subunit is configured to determine, among the N vertical super-resolution models, the vertical super-resolution model corresponding to the acquired degree of difference as the target vertical super-resolution model.
  • the display apparatus 200 may, according to the degree of difference between the image quality of the second video segment and the image quality of the first video segment, determine the target vertical super-score model corresponding to the degree of difference among the N vertical super-score models, so that the determination
  • the method of the target vertical super-resolution model is simpler, the efficiency of determining the target vertical super-resolution model is improved, and the efficiency of video processing is further improved.
  • the apparatus 200 further includes:
  • the bit rate recovery module is used to restore the bit rate of the target video played by the electronic device to the second bit rate when it is detected that the network state of the electronic device returns to normal, and stop the target vertical super-resolution model from performing the image frame of the target video. to be processed.
  • the display device 200 can stop the processing of the image frame of the target video played at the second bit rate by the target vertical super-resolution model in time, which can save the power consumption caused by unnecessary image processing by the target vertical super-resolution model. , to save the power of electronic equipment.
  • the display device in this embodiment of the present application may be a device, or may be a component, an integrated circuit, or a chip in a terminal.
  • the device can be a mobile electronic device or a non-mobile electronic device.
  • the mobile electronic device may be a mobile phone, a tablet computer, a notebook computer, a palmtop computer, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (personal digital assistant).
  • UMPC ultra-mobile personal computer
  • non-mobile electronic devices can be servers, network attached storage (Network Attached Storage, NAS), personal computer (personal computer, PC), television (television, TV), teller machine or self-service machine, etc., this application Examples are not specifically limited.
  • the display device in this embodiment of the present application may be a device having an operating system.
  • the operating system may be an Android (Android) operating system, an ios operating system, or other possible operating systems, which are not specifically limited in the embodiments of the present application.
  • the display device provided by the embodiment of the present application can implement each process implemented by the method embodiment of FIG. 1 , and to avoid repetition, details are not described here.
  • an embodiment of the present application further provides an electronic device 300 , including a processor 301 , a memory 302 , a program or instruction stored in the memory 302 and executable on the processor 301 , the program Or, when the instruction is executed by the processor 301, each process of the above-mentioned display method embodiment can be implemented, and the same technical effect can be achieved. In order to avoid repetition, details are not repeated here.
  • the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.
  • FIG. 4 is a schematic diagram of a hardware structure of an embodiment of another electronic device provided by the present application.
  • the electronic device 400 includes but is not limited to: a radio frequency unit 401, a network module 402, an audio output unit 403, an input unit 404, a sensor 405, a display unit 406, a user input unit 407, an interface unit 408, a memory 409, and a processor 410, etc. part.
  • the electronic device 400 may also include a power source (such as a battery) for supplying power to various components, and the power source may be logically connected to the processor 410 through a power management system, so that the power management system can manage charging, discharging, and power management. consumption management and other functions.
  • a power source such as a battery
  • the structure of the electronic device shown in FIG. 4 does not constitute a limitation to the electronic device.
  • the electronic device may include more or less components than the one shown, or combine some components, or arrange different components, which will not be repeated here. .
  • the processor 410 is used for:
  • Display unit 406 for:
  • the target image frame is displayed, and the target image frame is an image frame obtained by processing the image frame of the target video based on the target vertical superdivision model.
  • the electronic device 400 can process the image frame of the video played under the low bit rate by using the selected vertical super-resolution model, so as to improve the image quality of the video played under the low bit rate, thereby improving the electronic device when abnormal network conditions occur.
  • the processor 410 is further configured to:
  • the vertical super-score model corresponding to the video type of the target video is determined as the target vertical super-score model.
  • the electronic device 400 can determine a corresponding target vertical super-score model among the preset N vertical super-score models according to the video type of the target video, so that the selected vertical super-score model is more suitable for the first bit rate
  • the processing of the video frame of the target video to be played further improves the image quality of the video played at a low bit rate.
  • the processor 410 is further configured to:
  • the video type information is the information sent by the server when the video data of the target video is transmitted;
  • the video type of the target video is determined based on the image of the target image frame, where the target image frame is: a partial image frame of the target video.
  • the electronic device 400 can determine the video type of the target video according to the video type information transmitted by the server, so that the efficiency of video processing can be improved; it can also determine the video type of the target video according to the video frame of the target video, so as to obtain the video type of the target video.
  • the video type is flexible and can meet different needs.
  • the processor 410 is further configured to:
  • a second video clip is acquired, the second video clip is a video clip with the same content as the first video clip, and the image quality of the first video clip is higher than that of the first video clip The image quality of the two video clips;
  • a target vertical super-score model is determined among the N vertical super-score models.
  • the electronic device 400 may determine the above-mentioned among the N vertical super-score models according to the first video segment of the target video sent by the server and the second video segment played at the first bit rate with the same content as the first video segment
  • the target vertical super-score model makes the determined target vertical super-score model for processing the target video played at the first bit rate more suitable, and further improves the image quality of the target video played at the first bit rate.
  • the processor 410 is further configured to:
  • the image frames of the second video segment are respectively processed based on the N vertical super-resolution models, and N video segments to be compared corresponding to the N vertical super-resolution models are obtained;
  • the target video clip to be compared determines the target video clip to be compared whose image quality is the closest to the image quality of the first video clip
  • the vertical super-score model corresponding to the target video segment to be compared is determined as the target vertical super-score model.
  • the electronic device 400 may process the second video segment by using the preset N vertical super-score models, and compare the second video segment processed by each vertical super-score model (ie, the video segment to be compared) with the second video segment.
  • the video clips are compared, and the vertical super-score model whose image quality of the processed second video clip is the closest to the image quality of the first video clip is determined as the above-mentioned target vertical super-score model, so that it is possible to choose to play at a low bit rate.
  • the image quality processing effect of the video is the best, and the vertical super-score model is used as the target vertical super-score model, which further improves the image quality of the target video played at a low bit rate.
  • the processor 410 is further configured to:
  • the vertical super-resolution model corresponding to the obtained degree of difference is determined as the target vertical super-resolution model.
  • the electronic device 400 may, according to the degree of difference between the image quality of the second video segment and the image quality of the first video segment, determine the target vertical super-score model corresponding to the degree of difference among the N vertical super-score models, so that the determination
  • the method of the target vertical super-resolution model is simpler, the efficiency of determining the target vertical super-resolution model is improved, and the efficiency of video processing is further improved.
  • the processor 410 is further configured to:
  • the bit rate of the target video played by the electronic device is restored to the second bit rate, and the target vertical super-score model is stopped from processing the image frames of the target video.
  • the electronic device 400 can stop the processing of the image frame of the target video played at the second bit rate by the target vertical super-resolution model in time, which can save the power consumption caused by unnecessary image processing by the target vertical super-resolution model. , to save the power of electronic equipment.
  • the input unit 404 may include a graphics processor (Graphics Processing Unit, GPU) 4041 and a microphone 4042. Such as camera) to obtain still pictures or video image data for processing.
  • the display unit 406 may include a display panel 4061, which may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like.
  • the user input unit 407 includes a touch panel 4071 and other input devices 4072 .
  • the touch panel 4071 is also called a touch screen.
  • the touch panel 4071 may include two parts, a touch detection device and a touch controller.
  • Other input devices 4072 may include, but are not limited to, physical keyboards, function keys (such as volume control keys, switch keys, etc.), trackballs, mice, and joysticks, which are not described herein again.
  • Memory 409 may be used to store software programs as well as various data including, but not limited to, application programs and operating systems.
  • the processor 410 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and application programs, and the like, and the modem processor mainly handles wireless communication. It can be understood that, the above-mentioned modulation and demodulation processor may not be integrated into the processor 410.
  • the embodiments of the present application further provide a readable storage medium, where a program or an instruction is stored on the readable storage medium, and when the program or instruction is executed by a processor, each process of the above-mentioned display method embodiment can be realized, and the same technical effect can be achieved , in order to avoid repetition, it will not be repeated here.
  • the processor is the processor in the electronic device in the above embodiment.
  • Readable storage media including computer-readable storage media, such as computer read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disks or non-transitory computer-readable storage such as optical disks medium.
  • An embodiment of the present application further provides a chip, where the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run a program or an instruction, implement each process of the above display method embodiment, and can achieve the same technology The effect, in order to avoid repetition, is not repeated here.
  • the chip mentioned in the embodiments of the present application may also be referred to as a system-on-chip, a system-on-chip, a system-on-a-chip, or a system-on-a-chip, or the like.
  • the method of the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is better implementation.
  • the technical solution of the present application can be embodied in the form of a software product in essence or in a part that contributes to the prior art, and the computer software product is stored in a storage medium (such as ROM/RAM, magnetic disk, CD-ROM), including several instructions to make a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) execute the methods described in the various embodiments of this application.
  • a storage medium such as ROM/RAM, magnetic disk, CD-ROM

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Computer Security & Cryptography (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

La présente demande se rapporte au domaine technique du traitement vidéo. La divulgation concerne un procédé et un appareil d'affichage, ainsi qu'un dispositif électronique. Le procédé comprend les étapes suivantes : lorsqu'il est détecté qu'un état de réseau d'un dispositif électronique est anormal, et qu'un débit binaire auquel le dispositif électronique lit une vidéo cible est réduit à un premier débit binaire, déterminer, parmi N modèles de super-résolution verticale prédéfinis dans le dispositif électronique, un modèle de super-résolution verticale cible de la vidéo cible, N étant un nombre entier supérieur à 1 ; et lorsqu'une trame d'image de la vidéo cible est transmise au premier débit binaire, afficher une trame d'image cible, la trame d'image cible étant une trame d'image obtenue après que la trame d'image de la vidéo cible est traitée sur la base du modèle de super-résolution verticale cible.
PCT/CN2022/073465 2021-01-27 2022-01-24 Procédé et appareil d'affichage et dispositif électronique WO2022161310A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110113902.6 2021-01-27
CN202110113902.6A CN112910875B (zh) 2021-01-27 2021-01-27 显示方法及装置

Publications (1)

Publication Number Publication Date
WO2022161310A1 true WO2022161310A1 (fr) 2022-08-04

Family

ID=76119195

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/073465 WO2022161310A1 (fr) 2021-01-27 2022-01-24 Procédé et appareil d'affichage et dispositif électronique

Country Status (2)

Country Link
CN (1) CN112910875B (fr)
WO (1) WO2022161310A1 (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112910875B (zh) * 2021-01-27 2023-06-30 维沃移动通信有限公司 显示方法及装置

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190045248A1 (en) * 2018-05-31 2019-02-07 Intel Corporation Super resolution identifier mechanism
CN109688465A (zh) * 2018-11-27 2019-04-26 Oppo广东移动通信有限公司 视频增强控制方法、装置以及电子设备
KR20190117416A (ko) * 2019-09-26 2019-10-16 엘지전자 주식회사 동영상 프레임 해상도를 향상시키기 위한 방법 및 장치
CN110958469A (zh) * 2019-12-13 2020-04-03 联想(北京)有限公司 视频处理方法、装置、电子设备及存储介质
CN112910875A (zh) * 2021-01-27 2021-06-04 维沃移动通信有限公司 显示方法及装置

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4444354B2 (ja) * 2008-08-04 2010-03-31 株式会社東芝 画像処理装置、および画像処理方法
JP6689656B2 (ja) * 2016-04-18 2020-04-28 ルネサスエレクトロニクス株式会社 画像処理システム、画像処理方法及び画像送信装置
CN109862370A (zh) * 2017-11-30 2019-06-07 北京大学 视频超分辨率处理方法及装置
CN111784624B (zh) * 2019-04-02 2024-05-24 北京沃东天骏信息技术有限公司 目标检测方法、装置、设备及计算机可读存储介质
CN111667406B (zh) * 2020-04-24 2023-05-30 东北大学 一种基于时域相关性的视频图像超分辨率重建方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190045248A1 (en) * 2018-05-31 2019-02-07 Intel Corporation Super resolution identifier mechanism
CN109688465A (zh) * 2018-11-27 2019-04-26 Oppo广东移动通信有限公司 视频增强控制方法、装置以及电子设备
KR20190117416A (ko) * 2019-09-26 2019-10-16 엘지전자 주식회사 동영상 프레임 해상도를 향상시키기 위한 방법 및 장치
CN110958469A (zh) * 2019-12-13 2020-04-03 联想(北京)有限公司 视频处理方法、装置、电子设备及存储介质
CN112910875A (zh) * 2021-01-27 2021-06-04 维沃移动通信有限公司 显示方法及装置

Also Published As

Publication number Publication date
CN112910875A (zh) 2021-06-04
CN112910875B (zh) 2023-06-30

Similar Documents

Publication Publication Date Title
CN109783178B (zh) 一种界面组件的颜色调整方法、装置、设备和介质
CN108737908B (zh) 一种媒体播放方法、装置及存储介质
WO2022111672A1 (fr) Procédé et appareil de traitement de données et dispositif électronique
JP6404368B2 (ja) 動的フレームレートサポートを用いる電力最適化
WO2021175054A1 (fr) Procédé de traitement de données d'image, et appareil associé
CN111866433B (zh) 视频源切换方法、播放方法、装置、设备和存储介质
CN108536370B (zh) 一种输入界面切换方法及装置、电子设备
WO2022242213A1 (fr) Procédé de réglage du fréquence de rafraîchissement et dispositif électronique
WO2022161383A1 (fr) Procédé et appareil de commande de filmage, et dispositif électronique
US9781380B2 (en) Method, apparatus and terminal for playing multimedia content
WO2022116962A1 (fr) Procédé et appareil de lecture vidéo, et dispositif électronique
CN103488450A (zh) 一种投射图片的方法、装置及终端设备
WO2023061096A1 (fr) Procédé et appareil d'ajustement de la fréquence de trame de rafraîchissement d'un écran d'affichage, et processeur, puce et terminal
CN111949239B (zh) 一种屏幕共享方法、装置、存储介质及终端
WO2023045885A1 (fr) Procédé et appareil de synchronisation
WO2022161310A1 (fr) Procédé et appareil d'affichage et dispositif électronique
CN114189700A (zh) 直播卡顿提示方法、装置、计算机设备和存储介质
US20140240202A1 (en) Information processing method and apparatus for electronic device
US20240214443A1 (en) Methods, systems, and media for selecting video formats for adaptive video streaming
US20170064389A1 (en) Transmission apparatus, transmission method, reception apparatus, and reception method
US20230368338A1 (en) Image display method and apparatus, and electronic device
WO2023125553A1 (fr) Procédé et appareil d'interpolation de trame et dispositif électronique
JP2013015741A (ja) 画像出力装置、画像出力方法、およびプログラム
CN112199149A (zh) 界面渲染方法、装置及电子设备
CN107004018B (zh) 数据处理方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22745185

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22745185

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 17-01-2024)

122 Ep: pct application non-entry in european phase

Ref document number: 22745185

Country of ref document: EP

Kind code of ref document: A1