CN110062244B

CN110062244B - VoLTE video compensation method, terminal and readable storage medium

Info

Publication number: CN110062244B
Application number: CN201910359798.1A
Authority: CN
Inventors: 柴启蕾
Original assignee: Nubia Technology Co Ltd
Current assignee: Nubia Technology Co Ltd
Priority date: 2019-04-29
Filing date: 2019-04-29
Publication date: 2023-05-19
Anticipated expiration: 2039-04-29
Also published as: CN110062244A

Abstract

The invention discloses a VoLTE video compensation method, which comprises the following steps: when VoLTE video operation is currently carried out by the VoLTE terminal and data loss of a currently received VoLTE video frame is detected, a history frame is acquired based on a preset storage area, then a video compensation type is determined based on the VoLTE video frame and the history frame, the VoLTE video frame is updated based on the compensation type, and the history frame stored in the preset storage area is updated based on the updated VoLTE video frame. The invention also discloses a terminal and a readable storage medium. The VoLTE terminal caches the frame video, and when the current video frame is lost, the cached previous frame video frame is utilized to compensate according to a preset algorithm, so that the problem that the VoLTE video picture is blocked or a screen is jumped due to the data loss caused by network transmission is solved, and good experience is brought to users.

Description

VoLTE video compensation method, terminal and readable storage medium

Technical Field

The present invention relates to the field of terminal technologies, and in particular, to a VoLTE video compensation method, a terminal, and a readable storage medium.

Background

With the rapid development of communication services, voLTE applications will get more and more use, and VoLTE video frames mainly consist of two parts: firstly, a terminal A calls a front or rear camera of the terminal A, acquires a picture of the terminal A, displays the picture for the terminal A to see, packages the acquired picture information and sends the packaged picture information to another terminal B; secondly, the terminal A acquires and displays the picture data packet of the terminal B. Therefore, the first picture is related to the camera of the first picture, and whether the picture quality is clear or not needs the camera to apply the allocation parameters; the second is the picture packet information of another terminal acquired from the network, and the main reason for poor picture quality may be that the packets are delayed or lost in the network transmission process, so that a jam or screen is caused when the terminal B displays the picture, and in this case, the use experience of both video parties is greatly affected.

The foregoing is provided merely for the purpose of facilitating understanding of the technical solutions of the present invention and is not intended to represent an admission that the foregoing is prior art.

Disclosure of Invention

The invention mainly aims to provide a VoLTE video compensation method, a terminal and a readable storage medium, and aims to solve the technical problem of video picture blocking or screen display of VoLTE caused by data loss due to network transmission.

In order to achieve the above object, the present invention provides a VoLTE video compensation method applied to a VoLTE terminal, the VoLTE video compensation method comprising the steps of:

when VoLTE video operation is currently performed by the VoLTE terminal and data occurrence of a currently received VoLTE video frame is detected, acquiring a history frame based on a preset storage area;

determining a compensation type of a video based on the VoLTE video frame and the history frame;

updating the VoLTE video frame based on the compensation type, and updating the history frame stored in the preset storage area based on the updated VoLTE video frame.

In an embodiment, the step of determining the compensation type of the video based on the VoLTE video frame and the history frame comprises:

determining a pixel area corresponding to the VoLTE video frame, in which data loss occurs, and determining first position information corresponding to a preset selection area based on the pixel area;

determining an image fluctuation ratio based on the first location information, the pixel region, the VoLTE video frame, and the history frame;

a type of compensation for the video is determined based on the image fluctuation ratio.

In an embodiment, the step of determining an image fluctuation ratio based on the first location information, the pixel region, the VoLTE video frame, and the history frame comprises:

determining second position information corresponding to a target pixel area based on the first position information and the pixel area;

an image fluctuation ratio is determined based on the second location information, the VoLTE video frame, and the historical frame.

In an embodiment, the step of determining an image fluctuation ratio based on the second location information, the VoLTE video frame, and the history frame comprises:

acquiring first RGB data of each pixel point corresponding to the second position information based on the VoLTE video frame;

acquiring second RGB data of each pixel point corresponding to the second position information based on the history frame;

subtracting R component, G component and B component of the first RGB data and the second RGB data of the pixel respectively, solving absolute values, and obtaining R component fluctuation ratio, G component fluctuation ratio and B component fluctuation ratio corresponding to the pixel based on a preset algorithm;

the image fluctuation ratio is determined based on the R component fluctuation ratio, G component fluctuation ratio, and B component fluctuation ratio corresponding to the pixel.

In an embodiment, the compensation type includes a substitution compensation and a difference compensation, and the step of determining the compensation type of the video based on the image fluctuation ratio includes:

when the image fluctuation ratio is smaller than or equal to a preset fluctuation ratio, determining that the compensation type is substitution compensation;

and when the image fluctuation ratio is larger than a preset fluctuation ratio, determining the compensation type as difference compensation.

In an embodiment, the step of updating the VoLTE video frame based on the compensation type comprises:

determining first compensation region data based on the pixel region and the history frame when the compensation type is a replacement compensation;

updating the VoLTE video frame based on the first compensation zone data.

In an embodiment, the step of updating the VoLTE video frame based on the compensation type further comprises:

determining second compensation region data based on the image fluctuation ratio and the history frame when the compensation type is difference compensation;

updating the VoLTE video frame based on the second compensation zone data.

In an embodiment, the step of determining the second compensation area data based on the image fluctuation ratio and the history frame when the compensation type is difference compensation includes:

determining third position information corresponding to the pixel area, and acquiring RGB data of each pixel corresponding to the third position information based on the history frame;

the second compensation area data is determined based on RGB data of each pixel corresponding to the image fluctuation ratio and the third position information.

In addition, to achieve the above object, the present invention further provides a VoLTE terminal, including: the video compensation system comprises a memory, a processor and a VoLTE video compensation program stored on the memory and capable of running on the processor, wherein the VoLTE video compensation program realizes the steps of the VoLTE video compensation method when being executed by the processor.

In addition, to achieve the above object, the present invention further provides a readable storage medium having stored thereon a VoLTE video compensation program, which when executed by a processor, implements the steps of the VoLTE video compensation method described in any one of the above.

According to the method, when VoLTE video operation is currently carried out by the VoLTE terminal and data loss of the currently received VoLTE video frame is detected, a history frame is acquired based on a preset storage area, then a video compensation type is determined based on the VoLTE video frame and the history frame, the VoLTE video frame is updated based on the compensation type, and the history frame stored in the preset storage area is updated based on the updated VoLTE video frame. The VoLTE terminal caches the frame video, and when the current video frame is lost, the cached previous frame video frame is utilized to compensate according to a preset algorithm, so that the problem that the VoLTE video picture is blocked or a screen is jumped due to the data loss caused by network transmission is solved, and good experience is brought to users.

Drawings

FIG. 1 is a schematic diagram of a terminal in a hardware operating environment according to an embodiment of the present invention;

fig. 2 is a schematic diagram of a communication network system according to an embodiment of the present invention;

fig. 3 is a flowchart of a first embodiment of the VoLTE video compensation method according to the present invention;

fig. 4 is a schematic diagram of a pixel area and a preset selection area in an embodiment of a VoLTE video compensation method according to the present invention;

fig. 5 is a flowchart of a second embodiment of the VoLTE video compensation method according to the present invention.

The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.

Detailed Description

It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.

In the following description, suffixes such as "module", "component", or "unit" for representing elements are used only for facilitating the description of the present invention, and have no specific meaning per se. Thus, "module," "component," or "unit" may be used in combination.

The terminal may be implemented in various forms. For example, the terminals described in the present invention may include mobile terminals such as cell phones, tablet computers, notebook computers, palm computers, personal digital assistants (Personal Digital Assistant, PDA), portable media players (Portable Media Player, PMP), navigation devices, wearable devices, smart bracelets, pedometers, and fixed terminals such as digital TVs, desktop computers, and the like.

The following description will be given taking a mobile terminal as an example, and those skilled in the art will understand that the configuration according to the embodiment of the present invention can be applied to a fixed type terminal in addition to elements particularly used for a moving purpose.

Referring to fig. 1, which is a schematic diagram of a hardware structure of a mobile terminal implementing various embodiments of the present invention, the mobile terminal 100 may include: RF (Radio Frequency) unit 101, wi-Fi module 102, audio output unit 103, a/V (audio/video) input unit 104, sensor 105, display unit 106, user input unit 107, interface unit 108, memory 109, processor 110, and power supply 111. Those skilled in the art will appreciate that the mobile terminal 100 structure shown in fig. 1 does not constitute a limitation of the mobile terminal 100, and that the mobile terminal 100 may include more or fewer components than shown, or may combine certain components, or may have a different arrangement of components.

The various components of the mobile terminal 100 are described in detail below in conjunction with fig. 1:

the radio frequency unit 101 may be used for receiving and transmitting signals during the information receiving or communication process, specifically, after receiving downlink information of the base station, processing the downlink information by the processor 110; and, the uplink data is transmitted to the base station. Typically, the radio frequency unit 101 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like. In addition, the radio frequency unit 101 may also communicate with networks and other devices via wireless communications. The wireless communication may use any communication standard or protocol including, but not limited to, GSM (Global System of Mobile communication, global system for mobile communications), GPRS (General Packet Radio Service ), CDMA2000 (Code Division Multiple Access, CDMA 2000), WCDMA (Wideband Code Division Multiple Access ), TD-SCDMA (Time Division Synchronous Code Division Multiple Access, time division synchronous code division multiple access), FDD-LTE (Frequency Division Duplexing-Long Term Evolution, frequency division duplex long term evolution), TDD-LTE (Time Division Duplexing-Long Term Evolution, time division duplex long term evolution), and the like.

Wi-Fi belongs to a short-range wireless transmission technology, and the mobile terminal 100 can help a user to send and receive e-mail, browse web pages, access streaming media and the like through the Wi-Fi module 102, so that wireless broadband internet access is provided for the user. Although fig. 1 shows Wi-Fi module 102, it is to be understood that it does not belong to the necessary constitution of mobile terminal 100, and can be omitted entirely as required within the scope of not changing the essence of the invention.

The audio output unit 103 may convert audio data received by the radio frequency unit 101 or the Wi-Fi module 102 or stored in the memory 109 into an audio signal and output as sound when the mobile terminal 100 is in a call signal reception mode, a talk mode, a recording mode, a voice recognition mode, a broadcast reception mode, or the like. Also, the audio output unit 103 may also provide audio output (e.g., a call signal reception sound, a message reception sound, etc.) related to a specific function performed by the mobile terminal 100. The audio output unit 103 may include a speaker, a buzzer, and the like.

The a/V input unit 104 is used to receive an audio or video signal. The a/V input unit 104 may include a graphics processor (Graphics Processing Unit, GPU) 1041 and a microphone 1042, the graphics processor 1041 processing image data of still pictures or video obtained by an image capturing device (e.g., a camera) in a video capturing mode or an image capturing mode. The processed image frames may be displayed on the display unit 106. The image frames processed by the graphics processor 1041 may be stored in the memory 109 (or other storage medium) or transmitted via the radio frequency unit 101 or Wi-Fi module 102. The microphone 1042 can receive sound (audio data) via the microphone 1042 in a phone call mode, a recording mode, a voice recognition mode, and the like, and can process such sound into audio data. The processed audio (voice) data may be converted into a format output that can be transmitted to the mobile communication base station via the radio frequency unit 101 in the case of a telephone call mode. The microphone 1042 may implement various types of noise cancellation (or suppression) algorithms to cancel (or suppress) noise or interference generated in the course of receiving and transmitting the audio signal.

The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel 1061 according to the brightness of ambient light, and the proximity sensor can turn off the display panel 1061 and/or the backlight when the mobile terminal 100 moves to the ear. As one of the motion sensors, the accelerometer sensor can detect the acceleration in all directions (generally three axes), and can detect the gravity and direction when stationary, and can be used for applications of recognizing the gesture of a mobile phone (such as horizontal and vertical screen switching, related games, magnetometer gesture calibration), vibration recognition related functions (such as pedometer and knocking), and the like; as for other sensors such as fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc. that may also be configured in the mobile phone, the detailed description thereof will be omitted.

The display unit 106 is used to display information input by a user or information provided to the user. The display unit 106 may include a display panel 1061, and the display panel 1061 may be configured in the form of a liquid crystal display (Liquid Crystal Display, LCD), an Organic Light-Emitting Diode (OLED), or the like.

The user input unit 107 may be used to receive input numeric or character information and to generate key signal inputs related to user settings and function control of the mobile terminal. In particular, the user input unit 107 may include a touch panel 1071 and other input devices 1072. The touch panel 1071, also referred to as a touch screen, may collect touch operations thereon or thereabout by a user (e.g., operations of the user on the touch panel 1071 or thereabout by using any suitable object or accessory such as a finger, a stylus, etc.) and drive the corresponding connection device according to a predetermined program. The touch panel 1071 may include two parts of a touch detection device and a touch controller. The touch detection device detects the touch azimuth of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends the touch point coordinates to the processor 110, and can receive and execute commands sent from the processor 110. Further, the touch panel 1071 may be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. The user input unit 107 may include other input devices 1072 in addition to the touch panel 1071. In particular, other input devices 1072 may include, but are not limited to, one or more of a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, mouse, joystick, etc., as specifically not limited herein.

Further, the touch panel 1071 may overlay the display panel 1061, and when the touch panel 1071 detects a touch operation thereon or thereabout, the touch panel 1071 is transferred to the processor 110 to determine the type of touch event, and then the processor 110 provides a corresponding visual output on the display panel 1061 according to the type of touch event. Although in fig. 1, the touch panel 1071 and the display panel 1061 are two independent components for implementing the input and output functions of the mobile terminal, in some embodiments, the touch panel 1071 may be integrated with the display panel 1061 to implement the input and output functions of the mobile terminal, which is not limited herein.

The interface unit 108 serves as an interface through which at least one external device can be connected with the mobile terminal 100. For example, the external devices may include a wired or wireless headset port, an external power (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device having an identification module, an audio input/output (I/O) port, a video I/O port, an earphone port, and the like. The interface unit 108 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the mobile terminal 100 or may be used to transmit data between the mobile terminal 100 and an external device.

Memory 109 may be used to store software programs as well as various data. The memory 109 may mainly include a storage program area that may store an operating system, application programs required for at least one function (such as a sound playing function, an image playing function, etc.), and a storage data area; the storage data area may store data (such as audio data, phonebook, etc.) created according to the use of the handset, etc. In addition, memory 109 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device.

The processor 110 is a control center of the mobile terminal 100, connects various parts of the entire mobile terminal using various interfaces and lines, and performs various functions of the mobile terminal 100 and processes data by running or executing software programs and/or modules stored in the memory 109 and calling data stored in the memory 109, thereby performing overall monitoring of the mobile terminal 100. Processor 110 may include one or more processing units; preferably, the processor 110 may integrate an application processor that primarily handles operating systems, user interfaces, applications, etc., with a modem processor that primarily handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 110.

In addition, in the mobile terminal shown in fig. 1, the processor 110 is configured to invoke the wrist information reporting program stored in the memory 109, and execute the steps of the wrist information reporting method provided in the embodiments of the present application.

The mobile terminal 100 may further include a power source 111 (e.g., a battery) for supplying power to the respective components, and preferably, the power source 111 may be logically connected to the processor 110 through a power management system, so as to perform functions of managing charging, discharging, and power consumption management through the power management system.

Although not shown in fig. 1, the mobile terminal 100 may further include a bluetooth module or the like, which is not described herein.

In order to facilitate understanding of the embodiments of the present invention, a communication network system on which the mobile terminal of the present invention is based will be described below.

Referring to fig. 2, fig. 2 is a schematic diagram of a communication network system according to an embodiment of the present invention, where the communication network system is an LTE system of a general mobile communication technology, and the LTE system includes a UE (User Equipment) 201, an e-UTRAN (Evolved UMTS Terrestrial Radio Access Network ) 202, an epc (Evolved Packet Core, evolved packet core) 203, and an IP service 204 of an operator that are sequentially connected in communication.

Specifically, the UE201 may be the mobile terminal 100 described above, and will not be described herein.

The E-UTRAN202 includes eNodeB2021 and other eNodeB2022, etc. The eNodeB2021 may be connected with other eNodeB2022 by a backhaul (e.g., an X2 interface), the eNodeB2021 is connected to the EPC203, and the eNodeB2021 may provide access from the UE201 to the EPC 203.

EPC203 may include MME (Mobility Management Entity ) 2031, hss (Home Subscriber Server, home subscriber server) 2032, other MMEs 2033, SGW (Serving Gate Way) 2034, pgw (PDN Gate Way) 2035 and PCRF (Policy and Charging Rules Function, policy and tariff function entity) 2036, and so on. The MME2031 is a control node that handles signaling between the UE201 and EPC203, providing bearer and connection management. HSS2032 is used to provide registers to manage functions such as home location registers (not shown) and to hold user specific information about service characteristics, data rates, etc. All user data may be sent through SGW2034 and PGW2035 may provide IP address allocation and other functions for UE201, PCRF2036 is a policy and charging control policy decision point for traffic data flows and IP bearer resources, which selects and provides available policy and charging control decisions for a policy and charging enforcement function (not shown).

IP services 204 may include the internet, intranets, IMS (IP Multimedia Subsystem ), or other IP services, etc.

Although the LTE system is described above as an example, it should be understood by those skilled in the art that the present invention is not limited to LTE systems, but may be applied to other wireless communication systems, such as GSM, CDMA2000, WCDMA, TD-SCDMA, and future new network systems.

Based on the above terminal hardware structure and the communication network system, various embodiments of the VoLTE video compensation method of the present invention are presented.

The invention further provides a VoLTE video compensation method, referring to fig. 3, fig. 3 is a schematic flow chart of a first embodiment of the VoLTE video compensation method of the invention.

In this embodiment, the VoLTE video compensation method is applied to VoLTE terminals. The VoLTE video compensation method comprises the following steps:

step S100, when the VoLTE terminal currently performs VoLTE video operation and detects that data of a currently received VoLTE video frame occur, acquiring a history frame based on a preset storage area;

in this embodiment, voLTE (Voice over Long-Term Evolution) is a high-speed wireless communication standard for mobile phones and data terminals, which is based on an IP Multimedia Subsystem (IMS) network, and uses configuration files tailored to the control plane and the media plane of the Voice service on LTE, which enables the Voice service (control and media planes) to be transmitted as a data stream in the LTE data bearer network without maintaining and relying on the conventional circuit switched Voice network. VoLTE has more than three times the voice and data capacity of 3G UMTS and more than six times the voice and data capacity of 2G GSM.

Specifically, when a user performs video operation by using a VoLTE video service through a VoLTE terminal, the VoLTE terminal caches a received video frame in a preset storage area, where the preset storage area may be an internal memory or an external memory of the VoLTE terminal, and the present invention is not limited thereto. Further, when the VoLTE terminal receives a new video frame, the new received video frame is buffered in the preset storage area to replace the previously buffered video frame, that is, the preset storage area stores only one video frame, and the stored video frame is the last video frame of the currently received video frame.

Further, when the VoLTE terminal detects that the currently received VoLTE video frame has data loss, the VoLTE terminal acquires a history frame from the preset storage area, wherein the history frame is the last video frame already stored in the preset storage area.

The user application scene corresponding to the invention is a non-instant application scene, that is, the pictures of the two parties for video call have no large-area abrupt change, and the character image is mainly displayed.

Step 200, determining a compensation type of the video based on the VoLTE video frame and the history frame;

in this embodiment, when data occurs in the VoLTE video frame, compensation needs to be performed on the VoLTE video frame to prevent video frames displayed on the VoLTE terminal screen from being blocked or blurred, thereby affecting the video effect of the user. In the invention, the video compensation types comprise two types, namely substitution compensation and difference compensation, and the video compensation type needs to be determined according to the currently received VoLTE video frame and the history frame.

Specifically, step S200 includes:

step S210, determining a pixel area corresponding to the VoLTE video frame, in which data loss occurs, and determining first position information corresponding to a preset selection area based on the pixel area;

in this embodiment, when the VoLTE terminal detects that the currently received VoLTE video frame has data loss, the position of the pixel area where the data loss occurs is determined, where the pixel area where the data loss occurs includes a plurality of pixel points, and further, the position information of the pixel where the pixel area is located at the center point is determined. Then, the pixel at the center point of the pixel area is taken as the center of a preset selection area, and the position information of the preset selection area is determined, as shown in fig. 4, fig. 4 is a schematic diagram of the pixel area and the preset selection area, wherein the preset selection area can be rectangular, circular or square, and the like, the specific shape of the selection area is not limited in the embodiment, the size of the selection area is related to the size of the video frame, and the size adjustment of the selection area is supported according to a preset rule. The preset selection area is used for selecting the pixel points near the pixel area, further judging the change condition of the RGB values of the pixel points near the pixel area, and determining the compensation type of the video according to the change condition of the RGB values of the pixel points near the pixel area.

Step S220, determining an image fluctuation ratio based on the first location information, the pixel region, the VoLTE video frame, and the history frame;

in this embodiment, after determining the position information of the pixel where the pixel area is located at the center point on the VoLTE video frame, that is, the currently received video frame, the first position information corresponding to the preset selection area is further determined. The historical frame is the last frame data of the currently received video frame, so that the two formats are consistent, and meanwhile, the positions of a corresponding preset selected area and a pixel area can be determined on the historical frame, wherein the pixel points except the pixel area in the preset selected area are adjacent pixel points of the pixel area, and according to the front-back change conditions of the adjacent pixel points in the VoLTE video frame and the historical frame, the image baud rate can be calculated, namely the fluctuation condition of the adjacent pixel points is determined, so that the compensation type of the video is determined.

Step S230, determining a compensation type of the video based on the image fluctuation ratio.

In this embodiment, the types of compensation of the video include two types, that is, substitution compensation and difference compensation, respectively, and the types of compensation are determined according to the baud rate of the image.

Specifically, step S230 includes:

step S231, when the image fluctuation ratio is smaller than or equal to a preset fluctuation ratio, determining that the compensation type is substitution compensation;

step S232, when the image fluctuation ratio is larger than a preset fluctuation ratio, determining the compensation type as difference compensation.

In this embodiment, a pixel area and a preset selection area are determined on the VoLTE video frame and the history frame at the same time, and pixels except the pixel area in the preset selection area are adjacent pixels of the pixel area, and according to the front-back change conditions of the adjacent pixels in the VoLTE video frame and the history frame, the image baud rate can be calculated, that is, the fluctuation condition of the adjacent pixels is determined according to the front-back change conditions of the video frame.

Specifically, when the image fluctuation ratio is smaller than or equal to the preset fluctuation ratio, it is indicated that RGB data of adjacent pixels in the front and rear frames of the video do not fluctuate too much, so that the compensation type is determined to be the alternative compensation, that is, the adjacent pixels on the history frame can be directly used for directly compensating the currently received VoLTE video frame. In contrast, when the image fluctuation ratio is larger than the preset fluctuation ratio, the fact that the RGB value fluctuation of the adjacent pixel points in the frames before and after the video is larger is indicated, so that compensation cannot be directly performed by the adjacent pixel points, and the compensation type is determined to be difference compensation.

Step S300, updating the VoLTE video frame based on the compensation type, and updating the history frame stored in the preset storage area based on the updated VoLTE video frame.

Specifically, step S300 includes:

in this embodiment, after the compensation type is determined by comparing the image fluctuation ratio with the preset fluctuation ratio, the VoLTE video frame is further updated according to the compensation type, and then the updated VoLTE video frame is stored as a new history frame in the preset storage area.

Step S310, when the compensation type is substitution compensation, determining first compensation area data based on the pixel area and the history frame;

in this embodiment, when the compensation type is the replacement compensation, it is indicated that the RGB values of the adjacent pixels in the frames before and after the video do not greatly fluctuate, so that the compensation type is determined to be the replacement compensation, that is, the adjacent pixels on the history frame can be directly used to directly compensate the currently received VoLTE video frame. Specifically, an area corresponding to a pixel area of a VoLTE video frame is determined on a history frame, and RGB data of pixels of the area is taken as first compensation area data.

Step S320, updating the VoLTE video frame based on the first compensation area data.

In this embodiment, after the first compensation area data is determined on the history frame, that is, RGB data of the pixel points of the area corresponding to the pixel area of the VoLTE video frame on the history frame, the first compensation area data is updated to the pixel area of the VoLTE video frame in a one-to-one correspondence manner, so as to complete video compensation.

Step S330, when the compensation type is difference compensation, determining second compensation area data based on the image fluctuation ratio and the history frame;

in this embodiment, when the compensation type is difference compensation, it is indicated that the RGB values of the adjacent pixels in the frames before and after the video fluctuate greatly, and the second compensation area data is determined according to the image fluctuation ratio and the history frame.

Specifically, step S330 includes:

step S331, determining third position information corresponding to the pixel area, and acquiring RGB data of each pixel corresponding to the third position information based on the history frame;

in this embodiment, first, the position information of the pixel area in the VoLTE video frame, that is, the third position information, is determined, then, according to the third position information, the area corresponding to the pixel area of the VoLTE video frame is determined on the history frame, and the RGB data of each pixel in the area is acquired.

Step S332, determining the second compensation area data based on the image fluctuation ratio and RGB data of each pixel corresponding to the third position information.

Step S340, updating the VoLTE video frame based on the second compensation area data.

In this embodiment, first, R component, G component, and B component of each pixel corresponding to the third position information are multiplied by the image fluctuation ratio to obtain R component fluctuation data, G component fluctuation data, and B component fluctuation data, where the formulas are as follows:

r component fluctuation data=r component x image fluctuation ratio

G component fluctuation data=g component x image fluctuation ratio

B component fluctuation data=b component x image fluctuation ratio

Then, adding the R component, the G component and the B component with the R component fluctuation data, the G component fluctuation data and the B component fluctuation data respectively to obtain an updated R component, an updated G component and an updated B component, wherein the formula is as follows:

updated R component = R component + R component fluctuation data

Updated G component=g component+g component fluctuation data

Updated B component = B component + B component fluctuation data

The updated RGB data of each pixel corresponding to the third location information is the second compensation area data.

And finally, updating the second compensation area data to the pixel area of the VoLTE video frame in a one-to-one correspondence manner, and further completing video compensation.

According to the VoLTE video compensation method, when VoLTE video operation is currently performed by a VoLTE terminal and data loss of currently received VoLTE video frames is detected, historical frames are acquired based on a preset storage area, then a video compensation type is determined based on the VoLTE video frames and the historical frames, the VoLTE video frames are updated based on the compensation type, and the historical frames stored in the preset storage area are updated based on the updated VoLTE video frames. The VoLTE terminal caches the frame video, and when the current video frame is lost, the cached previous frame video frame is utilized to compensate according to a preset algorithm, so that the problem that the VoLTE video picture is blocked or a screen is jumped due to the data loss caused by network transmission is solved, and good experience is brought to users.

Based on the first embodiment, a second embodiment of the VoLTE video compensation method of the present invention is proposed, referring to fig. 5, in this embodiment, step S220 includes:

step S221, determining second location information corresponding to the target pixel area based on the first location information and the pixel area;

in this embodiment, a pixel at a center point of the pixel area is used as a center of a preset selection area, so as to determine first position information of the preset selection area, an area where the pixel points in the preset selection area except the pixel area are located is a target pixel area, and the position information of the target pixel area is second position information, wherein the pixel points in the target pixel area are adjacent pixel points of the pixel area.

Step S222, determining an image fluctuation ratio based on the second location information, the VoLTE video frame, and the history frame.

In this embodiment, the image fluctuation ratio is determined from the second position information, the VoLTE video frame, and the history frame.

Specifically, step S222 includes:

step a, acquiring first RGB data of each pixel point corresponding to the second position information based on the VoLTE video frame;

step b, obtaining second RGB data of each pixel point corresponding to the second position information based on the history frame;

in this embodiment, the image fluctuation ratio is calculated by calculating the RGB data of each pixel point in the target pixel area, where the pixel point in the target pixel area is used as the adjacent pixel point of the pixel area, that is, the fluctuation condition of the adjacent pixel point is determined according to the change condition of the frames before and after the video frame.

Specifically, first RGB data of each pixel corresponding to the second position information is obtained in the VoLTE video frame, and second RGB data of each pixel corresponding to the second position information is obtained in the history frame.

Step c, subtracting R component, G component and B component of the first RGB data and the second RGB data of the pixel respectively, solving absolute values, and obtaining R component fluctuation ratio, G component fluctuation ratio and B component fluctuation ratio corresponding to the pixel based on a preset algorithm;

in this embodiment, first, the R component, G component and B component of the first RGB data and the R component, G component and B component of the second RGB data of each pixel are subtracted and the absolute values are obtained, so as to obtain an R component difference value, a G component difference value and a B component difference value, where the formula is as follows:

r component difference value= |r component of first RGB data-R component of second RGB data|

G component difference= |g component of first RGB data-G component of second RGB data|

B component difference= |b component of first RGB data-B component of second RGB data|

Then, obtaining a maximum value from the R component of the first RGB data and the R component of the second RGB data to obtain a maximum value of the R component; obtaining a maximum value from the G component of the first RGB data and the G component of the second RGB data to obtain a G component maximum value; obtaining the maximum value from the B component of the first RGB data and the B component of the second RGB data to obtain the maximum value of the B component, wherein the formula is as follows:

r component maximum value=max (R component of first RGB data, R component of second RGB data)

Maximum value of G component=max (G component of first RGB data, G component of second RGB data)

Maximum value of B component=max (B component of first RGB data, B component of second RGB data)

Finally, dividing the R component difference value by the maximum value of the R component to obtain an R component fluctuation ratio; dividing the G component difference value by the maximum value of the G component to obtain a G component fluctuation ratio; the B component difference divided by the B component maximum value yields the B component fluctuation ratio as follows:

and d, determining an image fluctuation ratio based on the R component fluctuation ratio, the G component fluctuation ratio and the B component fluctuation ratio corresponding to the pixel.

In this embodiment, the R component fluctuation ratio, the G component fluctuation ratio, and the B component fluctuation ratio are averaged to obtain the image fluctuation ratio, and the formula is as follows:

image fluctuation ratio= (R component fluctuation ratio+g component fluctuation ratio+b component fluctuation ratio)/3

According to the VoLTE video compensation method, the image fluctuation ratio is determined according to the fluctuation condition of the pixel points near the pixel area where data loss occurs, the supplement type of the video frame is determined, and then the video is compensated, so that the problem that VoLTE video pictures are blocked or screen-printed due to data loss caused by network transmission is solved, and good experience is brought to users.

In addition, the embodiment of the invention also provides a readable storage medium, and the readable storage medium stores a VoLTE video compensation program, and the VoLTE video compensation program realizes the steps of the VoLTE video compensation method in each embodiment when being executed by a processor.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.

The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.

From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) as described above, comprising instructions for causing a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method according to the embodiments of the present invention.

The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.

Claims

1. The VoLTE video compensation method is characterized by being applied to a VoLTE terminal, and comprises the following steps of:

updating the VoLTE video frame based on the compensation type, and updating the history frame stored in the preset storage area based on the updated VoLTE video frame;

wherein the step of determining the compensation type of the video based on the VoLTE video frame and the history frame comprises:

2. The VoLTE video compensation method of claim 1, wherein the step of determining an image fluctuation ratio based on the first location information, the pixel region, the VoLTE video frame, and the history frame comprises:

3. The VoLTE video compensation method of claim 2, wherein the step of determining an image fluctuation ratio based on the second location information, the VoLTE video frames, and the history frames comprises:

4. The VoLTE video compensation method of claim 1, wherein the compensation type includes a substitution compensation and a difference compensation, and wherein the determining the compensation type of the video based on the image fluctuation ratio includes:

5. The VoLTE video compensation method of claim 4, wherein the step of updating the VoLTE video frame based on the compensation type comprises:

updating the VoLTE video frame based on the first compensation zone data.

6. The VoLTE video compensation method of claim 4, wherein the step of updating the VoLTE video frame based on the compensation type further comprises:

updating the VoLTE video frame based on the second compensation zone data.

7. The VoLTE video compensation method of claim 6, wherein said step of determining second compensation zone data based on said image fluctuation ratio and said history frame when said compensation type is difference compensation comprises:

8. A VoLTE terminal, characterized in that the VoLTE terminal comprises: memory, a processor and a VoLTE video compensation program stored on the memory and executable on the processor, which when executed by the processor implements the steps of the VoLTE video compensation method of any of claims 1 to 7.

9. A readable storage medium, characterized in that it has stored thereon a VoLTE video compensation program, which when executed by a processor implements the steps of the VoLTE video compensation method according to any of claims 1 to 7.