CN116567229A

CN116567229A - Image processing method, device, equipment and storage medium

Info

Publication number: CN116567229A
Application number: CN202210103023.XA
Authority: CN
Inventors: 曹健; 曹洪彬; 陈思佳; 黄永铖; 杨小祥; 张佳
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2022-01-27
Filing date: 2022-01-27
Publication date: 2023-08-08

Abstract

The application provides an image processing method, an image processing device and a storage medium, wherein the method comprises the following steps: the cloud server determines a first image area and a second image area in a first image frame; performing image quality enhancement processing on a first image area in the first image frame, and performing low-pass filtering processing on a second image area to obtain a second image frame; encoding the second image frame to obtain a code stream; outputting the code stream to terminal equipment, and analyzing the code stream by the terminal equipment to obtain a third image frame; determining a third image area in the third image frame, wherein the third image area is an image area subjected to low-pass filtering processing and encoding and decoding on the second image area; performing image quality enhancement processing on a third image area in the third image frame to obtain a fourth image frame; and finally displaying the fourth image frame. On one hand, the coding difficulty can be reduced, and on the other hand, the image quality can be improved.

Description

Image processing method, device, equipment and storage medium

Technical Field

The embodiment of the application relates to the technical field of video processing, in particular to an image processing method, an image processing device, image processing equipment and a storage medium.

Background

The video or image processing procedure in some cloud-based scenarios can now be as follows: as shown in fig. 1, a cloud server generates a video, performs video image acquisition, performs sharpening processing on the acquired video image, encodes the processed video image to obtain a code stream of the video image, and further, the cloud server may send the code stream to a terminal device, decodes the code stream by the terminal device, and finally displays the video image according to a decoding result.

It should be appreciated that the essence of the sharpening process is to enhance the edges of the image and the portion of the gray jump by compensating the contours of the image so that the image becomes clear. Therefore, the sharpening process increases the high-frequency components of the image, and under the condition of a certain coding code rate, the code rate of the encoder allocated to a non-sharpening area in the image is reduced due to the increase of the high-frequency components, on one hand, the coding difficulty of the encoder is high, and on the other hand, the image quality of the whole image is reduced. Especially, under the conditions of insufficient code rate, very complex video pictures or severe movement, the coding difficulty is increased and the image quality is obviously reduced.

Disclosure of Invention

The application provides an image processing method, an image processing device and a storage medium, which can reduce the coding difficulty on one hand and improve the image quality on the other hand.

In a first aspect, an image processing method is provided, where the method is applied to a cloud server, and the method includes: determining a first image area and a second image area in a first image frame, wherein the first image area is an image area to be introduced with high-frequency components, and the second image area is an image area to be filtered; performing image quality enhancement processing on a first image area in the first image frame, and performing low-pass filtering processing on a second image area to obtain a second image frame; encoding the second image frame to obtain a code stream; and outputting the code stream to the terminal equipment.

In a second aspect, there is provided an image processing method, the method being applied to a terminal device, the method comprising: acquiring a code stream; analyzing the code stream to obtain a third image frame; determining a third image region in a third image frame; performing image quality enhancement processing on a third image area in the third image frame to obtain a fourth image frame; displaying a fourth image frame; the third image frame is an image frame obtained by performing image quality enhancement processing on a first image area in the first image frame and performing low-pass filtering processing on a second image area in the first image frame, and is obtained by performing encoding and decoding on the second image area, and the third image area is an image area obtained by performing low-pass filtering processing on the second image area and performing encoding and decoding on the second image area.

In a third aspect, there is provided an image processing apparatus comprising: the device comprises a determining module, a processing module, an encoding module and an output module, wherein the determining module is used for determining a first image area and a second image area in a first image frame, the first image area is an image area to be introduced with high-frequency components, and the second image area is an image area to be filtered; the processing module is used for carrying out image quality enhancement processing on a first image area in the first image frame and carrying out low-pass filtering processing on a second image area to obtain a second image frame; the encoding module is used for encoding the second image frame to obtain a code stream; the output module is used for outputting the code stream to the terminal equipment.

In a fourth aspect, there is provided an image processing apparatus comprising: the device comprises a first acquisition module, an analysis module, a first determination module, a processing module and a display module, wherein the first acquisition module is used for acquiring a code stream; the analysis module is used for analyzing the code stream to obtain a third image frame; the first determining module is used for determining a third image area in a third image frame; the processing module is used for carrying out image quality enhancement processing on a third image area in the third image frame to obtain a fourth image frame; the display module is used for displaying a fourth image frame; the third image frame is an image frame obtained by performing image quality enhancement processing on a first image area in the first image frame and performing low-pass filtering processing on a second image area in the first image frame, and is obtained by performing encoding and decoding on the second image area, and the third image area is an image area obtained by performing low-pass filtering processing on the second image area and performing encoding and decoding on the second image area.

In a fifth aspect, there is provided an electronic device comprising: a processor and a memory for storing a computer program, the processor being for invoking and running the computer program stored in the memory for performing the method as in the first aspect, the second aspect or various implementations thereof.

In a sixth aspect, a computer readable storage medium is provided for storing a computer program for causing a computer to perform the method as in the first aspect, the second aspect or various implementations thereof.

In a seventh aspect, a computer program product is provided comprising computer program instructions for causing a computer to perform the method as in the first aspect, the second aspect or various implementations thereof.

In an eighth aspect, a computer program is provided, the computer program causing a computer to perform the method as in the first aspect, the second aspect or various implementations thereof.

According to the technical scheme, under the condition that high-frequency components are introduced into the image, low-pass filtering processing can be adopted for some areas in the image, and because some high-frequency components can be removed through the low-pass filtering processing, certain offset can be carried out on the removed high-frequency components and the high-frequency components introduced by the image, the high code rate is not required for the areas subjected to the low-pass filtering processing, so that on one hand, the coding difficulty of an encoder can be reduced, and on the other hand, the terminal equipment can carry out image quality enhancement processing on the areas subjected to the low-pass filtering processing, and therefore the image quality of the whole image can be improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 provides a flow chart of an image processing procedure;

FIG. 2 is a schematic diagram of a cloud game scenario provided in an embodiment of the present application;

fig. 3 is a flowchart of an image processing method according to an embodiment of the present application;

FIG. 4 is a flowchart of a method for determining whether a third image area is reasonable according to an embodiment of the present application;

FIG. 5 is a flowchart of a method for determining whether a third image area is reasonable according to another embodiment of the present application;

fig. 6 is a flowchart of a video rendering collaboration method according to an embodiment of the present application;

fig. 7 is a schematic diagram of an image processing apparatus according to an embodiment of the present application;

FIG. 8 is a schematic diagram of another image processing apparatus according to an embodiment of the present disclosure;

fig. 9 is a schematic block diagram of an electronic device 900 provided by an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or server that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

Before introducing the technical solution of the present application, the following will describe relevant knowledge of the present application:

video coding: the manner in which a file in the original video format is converted into a file in another video format by compression techniques may be referred to as a bitstream.

Video decoding: reverse process of video coding.

Sharpening: an image processing method is provided, which enhances the edge and the gray jump part of an image by compensating the outline of the image, so that the image becomes clear. Sharpening is to highlight edges, contours, of objects in an image, which can improve the contrast between the object and the surrounding background, and is therefore also referred to as edge enhancement.

It should be understood that the present application is not limited to a particular sharpening algorithm, such as: a sharpening mask (unsharp masking) sharpening algorithm may be employed.

Low pass filtering: a signal filtering method is provided, wherein the rule is that the low frequency signal can normally pass through, and the high frequency signal exceeding the set critical value is blocked and weakened. The low-pass filtering can play a role in denoising and blurring, and is one of the most common operations in the field of image processing.

It should be understood that the present application is not limited to a specific low pass filtering algorithm, such as: a bilateral filtering algorithm may be employed.

In this application, "rendering" is also referred to as "processing", for example: image rendering may be referred to as image processing.

The technical problems and the inventive concepts to be solved by the present application will be described below:

as described above, sharpening increases the high frequency components of an image, and in the case where the encoding rate is constant, the increase in the high frequency components results in a decrease in the rate at which the encoder is allocated to a non-sharpened region in the image, which on the one hand results in a greater difficulty in encoding by the encoder and on the other hand results in a decrease in the image quality of the entire image. Especially, under the conditions of insufficient code rate, very complex video pictures or severe movement, the coding difficulty is increased and the image quality is obviously reduced.

In order to solve the above technical problem, the present application proposes that, in the case of introducing a high-frequency component into an image, a low-pass filtering process may be used for some regions in the image, and since some high-frequency components may be removed by the low-pass filtering process, these removed high-frequency components and the high-frequency components introduced into the image may be offset to some extent, and a higher code rate is not required for the region of the low-pass filtering process, so, on the one hand, the coding difficulty of the encoder may be reduced, and on the other hand, for the region of the low-pass filtering process, the terminal device may perform image quality enhancement processing on the region of the low-pass filtering process, so as to improve the image quality of the whole image.

It should be appreciated that the technical scheme of the application can be applied to scenes such as cloud game scenes, interactive live broadcast, video conferences, video calls and the like, but is not limited to the following:

cloud gaming (Cloud gaming), which may also be referred to as game on demand, is an online gaming technology based on Cloud computing technology. Cloud gaming technology enables lightweight devices (thin clients) with relatively limited graphics processing and data computing capabilities to run high quality games. In a cloud game scene, the game is not run in a player game terminal, but is run in a cloud server, the cloud server renders the game scene into a video and audio stream, and the video and audio stream is transmitted to the player game terminal through a network. The player game terminal does not need to have strong graphic operation and data processing capability, and only needs to have basic streaming media playing capability and the capability of acquiring player input instructions and sending the player input instructions to the cloud server.

Fig. 2 is a schematic diagram of a cloud game scenario provided in an embodiment of the present application, as shown in fig. 2, a cloud server 210 may communicate with a player game terminal 220, the cloud server 210 may run a game, collect video images of the game, encode the collected video images to obtain a code stream of the video images, further, the cloud server may send the code stream to a terminal device, the terminal device decodes the code stream, and finally display the video images according to a decoding result.

Optionally, communication between the cloud server 210 and the player gaming terminal 220 may be implemented by long term evolution (Long Term Evolution, LTE), new Radio (NR) technology, wireless fidelity (Wireless Fidelity, wi-Fi) technology, or the like, but is not limited thereto.

In a cloud game scenario, a cloud server refers to a server running a game in the cloud, and has functions of video enhancement (pre-encoding processing), video encoding, and the like, but is not limited thereto.

The terminal equipment is equipment with rich man-machine interaction modes, internet access capability, various operating systems and stronger processing capability. The terminal device may be, but is not limited to, a smart phone, a living room television, a tablet computer, a vehicle-mounted terminal, a player gaming terminal, such as a palm game console, etc.

The technical scheme of the application will be described in detail as follows:

fig. 3 is a flowchart of an image processing method provided in the embodiment of the present application, where the method may be performed by a cloud server and a terminal device, for example, in a cloud game scenario, the cloud server may be the cloud server 210 in fig. 2, and the terminal device may be the player game terminal 220 in fig. 2, and in summary, the application does not limit the execution subject of the image processing method, as shown in fig. 3, where the method includes:

S310: the cloud server determines a first image area and a second image area in a first image frame;

s320: the cloud server performs image quality enhancement processing on a first image area in the first image frame, and performs low-pass filtering processing on a second image area to obtain a second image frame;

s330: the cloud server encodes the second image frame to obtain a code stream;

s340: the cloud server outputs the code stream to the terminal equipment;

s350: the terminal equipment analyzes the code stream to obtain a third image frame;

s360: the terminal equipment determines a third image area in a third image frame, wherein the third image area is an image area subjected to low-pass filtering processing and encoding and decoding on the second image area;

s370: the terminal equipment carries out image quality enhancement processing on a third image area in the third image frame to obtain a fourth image frame;

s380: the terminal device displays a fourth image frame.

It should be appreciated that the first image frame may be any image frame in the target video. The target video may be a cloud video in a cloud video scene, a live video in an interactive live broadcast, or a video in a video conference or a video call, which is not limited in this application.

Optionally, the first image frame is obtained from a video sequence uploaded in real time, which may be a virtual game video sequence; the first image frame includes an image frame header indicating coordinates of the first image region and the second image region.

It should be understood that the first image area is an image in the first image frame into which the high frequency component is to be introduced, for example, the first image area is an image area in the first image frame in which sharpening is required, and the second image area is an image area in the first image frame to be filtered. The first image area and the second image area may or may not constitute the entire image frame, that is, the other areas of the first image frame except the first image area are all the second image area, or the other areas of the first image frame except the first image area include: the second image area further includes other image areas that do not require image quality enhancement processing and low-pass filtering processing.

For example, in a cloud game scene, the first image area may be an image area in which a game character in a certain game image frame is located, and the second image area may be an image area in the game image frame excluding the first image area.

For example, in a cloud game scene, the first image area may be an image area where a game character in a certain game image frame is located, the second image area may be an edge area in the game image frame, and the game image frame includes other image areas in addition to the first image area and the second image area.

As described above, there is one case as follows: the cloud server may determine the first image area first, and then determine an area except the first image area in the first image frame as the second image area. Based on this, in this case, it is important to determine the first image area, wherein the cloud server may determine the first image area in the following manner, but is not limited thereto.

It should be appreciated that the image area through which the image quality enhancement process is required is related to the characteristics of the human eye vision system, such as: the human eye is more sensitive to the image area carrying the characters and the central area of the image. The image area that normally carries text may be at least one of the upper left corner, lower left corner, upper right corner, and lower right corner of the entire image frame, and thus, for the first image frame, at least one of the upper left corner area, lower right corner area, upper right corner area, lower right corner area, and center area of the image frame may be defaulted as the image area that needs sharpening processing, that is, these areas are all the first image area. Alternatively, the cloud server may detect image areas carrying text in a text detection manner, and determine the image areas as the first image area. Still alternatively, the cloud server may employ automated identification algorithms for some regions of interest to determine the first image region.

Optionally, the area sizes of the upper left corner area, the lower right corner area, the upper right corner area, the lower right corner area, and the central area may be negotiated by the cloud server and the terminal device, may be predefined, may be specified by the cloud server, or may be specified by the terminal device, which is not limited in this application.

It should be understood that in general the central region may be represented by:

(W*α，H*α)-(W*(1-α)，H*(1-α))

where W and H are the resolution of the first image frame and α represents the size parameter of the central region, where 0< α <0.5.

Alternatively, the present application does not limit the size of the central region, provided that the central region is represented by the above manner, that is, the present application does not limit α.

Optionally, α may be negotiated between the cloud server and the terminal device, may be predefined, may be specified by the cloud server, or may be specified by the terminal device, which is not limited in this application.

Optionally, assuming that the cloud server is the first image area determined by the area detection module, the area detection module may output the number and coordinates of the first image area after determining the first image area.

Alternatively, the coordinates of the first image region may include: coordinates of an upper left corner pixel point and coordinates of a lower right corner pixel point of the first image region, or includes: the coordinates of the upper right corner pixel point and the lower left corner pixel point of the first image region may further include at least three coordinates of: coordinates of an upper left corner pixel point, coordinates of an upper right corner pixel point, and coordinates of a lower right corner pixel point of the first image region. In summary, the present application does not limit the coordinates of the first image area as long as it can uniquely determine the first image area.

Illustratively, it is assumed that the region detection module determines n first image regions, and the coordinates of each first image region include: the output result of the region detection module is that: n, x_1, y_1, x_2, y_2, x_3, y_3, x_4, y_4, …, x_i, y_i, x_i+1, y_i+1, …, x_2n-1, y_2n-1, x_2n, y_2n, where n is the number of first image areas, (x_i, y_i), (x_i+1, y_i+1) respectively represent coordinates of an upper left corner pixel point and coordinates of a lower right corner pixel point of a certain first image area.

Optionally, assuming that the cloud server is the first image area determined by the area detection module, the area detection module may determine the second image area according to the first image area after determining the first image area, and may output the number and coordinates of the second image area.

Alternatively, the coordinates of the second image region may include: coordinates of an upper left corner pixel point and coordinates of a lower right corner pixel point of the second image area, or includes: the coordinates of the upper right corner pixel point and the lower left corner pixel point of the second image area may further include at least three coordinates of: coordinates of an upper left corner pixel point, coordinates of an upper right corner pixel point, and coordinates of a lower right corner pixel point of the second image region. In summary, the present application does not limit the coordinates of the second image area as long as it can uniquely determine the second image area.

Illustratively, it is assumed that the region detection module determines n second image regions, and the coordinates of each second image region include: the output result of the region detection module is that: n, x_1, y_1, x_2, y_2, x_3, y_3, x_4, y_4, …, x_i, y_i, x_i+1, y_i+1, …, x_2n-1, y_2n-1, x_2n, y_2n, where n is the number of second image areas, (x_i, y_i), (x_i+1, y_i+1) respectively represent coordinates of an upper left corner pixel point and coordinates of a lower right corner pixel point of a certain second image area.

Optionally, assuming that the cloud server is the first image area determined by the area detection module, the area detection module may determine the second image area according to the first image area after determining the first image area, and may output the number and coordinates of the first image area and the number and coordinates of the second image area.

It should be understood that, regarding the coordinate representation manners of the first image area and the second image area, and the output examples of the area detection module may refer to the above, which is not repeated herein.

It should be understood that, for the first image frame, the cloud server may perform image quality enhancement processing, such as sharpening processing, on the first image area in the image frame, and perform low-pass filtering processing on the second image area to obtain the second image frame, that is, if the first image frame is the second image area except the first image area, in this case, the cloud server performs image quality enhancement processing and low-pass filtering processing on the first image area and the second image area in the first image frame, respectively, so as to obtain the second image frame. If the first image frame includes other image areas in addition to the first image area and the second image area, the cloud server performs image quality enhancement processing and low-pass filtering processing on the first image area and the second image area in the first image frame, respectively, and does not perform any processing on the other image frames, so that the second image frame can be obtained.

It should be understood that the encoding mode adopted by the encoder in the cloud server is not limited in the application. The analysis mode of the code rate by the terminal equipment is not limited.

As described above, for the image area which is subjected to the low-pass filtering processing and is encoded and decoded in the first image frame, that is, the third image area, in order to improve the image quality of the image area, the terminal device needs to perform the image quality enhancement processing, such as the sharpening processing, on the third image area, and based on this, the terminal device needs to identify the third image area in the third image frame first, which may be implemented, but is not limited to:

in one implementation, the terminal device may employ a detection algorithm to detect the third image area, for example: because the code rate corresponding to the third image area is lower, the terminal device can determine the image area with the code rate lower than the preset code rate in the third image frame as the third image area. The present application does not limit the detection algorithm.

Optionally, the preset code rate may be negotiated between the cloud server and the terminal device, may be predefined, may be specified by the cloud server, or may be specified by the terminal device, which is not limited in this application.

In the second implementation manner, the cloud server may send information of the third image area to the terminal device, so that the terminal device identifies the third image area according to the information of the third area information.

Optionally, the information of the third image area may include: identification of the third image region. Alternatively, the information of the third image area includes: coordinates of the third image area. Alternatively, the information of the third image area includes: the number of areas of the third image area and the coordinates of the third image area.

It should be appreciated that the identification of the third image area is used to uniquely identify the third image area, which may be, but is not limited to, an index to which the third image area corresponds.

Alternatively, the coordinates of the third image region may include: coordinates of an upper left corner pixel point and coordinates of a lower right corner pixel point of the third image area, or includes: the coordinates of the upper right corner pixel point and the lower left corner pixel point of the third image area may further include at least three coordinates of: coordinates of an upper left corner pixel point, coordinates of an upper right corner pixel point, and coordinates of a lower right corner pixel point of the third image region. In summary, the coordinates of the third image area are not limited in this application as long as it can uniquely determine the third image area.

Optionally, after determining the third image area, the terminal device may detect whether the third image area is reasonable, which may include the following cases:

case one: assume that the terminal device acquires information of a third image area, and the information of the third image area includes: the terminal device detects whether the coordinates of the third image area are reasonable or not, i.e. whether the coordinates of the third image area are reasonable or not.

And a second case: assume that the terminal device acquires information of a third image area, and the information of the third image area includes: the terminal device detects whether the third image area is reasonable or not, that is, whether the coordinates of the third image area are reasonable or not and whether the number of the areas of the third image area is reasonable or not.

It should be noted that the information of the third image area may be used to determine the third image area and/or detect whether the third image area is reasonable.

Illustratively, it is assumed that the information of the third image region includes: the identification of the third image area, which cannot then be used to detect whether the third image area is reasonable. Alternatively, even if the information of the third image area includes: the coordinates of the third image area, or the information of the third image area, includes: the number of areas of the third image area and coordinates of the third image area, but the information of the third image area may not be used to detect whether the third image area is reasonable. Alternatively, it is assumed that the information of the third image area includes: the coordinates of the third image area, or the information of the third image area, includes: the number of areas of the third image area and coordinates of the third image area, the information of the third image area may be used to detect whether the third image area is reasonable or not, but the terminal device does not determine the third image area from the information of the third image area. Alternatively, it is assumed that the information of the third image area includes: the coordinates of the third image area, or the information of the third image area, includes: the number of areas of the third image area and coordinates of the third image area, the information of the third image area may be used to detect whether the third image area is reasonable or not, and may also be used to determine the third image area.

Description is made for case one:

it should be understood that whether the terminal device detects the coordinates of the third image area reasonably refers to whether the terminal device detects that the coordinates of the third image area fall within the third image frame range.

Description is made for case two:

fig. 4 is a flowchart of a method for determining whether a third image area is reasonable according to an embodiment of the present application, where the method may be performed by a terminal device, which may be, but is not limited to, the player gaming terminal 220 in fig. 2, and as shown in fig. 4, the method includes:

s410: determining the coordinate number of the third image area according to the area number of the third image area;

s420: and determining whether the third image area is reasonable according to whether the coordinate number of the third image area is consistent with the coordinate number actually included in the information of the third image area and whether the coordinate of the third image area falls into the third image frame range.

Optionally, if the number of coordinates of the third image area is consistent with the number of coordinates actually included in the information of the third image area, and the coordinates of the third image area fall within the third image frame range, determining that the third image area is reasonable; if the number of coordinates of the third image area is inconsistent with the number of coordinates actually included in the information of the third image area, or at least one coordinate of the third image area does not fall into the third image frame range, determining that the third image area is unreasonable.

As described above, there are various coordinate representation modes of the third image area, and if the coordinate representation method of the third image area is fixed, the number of coordinates included in the third image area can be determined, for example: the coordinates of the third image region are assumed to include: the coordinates of the upper left corner pixel point and the lower right corner pixel point of the third image area are two-dimensional coordinates, and thus, it can be known that the coordinates of the third image area include: coordinates of the upper left corner pixel point and coordinates of the lower right corner pixel point, namely 4 data. That is, the number of coordinates of each third image area is 4. And assuming that x_1 third image areas exist, the number of coordinates of x_1 third image areas should be 4 x_1.

In addition, the terminal device may obtain the number of coordinates and the coordinates of the third image areas from the cloud server, and assume that x_1 third image areas exist, and the coordinates of each third image area include: the terminal device may obtain the following data sequence if the coordinates of the upper left corner pixel point and the coordinates of the lower right corner pixel point of the third image area are: x_1, x_2, x_3 … … x_n, where x_1 is the number of third image areas, and x_2, x_3 … … x_n represent the coordinates of these third image areas, respectively, for example: (x_2, x_3) is the coordinates of the upper left corner pixel point of a certain third image area, and (x_4, x_5) is the coordinates of the lower right corner pixel point of the third image area. That is, as can be seen from the data sequence, the number of coordinates of the third image region should be n-1. If 4xx1=n-1, the number of coordinates representing the third image area is consistent with the number of coordinates actually included in the information of the third image area, otherwise, the number of coordinates representing the third image area is inconsistent with the number of coordinates actually included in the information of the third image area.

It should be understood that, since the third image frame has a certain resolution, for example, w×h, W represents the width of the third image frame, that is, the number of pixels included in the width direction, and H represents the height of the third image frame, that is, the number of pixels included in the height direction. Based on this, it is determined whether or not the coordinates of the third image area fall within the third image frame range, that is, the coordinates of the third image area need to be within the width range and the height range of the third image frame.

Illustratively, fig. 5 is a flowchart of a method for determining whether a third image area is reasonable according to another embodiment of the present application, where the method may be performed by a terminal device, which may be, but is not limited to, the player gaming terminal 220 in fig. 2, and as shown in fig. 5, the method includes:

s510: information of the third image area is acquired, which is x_1, x_2, x_3 … … x_n.

Where x_1 is the number of areas of the third image area, and x_2, x_ … … x_n respectively represent coordinates of the third image area, for example: (x_2, x_3) is the coordinates of the upper left corner pixel point of a certain third image area, and (x_4, x_5) is the coordinates of the lower right corner pixel point of the third image area.

S520: judging whether 4 x_1 is equal to n-1, if 4 x_1 is equal to n-1, executing S530, otherwise, executing S570;

that is, S520 is configured to determine whether the number of coordinates of the third image area is identical to the number of coordinates actually included in the information of the third image area, if 4×x_1 is equal to n-1, the number of coordinates of the third image area is identical to the number of coordinates actually included in the information of the third image area, and if 4×x_1 is not equal to n-1, the number of coordinates of the third image area is not identical to the number of coordinates actually included in the information of the third image area.

S530: let i=1;

s540: judging whether i+4 is less than or equal to n, if i+4 is less than or equal to n, executing S550, otherwise, executing S580;

s550: judging whether the following conditions are all satisfied:

0≤x_(i+1)≤W-1；

0≤x_(i+3)≤W-1；

0≤x_(i+2)≤H-1；

0≤x_(i+4)≤H-1；

if the above conditions are satisfied, S560 is executed, otherwise S570 is executed;

where W represents the width of the third image frame, i.e., the number of pixels included in the width direction, and H represents the height of the third image frame, i.e., the number of pixels included in the height direction.

It should be understood that, here, assuming that the upper left corner pixel of the third image frame is the origin of coordinates, and that the positive x-axis direction is right and the positive y-axis direction is downward in the coordinates, the judgment of the above three conditions is actually to judge whether the coordinates of the third image region are within the range of the third image frame.

S560: let i=i+4, and continue to execute S540;

s570: determining that the third image area is unreasonable;

s580: the third image area is determined to be reasonable.

It should be noted that, determining whether the third image area is reasonable may be performed before S370.

It should be understood that, for the third image frame, the terminal device may perform image quality enhancement processing on the third image area in the image frame, and perform no processing on other image areas, to obtain the fourth image frame.

It will be appreciated that the terminal device, after deriving the fourth image frame, may perform image rendering to display the fourth image frame. The rendering mode adopted by the terminal equipment is not limited.

In summary, the present application provides an image processing method, where a high-frequency component is introduced into an image, a low-pass filtering process may be used for some regions in the image, and since some high-frequency components may be removed by the low-pass filtering process, the removed high-frequency components and the high-frequency components introduced into the image may be offset to some extent, and a higher code rate is not required for the region of the low-pass filtering process, so on one hand, the coding difficulty of the encoder may be reduced, and on the other hand, for the region of the low-pass filtering process, the terminal device may perform an image quality enhancement process, such as a sharpening process, on the other hand, so that the image quality of the whole image may be improved.

Further, before the image quality enhancement processing is performed on the third image area, the terminal device may detect whether the third image area is reasonable, and perform the image quality enhancement processing on the third image area only if the third image area is reasonable, and not perform the image quality enhancement processing on the third image area if the third image area is not reasonable, so that the reliability of the image processing may be improved.

Alternatively, since the terminal device is required to perform image quality enhancement processing on the third image area, the video rendering capability of the terminal device needs to satisfy the image quality enhancement processing requirement of the third image area.

Optionally, the cloud server may send a video rendering capability request to the terminal device; receiving a video rendering capability response of the terminal device, wherein the video rendering capability response comprises: video rendering capabilities of the terminal device.

Optionally, as shown in fig. 6, the cloud server may send a video rendering capability request to the terminal device through a client installed in the terminal device, and the terminal device may also return a video rendering capability response to the cloud server through the client. Wherein, in a cloud game scenario, the client may be a cloud game client.

Optionally, the video rendering capability request is used to request acquisition of video rendering capabilities of the terminal device.

Optionally, the video rendering capability request includes at least one of, but is not limited to: protocol version number, video resolution, video frame rate, and type of rendering algorithm queried.

Optionally, the protocol version number refers to a lowest protocol version supported by the cloud server, which may be a rendering protocol.

Alternatively, the video resolution, i.e., video size, may be the resolution of the video source to be rendered, such as 1080p.

Alternatively, the video frame rate may be the frame rate of the video source to be rendered, such as 60fps.

Alternatively, the type of rendering algorithm for the query may be at least one of, but not limited to: sharpening processing algorithms, noise reduction processing algorithms, blurring processing algorithms, video high dynamic range imaging (High Dynamic Range Imaging, HDR) enhancement capability algorithms, and the like.

Alternatively, different video resolutions may be defined by enumeration, as shown in table 1:

TABLE 1

Video resolution	Enumeration definition
		360p	0x1
576p	0x2
		720p	0x4
1080p	0x8
		2k	0x10
4k	0x20

Alternatively, the different video frame rates may be defined by enumeration, as shown in table 2:

TABLE 2

Video frame rate	Enumeration definition
		30fps	0x1
40fps	0x2
		50fps	0x4
60fps	0x8
		90fps	0x10
120fps	0x20

Alternatively, different rendering algorithms may be defined by enumeration, as shown in table 3:

TABLE 3 Table 3

/>

Illustratively, the code implementation of the video rendering capability request may be as follows:

{

"render_ability":{

"version":"1.0",

"resolution":"8",

"framerate":"8",

"type":"1,2"

}

for an explanation of the respective data structures in the code, refer to table 4 below, which is not repeated herein.

The data structure of the video rendering capability of the terminal device may be as shown in table 4:

TABLE 4 Table 4

/>

Optionally, the video rendering capability response may include at least one of, but is not limited to: whether the type of the rendering algorithm to be queried by the cloud server is successfully queried, a protocol version number supported by the terminal equipment, video rendering capability of the terminal equipment and the like.

Optionally, if the query of the rendering algorithm type to be queried for the cloud server is successful, the identifier of whether the query of the rendering algorithm type to be queried for the cloud server is successful may be represented by 0, and if the query of the rendering algorithm type to be queried for the cloud server is unsuccessful, the identifier of whether the query of the rendering algorithm type to be queried for the cloud server is successful may be represented by an error code, such as 001.

Optionally, the protocol version number refers to the lowest protocol version supported by the terminal device, which may be a rendering protocol.

Optionally, the video rendering capability of the terminal device includes at least one of, but is not limited to: the type of rendering algorithm supported by the terminal device and the performance of the rendering algorithm.

Optionally, the performance of the rendering algorithm includes at least one of, but is not limited to: the video size, frame rate, and latency that the algorithm can handle.

Illustratively, the code implementation of the video rendering capability response may be as follows:

{

"render_ability":{

"state":"0",

"version":"1.0",

"renders":"2"

},

"render1":{

"type":"1",

"performances":"1",

"performance1":"8,8,10"

},

"render2":{

"type":"2",

"performances":"1",

"performance1":"8,8,5"

}

illustratively, a code implementation of a video rendering capability response (supporting only partial rendering capabilities) may be as follows:

{

"render_ability":{

"state":"0",

"version":"1.0",

"renders":"1"

},

"render1":{

"type":"2",

"performances":"1",

"performance1":"8,8,5"

}

illustratively, the code implementation of the video rendering capability response (without supporting rendering capability) may be as follows:

{

"render_ability":{

"state":"0",

"version":"1.0",

"renders":"0"

}

illustratively, the code implementation of the video rendering capability response (protocol request failure) may be as follows:

{

"render_ability":{

"state":"-1",

"version":"0.9"

}

it should be understood that, for an explanation of each data structure in these codes, reference may be made to table 4, which is not repeated herein.

Fig. 7 is a schematic diagram of an image processing apparatus according to an embodiment of the present application, as shown in fig. 7, where the apparatus includes: the device comprises a determining module 710, a processing module 720, an encoding module 730 and an output module 740, wherein the determining module 710 is configured to determine a first image area and a second image area in a first image frame, the first image area is an image area to be introduced with high frequency components, and the second image area is an image area to be filtered; the processing module 720 is configured to perform image quality enhancement processing on a first image area in the first image frame, and perform low-pass filtering processing on a second image area to obtain a second image frame; the encoding module 730 is configured to encode the second image frame to obtain a code stream; the output module 740 is configured to output the code stream to a terminal device.

Optionally, the apparatus further comprises: a sending module 750, configured to perform image quality enhancement processing on a first image area in the first image frame by using the processing module 720, perform low-pass filtering processing on a second image area, and send information of a third image area to the terminal device after obtaining the second image frame; the third image area is an image area after the second image area is subjected to low-pass filtering processing and encoding and decoding.

Optionally, the information of the third image area includes: the number of areas of the third image area and the coordinates of the third image area.

Optionally, the apparatus further comprises: a receiving module 760, wherein the sending module 750 is further configured to send a video rendering capability request to a terminal device; the receiving module 760 is configured to receive a video rendering capability response of the terminal device, where the video rendering capability response includes: the video rendering capability of the terminal device, correspondingly, the sending module 750 is specifically configured to: and when the video rendering capability of the terminal equipment meets the image quality enhancement processing requirement of the third image area, transmitting the information of the third image area to the terminal equipment.

Optionally, the determining module 710 is specifically configured to: determining a first image area according to a preset rule; determining the area except the first image area in the first image frame as a second image area, wherein the preset rule is that at least one of the following areas in the first image frame is the first image area by default: upper left corner region, lower right corner region, upper right corner region, lower right corner region, and center region.

Optionally, the first image frame is acquired from a video sequence uploaded in real time, the video sequence being a virtual game video sequence; the first image frame includes an image frame header indicating coordinates of the first image region and the second image region.

It should be understood that apparatus embodiments and method embodiments may correspond with each other and that similar descriptions may refer to the method embodiments. To avoid repetition, no further description is provided here. Specifically, the apparatus shown in fig. 7 may execute the method embodiment corresponding to the cloud server, and the foregoing and other operations and/or functions of each module in the apparatus are respectively for implementing the corresponding flow in each method corresponding to the cloud server, which is not described herein for brevity.

The apparatus of the embodiments of the present application are described above in terms of functional modules in conjunction with the accompanying drawings. It should be understood that the functional module may be implemented in hardware, or may be implemented by instructions in software, or may be implemented by a combination of hardware and software modules. Specifically, each step of the method embodiments in the embodiments of the present application may be implemented by an integrated logic circuit of hardware in a processor and/or an instruction in software form, and the steps of the method disclosed in connection with the embodiments of the present application may be directly implemented as a hardware decoding processor or implemented by a combination of hardware and software modules in the decoding processor. Alternatively, the software modules may be located in a well-established storage medium in the art such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, and the like. The storage medium is located in a memory, and the processor reads information in the memory, and in combination with hardware, performs the steps in the above method embodiments.

Fig. 8 is a schematic diagram of another image processing apparatus according to an embodiment of the present application, as shown in fig. 8, including: the device comprises a first acquisition module 810, an analysis module 820, a first determination module 830, a processing module 840 and a display module 850, wherein the first acquisition module 810 is configured to acquire a code stream; the parsing module 820 is configured to parse the code stream to obtain a third image frame; the first determining module 830 is configured to determine a third image area in a third image frame; the processing module 840 is configured to perform image quality enhancement processing on a third image area in the third image frame to obtain a fourth image frame; the display module 850 is used for displaying a fourth image frame; the second image frame is an image frame obtained by performing image quality enhancement processing on a first image area in the first image frame and performing low-pass filtering processing on a second image area in the first image frame, and the third image area is an image area obtained by performing low-pass filtering processing on the second image area and performing encoding and decoding on the second image area.

Optionally, the apparatus further comprises: the second obtaining module 860 is configured to obtain information of the third image area.

Optionally, the first determining module 830 is specifically configured to: a third image region in the third image frame is determined from the information of the third image region.

Optionally, the apparatus further includes a second determining module 870, configured to determine, before the processing module 840 performs image quality enhancement processing on the third image area in the third image frame to obtain the fourth image frame, whether the third image area is reasonable according to the information of the third image area.

Optionally, the information of the third image area includes: the number of areas of the third image area and coordinates of the third image area; accordingly, the second determining module 870 is specifically configured to: determining the coordinate number of the third image area according to the area number of the third image area; and determining whether the third image area is reasonable according to whether the coordinate number of the third image area is consistent with the coordinate number actually included in the information of the third image area and whether the coordinate of the third image area falls into the third image frame range.

Optionally, the second determining module 870 is specifically configured to: if the coordinate number of the third image area is consistent with the coordinate number actually included by the information of the third image area, and the coordinate of the third image area falls into the third image frame range, determining that the third image area is reasonable; if the number of coordinates of the third image area is inconsistent with the number of coordinates actually included in the information of the third image area, or at least one coordinate of the third image area does not fall into the third image frame range, determining that the third image area is unreasonable.

Optionally, the apparatus further comprises: the communication module 880 is used for receiving the video rendering capability request sent by the cloud server; sending a video rendering capability response to the cloud server, the video rendering capability response comprising: video rendering capabilities of the terminal device.

It should be understood that apparatus embodiments and method embodiments may correspond with each other and that similar descriptions may refer to the method embodiments. To avoid repetition, no further description is provided here. Specifically, the apparatus shown in fig. 8 may execute the method embodiment corresponding to the above terminal device, and the foregoing and other operations and/or functions of each module in the apparatus are respectively for implementing the corresponding flow in each method corresponding to the above terminal device, which is not described herein for brevity.

Fig. 9 is a schematic block diagram of an electronic device 900 provided by an embodiment of the present application. The electronic device may be the cloud server or the terminal device.

As shown in fig. 9, the electronic device 900 may include:

a memory 910 and a processor 920, the memory 910 being configured to store a computer program and to transfer the program code to the processor 920. In other words, the processor 920 may call and run a computer program from the memory 910 to implement the methods in the embodiments of the present application.

For example, the processor 920 may be configured to perform the above-described method embodiments according to instructions in the computer program.

In some embodiments of the present application, the processor 920 may include, but is not limited to:

a general purpose processor, digital signal processor (Digital Signal Processor, DSP), application specific integrated circuit (Application Specific Integrated Circuit, ASIC), field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like.

In some embodiments of the present application, the memory 910 includes, but is not limited to:

volatile memory and/or nonvolatile memory. The nonvolatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable EPROM (EEPROM), or a flash Memory. The volatile memory may be random access memory (Random Access Memory, RAM) which acts as an external cache. By way of example, and not limitation, many forms of RAM are available, such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (Double Data Rate SDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), and Direct memory bus RAM (DR RAM).

In some embodiments of the present application, the computer program may be partitioned into one or more modules that are stored in the memory 910 and executed by the processor 920 to perform the methods provided herein. The one or more modules may be a series of computer program instruction segments capable of performing the specified functions, which are used to describe the execution of the computer program in the electronic device.

As shown in fig. 9, the electronic device may further include:

a transceiver 930, the transceiver 930 being connectable to the processor 920 or the memory 910.

The processor 920 may control the transceiver 930 to communicate with other devices, and in particular, may send information or data to other devices or receive information or data sent by other devices. Transceiver 930 may include a transmitter and a receiver. Transceiver 930 may further include antennas, the number of which may be one or more.

It will be appreciated that the various components in the electronic device are connected by a bus system that includes, in addition to a data bus, a power bus, a control bus, and a status signal bus.

The present application also provides a computer storage medium having stored thereon a computer program which, when executed by a computer, enables the computer to perform the method of the above-described method embodiments. Alternatively, embodiments of the present application also provide a computer program product comprising instructions which, when executed by a computer, cause the computer to perform the method of the method embodiments described above.

When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces, in whole or in part, a flow or function consistent with embodiments of the present application. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by a wired (e.g., coaxial cable, fiber optic, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a digital video disc (digital video disc, DVD)), or a semiconductor medium (e.g., a Solid State Disk (SSD)), or the like.

Those of ordinary skill in the art will appreciate that the various illustrative modules and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, and for example, the division of the modules is merely a logical function division, and there may be additional divisions when actually implemented, for example, multiple modules or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or modules, which may be in electrical, mechanical, or other forms.

The modules illustrated as separate components may or may not be physically separate, and components shown as modules may or may not be physical modules, i.e., may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. For example, functional modules in the embodiments of the present application may be integrated into one processing module, or each module may exist alone physically, or two or more modules may be integrated into one module.

The foregoing is merely a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think about changes or substitutions within the technical scope of the present application, and the changes and substitutions are intended to be covered in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An image processing method, wherein the method is applied to a cloud server, and the method comprises:

determining a first image area and a second image area in a first image frame, wherein the first image area is an image area to be introduced with high-frequency components, and the second image area is an image area to be filtered;

Performing image quality enhancement processing on the first image area in the first image frame, and performing low-pass filtering processing on the second image area to obtain a second image frame;

encoding the second image frame to obtain a code stream;

and outputting the code stream to terminal equipment.

2. The method according to claim 1, wherein the performing image quality enhancement processing on the first image area in the first image frame and performing low-pass filtering processing on the second image area to obtain a second image frame further comprises:

transmitting information of a third image area to the terminal equipment;

the third image area is an image area subjected to low-pass filtering processing and encoding and decoding on the second image area.

3. The method of claim 2, wherein the information of the third image region comprises: the number of areas of the third image area and coordinates of the third image area.

4. A method according to claim 2 or 3, characterized in that before said sending information of the third image area to the terminal device, further comprises:

sending a video rendering capability request to the terminal equipment;

Receiving a video rendering capability response of the terminal device, wherein the video rendering capability response comprises: video rendering capabilities of the terminal device;

the sending the information of the third image area to the terminal device includes:

and when the video rendering capability of the terminal equipment meets the image quality enhancement processing requirement of the third image area, transmitting the information of the third image area to the terminal equipment.

5. A method according to any of claims 1-3, wherein said determining a first image area and a second image area in a first image frame comprises:

determining the first image area according to a preset rule;

determining a region of the first image frame other than the first image region as the second image region;

wherein the preset rule defaults to at least one of the following areas in the first image frame being the first image area: upper left corner region, lower right corner region, upper right corner region, lower right corner region, and center region.

6. A method according to any of claims 1-3, wherein the first image frame is obtained from a video sequence uploaded in real time, the video sequence being a virtual game video sequence;

The first image frame includes an image frame header indicating coordinates of the first image region and the second image region.

7. An image processing method, wherein the method is applied to a terminal device, the method comprising:

acquiring a code stream;

analyzing the code stream to obtain a third image frame;

determining a third image region in the third image frame;

performing image quality enhancement processing on the third image area in the third image frame to obtain a fourth image frame;

displaying the fourth image frame;

the third image frame is an image frame obtained by performing image quality enhancement processing on a first image area in a first image frame and performing low-pass filtering processing on a second image area in the first image frame, and is obtained by performing encoding and decoding on the second image area, and the third image area is an image area obtained by performing low-pass filtering processing on the second image area and performing encoding and decoding on the second image area.

8. The method as recited in claim 7, further comprising:

and acquiring information of the third image area.

9. The method of claim 8, wherein the determining a third image region in the third image frame comprises:

And determining the third image area in the third image frame according to the information of the third image area.

10. The method of claim 8, wherein the performing image quality enhancement processing on the third image area in the third image frame, before obtaining a fourth image frame, further comprises:

and determining whether the third image area is reasonable according to the information of the third image area.

11. The method of claim 10, wherein the information of the third image region comprises: the number of areas of the third image area and coordinates of the third image area;

the determining whether the third image area is reasonable according to the information of the third image area includes:

determining the coordinate number of the third image area according to the area number of the third image area;

and determining whether the third image area is reasonable according to whether the coordinate number of the third image area is consistent with the coordinate number actually included by the information of the third image area and whether the coordinate of the third image area falls into the third image frame range.

12. The method of claim 11, wherein determining whether the third image area is reasonable based on whether the number of coordinates of the third image area matches the number of coordinates actually included in the information of the third image area, and whether the coordinates of the third image area fall within the third image frame range, comprises:

If the number of coordinates of the third image area is consistent with the number of coordinates actually included in the information of the third image area, and the coordinates of the third image area fall within the third image frame range, determining that the third image area is reasonable;

and if the number of coordinates of the third image area is inconsistent with the number of coordinates actually included in the information of the third image area, or at least one coordinate of the third image area does not fall into the third image frame range, determining that the third image area is unreasonable.

13. The method according to any one of claims 7-12, further comprising:

receiving a video rendering capability request sent by a cloud server;

transmitting a video rendering capability response to the cloud server, the video rendering capability response comprising: video rendering capabilities of the terminal device.

14. The method according to any of claims 7-12, wherein the first image frame is obtained from a video sequence uploaded in real time, the video sequence being a virtual game video sequence;

15. An image processing apparatus, comprising:

the device comprises a determining module, a filtering module and a filtering module, wherein the determining module is used for determining a first image area and a second image area in a first image frame, the first image area is an image area to be introduced with high-frequency components, and the second image area is an image area to be filtered;

the processing module is used for carrying out image quality enhancement processing on the first image area in the first image frame and carrying out low-pass filtering processing on the second image area to obtain a second image frame;

the encoding module is used for encoding the second image frame to obtain a code stream;

and the output module is used for outputting the code stream to terminal equipment.

16. An image processing apparatus, comprising:

the first acquisition module is used for acquiring the code stream;

the analysis module is used for analyzing the code stream to obtain a third image frame;

a first determining module configured to determine a third image area in the third image frame;

the processing module is used for carrying out image quality enhancement processing on the third image area in the third image frame to obtain a fourth image frame;

a display module for displaying the fourth image frame;

17. An electronic device, comprising:

a processor and a memory for storing a computer program, the processor being for invoking and running the computer program stored in the memory to perform the method of any of claims 1 to 14.

18. A computer readable storage medium storing a computer program for causing a computer to perform the method of any one of claims 1 to 14.