CN116726488A - Data processing method, device, equipment and readable storage medium - Google Patents

Abstract

The embodiment of the application discloses a data processing method, a device, equipment and a readable storage medium, and relates to the technical field of cloud, wherein the method comprises the following steps: receiving first image data sent by a server, and decoding the first image data to obtain a first image frame corresponding to the first image data; detecting whether the first image data is image data of a target type; and if the first image data is the image data of the target type, sending a refreshing instruction to the operating system, so that the operating system refreshes a display screen of the terminal equipment when receiving the refreshing instruction, and displaying the first image frame on the display screen, wherein the refreshing instruction carries the first image frame. By adopting the embodiment of the application, the image display delay can be reduced, and the user experience can be improved.

Description

Data processing method, device, equipment and readable storage medium

Technical Field

The present application relates to the field of cloud technologies, and in particular, to a data processing method, apparatus, device, and readable storage medium.

Background

When the server and the terminal equipment perform data interaction, the server has a data transmission period, the terminal equipment also has a self refresh period, and when the server periodically transmits image data to the terminal equipment, the terminal equipment processes the image data and needs to refresh the display screen according to the self refresh period of the terminal equipment and then display image frames corresponding to the image data on the display screen.

After processing the image data sent by the server, the current terminal device refreshes the display screen according to the self refresh period, and displays the image frames corresponding to the image data on the display screen. Since the period in which the server transmits the image data and the refresh period of the terminal device itself are generally not synchronized, a delay may occur in the image frames displayed on the display screen of the terminal device. It can be seen that how to reduce the image display delay is a problem to be solved.

Disclosure of Invention

The embodiment of the application provides a data processing method, a device, equipment and a readable storage medium, which can reduce image display delay and improve user experience.

In a first aspect, the present application provides a data processing method, including:

receiving first image data sent by a server, and decoding the first image data to obtain a first image frame corresponding to the first image data;

detecting whether the first image data is image data of a target type;

and if the first image data is the image data of the target type, sending a refreshing instruction to the operating system, so that the operating system refreshes a display screen of the terminal equipment when receiving the refreshing instruction, and the first image frame is displayed on the display screen, wherein the refreshing instruction carries the first image frame.

In a second aspect, the present application provides a data processing apparatus comprising:

the data receiving unit is used for receiving the first image data sent by the server and decoding the first image data to obtain a first image frame corresponding to the first image data;

a data detection unit configured to detect whether the first image data is image data of a target type;

and the image display unit is used for sending a refreshing instruction to the operating system if the first image data is the image data of the target type, so that the operating system refreshes a display screen of the terminal device when receiving the refreshing instruction, and the first image frame is displayed on the display screen, and the refreshing instruction carries the first image frame.

In a third aspect, the present application provides a computer device comprising: a processor, a memory, a network interface;

the processor is connected to a memory for providing data communication functions, a network interface for storing a computer program, and for calling the computer program to cause a computer device comprising the processor to execute the data processing method.

In a fourth aspect, the present application provides a computer readable storage medium having stored therein a computer program adapted to be loaded and executed by a processor to cause a computer device having the processor to perform the above-described data processing method.

In a fifth aspect, the present application provides a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the data processing method provided in the various alternatives in the first aspect of the application.

In the embodiment of the application, when first image data sent by a server is received, decoding processing is carried out on the first image data to obtain a first image frame corresponding to the first image data; further, whether the first image data is the image data of the target type is detected, if the first image data is the image data of the target type, a refreshing instruction is sent to the operating system, so that the operating system refreshes a display screen of the terminal device, and the first image frame is displayed on the display screen. Because the first image data is the image data of the target type, when the first image frame is acquired, a refreshing instruction can be sent to the operating system so as to immediately refresh the display screen of the terminal device, the first image frame is displayed on the display screen, refreshing is not required to be carried out after the refreshing period of the terminal device comes, the image can be rapidly displayed on the display screen, the image display delay is reduced, and therefore user experience is improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic diagram of image data transmission according to an embodiment of the present application;

FIG. 2 is a schematic diagram of an image refresh according to an embodiment of the present application;

FIG. 3 is a network architecture diagram of a data processing system provided in an embodiment of the present application;

fig. 4 is an application scenario schematic diagram of a data processing method according to an embodiment of the present application;

FIG. 5 is a schematic flow chart of a data processing method according to an embodiment of the present application;

FIG. 6 is a schematic diagram of another image refresh provided by an embodiment of the present application;

FIG. 7 is a schematic diagram of image data transmission according to an embodiment of the present application;

FIG. 8 is a flowchart of another data processing method according to an embodiment of the present application;

FIG. 9 is a flowchart of another data processing method according to an embodiment of the present application;

FIG. 10 is a schematic diagram of still another image refresh provided by an embodiment of the present application;

FIG. 11 is a schematic diagram of a data processing apparatus according to an embodiment of the present application;

fig. 12 is a schematic diagram of a composition structure of a computer device according to an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

Cloud computing (clouding) is a computing model that distributes computing tasks across a large pool of computers, enabling various application systems to acquire computing power, storage space, and information services as needed. The network that provides the resources is referred to as the "cloud". Resources in the cloud are infinitely expandable in the sense of users, and can be acquired at any time, used as needed, expanded at any time and paid for use as needed. The cloud application is a subset of cloud computing and is the embodiment of cloud computing technology in an application layer, and the working principle of the cloud application is a novel application which changes the use mode of local installation and local operation of traditional software into a service which is used as a service, and the cloud application is connected with and controls a remote server cluster through the Internet or a local area network to complete business logic or operation tasks. The cloud application has the advantages that the application program of the cloud application runs on the cloud, the application program is not required to be installed locally, operation is not required, the storage space consumption and the power consumption expenditure of the terminal are saved, and therefore the running speed of the terminal can be improved. The cloud application refers to an application in which terminal equipment interacts with a server (or called cloud), and the terminal equipment operates to synchronize the cloud, so that the terminal data is backed up by the cloud while occupying local space.

Among them, cloud games have received increasing attention in recent years as a typical one in cloud applications. Cloud gaming (Cloud gaming), which may also be referred to as game on demand, is an online gaming technology based on Cloud computing technology. Cloud gaming technology enables lightweight devices with relatively limited graphics processing and data computing capabilities to run high quality games. In a cloud game scene, the game is not run in a player game terminal, but is run in a cloud server, the cloud server renders the game scene into a video and audio stream, and the video and audio stream is transmitted to the player game terminal through a network. The player game terminal does not need to have strong graphic operation and data processing capability, and only needs to have basic streaming media playing capability and the capability of acquiring player input instructions and sending the player input instructions to the cloud server.

The data related to the user information in the embodiment of the application are all data after the user authorization.

The current cloud application generally comprises that a server periodically sends image data to a terminal device, the terminal device processes the image data to obtain image frames, and the image frames are displayed on a display screen after refreshing the display screen according to a refreshing period of the terminal device. Typical cloud applications today may include medical clouds, cloud security, cloud gaming, cloud security, cloud education, cloud social interactions, and so forth. For example, taking cloud application as a cloud game, after the cloud game is started, the server periodically sends game data to the terminal device, and the terminal device processes the game data after receiving the game data to obtain a game frame, namely a game picture. When the refresh period of the terminal equipment is satisfied, the display screen of the terminal equipment is refreshed to display the game frame on the display screen of the terminal equipment, so that a user can view the game picture through the display screen of the terminal equipment. However, generally, since the period in which the server transmits the game data does not coincide with the refresh period of the terminal device, the server and the terminal device keep the respective refresh periods continuously circulating, and the terminal device such as a mobile phone or a television continuously refreshes the screen at a rate of 60 frames per second. Because the period of the server is inconsistent with the refresh period of the terminal equipment, the terminal equipment can process the game data to obtain the game frame, and then the display screen of the terminal equipment is refreshed and the game frame is displayed when the refresh period of the terminal equipment is required to be met. Because the network itself has fluctuation, network delay may be caused, if the network delay corresponding to a certain frame of game data is larger, the terminal device misses the refresh point (i.e. refresh time) of the terminal device when processing the game data to obtain a game frame, and the game frame needs to be displayed after being refreshed by the display screen when the next refresh period arrives, which may cause delay of the displayed image.

Referring to fig. 1 for describing the current image display mode with reference to fig. 1, fig. 1 is a schematic diagram of image data transmission provided in an embodiment of the present application, as shown in fig. 1, in which an application running on a server 11 has its own screen update period, and the server 11 renders and sends image data according to its own screen update period. When the screen update period of the server 11 is satisfied, the server 11 transmits image data to the terminal device 12 through network transmission, and the terminal device 12 processes the image data to obtain an image frame after receiving the image data. Further, when the refresh period of the terminal device 12 is satisfied, the screen of the display screen of the terminal device 12 is refreshed, and the image frame is displayed on the display screen, through which the user can view the game screen.

As can be seen from fig. 1, the period of updating the picture of the server and the image data received by the terminal device from the network are causal, and the transmission of each frame of image data by the server corresponds to one cycle of receiving the image data by the terminal device, and the period and the server keep strict agreement. The display screen refresh period of the terminal device and the period of receiving the image data by the terminal device are not strictly consistent, that is, the display screen refresh of the terminal device and the screen update period of the server are not consistent, which may cause an increase in the image refresh delay.

Further, referring to fig. 2 for explaining the current image display mode with reference to fig. 2, fig. 2 is a schematic image refreshing diagram provided in an embodiment of the present application, where a server sends image data to a terminal device according to a self-frame update period, that is, the server periodically sends the image data. Due to factors such as network delay fluctuation, the time when the terminal equipment receives the image data and the time when the server transmits the image data are delayed, and the terminal equipment needs to process the image data after receiving the image data to obtain an image frame. And when the refresh period of the terminal equipment is met, the terminal equipment periodically refreshes the display screen and sends the image frames to the display screen for display. If the time of the terminal device for processing the image data to obtain the image frame is not equal to the time point corresponding to the refresh period of the terminal device, the terminal device needs to refresh the display screen when the refresh period arrives, so that extra delay is generated, and the delay of image display is increased.

Further, since the server periodically sends the image data, the terminal device processes the image data to obtain an image frame corresponding to each period, if a certain image frame misses a refresh point corresponding to the image frame, when the next refresh period of the terminal device arrives, no matter the image frame is displayed or the next image frame is displayed, the frame loss, that is, the image frame is lost, so that the user experience is poor.

In view of this, the technical solution of the present application provides a method for reducing image display delay on a terminal device, which can reduce the problem of image display delay caused by inconsistent refresh period of the terminal device and period of sending game data by a server by removing refresh period of the terminal device, thereby improving user experience.

Referring to fig. 3, fig. 3 is a network architecture diagram of a data processing system according to an embodiment of the present application, as shown in fig. 3, a server 31 may perform data interaction with terminal devices 32, and the number of terminal devices 32 may be one or more. If the number of the terminal devices 32 is plural, the image data transmitted from the server 31 to each terminal device is the same, and the image data is processed in the same manner by each terminal device, so that the same image frame can be displayed simultaneously at each terminal device when the image data transmitted from the server 31 is received. Wherein the server 31 may send image data, such as game data, to the terminal device 32, the terminal device 32 may include a client, an operating system and a display screen. In the embodiment of the present application, when receiving the image data sent by the server 31, the display screen of the terminal device may be refreshed according to the initial refresh period of the terminal device 32, and the image frame corresponding to the image data is displayed on the display screen of the terminal device 32. Alternatively, when the image data transmitted from the server 31 is received, the display screen of the terminal device 32 may be refreshed immediately without waiting for the refresh period of the terminal device 32 to come, and the image frame corresponding to the image data may be displayed on the display screen of the terminal device 32. After receiving the image data, the terminal device 32 may process the image data, for example, decode the image data to obtain an image frame. That is, if the terminal device immediately refreshes the display screen of the terminal device after processing the image data, it is not necessary to refresh the display screen after waiting for the refresh period of the terminal device to come, so that the delay of image display can be reduced and the user experience can be improved.

It will be understood that the terminal device mentioned in the embodiments of the present application may be a hardware device for receiving input information from an input device and feeding back information such as sound and image to a user. The terminal device may be an electronic device including, but not limited to, a cell phone, a television, a tablet, a desktop computer, a notebook computer, a palm top computer, a vehicle-mounted device, an augmented Reality/Virtual Reality (AR/VR) device, a head mounted display, a wearable device, a smart speaker, a digital camera, a camera, and other mobile internet devices (mobile internet device, MID) with network access capabilities, etc. The server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, vehicle-road collaboration, a content distribution network (Content Delivery Network, CDN), basic cloud computing services such as big data and an artificial intelligent platform.

Further, referring to fig. 4, fig. 4 is a schematic application scenario diagram of a data processing method according to an embodiment of the present application. As shown in fig. 4, for example, the target application may refer to a game application, the image data may refer to game data, and the image frame may refer to a game frame. In a possible case, when the terminal device 42 receives the first game data sent by the server 41 during the starting process of the game application, the first game data may be decoded to obtain a first game frame, and if the first game data is not the target type of game data, for example, the first game data is interface data of the entering game application, or the initial game data, etc., the display screen of the terminal device 42 is refreshed when the initial refresh period of the terminal device 42 is satisfied, that is, the display screen is refreshed periodically, and the first game frame 43 is displayed on the display screen. In another possible case, for example, after the target function of the game application is started, the terminal device 42 receives the first game data sent by the server 41, and may perform decoding processing on the first game data to obtain a first game frame, if the first game data is image data of the target type, a refresh command is sent to the operating system of the terminal device 42, so that the operating system refreshes the display screen of the terminal device 42 when receiving the refresh command, and displays the second game frame 44 on the display screen. The image data of the target type may be, for example, image data after the target function of the game application is started, such as interaction data of a player during the course of performing a game operation. The display screen can be refreshed and the first game frame can be displayed without waiting for the initial refresh period, so that the received first game data can be updated and displayed immediately, and the image display delay is reduced.

Further, referring to fig. 5, fig. 5 is a flow chart of a data processing method according to an embodiment of the present application; as shown in fig. 5, the data processing method may be applied to a terminal device, and the data processing method includes, but is not limited to, the following steps:

s101, receiving first image data sent by a server, and decoding the first image data to obtain a first image frame corresponding to the first image data.

In the embodiment of the application, the terminal equipment can receive the first image data sent by the server and decode the first image data to obtain the first image frame corresponding to the first image data. Specifically, the cloud application may be run on the server, image data is generated and sent to the terminal device, the terminal device receives the image data sent by the server, and decoding processing may be performed on the image data to obtain an image frame, so as to display the image frame on a display screen of the terminal device. Wherein, the first image data may include image data before the cloud application is started (such as data outside the cloud application); or, starting image data (such as first page data or initial data of the cloud application) in the process of cloud application; alternatively, the first image data may also include image data after the target application is launched (e.g., data during a game operation, such as interaction data between game players). The cloud application may include a target application, which may be a specific type of application or a preset application. The target application may include, but is not limited to, a cloud game, a cloud live, etc., then the first image data may include game data, live data, etc.

S102, detecting whether the first image data is the image data of the target type.

The target type may be preset or determined in other manners. For example, the image data of the target type may be image data after the target function of the target application is started. The target application may include a variety of functions such as a game interface entry function, a game initiation data setup function, a game competition function, and the like. The target function may refer to, for example, a specific function of the target application, for example, the target application is a game application, the target function may refer to a function of a game competition, a game operation and a game competition may be performed by a player in the game application through a game character after the target function is started, and the like. Correspondingly, the image data of the object type may include interaction data between players during the game operation, such as competition data between players, character state change data, game scene data, and the like; game outcome data, such as game win or loss conditions, game score ranking data, and the like, may also be included. Alternatively, the image data of the target type may refer to a continuous large amount of image data, for example, the number of image frames is greater than the target threshold in a preset time, that is, the terminal device needs to display the image data of the target type immediately, or the image data of the target type may include other image data after the start of the target application. Because the image data of the target type is continuous and large in quantity, namely the image data of the target type can change greatly in a short time, the terminal equipment needs to be updated immediately when receiving the image data of the target type, so that a player can accurately see the current game battle situation.

Since the server can send the first image data to the terminal device based on the server's own picture update period, the terminal device also has an initial refresh period. In general, when a frame update period of a server is inconsistent with an initial refresh period of a terminal device, and when the terminal device receives first image data sent by the server and decodes the first image data to obtain a first image frame, and the first image data is image data of a target type, if a display screen of the terminal device is refreshed according to the initial refresh period of the terminal device and then displayed, that is, when the initial refresh period of the terminal device is met, the display screen is refreshed and the first image frame is displayed, additional delay (that is, time waiting for the initial refresh period) is caused, so that an image frame cannot be displayed on the display screen immediately. Therefore, in the embodiment of the application, when the image data of the target type is received, the initial refreshing period of the terminal equipment is removed, and when the terminal equipment decodes the image data of the target type to obtain the first image frame, the display screen is refreshed immediately, and the display screen is not refreshed when the initial refreshing period arrives, so that the extra delay is reduced, the image display delay is further reduced, and the user experience is improved. Alternatively, if the first image data is not the image data of the target type, the first image data may be the image data before the target function of the target application is started, for example, the first image data may include the image data before the cloud application is started or the image data during the cloud application is started. When the first image data is the image data of the target application, the first image data may be the image data in the process of starting the target application. For example, the first image data may include data in a start interface of the target application, such as an identification of the application and a manufacturer of the application, or application initial data after the target application is started, such as game character profile data, game equipment profile data, current account number level data, and so on. The terminal device decodes the first image data to obtain a first image frame corresponding to the first image data, where the first image frame may refer to an image picture, that is, the image data is displayed in a picture form. Alternatively, the first image data may refer to a discontinuous large amount of image data, for example, the number of frames of the first image frame is less than or equal to the target threshold in a preset time, that is, the first image frame is displayed after the display screen of the terminal device is refreshed when the refresh period of the terminal device is satisfied. That is, after the server sends the first image data to the terminal device, the terminal device performs refresh processing on the display screen of the terminal device according to the refresh period of the terminal device, and then displays the first image frame, so that delay in image display is not caused.

Optionally, if the first image data is not the image data of the target type, the display screen is refreshed according to the initial refresh period of the terminal device, so as to display the first image frame on the display screen. The terminal equipment can comprise a client, a display screen and an operating system. Because the terminal equipment has an initial refresh period, when the terminal equipment needs to display the image frames on the display screen, the display screen of the terminal equipment can be refreshed according to the initial refresh period of the terminal equipment so as to display the image frames on the display screen. It can be understood that, before or during the starting of the target application, that is, before the starting of the target function of the target application, if the terminal device receives the image data sent by the server, the first image frame may be displayed on the display screen after the refresh processing is performed on the display screen according to the initial refresh period of the terminal device. Because the image data sent by the server is discontinuous and large amount of image data, such as preset data or data which does not need to be updated immediately, when the terminal equipment receives the data, the display screen can be refreshed according to the initial refresh period of the terminal equipment, and the timeliness of image display is not affected, namely the image display delay is not caused.

And S103, if the first image data is the image data of the target type, sending a refreshing instruction to the operating system so that the operating system refreshes a display screen of the terminal equipment when receiving the refreshing instruction to display the first image frame on the display screen.

In the embodiment of the application, if the first image data is the image data of the target type, the terminal equipment can send the refreshing instruction to the operating system, so that the operating system refreshes the display screen of the terminal equipment when receiving the refreshing instruction, and the refreshing instruction carries the first image frame so as to display the first image frame on the display screen. When the terminal device acquires a first image frame corresponding to the first image data, the terminal device can immediately send a refreshing instruction to the operating system, so that the operating system refreshes the display screen and displays the first image frame. That is, after the terminal device decodes the first image data to obtain the first image frame, the terminal device may immediately refresh the display screen by sending a refresh command to the operating system, and display the first image frame on the display screen. Because the display screen is not required to be refreshed after waiting for the initial refresh period, the extra delay caused by waiting for the refresh period can be reduced, the image display delay can be further reduced, and the user experience can be improved.

Optionally, the terminal device may further obtain a size of a picture display area of the display screen, so that the image frame is adjusted based on the size of the picture display area and then displayed, so as to improve an image display effect. Specifically, the terminal device may acquire a size of a picture display area of the display screen, and acquire a size of an image frame, where the image frame includes one or more of a first image frame, a second image frame, and a third image frame; if the size of the picture display area does not match the size of the image frame, the size of the image frame is adjusted based on the size of the picture display area. If the size of the image display area is equal to the size of the image frame, the size of the image display area is determined to be matched with the size of the image frame, and the image frame is displayed in the display screen. If the size of the picture display area is larger or smaller than the size of the image frame, the size of the picture display area is not matched with the size of the image frame, and the size of the terminal equipment can be adjusted based on the size of the picture display area, so that the size of the picture display area is equal to the size of the image frame, and the image frame is displayed in the display screen. The size of the picture display area of the display screen and the size of the image frame are matched and adjusted, so that the displayed image frame meets the display requirement of the equipment, and the user experience is improved.

Referring to fig. 6, fig. 6 is another image refreshing schematic diagram provided in an embodiment of the present application, in which a server sends first image data to a terminal device according to a self-frame update period, that is, periodically sends the first image data, where the first image data is image data of a target type. Due to factors such as network delay fluctuation, the time when the terminal device receives the first image data and the time when the server transmits the first image data have delays. Further, after receiving the first image data, the terminal device needs to process the first image data to obtain a first image frame, and at this time, the terminal device sends a refresh command to the operating system, so that the operating system immediately refreshes the display screen, and sends the first image frame to the display screen for display. When the terminal equipment decodes the first image frame, the display screen is refreshed immediately through the operating system, and the first image frame is displayed on the display screen, so that the terminal equipment can refresh the first image frame without waiting for the refresh period of the terminal equipment to arrive, the period waiting time can be reduced, namely, the extra delay is reduced, the image display delay is reduced, and the user experience is improved.

Further, the server periodically sends the first image data, and the terminal device processes the first image data to obtain the first image frame corresponding to each period, so that the display screen is refreshed immediately after the first image frame is obtained without waiting for the refresh period of the terminal device to arrive, the problem that the next refresh point needs to be waited for to arrive due to the fact that the previous first image frame misses the refresh point corresponding to the previous first image frame can be avoided, and therefore the problem that a plurality of image frames fall in the refresh period of the same terminal device to generate frame loss can be avoided, and the continuity of image display is improved.

Referring to fig. 7, fig. 7 is another image data transmission schematic diagram provided in an embodiment of the present application, as shown in fig. 7, in which an application running on a server 71 has its own screen update period, and the server 71 renders and transmits image data according to its own screen update period. When the screen update period of the server 71 is satisfied, the server 71 transmits image data to the terminal device 72 through a network transmission, the terminal device 72 decodes the image data to obtain an image frame after receiving the image data, and if the image data is the image data of the target type, the terminal device 72 transmits a refresh command to the operating system to refresh the display screen, the image frame is displayed on the display screen, and the user can view the game screen through the display screen. It can be seen that with this method the refresh period of the terminal device is discarded and the only period control in the whole system originates from the server. The terminal equipment does not wait after processing the obtained image frames and immediately refreshes the image frames on the display screen, and the user can more quickly see the image frames because the process of waiting for the periodical refreshing points of the terminal equipment to arrive is eliminated.

Alternatively, since the image data of the target type may be image data after the function of the target application is started, the image data of the target type may refer to a continuous large amount of image data sent by the server, for example, the number of frames of the image frames is greater than the target threshold value in a preset time, and when the terminal device receives the image data of the target type, the terminal device needs to display the image data immediately, so that the user can quickly view the image data. Specifically, after the terminal device stops receiving the image data of the target type, the refresh period of the terminal device is adjusted to an initial refresh period; and refreshing the display screen according to the initial refreshing period when the second image data is acquired, so as to display a second image frame corresponding to the second image data on the display screen.

The second image data may include the first image data, or may include image data other than the target application. That is, after the terminal device stops receiving the image data of the target type, the refresh period of the terminal device may be adjusted to an initial refresh period, at this time, if the terminal device receives the second image data, the terminal device may perform decoding processing on the second image data to obtain a second image frame corresponding to the second image data, and refresh the display screen according to the initial refresh period and display the second image frame. Optionally, if the terminal device does not receive the image data of the target type within the target period, it may be determined to stop receiving the image data of the target type, and the refresh period of the terminal device is adjusted to be the initial refresh period.

Optionally, the server may further send a display end instruction to the terminal device, and the terminal device may resume the initial refresh period of the terminal device based on the display end instruction. Specifically, when the terminal device receives a display end instruction sent by the server; the refresh period of the terminal equipment is adjusted to be an initial refresh period; and refreshing the display screen according to the initial refreshing period when the second image data is acquired, so as to display a second image frame corresponding to the second image data on the display screen.

Optionally, when the server exits the target application, a display ending instruction may be sent to the terminal device; alternatively, the server may transmit the display end instruction to the terminal device when the image data is not transmitted to the terminal device within the target period of time (for example, when the server does not receive the operation instruction for the target application within the target period of time); or, the server transmits a display end instruction to the terminal device after stopping transmitting the image data of the target type. That is, when the terminal device receives the display end instruction sent by the server, the refresh period of the terminal device may be adjusted to an initial refresh period, and when the terminal device acquires the second image data, the terminal device performs decoding processing on the second image data to obtain a second image frame; and refreshing the display screen when the initial refresh period is met so as to display a second image frame on the display screen. The second image data may include the first image data, or may include image data other than the target application.

Since the current terminal devices such as televisions, mobile phones and the like bear more other user functions of the terminal devices, if the refresh period in the terminal devices is completely removed, the other user functions of the terminal devices cannot be used normally. Therefore, the mechanism for controlling the refreshing of the terminal equipment by the server should take effect in the process of starting the target application by the user, and the terminal equipment is closed in time after the target application is exited, namely the refreshing period of the terminal equipment is recovered after the target application is exited, so that the normal operation of the terminal equipment is ensured.

In the embodiment of the application, when first image data sent by a server is received, decoding processing is carried out on the first image data to obtain a first image frame corresponding to the first image data; further, whether the first image data is the image data of the target type is detected, if the first image data is the image data of the target type, a refreshing instruction is sent to the operating system, so that the operating system refreshes a display screen of the terminal device, and the first image frame is displayed on the display screen. The terminal equipment processes the image data in different modes due to different types of the image data sent by the received server. When the image data which is not of the target type is acquired, the initial refreshing period of the terminal equipment can be waited for to refresh the display screen of the terminal equipment so as to display the image frames corresponding to the image data on the display screen. When the image frames corresponding to the image data of the target type are acquired, a refreshing instruction can be sent to the operating system so as to refresh the display screen immediately, the image frames are displayed on the display screen, refreshing is not required to be carried out after the refreshing period of the terminal equipment comes, images can be displayed on the display screen rapidly, image display delay is reduced, and therefore user experience is improved.

Optionally, referring to fig. 8, fig. 8 is a flow chart of another data processing method according to an embodiment of the application. The data processing method can be applied to terminal equipment; as shown in fig. 8, the data processing method includes, but is not limited to, the steps of:

s201, receiving first image data sent by a server, and decoding the first image data to obtain a first image frame corresponding to the first image data.

S202, detecting whether the first image data is the image data of the target type.

In the embodiment of the present application, if the first image data is the image data of the target type, step S203 is executed, that is, a refresh command is sent to the operating system, so that the operating system refreshes the display screen of the terminal device when receiving the refresh command, so as to display the first image frame on the display screen; if the first image data is not the target type image data, step S204 is executed, namely, the display screen is refreshed according to the initial refresh period of the terminal device, so as to display the first image frame on the display screen.

And S203, sending a refreshing instruction to the operating system so that the operating system refreshes a display screen of the terminal equipment when receiving the refreshing instruction to display a first image frame on the display screen.

S204, refreshing the display screen according to the initial refreshing period of the terminal equipment so as to display the first image frame on the display screen.

In the embodiment of the application, the types of the received image data sent by the server are different, so that the terminal equipment processes the image data in different modes. Different processing modes are adopted for different types of image data, so that image frames can be better displayed in a display screen, the image display method is more flexible, and user experience is improved.

Optionally, referring to fig. 9, fig. 9 is a flow chart of another data processing method according to an embodiment of the application. The data processing method can be applied to terminal equipment; as shown in fig. 9, the data processing method includes, but is not limited to, the steps of:

s301, detecting a network quality parameter value of the terminal equipment.

In the embodiment of the application, the terminal equipment can detect the network quality parameter value of the terminal equipment, the network quality parameter value can reflect the quality of the network quality, and the larger the network quality parameter value is, the better the network quality is. If the network quality parameter value is less than or equal to the quality threshold value, it may indicate that the network quality is poor. If the network quality parameter value is greater than the quality threshold, it may indicate that the network quality is good.

Alternatively, the terminal device may determine the network quality parameter value based on at least one of a network signal level, a data transmission speed, and a packet loss rate. The packet loss rate refers to the ratio of the number of lost data packets to the transmitted data set. The higher the network signal level is, the better the network signal is, and the larger the network quality parameter value is; the larger the data transmission speed is, the better the network signal is, and the larger the network quality parameter value is; the smaller the packet loss rate, the better the network signal, and the larger the network quality parameter value. Optionally, a correspondence between the network signal level and the network quality parameter value may be preset, a correspondence between the data transmission speed and the network quality parameter value may be preset, and a correspondence between the packet loss rate and the network quality parameter value may be preset, so that when the terminal device obtains at least one of the network signal level, the data transmission speed, and the packet loss rate, the network quality parameter value may be determined.

S302, if the network quality parameter value is smaller than or equal to the quality threshold value and the image data of the target type sent by the server is received, decoding the first image data to obtain a first image frame corresponding to the first image data.

In the embodiment of the application, if the network quality parameter value is smaller than or equal to the quality threshold value, the network quality can be represented as poor, and if the terminal equipment receives the first image data sent by the server and the first image data is the image data of the target type, the first image data is decoded to obtain the first image frame corresponding to the first image data.

S303, sending a refreshing instruction to the operating system so that the operating system refreshes the display screen when receiving the refreshing instruction to display the first image frame on the display screen.

In the embodiment of the application, when the network quality is poor, the refreshing period of the terminal equipment is removed, and when the first image frame is acquired, the refreshing instruction is sent by the operating system to refresh the display screen, so that the display on the display screen can be immediately carried out when the first image frame is received, the period waiting time is reduced, and the image display delay is further reduced.

And S304, if the network quality parameter value is greater than the quality threshold value, acquiring the delay time length of the third image frame by the terminal equipment, and updating the initial refresh period based on the delay time length to obtain the target refresh period.

In the embodiment of the application, if the network quality parameter value is greater than the quality threshold value, the network quality can be better, the delay time for the terminal equipment to acquire the third image frame can be acquired, and the initial refresh period of the terminal equipment is updated based on the delay time, so that the target refresh period is obtained.

Alternatively, the third image frame may refer to an image frame preceding the first image frame, for example, a previous image frame of the first image frame, and the delay period of the third image frame may include a network delay period and a processing period of the third image data by the terminal device. The third image frame is the image frame corresponding to the image data of the target application, and the first image data is the image data after the target function of the target application is started. Specifically, the terminal device may acquire a first time point when the server transmits the third image data, and a second time point when the terminal device receives the third image data; determining a network delay duration based on the first point in time and the second point in time; and acquiring the time length of decoding the third image data as the processing time length of the third image data. Since there is a fluctuation in the network delay, the time at which the server transmits the third image data to the terminal device may be different from the time at which the terminal device receives the third image data, and thus the network delay period is determined based on the difference between the second time and the first time by acquiring the first time at which the server transmits the third image data and the second time at which the terminal device receives the third image data. Further, by acquiring the duration of decoding processing of the third image data by the terminal device as the processing duration of the third image data, the delay duration of the third image frame may be further determined based on the sum of the network delay duration and the processing duration of the third image data by the terminal device. It can be understood that if the network quality is good, the time intervals of the terminal device receiving the image data are approximately equal, that is, the delay time of the image frames is equal, the terminal device can update the initial refresh period based on the delay time of any one of the image frames before the first image frame by acquiring the delay time of the image frame, so as to obtain the target refresh period, further refresh the display screen based on the target refresh period, and display the image frames after the image frames on the display screen. For example, updating the initial refresh period based on the delay time period may refer to advancing the initial refresh period of the terminal device to a time point when the terminal device decodes the third image data to obtain the third image frame, that is, a time point corresponding to a sum of a time point when the server transmits the third image data and the delay time period of the third image frame.

For example, if the initial refresh period of the terminal device is 20 seconds, for example, the refresh time of the terminal device in a certain period is 14:00:30, 14:00:50, 14:01:10 in sequence, and so on. The first time point when the server sends the third image data is 14:00:00, the second time point when the terminal device receives the third image data is 14:00:15, the network delay time is 15 seconds, the time length when the terminal device decodes the third image data is 5 seconds, the delay time of the third image frame is 20 seconds, the target refresh period can be obtained by advancing the initial refresh period to the time (14:00:20) when the terminal device decodes the third image data to obtain the third image frame, the refresh time corresponding to the target refresh period in the time period is 14:00:20, 14:00:40, 14:01:00 in sequence, and the like, namely, the initial refresh period of the terminal device is advanced by 10 seconds. That is, the target refresh period may be obtained by adjusting, e.g., advancing, the initial refresh period of the terminal device. Optionally, the time interval of the initial refresh period of the terminal device may also be adjusted to obtain the target refresh period. For example, the time interval of the target refresh period may be equal to the time interval of the server's picture update period.

Optionally, the network quality parameter value may also be determined based on the network delay duration. Specifically, a correspondence between a network delay time length and a network quality parameter value may be preset, when the terminal device obtains the network delay time length, the network quality parameter value may be determined based on the correspondence between the network delay time length and the network quality parameter value, and if the network quality parameter value is greater than a quality threshold, the delay time length of the third image frame is determined based on the network delay time length and a time length of decoding processing of the third image data by the terminal device, and then an initial refresh period of the terminal device is updated based on the delay time length of the third image frame, so as to obtain a target refresh period. Because the network quality parameter value can be determined based on the network delay time length, when the network quality parameter value is greater than the quality threshold value, the delay time length of the third image frame can be determined directly based on the network delay time length and the time length of decoding processing of the third image data by the terminal equipment, the network delay time length is not required to be acquired again, and the data processing efficiency can be improved.

S305, refreshing the display screen according to the target refreshing period to display the first image frame on the display screen.

In the embodiment of the application, the terminal equipment performs refreshing processing on the display screen according to the target refreshing period so as to display the first image frame on the display screen. That is, since the refresh period of the terminal device is adjusted to the target refresh period, when the time point corresponding to the target refresh period is satisfied, the terminal device performs refresh processing on the display screen, thereby displaying the first image frame on the display screen. That is, when the terminal device needs to display the first image frame in the display screen under the condition of good network quality, the initial refresh period of the terminal device is updated based on the delay time of the image frame (such as the third image frame) before the first image frame, so as to obtain the target refresh period, thereby refreshing the display screen according to the target refresh period and displaying the first image frame. Because the network quality is good, the delay time length of each image frame received by the terminal equipment is considered to be equal, and the time length of decoding processing of each image data by the terminal equipment is considered to be equal, the initial refresh period can be updated based on the delay time length of the image frame before the first image frame, so that the display screen is refreshed based on the target refresh period, and the first image frame is displayed on the display screen. Or updating the initial refresh period based on the delay time of the first image frame, so that the display screen is refreshed based on the target refresh period, and the image frames after the first image frame are displayed on the display screen.

Optionally, if the network quality parameter value is greater than the quality threshold, that is, if the network quality is good, the image data may be processed and displayed by referring to the manner of processing the first image data in the foregoing embodiment, that is, the initial refresh period of the terminal device is removed, and when the first image frame is acquired, a refresh command is sent to the operating system to refresh, and the first image frame is displayed on the display screen, so that the image display delay is reduced.

Referring to fig. 10, fig. 10 is a schematic diagram of still another image refresh provided in an embodiment of the present application, in which the server sends third image data to the terminal device according to its own screen update period, that is, periodically sends the third image data. Due to the existence of network delay fluctuation and other factors, the time when the terminal device receives the third image data and the time when the server transmits the third image data have delays. And after receiving the third image data, the terminal device needs to process the third image data to obtain a third image frame, updates the initial refresh period of the terminal device based on the delay time of the third image frame by obtaining the delay time of the third image frame, obtains a target refresh period, and refreshes the display screen according to the target refresh period to display the first image frame on the display screen.

It can be seen that, since the refresh period of the terminal device is adjusted, which is equivalent to advancing the refresh period to the time point when the terminal device decodes to obtain the image frame, the display screen is refreshed immediately and the image frame is displayed in the display screen when the decoding obtains the image frame, so that the refresh waiting time can be reduced, the image display delay can be reduced, and further the user experience can be improved. The terminal equipment can realize periodic refreshing of the display screen based on the target refreshing period, so that the original stable refreshing characteristic of the terminal equipment is ensured, stable refreshing of pictures on the display screen is ensured, and smooth visual experience is brought.

In some embodiments, the terminal device may acquire a delay time of the image frame, update an initial refresh period of the terminal device based on the delay time of the image frame, and obtain a target refresh period, so as to refresh the display screen according to the target refresh period, and display the image data on the display screen. The delay time of the image frame may include a network delay time and a time for which the terminal device performs decoding processing on the image data. Optionally, the terminal device may further determine a delay duration of the image frame in combination with the network situation, so as to update the initial refresh period of the terminal device. Specifically, a network quality parameter value of the terminal device may be detected; if the network quality parameter value is greater than the quality threshold value, acquiring the delay time of the image frame, updating the initial refresh period of the terminal equipment based on the delay time of the image frame, and obtaining a target refresh period, so that the display screen is refreshed according to the target refresh period, and the image data is displayed on the display screen.

In the embodiment of the application, the current network quality condition can be determined by detecting the network quality parameter value of the terminal equipment, if the network quality parameter value is larger than the quality threshold value, the network quality is better, the initial refresh period of the terminal equipment can be updated based on the delay time of the image frame before the first image frame, and the display screen of the terminal equipment is refreshed according to the updated initial refresh period so as to display the first image frame. That is, by adjusting the refresh period of the terminal device to the time when the terminal device acquires the image frame, the display screen can be refreshed immediately after the image frame is acquired, the refresh waiting time is reduced, the image display delay is further reduced, and the user experience is improved.

The method of the embodiment of the application is described above, and the device of the embodiment of the application is described below.

Referring to fig. 11, fig. 11 is a schematic diagram showing the composition of a data processing apparatus according to an embodiment of the present application, where the data processing apparatus may be a computer program (including program code) running in a computer device, and for example, the data processing apparatus is an application software; the data processing device may be used to execute corresponding steps in the data processing method provided by the embodiment of the present application. The data processing apparatus 110 includes:

A data receiving unit 1101, configured to receive first image data sent by a server, and decode the first image data to obtain a first image frame corresponding to the first image data;

a data detection unit 1102 for detecting whether the first image data is image data of a target type;

the image display unit 1103 is configured to send a refresh command to the operating system if the first image data is the target type image data, so that the operating system refreshes a display screen of the terminal device when receiving the refresh command, so as to display the first image frame on the display screen, where the refresh command carries the first image frame.

Optionally, the image display unit 1103 is further configured to, if the first image data is not the image data of the target type, perform a refresh process on the display screen according to an initial refresh period of the terminal device, so as to display the first image frame on the display screen.

Optionally, the image data of the target type is image data after a target function of the target application is started.

Optionally, the data processing device 110 further includes: an end display unit 1104 for:

after stopping receiving the image data of the target type, or when receiving a display end instruction sent by the server, adjusting the refresh period of the terminal device to be an initial refresh period;

And when the second image data is acquired, refreshing the display screen according to the initial refreshing period so as to display a second image frame corresponding to the second image data on the display screen.

Optionally, the data processing device 110 further includes: a network detection unit 1105 for:

detecting a network quality parameter value of the terminal equipment;

and if the network quality parameter value is smaller than or equal to the quality threshold value and the image data of the target type is received, sending a refreshing instruction to an operating system, so that the operating system refreshes the display screen when receiving the refreshing instruction, and displaying the first image frame on the display screen.

Optionally, the network detection unit 1105 is further configured to:

if the network quality parameter value is greater than the quality threshold value, acquiring delay time length of the terminal equipment for acquiring a third image frame, and updating the initial refresh period based on the delay time length to obtain a target refresh period;

and carrying out refreshing processing on the display screen according to the target refreshing period so as to display the first image frame on the display screen.

Optionally, the delay time of the third image frame includes a network delay time and a processing time of the third image data; the network detection unit 1105 is specifically configured to:

Acquiring a first time point when the server transmits the third image data and a second time point when the terminal equipment receives the third image data;

determining the network delay duration based on the first point in time and the second point in time;

and acquiring the time length of decoding the third image data as the processing time length of the third image data.

It should be noted that, in the embodiment corresponding to fig. 11, the content not mentioned may be referred to the description of the method embodiment, and will not be repeated here.

In the embodiment of the application, when first image data sent by a server is received, decoding processing is carried out on the first image data to obtain a first image frame corresponding to the first image data; further, whether the first image data is the image data of the target type is detected, if the first image data is the image data of the target type, a refreshing instruction is sent to the operating system, so that the operating system refreshes a display screen of the terminal device, and the first image frame is displayed on the display screen. Because the first image data is the image data of the target type, when the first image frame is acquired, a refreshing instruction can be sent to the operating system so as to immediately refresh the display screen of the terminal device, the first image frame is displayed on the display screen, refreshing is not required to be carried out after the refreshing period of the terminal device comes, the image can be rapidly displayed on the display screen, the image display delay is reduced, and therefore user experience is improved.

Referring to fig. 12, fig. 12 is a schematic diagram of a composition structure of a computer device according to an embodiment of the present application. As shown in fig. 12, the computer device 120 may include: a processor 1201, a memory 1202, and a network interface 1203. The processor 1201 is connected to the memory 1202 and the network interface 1203, for example, the processor 1201 may be connected to the memory 1202 and the network interface 1203 by a bus. The computer device may be a terminal device or a server.

Processor 1201 is configured to support the data processing apparatus to perform the corresponding functions in the data processing methods described above. The processor 1201 may be a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), a hardware chip, or any combination thereof. The hardware chip may be an Application-specific integrated circuit (ASIC), a programmable logic device (Programmable Logic Device, PLD), or a combination thereof. The PLD may be a complex programmable logic device (Complex Programmable Logic Device, CPLD), a Field programmable gate array (Field-Programmable Gate Array, FPGA), general array logic (Generic Array Logic, GAL), or any combination thereof.

The memory 1202 is used for storing program codes and the like. The Memory 1202 may include Volatile Memory (VM), such as random access Memory (Random Access Memory, RAM); the Memory 1202 may also include a Non-Volatile Memory (NVM), such as Read-Only Memory (ROM), flash Memory (flash Memory), hard Disk (HDD) or Solid State Drive (SSD); memory 1202 may also include combinations of the above types of memory. In the embodiment of the present invention, the memory 1202 is used for storing a program for website security detection, interactive traffic data, and the like.

The network interface 1203 is configured to provide network communication functions.

The processor 1201 may call the program code to:

receiving first image data sent by a server, and decoding the first image data to obtain a first image frame corresponding to the first image data;

detecting whether the first image data is image data of a target type;

and if the first image data is the image data of the target type, sending a refreshing instruction to the operating system, so that the operating system refreshes a display screen of the terminal equipment when receiving the refreshing instruction, and the first image frame is displayed on the display screen, wherein the refreshing instruction carries the first image frame.

It should be understood that the computer device 120 described in the embodiment of the present application may perform the description of the data processing method in the embodiment corresponding to fig. 5, 8 and 9, and may also perform the description of the data processing apparatus in the embodiment corresponding to fig. 11, which is not repeated herein. In addition, the description of the beneficial effects of the same method is omitted.

The embodiments of the present application also provide a computer readable storage medium storing a computer program comprising program instructions which, when executed by a computer, cause the computer to perform a method as in the previous embodiments, the computer being part of a computer device as mentioned above. Such as the processor 1201 described above. As an example, the program instructions may be executed on one computer device or on multiple computer devices located at one site, or alternatively, on multiple computer devices distributed across multiple sites and interconnected by a communication network, which may constitute a blockchain network.

Embodiments of the present application also provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The computer instructions may be read by a processor of a computer device from a computer-readable storage medium and executed by the processor, such that the computer device performs the steps performed in the embodiments of the methods described above.

Those skilled in the art will appreciate that all or part of the processes in the methods of the above embodiments may be implemented by a computer program for instructing relevant hardware, where the program may be stored in a computer readable storage medium, and where the program, when executed, may include processes of the embodiments of the methods as described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a random-access Memory (Random Access Memory, RAM), or the like.

The foregoing disclosure is illustrative of the present application and is not to be construed as limiting the scope of the application, which is defined by the appended claims.

Claims (10)

1. A method of data processing, comprising:

receiving first image data sent by a server, and decoding the first image data to obtain a first image frame corresponding to the first image data;

detecting whether the first image data is image data of a target type;

and if the first image data is the image data of the target type, sending a refreshing instruction to an operating system, so that the operating system refreshes a display screen of the terminal device when receiving the refreshing instruction to display the first image frame on the display screen, wherein the refreshing instruction carries the first image frame.

2. The method according to claim 1, wherein the method further comprises:

and if the first image data is not the image data of the target type, carrying out refreshing processing on the display screen according to the initial refreshing period of the terminal equipment so as to display the first image frame on the display screen.

3. A method according to claim 1 or 2, characterized in that the image data of the target type is image data after the start of a target function of a target application.

4. The method according to claim 1, wherein the method further comprises:

After stopping receiving the image data of the target type, or when receiving a display end instruction sent by the server, adjusting the refresh period of the terminal equipment to be an initial refresh period;

and refreshing the display screen according to the initial refreshing period when the second image data is acquired, so as to display a second image frame corresponding to the second image data on the display screen.

5. The method according to claim 1, wherein the method further comprises:

detecting a network quality parameter value of the terminal equipment;

and if the network quality parameter value is smaller than or equal to a quality threshold value and the image data of the target type is received, executing the step of sending a refreshing instruction to an operating system so that the operating system refreshes the display screen when receiving the refreshing instruction to display the first image frame on the display screen.

6. The method of claim 5, wherein the method further comprises:

if the network quality parameter value is larger than the quality threshold value, acquiring delay time length of the terminal equipment for acquiring a third image frame, and updating the initial refresh period based on the delay time length to obtain a target refresh period;

And carrying out refreshing processing on the display screen according to the target refreshing period so as to display the first image frame on the display screen.

7. The method of claim 6, wherein the delay duration of the third image frame comprises a network delay duration and a processing duration of the third image data; the obtaining the delay time length of the terminal equipment to obtain the third image frame includes:

acquiring a first time point when the server transmits the third image data and a second time point when the terminal equipment receives the third image data;

determining the network delay duration based on the first point in time and the second point in time;

and acquiring the time length of decoding the third image data as the processing time length of the third image data.

8. A data processing apparatus, comprising:

the data receiving unit is used for receiving first image data sent by the server, and decoding the first image data to obtain a first image frame corresponding to the first image data;

a data detection unit configured to detect whether the first image data is image data of a target type;

And the image display unit is used for sending a refreshing instruction to an operating system if the first image data is the image data of the target type, so that the operating system refreshes a display screen of the terminal device when receiving the refreshing instruction to display the first image frame on the display screen, and the refreshing instruction carries the first image frame.

9. A computer device, comprising: a processor, a memory, and a network interface;

the processor is connected to the memory, the network interface for providing data communication functions, the memory for storing program code, the processor for invoking the program code to cause the computer device to perform the method of any of claims 1-7.

10. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program adapted to be loaded and executed by a processor to cause a computer device having the processor to perform the method of any of claims 1-7.