CN112954344A

CN112954344A - Encoding and decoding method, device and system

Info

Publication number: CN112954344A
Application number: CN202110075906.XA
Authority: CN
Inventors: 王知明
Original assignee: Xian Wanxiang Electronics Technology Co Ltd
Current assignee: Xian Wanxiang Electronics Technology Co Ltd
Priority date: 2021-01-20
Filing date: 2021-01-20
Publication date: 2021-06-11

Abstract

The present disclosure provides a coding and decoding method, device and system, relating to the technical field of images, wherein the method comprises the steps of obtaining a target application program displayed at the most front end currently; when the target application program is determined to be the low-frequency dynamic application, dividing the current frame image into a task bar area image and a non-task bar area image; when the switching of the application program is determined and the current coding scene can be simplified, sending current code stream data to the decoding end equipment; the current code stream data comprises a target number and first coded data, so that the decoding end equipment acquires a target frame image corresponding to a target application program and determines the target frame image as a decoded image of a non-taskbar area image. The decoding end device directly determines the latest frame image of the non-task bar area stored before as the decoding image of the non-task bar area to be displayed, and does not need to execute the decoding operation of the image of the non-task bar area, so that the calculation power consumption of the decoding end device is saved, and the decoding efficiency of the decoding end device is improved.

Description

Encoding and decoding method, device and system

Technical Field

The present disclosure relates to the field of image technologies, and in particular, to a method, device, and system for encoding and decoding.

Background

The coding and decoding principles of the existing computer pictures are as follows: each frame is coded with reference to its previous frame or the nearest previous I frame (I frame, also called intra picture).

In the related art, when a user switches an application scene, an encoding end device may regard a current frame as scene switching, that is, a previous frame or an I frame cannot be used as a reference frame of the current frame, and the current frame needs to be used as a new I frame for encoding.

However, in a cloud office scene, desktop application switching can be frequently encountered, when a user continuously switches back to a certain application program, the encoding end device will continuously re-encode the I frame, and the decoding end device also needs to continuously decode the I frame, so that the computational power consumption of the decoding end device is increased, and the decoding efficiency of the decoding end device is reduced.

Disclosure of Invention

The embodiment of the disclosure provides a coding and decoding method, device and system, which can solve the problems that the computational power consumption of a decoding end device is increased and the decoding efficiency of the decoding end device is reduced in the prior art. The technical scheme is as follows:

according to a first aspect of the embodiments of the present disclosure, there is provided a coding and decoding method applied to a decoding-end device, the method including:

acquiring a target application program displayed at the most front end currently;

when the target application program is determined to be low-frequency dynamic application, dividing the current frame image into a task bar area image and a non-task bar area image;

determining whether the application program is switched or not according to the non-taskbar area image;

when the switching of the application program is determined, determining whether the current coding scene is a simplified coding scene;

when the current coding scene is determined to be the simplified coding scene, acquiring a target number of the target application program, and coding the image of the taskbar area to obtain first coded data;

sending current code stream data to decoding end equipment; the current code stream data comprises the target number of the target application program and the first coded data, so that the decoding end equipment acquires a target frame image corresponding to the target application program and determines the target frame image as a decoded image of a non-taskbar region image.

The embodiment of the disclosure provides a coding and decoding method, wherein when a coding end device determines that a target application program is a low-frequency dynamic application, a current frame image is divided into a task bar area image and a non-task bar area image; when the switching of the application program is determined, determining whether the current coding scene is a simplified coding scene, when the current coding scene is determined, acquiring a target number of a target application program, coding a taskbar area image to obtain first coded data, adding the target number and the first coded data into current code stream data, and sending the current code stream data to a decoding end device, so that the decoding end device determines whether the current code stream data only contains the target number of the target application program and the first coded data when receiving the current code stream data sent by the coding end device, and acquires a latest frame image of a non-taskbar area of the target application program, namely acquires the target frame image, and further determines the target frame image as a decoded image of the non-taskbar area when determining that the current code stream data only contains the target number of the target application program and the first coded data, and merging the target frame image with the decoded data corresponding to the first encoded data and displaying the merged target frame image. Therefore, the decoding end device can directly determine the latest frame image of the non-taskbar area stored before as the decoded image of the non-taskbar area of the current frame image to be displayed, and does not need to perform the decoding operation of the non-taskbar area image of the current frame image, so that the computational power consumption of the decoding end device is saved, and the decoding efficiency of the decoding end device is improved.

According to a second aspect of the embodiments of the present disclosure, there is provided a coding and decoding method applied to a coding end device, the method including:

receiving current code stream data sent by encoding end equipment;

determining whether the current code stream data only contains a target number and first coded data of a target application program; the first coded data are coded data of a taskbar region of a current frame image of the target application program;

when the current code stream data only contains the target number and the first coding data, acquiring a target frame image corresponding to the target application program; the target frame image is a latest frame image of a non-taskbar area of the target application program stored before;

and merging the decoding data corresponding to the first encoding data with the target frame image and displaying the merged decoding data.

The embodiment of the disclosure provides a coding and decoding method, wherein when receiving current code stream data sent by a coding end device, a decoding end device determines whether the current code stream data only contains a target number and first coded data of a target application program, and when determining that the current code stream data only contains the target number and the first coded data of the target application program, obtains a latest frame image of a non-taskbar region of the target application program, that is, obtains the target frame image, further determines the target frame image as a decoded image of the non-taskbar region, and combines and displays the target frame image and the decoded data corresponding to the first coded data. Therefore, the decoding end device can directly determine the latest frame image of the non-taskbar area stored before as the decoded image of the non-taskbar area of the current frame image to be displayed, and does not need to perform the decoding operation of the non-taskbar area image of the current frame image, so that the computational power consumption of the decoding end device is saved, and the decoding efficiency of the decoding end device is improved.

According to a third aspect of the embodiments of the present disclosure, there is provided an encoding side apparatus, including:

the first acquisition module is used for acquiring a target application program displayed at the most front end currently;

the division module is used for dividing the current frame image into a task bar area image and a non-task bar area image when the target application program is determined to be the low-frequency dynamic application;

the first determining module is used for determining whether the application program is switched or not according to the non-taskbar area image;

the second determining module is used for determining whether the current coding scene is a simplified coding scene when the switching of the application program is determined;

the first coding module is used for acquiring a target number of the target application program when the current coding scene is determined to be the simplified coding scene, and coding the image of the taskbar area to obtain first coded data;

the first sending module is used for sending current code stream data to the decoding end equipment; the current code stream data comprises the target number of the target application program and the first coded data, so that the decoding end equipment acquires a target frame image corresponding to the target application program and determines the target frame image as a decoded image of a non-taskbar region image.

According to a fourth aspect of the embodiments of the present disclosure, there is provided a decoding-side apparatus including:

the receiving module is used for receiving current code stream data sent by the encoding end equipment;

a third determining module, configured to determine whether the current code stream data only includes a target number of a target application program and first encoded data; the first coded data are coded data of a taskbar region of a current frame image of the target application program;

the second obtaining module is used for obtaining a target frame image corresponding to the target application program when the current code stream data only contains the target number and the first coding data; the target frame image is a latest frame image of a non-taskbar area of the target application program stored before;

and the first display module is used for merging the decoded data corresponding to the first coded data and the target frame image and then displaying the merged decoded data and the target frame image.

According to a fifth aspect of the embodiments of the present disclosure, there is provided a coding and decoding system, including the decoding-side device according to any of the above embodiments and the encoding-side device according to any of the above embodiments.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.

Fig. 1 is a flowchart of a coding and decoding method provided in an embodiment of the present disclosure;

FIG. 2 is a schematic diagram of a desktop interface provided by an embodiment of the present disclosure;

fig. 3 is a flowchart of a coding and decoding method provided in an embodiment of the present disclosure;

fig. 4 is a flowchart of a coding and decoding method provided in an embodiment of the present disclosure;

fig. 5 is a flowchart of a coding and decoding method provided in an embodiment of the present disclosure;

fig. 6 is a flowchart of a coding and decoding method provided in an embodiment of the present disclosure;

fig. 7 is an interaction diagram of a coding and decoding method provided by an embodiment of the present disclosure;

fig. 8a is a schematic structural diagram of an encoding-side device according to an embodiment of the present disclosure;

fig. 8b is a schematic structural diagram of an encoding-side device according to an embodiment of the present disclosure;

fig. 8c is a schematic structural diagram of an encoding-side device according to an embodiment of the present disclosure;

fig. 8d is a schematic structural diagram of an encoding-side device according to an embodiment of the present disclosure;

fig. 8e is a schematic structural diagram of an encoding-side device according to an embodiment of the present disclosure;

fig. 8f is a schematic structural diagram of an encoding-side device according to an embodiment of the present disclosure;

fig. 8g is a schematic structural diagram of an encoding-side device according to an embodiment of the present disclosure;

fig. 8h is a schematic structural diagram of an encoding-side device according to an embodiment of the present disclosure;

fig. 8i is a schematic structural diagram of an encoding-side device according to an embodiment of the present disclosure;

fig. 9a is a schematic structural diagram of a decoding-end device provided in an embodiment of the present disclosure;

fig. 9b is a schematic structural diagram of a decoding-end device provided in an embodiment of the present disclosure;

fig. 9c is a schematic structural diagram of a decoding-end device provided in an embodiment of the present disclosure;

fig. 9d is a schematic structural diagram of a decoding-end device provided in an embodiment of the present disclosure;

fig. 9e is a schematic structural diagram of a decoding-end device provided in an embodiment of the present disclosure;

fig. 9f is a schematic structural diagram of a decoding-end device according to an embodiment of the present disclosure;

fig. 10 is a block diagram of a coding/decoding system provided in an embodiment of the present disclosure;

fig. 11 is a block diagram of a coding/decoding system according to an embodiment of the present disclosure.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

The disclosed embodiment provides a coding and decoding method, which is applied to a coding end device, as shown in fig. 1, the method includes the following steps:

step 101, obtaining a target application program displayed at the front end at present.

In an example, the encoding end device obtains an application program displayed at the front end at present, namely, a target application program, and then determines whether the target application program is a low-frequency dynamic application or a high-frequency dynamic application, wherein the low-frequency dynamic application is an application in which an image changes slowly or only when a user operates the application, for example, a word, a browser, instant messaging software and the like are low-frequency dynamic applications; the high-frequency dynamic application is an application in which an image frame changes rapidly with time, and for example, a video playing application, a game application, and the like are all high-frequency dynamic applications.

It should be noted that the application program displayed at the frontmost end is the topmost application program displayed on the desktop, and is also the application program currently selected by the user.

Further, before executing step 101, all currently running applications are detected; determining at least one low-frequency dynamic application program from all the application programs; and numbering each low-frequency dynamic application program to form a numbering list, and storing the numbering list.

And the number list stores the mapping relation between the identification information and the number of each low-frequency dynamic application program.

The method for determining at least one low-frequency dynamic application program from all the application programs specifically comprises the following steps:

and determining at least one low-frequency dynamic application program in all the application programs according to a pre-stored application list.

And the application list stores identification information of each low-frequency dynamic application program.

For example, a user may determine the type of each application program in advance, determine all low-frequency dynamic application programs, then store identification information of all low-frequency dynamic application programs to form an application list, when the encoding end device detects all currently running application programs, search for a low-frequency dynamic application program of all application programs from the application list to obtain at least one low-frequency dynamic application program, then number each determined low-frequency dynamic application program to form a numbering list, and store the numbering list, thereby facilitating subsequent search. Specifically, each low-frequency dynamic application program can be numbered directly according to a preset numbering rule to generate a numbering list, for example, the preset numbering rule is that each low-frequency dynamic application program is numbered according to letter sorting order, and if three low-frequency dynamic application programs exist, the low-frequency dynamic application programs are respectively numbered as a low-frequency dynamic application program a, a low-frequency dynamic application program b and a low-frequency dynamic application program c; or, determining all frequently-used or available low-frequency dynamic applications in advance, fixedly allocating a number to each low-frequency dynamic application, and correspondingly storing the number of each low-frequency dynamic application and the identification information to generate a number list. For example, as shown in table 1, it is a schematic diagram of a numbering list, where in table 1, the number of the application Excel is s1, the number of the application Word is s2, the number of the application PPT is s3, and so on, the number of the application MV is sn, where n is an integer greater than 1.

TABLE 1

The preset numbering rule may be other rules, for example, the preset numbering rule is to number each application program in a descending order of numbers, or number each application program according to different numbers generated randomly, and the disclosure is not limited thereto.

It should be noted that some application programs may open multiple files simultaneously, for example, a word application program may open multiple documents simultaneously, for this type of application program, not only the number of the application program but also the number of each file need to be given during numbering, for example, for the case where word software opens two documents, the display interfaces for the two documents are numbered as s2-01 and s2-02, respectively. In this case, the two numbers are also different numbers.

And 102, when the target application program is determined to be the low-frequency dynamic application, dividing the current frame image into a task bar area image and a non-task bar area image.

Acquiring the resolution of the current frame image; determining the display scale of the taskbar area according to the resolution; and determining the non-taskbar area image and the taskbar area image according to the display scale.

The taskbar area is a strip at the bottom of the desktop, the image of the taskbar area is an image displayed in the strip at the bottom of the desktop, as shown in fig. 2, the black strip area at the bottom is the taskbar area, and the rest is a non-taskbar area.

For example, when determining that the target application program is a low-frequency dynamic application, the encoding-side device acquires a current frame image of the target application program and acquires the resolution of the current frame image, and since the resolution of the current frame image is determined, the display proportion of the taskbar region is also fixed under the condition that the resolution is determined, the display position of the taskbar region image can be directly calculated according to the display proportion of the taskbar region, and then the taskbar region image is determined according to the display position of the taskbar region image; and subtracting the display position of the image of the task bar area from the position area of the current frame image to obtain the display position of the image of the non-task bar area, and determining the image of the non-task bar area according to the display position of the image of the non-task bar area.

In addition, since the division of the taskbar region and the non-taskbar region at the time of encoding is performed because the image change of the taskbar region is relatively frequent and the temporal display of the lower right corner changes frequently even if other display items are not changed, it is necessary to consider the taskbar region and the non-taskbar region separately.

And 103, determining whether the application program is switched or not according to the non-taskbar area image.

Optionally, acquiring a non-taskbar area image of a previous frame of image; and comparing the non-taskbar area image of the previous frame image with the non-taskbar area image of the current frame image to determine whether the application program is switched.

And step 104, when the switching of the application program is determined, determining whether the current coding scene is a reducible coding scene.

Optionally, acquiring a target frame image; determining whether the non-taskbar region image is the same as the target frame image; determining that the reducible encoding scene is currently present upon determining that the non-taskbar region image is the same as the target frame image.

The target frame image is a latest frame image of a non-taskbar area of the target application program which is stored before; an image list is stored in the encoding end device in advance, a mapping relationship between identification information of the application program and a target frame image of the application program is stored in the image list, and the specific creation of the image list can refer to a method for creating the image list in the decoding end device, which is not described herein again.

Illustratively, when determining that an application program is switched, an encoding end device acquires a corresponding target frame image from a pre-stored image list according to identification information of the target application program, compares a non-taskbar region image of a current frame image with the target frame image, determines whether the non-taskbar region image of the current frame image is the same as the target frame image, and determines that a current encoding scene can be simplified when determining that the non-taskbar region image of the current frame image is the same as the target frame image; when the image of the non-taskbar area of the current frame image is determined to be different from the image of the target frame image, the target application program is displayed in a non-full screen mode at the moment, and the display parts of other application programs positioned at the bottom layer of the target application program are changed, so that the current coding scene is determined to be an un-reducible coding scene.

Specifically, determining whether the non-taskbar area image is the same as the target frame image may be accomplished by:

determining a first hash fingerprint of a non-taskbar region image; determining a second hash fingerprint of the target frame image; comparing whether the first hash fingerprint and the second hash fingerprint at the corresponding positions are the same; and determining that the non-taskbar area image is the same as the target frame image when the first hash fingerprint and the second hash fingerprint at the corresponding positions are determined to be the same.

Specifically, determining the first hashed fingerprint of the non-taskbar region image may be accomplished by:

carrying out reduction processing on the non-taskbar area image to obtain a new non-taskbar area image; converting the new non-taskbar area image into a grayscale image; calculating a difference value between two adjacent pixels in each row of pixels in the gray-scale image; determining the magnitude of two adjacent difference values in each row difference value; and determining the first hash fingerprint according to the sizes of two adjacent difference values.

Illustratively, the non-taskbar region image is downscaled by down-sampling, for example, dividing the non-taskbar region image into macroblocks of a specific size (8 × 8), averaging the pixels of each macroblock, forming a new non-taskbar region image from the pixel averages, and applying the formula IMG_grey＝0.299*IMG_R+0.587*IMG_G+0.114*IMG_BConverting the color image of the new non-taskbar area image into a gray image; wherein, IMG_RImage representing the red channel, IMG_GImage representing the green channel, IMG_BImage representing blue channel, IMG_greyRepresenting the gray image, and calculating the difference value between two adjacent pixels in each row of pixels in the gray image, for example, as shown in table 2, each row of the gray image has 9 pixels, the original pixel values are 109, 139, 109, 92, 46, 50, 61, and 179, and the difference values are-20, 17, 0, 46, -4, -11, and-118, respectively. Finally, determining the size of two adjacent difference values in the difference values of each row, and recording the gray value on the left side as 1 if the gray value on the left side is larger than the gray value on the right side; if the grayscale value on the right side is greater than the grayscale value on the left side, then it is recorded as 0, and the first hash fingerprint finally generated is shown in table 3.

TABLE 2

TABLE 3

It should be noted that, the calculation method of the second hash fingerprint is the same as the calculation method of the first hash fingerprint, and reference may be specifically made to the calculation method of the first hash fingerprint, which is not described herein again.

It should be noted that, determining whether the non-taskbar region image is the same as the target frame image may also be comparing each pixel of the non-taskbar region image with each pixel of the target frame image one by one, and if each pixel of the non-taskbar region image is completely the same as each pixel at the corresponding position of the target frame image, determining that the non-taskbar region image is the same as the target frame image; and if each pixel of the non-task bar area image is not identical to each pixel of the corresponding position of the target frame image, determining that the non-task bar area image is not identical to the target frame image.

And 105, when the current coding scene is determined to be the simplified coding scene, acquiring a target number of the target application program, and coding the image of the taskbar area to obtain first coded data.

For example, when determining that the current coding scene is a simplified coding scene, the coding end device directly searches a target number corresponding to the target application program from a pre-stored number list, assuming that the target application program is Word, the target number is s2, and codes the image in the taskbar region by using any coding method in the prior art to obtain first coded data.

Step 106, sending current code stream data to decoding end equipment; and enabling the decoding end equipment to acquire a target frame image corresponding to the target application program and determine the target frame image as a decoded image of a non-taskbar area image.

And the current code stream data comprises a target number of the target application program and the first coded data.

For example, when determining that the application program is switched, the encoding end device sends current code stream data including only the target number of the target application program and the first encoded data to the decoding end device, so that when determining that the current code stream data includes only the target number of the target application program and the first encoded data, the decoding end device determines that the application program is switched, acquires a target frame image corresponding to the target application program at this time, and determines the target frame image as a decoded image of a non-taskbar region image of the current frame image to be displayed.

Further, as shown in fig. 3, the method further includes the following step 107:

step 107, when it is determined that the application program is not switched or when it is determined that the current coding scene is not the simplified coding scene, coding the image of the non-taskbar area to obtain second coded data; the current code stream data further comprises second coded data, so that when the decoding end equipment determines that the current code stream data comprises the target number, the first coded data and the second coded data are both decoded and displayed.

For example, when it is determined that the application program is not switched or when it is determined that the current coding scene is not the simplified coding scene, the encoding end device further encodes the non-taskbar region image by using any one of the encoding methods in the prior art to obtain second encoded data, and packages the second encoded data, the first encoded data, and the target number together to generate current code stream data, so that when the decoding end device obtains the second encoded data, the decoding end device decodes both the first encoded data and the second encoded data, and displays the obtained decoded data.

Further, as shown in fig. 4, the method further includes the following steps 108 and 109:

and 108, when the switching of the application program is determined, adding mark information into the coded data of the reference frame image to obtain the coded data of the reference frame image.

Wherein the mark information is used for indicating that the reference frame image is the last frame image of the non-taskbar area of the application program before switching.

For example, each time the encoding end device detects that the application program is switched, the encoding end device adds mark information to the encoded data of the reference frame image, that is, adds mark information to the encoded data of the last frame image of the non-taskbar area of the application program before switching, where the mark information is used to indicate that the reference frame image is the last frame image of the non-taskbar area of the application program before switching, and obtains the encoded data of the reference frame image.

Step 109, sending the reference frame image coded data to a decoding end device, so that the decoding end device correspondingly stores the decoded image corresponding to the reference frame image and the identification information of the application program before switching in an image list.

For example, when obtaining the reference frame image encoded data, the encoding end device sends the reference frame image encoded data to the decoding end device, so that when receiving the reference frame image encoded data, the decoding end device parses the reference frame image encoded data to obtain encoded data added with the tag information, and further saves the decoded image corresponding to the reference frame image as the latest frame image of the non-task bar area of the application before switching according to the instruction of the tag information.

The embodiment of the disclosure provides a coding and decoding method, wherein when a coding end device determines that a target application program is a low-frequency dynamic application, a current frame image is divided into a task bar area image and a non-task bar area image; when the switching of the application program is determined, determining whether the current coding scene is a simplified coding scene, when the current coding scene is determined, acquiring a target number of a target application program, coding a taskbar area image to obtain first coded data, adding the target number and the first coded data into current code stream data, and sending the current code stream data to a decoding end device, so that the decoding end device determines whether the current code stream data only contains the target number of the target application program and the first coded data when receiving the current code stream data sent by the coding end device, and acquires a latest frame image of a non-taskbar area of the target application program, namely acquires the target frame image, and further determines the target frame image as a decoded image of the non-taskbar area when determining that the current code stream data only contains the target number of the target application program and the first coded data, and merging the target frame image with the decoded data corresponding to the first encoded data and displaying the merged target frame image. Therefore, the decoding end device can directly determine the latest frame image of the non-taskbar area stored before as the decoded image of the non-taskbar area of the current frame image to be displayed, and does not need to perform the decoding operation of the non-taskbar area image of the current frame image, so that the computational power consumption of the decoding end device is saved, and the decoding efficiency of the decoding end device is improved. In addition, the judgment of whether the current coding scene is the simplified coding scene is to improve the accuracy of the simplified coding and avoid the adoption of error coding and decoding caused by the simplified coding when the target application program is not displayed in a full screen and other program images are changed.

The disclosed embodiment provides a coding and decoding method, which is applied to a decoding end device, and as shown in fig. 5, the method includes the following steps:

and step 501, receiving current code stream data sent by the encoding end equipment.

Step 502, determining whether the current code stream data only contains the target number and the first coded data of the target application program.

The first coded data is coded data of a taskbar area of a current frame image of the target application program.

Illustratively, when receiving current code stream data, the decoding-end device analyzes the current code stream data, and determines whether the data obtained by analysis only contains a target number and first coded data of a target application program.

It should be noted that, according to different analysis results of current code stream data, the following executed steps are also different, and when it is determined that the current code stream data only includes the target number and the first encoded data, step 503 and step 504 are executed; and executing step 505 when it is determined that the current code stream data includes the target number, the first encoded data and the second encoded data.

Step 503, when it is determined that the current code stream data only contains the target number and the first encoded data, obtaining a target frame image corresponding to the target application program.

The target frame image is a latest frame image of a non-taskbar area of the target application program which is stored before.

Optionally, a target frame image corresponding to the target application program is acquired from a pre-stored image list; the image list stores the mapping relation between the identification information of the application program and the latest frame image.

For example, when determining that only the target number and the first encoded data of the target application are obtained by analysis, the decoding-side device indicates that the application is switched at this time, directly searches for a frame image corresponding to the target application from a pre-stored image list, and determines the frame image as the target frame image, where the target frame image is the latest frame image of the non-taskbar region of the target application stored previously, so as to obtain the target frame image corresponding to the target application.

Specifically, the image list may be created by:

determining a latest frame image of a non-taskbar area of each application; and establishing a corresponding relation between the latest frame of image of the non-taskbar area of each application program and the identification information of the application program to obtain the image list.

For example, determining the latest frame of image of the non-taskbar region of each application may be accomplished by two methods:

the first method comprises the steps of receiving reference frame image coded data sent by a coding end device; the reference frame image coded data is data obtained by adding marking information in the coded data of the reference frame image; and determining the decoded image corresponding to the reference frame image as the latest frame image of the non-task bar area of the application program before switching.

For example, the encoding end device compares the content of the non-task bar area of the previous frame image collected with the content of the non-task bar area of the current frame image to determine whether the application program is switched, when it is determined that the application program is switched, the coding data of the reference frame image is obtained by adding the marking information to the coding data of the reference frame image, and the coding data of the reference frame image is sent to the decoding end device, when the decoding end device receives the coding data of the reference frame image, the coding data added with the marking information is obtained by analyzing the coding data of the reference frame image, the decoding image corresponding to the reference frame image is taken as the newest frame image of the non-task bar area of the application program before switching according to the indication of the marking information, and the newest frame image of the non-task bar area of the application program before switching is correspondingly stored with the identification information of the application program before switching, an image list is formed.

The second method comprises the steps of obtaining the number of an application program corresponding to the last coding frame information; comparing the number of the application program corresponding to the last coded frame information with the target number; and when the number of the application program corresponding to the last coding frame information is determined to be not the same as the target number, determining the decoded image corresponding to the last coding frame information as the latest frame image of the non-task bar area corresponding to the application program.

For example, the decoding end device may further determine whether the application program is switched according to a comparison between a number of the application program corresponding to the received previous code stream data and a target number of the target application program, when it is determined that the number of the application program corresponding to the previous code stream data is different from the target number, it indicates that the application program is switched, and at this time, determine the decoded image of the non-taskbar region corresponding to the previous code stream data as a latest frame image of the non-taskbar region of the application program before switching, and correspondingly store the latest frame image of the non-taskbar region of the application program corresponding to the previous code stream data and the identification information of the application program corresponding to the previous code stream data to form an image list.

And step 504, merging the decoded data corresponding to the first encoded data with the target frame image and displaying the merged decoded data.

In an example, when the decoding-end device acquires the target frame image, the target frame image is determined to be a decoded image corresponding to the non-taskbar-region image of the current frame image, the target frame image and decoded data corresponding to the first encoded data are combined to generate and display a complete display image of the current frame image, and the target frame image is directly displayed without decoding the encoded data of the non-taskbar-region image of the current frame image. In this coding mode, if the user continuously switches between a plurality of applications, when the switching occurs, the decoding side device does not need to decode the image in the non-taskbar region of the current frame image, but directly displays the last display picture (the latest frame image) of the non-taskbar region of the application stored before as the decoded image.

Further, as shown in fig. 6, the method further includes the following steps 505:

and 505, when it is determined that the current code stream data includes the target number, the first encoded data and the second encoded data, decoding and displaying the first encoded data and the second encoded data.

For example, when the decoding-end device analyzes current code stream data to obtain a target number, first encoded data and second encoded data, it indicates that the application program is not switched, and at this time, the received first encoded data and second encoded data are both decoded and then displayed.

The disclosed embodiment provides a coding and decoding method, which is applied to a coding end device and a decoding end device, and as shown in fig. 7, the method includes the following steps:

and 701, acquiring the current target application program displayed at the front end.

Step 702, when the target application program is determined to be the low-frequency dynamic application, dividing the current frame image into a taskbar region image and a non-taskbar region image.

And 703, determining whether the application program is switched or not according to the non-taskbar area image.

Step 704, when it is determined that the application program is switched, it is determined whether the current coding scene is a reducible coding scene.

Step 705, when it is determined that the current coding scene is the simplified coding scene, acquiring a target number of the target application program, and coding the image of the taskbar area to obtain first coded data.

And step 706, sending the current code stream data to the decoding end equipment.

It should be noted that, according to different information contained in the current code stream data, the following execution steps are also different, and when it is determined that the current code stream data only contains the target number and the first encoded data, step 707 and step 709 are executed; when it is determined that the current code stream data includes the target number, the first encoded data, and the second encoded data, step 710 is executed.

And step 707, determining whether the current code stream data only contains the target number and the first encoded data of the target application program.

Step 708, when it is determined that the current code stream data only contains the target number and the first encoded data, obtaining a target frame image corresponding to the target application program.

And 709, merging the decoded data corresponding to the first encoded data with the target frame image and displaying the merged decoded data.

And 710, when it is determined that the current code stream data includes the target number, the first coded data and the second coded data, decoding and displaying the first coded data and the second coded data.

Based on the encoding and decoding methods described in the above embodiments, the following are embodiments of the apparatus of the present disclosure, which can be used to implement the embodiments of the method of the present disclosure.

The embodiment of the present disclosure provides an encoding end device, as shown in fig. 8a, the encoding end device 80 includes: a first obtaining module 801, a dividing module 802, a first determining module 803, a second determining module 804, a first encoding module 805 and a first transmitting module 806.

The first obtaining module 801 is configured to obtain a current target application program displayed at the frontmost end.

A dividing module 802, configured to divide the current frame image into a taskbar region image and a non-taskbar region image when it is determined that the target application is a low-frequency dynamic application.

A first determining module 803, configured to determine whether an application program is switched according to the non-taskbar region image.

A second determining module 804, configured to determine whether the current coding scenario is a reducible coding scenario when it is determined that the application is switched.

A first encoding module 805, configured to, when it is determined that the current coding scene is the simplified coding scene, obtain a target number of the target application, and encode the image of the taskbar region to obtain first encoded data.

A first sending module 806, configured to send current code stream data to a decoding-end device; the current code stream data comprises the target number of the target application program and the first coded data, so that the decoding end equipment acquires a target frame image corresponding to the target application program and determines the target frame image as a decoded image of a non-taskbar region image.

In one embodiment, as shown in fig. 8b, the apparatus further comprises a second encoding module 807.

The second encoding module 807 is configured to encode the image of the non-taskbar region to obtain second encoded data when it is determined that an application is not switched or when it is determined that the current coding scene is not the simplified coding scene; the current code stream data further comprises second coded data, so that when the decoding end equipment determines that the current code stream data comprises the target number, the first coded data and the second coded data are both decoded and displayed.

In one embodiment, as shown in FIG. 8c, the second determination module 804 includes a first acquisition sub-module 8041, a first determination sub-module 8042, and a second determination sub-module 8043.

The first obtaining sub-module 8041 is configured to obtain a target frame image; the target frame image is a latest frame image of a previously stored non-taskbar region of the target application.

The first determining sub-module 8042 is configured to determine whether the non-taskbar area image is the same as the target frame image.

The second determining sub-module 8043 is configured to determine that the simplified coding scene is currently determined when the non-taskbar region image is determined to be the same as the target frame image.

In one embodiment, as shown in fig. 8d, the first determining submodule 8042 comprises a first determining unit 80421, a second determining unit 80422, an alignment unit 80423 and a third determining unit 80424.

Wherein the first determining unit 80421 is configured to determine a first hashed fingerprint of the non-taskbar area image.

The second determining unit 80422 is configured to determine a second hashed fingerprint of the target frame image.

The comparing unit 80423 is configured to compare whether the first hashed fingerprint and the second hashed fingerprint at corresponding positions are the same.

The third determining unit 80424 is configured to determine that the non-taskbar area image is the same as the target frame image when it is determined that the first hash fingerprint and the second hash fingerprint at corresponding positions are the same.

In one embodiment, as shown in fig. 8e, the first determination module 803 includes a second acquisition sub-module 8031 and a third determination sub-module 8032.

The second obtaining sub-module 8031 is configured to obtain a non-taskbar area image of a previous frame of image.

The third determining sub-module 8032 is configured to compare the non-taskbar area image of the previous frame image with the non-taskbar area image of the current frame image to determine whether an application is switched.

In one embodiment, as shown in fig. 8f, the apparatus further comprises an adding module 808 and a second sending module 809.

The adding module 808 is configured to add mark information to the encoded data of the reference frame image to obtain encoded data of the reference frame image when it is determined that the application program is switched; the mark information is used for indicating that the reference frame image is the last frame image of the non-taskbar area of the application program before switching.

The second sending module 809 is configured to send the reference frame image coded data to a decoding end device, so that the decoding end device correspondingly stores a decoded image corresponding to the reference frame image and identification information of an application program before switching in an image list.

In one embodiment, as shown in fig. 8g, the apparatus further comprises a detection module 810, a third determination module 811 and a numbering module 812, and the first encoding module 805 comprises a third obtaining sub-module 8051.

The detecting module 810 is configured to detect all currently running applications.

The third determining module 811 is configured to determine at least one low frequency dynamic application from all the applications.

The numbering module 812 is configured to number each low-frequency dynamic application program to form a numbering list; the mapping relation between the identification information and the number of each low-frequency dynamic application program is stored in the number list.

The third obtaining sub-module 8051 is configured to obtain a target number of the target application program from the number list stored in advance.

In one embodiment, as shown in fig. 8h, the third determination module 811 includes a fourth determination submodule 8111.

The fourth determining submodule 8111 is configured to determine at least one low-frequency dynamic application program of the all application programs according to a pre-stored application list; the application list stores identification information of each low-frequency dynamic application program.

In one embodiment, as shown in fig. 8i, the partitioning module 802 includes a fourth obtaining sub-module 8021, a fifth determining sub-module 8022, and a sixth determining sub-module 8023.

The fourth obtaining sub-module 8021 is configured to obtain a resolution of the current frame image.

The fifth determining sub-module 8022 is configured to determine a display scale of the taskbar region according to the resolution.

The sixth determining submodule 8023 is configured to determine the non-taskbar area image and the taskbar area image according to the display scale.

The embodiment of the disclosure provides a coding end device, which divides a current frame image into a taskbar region image and a non-taskbar region image when determining that a target application program is a low-frequency dynamic application; when the switching of the application program is determined, determining whether the current coding scene is a simplified coding scene, when the current coding scene is determined, acquiring a target number of a target application program, coding a taskbar area image to obtain first coded data, adding the target number and the first coded data into current code stream data, and sending the current code stream data to a decoding end device, so that the decoding end device determines whether the current code stream data only contains the target number of the target application program and the first coded data when receiving the current code stream data sent by the coding end device, and acquires a latest frame image of a non-taskbar area of the target application program, namely acquires the target frame image, and further determines the target frame image as a decoded image of the non-taskbar area when determining that the current code stream data only contains the target number of the target application program and the first coded data, and merging the target frame image with the decoded data corresponding to the first encoded data and displaying the merged target frame image. Therefore, the decoding end device can directly determine the latest frame image of the non-taskbar area stored before as the decoded image of the non-taskbar area of the current frame image to be displayed, and does not need to perform the decoding operation of the non-taskbar area image of the current frame image, so that the computational power consumption of the decoding end device is saved, and the decoding efficiency of the decoding end device is improved.

An embodiment of the present disclosure provides a decoding-side device, as shown in fig. 9a, the decoding-side device 90 includes: a receiving module 901, a third determining module 902, a second obtaining module 903 and a first displaying module 904.

The receiving module 901 is configured to receive current code stream data sent by an encoding end device.

A third determining module 902, configured to determine whether the current code stream data only includes a target number of a target application program and first encoded data; the first coded data is coded data of a taskbar region of a current frame image of the target application program.

A second obtaining module 903, configured to obtain a target frame image corresponding to the target application program when it is determined that the current code stream data only includes the target number and the first encoded data; the target frame image is a latest frame image of a previously stored non-taskbar region of the target application.

A first display module 904, configured to merge the decoded data corresponding to the first encoded data with the target frame image and display the merged decoded data.

In one embodiment, as shown in fig. 9b, the apparatus further comprises a second display module 905.

The second display module 905 is configured to decode and display the first encoded data and the second encoded data when it is determined that the current code stream data includes the target number, the first encoded data, and the second encoded data.

In one embodiment, as shown in fig. 9c, the second fetch module 903 comprises a fifth fetch submodule 9031.

The fifth obtaining sub-module 9031 is configured to obtain a target frame image corresponding to the target application program from a pre-stored image list; the image list stores the mapping relation between the identification information of the application program and the latest frame image.

In one embodiment, as shown in fig. 9d, the apparatus further comprises a fourth determining module 906 and a establishing module 907.

Wherein the fourth determining module 906 is configured to determine a latest frame image of the non-taskbar area of each application.

The establishing module 907 is configured to establish a correspondence between the latest frame of image in the non-taskbar area of each application and the identification information of the application, so as to obtain the image list.

In one embodiment, as shown in fig. 9e, the fourth determination module 906 includes a receiving submodule 9061 and a seventh determination submodule 9062.

The receiving submodule 9061 is configured to receive reference frame image encoded data sent by an encoding end device; the reference frame image coded data is data obtained by adding marking information in the coded data of the reference frame image; the mark information is used for indicating that the reference frame image is the last frame image of the non-taskbar area of the application program before switching.

The seventh determining sub-module 9062 is configured to determine the decoded image corresponding to the reference frame image as the latest frame image of the non-taskbar region of the application before switching.

In one embodiment, as shown in fig. 9f, the fourth determination module 906 includes a sixth obtaining submodule 9063, a comparison submodule 9064 and an eighth determination submodule 9065.

The sixth obtaining sub-module 9063 is configured to obtain a number of an application program corresponding to the last encoded frame information.

The comparison submodule 9064 is configured to compare the number of the application program corresponding to the previous encoded frame information with the target number.

The eighth determining sub-module 9065 is configured to determine, when it is determined that the number of the application program corresponding to the previous encoded frame information is not the same as the target number, the decoded image corresponding to the previous encoded frame information as the latest frame image of the non-taskbar region corresponding to the application program.

The embodiment of the disclosure provides a decoding end device, which determines whether current code stream data only contains a target number and first coded data of a target application program when receiving the current code stream data sent by an encoding end device, acquires a latest frame image of a non-taskbar region of the target application program when determining that the current code stream data only contains the target number and the first coded data of the target application program, namely acquires the target frame image, determines the target frame image as a decoded image of the non-taskbar region, and combines and displays the target frame image and the decoded data corresponding to the first coded data. Therefore, the decoding end device can directly determine the latest frame image of the non-taskbar area stored before as the decoded image of the non-taskbar area of the current frame image to be displayed, and does not need to perform the decoding operation of the non-taskbar area image of the current frame image, so that the computational power consumption of the decoding end device is saved, and the decoding efficiency of the decoding end device is improved.

Fig. 10 is a block diagram of a coding and decoding system according to an embodiment of the present disclosure, where the coding and decoding system shown in fig. 10 includes a coding end device 1001 according to any of the embodiments and a decoding end device 1002 according to any of the embodiments.

The encoding end device 1001 is a client device such as a cloud computer and a zero terminal, a decoding module is arranged on the client device, the decoding end device 1002 is a virtual machine which runs on a cloud server and corresponds to the client device, an encoding module is arranged on the virtual machine, and the virtual machine is correspondingly connected with the client device. The present disclosure is applied to a cloud office scenario, and based on a VDI (Virtual Desktop Infrastructure) technology, a VDI architecture is as shown in fig. 11, and includes a cloud server and at least one client device, where the client device runs at least one Virtual machine, and an operating system runs on the cloud server of a data center, so as to virtualize a Desktop of a user. And accessing the corresponding virtual machine through the client equipment by the user so as to obtain the desktop image, and controlling the obtained desktop image in a reverse control mode.

Based on the encoding and decoding method described in the embodiment corresponding to fig. 1, an embodiment of the present disclosure further provides a computer-readable storage medium, for example, the non-transitory computer-readable storage medium may be a Read Only Memory (ROM), a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like. The storage medium stores computer instructions for executing the encoding and decoding method described in the embodiment corresponding to fig. 1, which is not described herein again.

Based on the encoding and decoding method described in the embodiment corresponding to fig. 3, the embodiment of the present disclosure further provides a computer-readable storage medium, for example, the non-transitory computer-readable storage medium may be a read-only memory, a random access memory, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like. The storage medium stores computer instructions for executing the encoding and decoding method described in the embodiment corresponding to fig. 3, which is not described herein again.

Claims

1. A coding and decoding method is applied to a coding end device, and the method comprises the following steps:

2. The method of claim 1, further comprising:

when the application program is determined not to be switched or when the simplified coding scene is determined not to be currently generated, coding the image of the non-task bar area to obtain second coded data; the current code stream data further comprises second coded data, so that when the decoding end equipment determines that the current code stream data comprises the target number, the first coded data and the second coded data are both decoded and displayed.

3. The method of claim 2, wherein the determining whether the current coding scene is a reducible coding scene comprises:

acquiring a target frame image; the target frame image is a latest frame image of a non-taskbar area of the target application program stored before;

determining whether the non-taskbar region image is the same as the target frame image;

determining that the reducible encoding scene is currently present upon determining that the non-taskbar region image is the same as the target frame image.

4. The method of claim 3, wherein determining whether the non-taskbar region image and the target frame image are the same comprises:

determining a first hashed fingerprint of the non-taskbar region image;

determining a second hashed fingerprint of the target frame image;

comparing whether the first hash fingerprint and the second hash fingerprint at the corresponding positions are the same;

determining that the non-taskbar area image is identical to the target frame image upon determining that the first and second hashed fingerprints at corresponding locations are identical.

5. The method of claim 1, wherein determining whether an application has been toggled based on the non-taskbar region image comprises:

acquiring a non-taskbar area image of a previous frame of image;

and comparing the non-taskbar area image of the previous frame image with the non-taskbar area image of the current frame image to determine whether the application program is switched.

6. The method of claim 1, further comprising:

when the switching of the application program is determined, adding mark information into the coded data of the reference frame image to obtain the coded data of the reference frame image; the mark information is used for indicating that the reference frame image is the last frame image of the non-taskbar area of the application program before switching;

and sending the reference frame image coded data to a decoding end device, so that the decoding end device correspondingly stores the decoded image corresponding to the reference frame image and the identification information of the application program before switching in an image list.

7. The method of claim 1, further comprising:

detecting all currently running application programs;

determining at least one low-frequency dynamic application program from all the application programs;

numbering each low-frequency dynamic application program to form a numbering list; the mapping relation between the identification information and the serial number of each low-frequency dynamic application program is stored in the serial number list;

the acquiring the target number of the target application program comprises:

and acquiring the target number of the target application program from the number list stored in advance.

8. The method of claim 7, wherein the determining at least one low frequency dynamic application from the plurality of applications comprises:

determining at least one low-frequency dynamic application program in all the application programs according to a pre-stored application list; the application list stores identification information of each low-frequency dynamic application program.

9. The method of claim 1, wherein the dividing the current frame image into a taskbar region image and a non-taskbar region image comprises:

acquiring the resolution of the current frame image;

determining the display scale of the taskbar area according to the resolution;

and determining the non-taskbar area image and the taskbar area image according to the display scale.

10. A coding and decoding method is applied to a decoding end device, and the method comprises the following steps:

receiving current code stream data sent by encoding end equipment;

11. The method of claim 10, further comprising:

and when the current code stream data is determined to comprise the target number, the first coded data and the second coded data, decoding and displaying the first coded data and the second coded data.

12. The method of claim 10, wherein the obtaining the target frame image corresponding to the target application comprises:

acquiring a target frame image corresponding to the target application program from a pre-stored image list; the image list stores the mapping relation between the identification information of the application program and the latest frame image.

13. The method of claim 12, further comprising:

determining a latest frame image of a non-taskbar area of each application;

and establishing a corresponding relation between the latest frame of image of the non-taskbar area of each application program and the identification information of the application program to obtain the image list.

14. The method of claim 13, wherein determining the latest frame of image for the non-taskbar region for each application comprises:

receiving reference frame image coded data sent by coding end equipment; the reference frame image coded data is data obtained by adding marking information in the coded data of the reference frame image; the mark information is used for indicating that the reference frame image is the last frame image of the non-taskbar area of the application program before switching;

and determining the decoded image corresponding to the reference frame image as the latest frame image of the non-task bar area of the application program before switching.

15. The method of claim 13, wherein determining the latest frame of image for the non-taskbar region for each application comprises:

acquiring the number of an application program corresponding to the last coded frame information;

comparing the number of the application program corresponding to the last coded frame information with the target number;

and when the number of the application program corresponding to the last coding frame information is determined to be not the same as the target number, determining the decoded image corresponding to the last coding frame information as the latest frame image of the non-task bar area corresponding to the application program.

16. An encoding side device, comprising:

17. A decoding-side apparatus, comprising:

18. A coding/decoding system comprising the decoding-side apparatus of claim 17 and the encoding-side apparatus of claim 16.