WO2023071584A1

WO2023071584A1 - Image processing method and apparatus, and device, storage medium and program product

Info

Publication number: WO2023071584A1
Application number: PCT/CN2022/118849
Authority: WO
Inventors: 黎凌宇; 张莉; 王悦
Original assignee: 北京字节跳动科技有限公司
Priority date: 2021-10-29
Filing date: 2022-09-14
Publication date: 2023-05-04
Also published as: CN114007076A

Abstract

Provided in the embodiments of the present application are an image processing method and apparatus, and a device, a storage medium and a program product. The method comprises: receiving coded information, which is sent by a server, wherein the coded information comprises coded images corresponding to an original image; acquiring an operation instruction; according to the operation instruction and the coded information, determining, from among the coded images, an image to be decoded, wherein the image to be decoded is a coded image, which corresponds to a target original image in a marked area, in the original image; and decoding the image to be decoded, so as to obtain the target original image, and then displaying the target original image in the marked area. The image processing method and apparatus, and the device, the storage medium and the program product provided in the embodiments of the present application are used for saving on a memory space of a terminal device, so as to avoid lagging, crashing, etc. of an information application program.

Description

Image processing method, device, equipment, storage medium and program product

This application claims priority to a Chinese patent application with application number 202111267729.1 and titled "Image processing method, device, device, storage medium, and program product" filed on October 29, 2021, the entire contents of which are incorporated by reference This article.

technical field

The present application relates to the technical field of image processing, and in particular to an image processing method, device, equipment, storage medium and program product.

Background technique

Currently, an information application program may include a long picture with rich graphic content, and the terminal device may display the long picture included in the information application program to the user.

In related technologies, the terminal device corresponding to the information application program can send the coded information corresponding to the long picture to the terminal device, and when the user browses the information application program through the terminal device, the terminal device can decode the coded information to obtain the long picture, Then display the long picture to the user.

In practical applications, the user usually only browses part of the images in the long image (such as the image at the beginning), so if the terminal device decodes the coded information to obtain the long image, it will cause the terminal device to waste more memory space, thereby causing The news app freezes or crashes, etc.

Contents of the invention

Embodiments of the present application provide an image processing method, device, device, storage medium, and program product, which are used to solve problems such as waste of a lot of memory space in a terminal device, and further cause information application programs to freeze or crash.

In the first aspect, the embodiment of the present application provides an image processing method, which is applied to a terminal device, and the method includes:

receiving the encoded information sent by the server; the encoded information includes the encoded image corresponding to the original image;

Obtain operation instructions;

According to the operation instruction and the encoding information, determine the image to be decoded in the encoded image; the image to be decoded is the encoded image corresponding to the target original image in the marked area in the original image;

Decode the image to be decoded to obtain the target original image, and display the target original image in the marked area.

Optionally, the operation instruction includes the starting position, the first height, and the first width of the marked area, and the encoding information includes a sequence parameter set; according to the operation instruction and the encoding information, determining the image to be decoded in the encoded image includes: judging the start Whether the initial position, the first height and the first width are all preset values;

If so, analyze and process the sequence parameter set to obtain decoding information, which includes the second height and second width of the encoded image; and according to the starting position, the first height, the first width, the second height and the second Width, to determine the image to be decoded in the encoded image.

Optionally, determining the image to be decoded in the coded image according to the starting position, the first height, the first width, the second height and the second width includes: according to the starting position, the first height, the first width, the second width Two heights and a second width, judging whether the marked area is located in the coded image;

If so, determine the image to be decoded in the coded image according to the starting position, the first height and the second height.

Optionally, according to the starting position, the first height, the first width, the second height and the second width, judging whether the marked area is located in the coded image includes:

If the starting position is located in the coded image, the first height is smaller than the second height, and the first width is smaller than the second width, the marked area is located in the coded image;

Otherwise, the labeled region is not in the coded image.

Optionally, determining the image to be decoded in the coded image according to the starting position, the first height and the second height includes: determining the region to be decoded according to the starting position, the first height and the second height; The image in the area to be decoded is determined as the image to be decoded.

Optionally, the starting position includes a height starting position; according to the starting position, the first height and the second height, determining the area to be decoded includes: determining the minimum boundary value according to the height starting position; according to the first height, the second height 2. Determine the maximum boundary value for the height and the starting position of the height; determine the CTU line from the minimum boundary value to the maximum boundary value in the coded image as the image to be decoded.

Optionally, determining the minimum boundary value according to the starting position of the height includes: moving the starting position of the height to the right by a third preset value; the third preset value is a preset value corresponding to the encoding method corresponding to the encoded image; A preset value and the maximum value of the difference between the height starting position after the right shift and the second preset value are determined as the minimum boundary value.

Optionally, determining the maximum boundary value according to the first height, the second height, and the starting position of the height includes: shifting the third preset value to the left by the tenth preset value; determining the starting position of the height, the first height, the left The difference between the sum of the third preset value after shifting and the tenth preset value; shifting the difference to the third preset value to the right; and determining the minimum value of the difference between the second height and the right shifting is the maximum boundary value.

Optionally, decoding the image to be decoded to obtain the target original image includes: acquiring decoding reference information; decoding the image to be decoded according to the decoding reference information to obtain the target original image.

Optionally, the decoding information includes the switch identification of the filtering processing tool; obtaining the decoding reference information includes: judging whether the filtering processing tool is turned on according to the switch identification;

If so, the fourth preset value of CTU lines located above the image to be decoded and the fifth preset value of CTU lines located below the image to be decoded in the coded image are determined as the first reference decoded image, and according to the first reference Decode the image and determine the decoding reference information;

Otherwise, the sixth preset value of CTU lines located above the image to be decoded and the seventh preset value of CTU lines located below the image to be decoded in the coded image are determined as the second reference decoded image, and according to the second reference Decode the image and determine the decoding reference information.

Optionally, the fourth preset number of CTU rows above the image to be decoded includes the first CTU row and the second CTU row, and the fifth preset number of CTU rows below the image to be decoded includes the third CTU row; according to Determining decoding reference information for the first reference decoded image includes: decoding the first CTU line, the second CTU line and the third CTU line respectively to obtain the first original pixel set, the second original pixel set and the third original pixel set ; Using a filter processing tool to perform in-loop filtering on the first original pixel set and the second original pixel set, respectively, to obtain the first processed pixel set and the second processed pixel set; according to the first processed pixel set, the second processed pixel set and The third original pixel set is used to determine decoding reference information.

Optionally, the first processed pixel set and the second processed pixel set respectively include N rows of pixel data, and N is an integer greater than or equal to 2; according to the first processed pixel set, the second processed pixel set and the third original pixel set, Determining the decoding reference information includes: using the last M rows of pixel data in the N rows of pixel data in the second processing pixel set to replace the last M rows of pixel data in the N rows of pixel data in the first processing pixel set to obtain the corresponding M is a positive integer less than N; the second processed pixel set, the third processed pixel set and the third original pixel set are determined as decoding reference information.

Optionally, the sixth preset number of CTU rows above the image to be decoded includes the first CTU row, and the seventh preset number of CTU rows below the image to be decoded includes the second CTU row; according to the second reference decoded image , to determine the decoding reference information, including: respectively decoding the first CTU line and the second CTU line to obtain the first original pixel set and the second original pixel set; the first original pixel set includes N rows of original pixel data; using the first The last M lines of original pixel data of N lines of original pixel data in the original pixel set are replaced with the first M lines of original pixel data of N lines of original pixel data in the first original pixel set to obtain a second pixel set; the second pixel set and the second original Pixel set, determined as decoding reference information.

Optionally, decoding the image to be decoded according to the decoding reference information to obtain the target original image includes: decoding the image to be decoded according to the decoding reference information to obtain reconstructed pixel values; and storing the reconstructed pixel values in a preset area to obtain The image is reconstructed; the size of the preset area is the same as that of the image to be decoded; and the target original image is determined in the reconstructed image according to the starting position, the first height, and the first width.

Optionally, according to the initial position, the first height, and the first width, the target original image is determined in the reconstructed image, including: the initial position of the height, and the left shift value after the third preset value is shifted to the left by the minimum boundary value The difference is determined as the top offset; determine the sum of the maximum boundary value and the ninth preset value, shift the third preset value to the left and the value after the value, and the second height and the initial position of the height and the first The minimum value of the height difference is determined as the bottom offset; the width start position of the starting position is determined as the left offset; the second width is summed with the left offset and the first width The difference is determined as the right offset; the top offset, bottom offset, left offset, and right offset indicate the image to be deleted in the reconstructed image; delete the image to be deleted in the reconstructed image image to get the target original image.

Optionally, determining the area to be decoded according to the starting position, the first height, and the second height includes: analyzing and processing the image parameter set included in the encoding information to obtain the position area and the location area of the image slice included in the encoding image Number of slices; if the number of slices is equal to the preset value, then determine the area to be decoded according to the starting position, first height and second height; if the number of slices is not equal to the preset value, delete and mark in the encoded image There are no overlapping image slices, and according to the starting position, the first height and the second height, the area to be decoded is determined in the coded image after the image slice is deleted.

Optionally, the operation instruction is an instruction corresponding to the slide operation; according to the operation instruction and the encoding information, determining the image to be decoded in the encoded image includes: detecting the pause time corresponding to the slide operation; when the pause time exceeds a preset value, at the current The marked area is determined in the display interface; according to the marked area and the coding information, the image to be decoded corresponding to the marked area is determined in the coded image.

In the second aspect, the embodiment of the present application provides an image processing device, which is applied to a terminal device, and the device includes:

The receiving module is used to receive the encoded information sent by the server; the encoded information includes the encoded image corresponding to the original image;

an acquisition module, configured to acquire operation instructions;

The determination module is used to determine the image to be decoded in the encoded image according to the operation instruction and the encoding information; the image to be decoded is the encoded image corresponding to the target original image in the marked area in the original image;

The processing module is used to decode the image to be decoded to obtain the target original image;

The display module is used to display the target original image in the calibration area.

Optionally, the operation instruction includes the starting position, the first height and the first width of the marking area, and the encoding information includes a sequence parameter set; the determination module is specifically used to: judge whether the starting position, the first height and the first width are all is the default value;

Optionally, the determining module is specifically configured to: judge whether the marked area is located in the coded image according to the starting position, the first height, the first width, the second height, and the second width;

Optionally, identify modules specifically for:

Otherwise, the labeled region is not in the coded image.

Optionally, the determining module is specifically configured to: determine the area to be decoded according to the starting position, the first height and the second height; determine the image in the area to be decoded in the coded image as the image to be decoded.

Optionally, the starting position includes a height starting position; the determination module is specifically used to: determine the minimum boundary value according to the height starting position; determine the maximum boundary value according to the first height, the second height and the height starting position; The CTU line from the minimum boundary value to the maximum boundary value in the coded image is determined as the image to be decoded.

Optionally, the determining module is specifically configured to: move the height starting position to the right by a third preset value; the third preset value is a preset value corresponding to the encoding mode corresponding to the encoded image; the first preset value, and the right The maximum value of the difference between the height starting position after the shift and the second preset value is determined as the minimum boundary value.

Optionally, the determination module is specifically used to: shift the third preset value to the left by the tenth preset value; The difference of the preset value; shifting the difference to the third preset value to the right; determining the minimum value of the difference between the second height and the right shifted value as the maximum boundary value.

Optionally, the processing module is specifically configured to: acquire decoding reference information; decode the image to be decoded according to the decoding reference information to obtain the target original image.

Optionally, the decoding information includes a switch identification of the filter processing tool; the processing module is specifically used to: judge whether the filter processing tool is enabled according to the switch identification;

Optionally, the fourth preset number of CTU rows above the image to be decoded includes the first CTU row and the second CTU row, and the fifth preset number of CTU rows below the image to be decoded includes the third CTU row; The module is specifically used to: respectively decode the first CTU line, the second CTU line and the third CTU line to obtain the first original pixel set, the second original pixel set and the third original pixel set; An original pixel set and a second original pixel set are subjected to in-loop filtering processing to obtain a first processed pixel set and a second processed pixel set; according to the first processed pixel set, the second processed pixel set and the third original pixel set, determine the decoding Reference Information.

Optionally, the first processing pixel set and the second processing pixel set respectively include N rows of pixel data, and N is an integer greater than or equal to 2; the processing module is specifically configured to: use the last row of pixel data in the second processing pixel set M rows of pixel data are replaced with the last M rows of pixel data in the N rows of pixel data in the first processing pixel set to obtain the third processing pixel set corresponding to the first processing pixel set; M is a positive integer less than N; the second processing pixel set, the third processed pixel set and the third original pixel set are determined as decoding reference information.

Optionally, the sixth preset value of CTU rows located above the image to be decoded includes the first CTU row, and the seventh preset value of CTU rows located below the image to be decoded includes the second CTU row; the processing module is specifically used for: Decode the first CTU line and the second CTU line respectively to obtain the first original pixel set and the second original pixel set; the first original pixel set includes N lines of original pixel data; adopt N lines of original pixel data in the first original pixel set The last M lines of original pixel data of the first original pixel set are replaced with the first M lines of original pixel data of N lines of original pixel data to obtain a second pixel set; the second pixel set and the second original pixel set are determined as decoding reference information .

Optionally, the processing module is specifically used to: decode the image to be decoded according to the decoding reference information to obtain the reconstructed pixel value; store the reconstructed pixel value in a preset area to obtain the reconstructed image; the size of the preset area and the image to be decoded Same; determine the target original image in the reconstructed image according to the starting position, the first height, and the first width.

Optionally, the processing module is specifically used to: determine the difference between the initial height position and the left shift value after the third preset value is shifted to the left by the minimum boundary value as the top offset; determine the maximum boundary value and the ninth The sum of the preset values, the value after the third preset value is shifted to the left and the minimum value of the difference between the second height and the initial position of the height and the first height is determined as the bottom offset; The width starting position of the starting position is determined as the left offset; the difference between the second width and the sum of the left offset and the first width is determined as the right offset; the top offset, bottom The offset, the left offset and the right offset indicate the image to be deleted in the reconstructed image; the image to be deleted is deleted in the reconstructed image to obtain the target original image.

Optionally, the determining module is specifically configured to: analyze and process the image parameter set included in the encoding information to obtain the location area and the number of slices of the image slices included in the encoded image; if the number of slices is equal to a preset value, then According to the starting position, the first height and the second height, determine the area to be decoded; if the number of slices is not equal to the preset value, delete the image slices that do not overlap with the marked area in the coded image, and according to the starting position, The first height and the second height determine the area to be decoded in the coded image after the image slice is deleted.

Optionally, the operation instruction is an instruction corresponding to the sliding operation; the determination module is specifically used to: detect the pause time corresponding to the sliding operation; when the pause time exceeds a preset value, determine the calibration area in the current display interface; according to the calibration area and code Information to determine the image to be decoded corresponding to the marked area in the coded image.

In a third aspect, the embodiment of the present application further provides a terminal device, including: a processor, and a memory communicatively connected to the processor;

the memory stores computer-executable instructions;

The processor executes the computer-implemented instructions stored in the memory to implement the method of any one of the first aspects.

In a fourth aspect, embodiments of the present application provide a computer-readable storage medium, in which computer-executable instructions are stored, and the computer-executable instructions are used to implement the method in any one of the first aspects when executed by a processor.

In a fifth aspect, an embodiment of the present application provides a computer program product, including: a computer program; when the computer program is executed by a processor, the method in any one of the first aspects is implemented.

In a sixth aspect, the embodiments of the present application provide a computer program, which implements the method in any one of the first aspects when the computer program is executed by a processor.

Embodiments of the present application provide an image processing method, device, device, storage medium, and program product, wherein the image processing method includes: receiving encoding information sent by a server; the encoding information includes an encoded image corresponding to the original image; acquiring an operation instruction; According to the operation instruction and encoding information, determine the image to be decoded in the encoded image; the image to be decoded is the encoded image corresponding to the target original image in the marked area in the original image; decode the image to be decoded to obtain the target original image, and place it in the marked area The original image of the target is displayed inside. In the above method, according to the operation instruction and the encoding information, the image to be decoded is determined in the encoded image, the image to be decoded is decoded, the target original image is obtained, and the target original image is displayed in the marked area, so that the terminal device can realize only the encoded image Part of the coded images in (that is, the images to be decoded) are decoded to obtain the target original images that the user wants to browse, thereby saving the memory space of the terminal device and avoiding the freeze or crash of the information application program.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description serve to explain the principles of the application.

FIG. 1 is a schematic diagram of a scenario proposed in an embodiment of the present application;

FIG. 2 is a flowchart of an image processing method provided in an embodiment of the present application;

FIG. 3 is a flow chart 1 of a method for determining an image to be decoded provided by an embodiment of the present application;

Fig. 4 provides a schematic diagram of determining an image to be decoded according to an embodiment of the present application;

FIG. 5 is another schematic diagram for determining an image to be decoded according to an embodiment of the present application;

FIG. 6 is a flowchart of a method for obtaining an original image of a target provided by an embodiment of the present application;

FIG. 7 is a schematic diagram of determining decoding reference information provided by an embodiment of the present application;

FIG. 8 is another schematic diagram of determining decoding reference information provided by the embodiment of the present application;

FIG. 9 is a schematic diagram of obtaining a reconstructed image provided by an embodiment of the present application;

FIG. 10 is a schematic diagram of determining an original image of a target in a reconstructed image according to an embodiment of the present application;

FIG. 11 is the second flow chart of the method for determining the image to be decoded provided by the embodiment of the present application;

FIG. 12 is a schematic diagram of a coded image provided by an embodiment of the present application including multiple tiles;

FIG. 13 is a schematic diagram of a coded image provided by an embodiment of the present application including multiple slices;

FIG. 14 is a schematic structural diagram of an image processing device provided by an embodiment of the present application;

FIG. 15 is a schematic hardware diagram of a terminal device provided by an embodiment of the present application.

By means of the above drawings, specific embodiments of the present application have been shown, which will be described in more detail hereinafter. These drawings and text descriptions are not intended to limit the scope of the concept of the application in any way, but to illustrate the concept of the application for those skilled in the art by referring to specific embodiments.

Detailed ways

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with aspects of the present application as recited in the appended claims.

First, the prior art is briefly described.

In the prior art, when a user browses a long picture (or high-definition video) included in an information application through a terminal device, the terminal device will decode the encoded information corresponding to the long picture (or high-definition video) delivered by the terminal device, To get the completed long picture (or high-definition video), and display the completed long picture (or high-definition video).

In practice, through investigation and research on users browsing long pictures (or high-definition videos), it is found that users usually only browse the beginning or middle part of long pictures (or high-definition videos) with a high probability. The device decodes the coded information to obtain the completed long picture (or high-definition video), which will cause the terminal device to waste more memory space, and then cause the information application program to freeze or crash.

In this application, in order to save the memory space of the terminal device and avoid the stalling or crash of the information application program, the inventor thought of: during the browsing process of the user, the terminal device only decodes the image to be encoded to obtain a long image (or High-definition video) part of the image that the user wants to browse, so as to save the memory space of the terminal device, thereby avoiding the freeze or crash of the information application program.

The application scenario of the image processing method provided by the embodiment of the present application will be described below with reference to FIG. 1 .

FIG. 1 is a schematic diagram of a scenario provided in an embodiment of the present application. As shown in FIG. 1 , it includes: a terminal device and at least one terminal device.

An information application program may be installed in the terminal device, and the terminal device is a terminal device corresponding to the information application program.

In order to save the transmission traffic of the terminal device sending the original image to the terminal device, the terminal device usually encodes the visible original image to obtain the encoding information corresponding to the original image, and sends the encoding information to the terminal device.

After receiving the encoded information, the terminal device determines the image to be decoded according to the operation instruction and the encoded information, and then decodes the image to be decoded to obtain the target original image and display the target original image. Wherein, the target original image is a part of the original image that the user wants to browse.

In the above process, the terminal device determines the image to be decoded according to the operation instruction and the encoding information, and then decodes the image to be decoded to obtain the target original image, which can save the memory space of the terminal device and prevent the information application program from freezing or crashing.

The technical solution of the present application and how the technical solution of the present application solves the above technical problems will be described in detail below with specific embodiments. The following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present application will be described below in conjunction with the accompanying drawings.

FIG. 2 is a flowchart of an image processing method provided by an embodiment of the present application. As shown in Figure 2, the method includes:

S201. Receive encoding information sent by a server; the encoding information includes an encoded image corresponding to an original image.

Optionally, the execution subject of the embodiment of the present application may be a terminal device, or may be an image processing device installed in the terminal device, and the image processing device may be implemented through a combination of software and/or hardware.

The aforementioned terminal device may be a wireless terminal or a wired terminal. A wireless terminal may be a device that provides voice and/or other business data connectivity to a user, a handheld device with a wireless connection function, or other processing device connected to a wireless modem. The wireless terminal can communicate with one or more core network devices via the radio access network (Radio Access Network, referred to as RAN). The wireless terminal can be a mobile terminal, such as a mobile phone (or called a "cellular" phone, or a smart phone) ) and a computer with a mobile terminal. For example, the wireless terminal may also be a Personal Communication Service (PCS for short) phone, a cordless phone, a Session Initiation Protocol (SIP for short) phone, a Wireless Local Loop (WLL for short) station, Personal digital assistant (Personal Digital Assistant, referred to as PDA) and other equipment. The wireless terminal may also be called a system, a user terminal (User Terminal), or a user device (User Device or User Equipment), which is not limited here. Optionally, the above-mentioned terminal device may also be a device such as a tablet computer.

The encoding information may include an encoded image corresponding to the original image, a sequence parameter set (slice parameter set, SPS), and a picture parameter set (picture parameter set, PPS).

The original image is a long image or a high-definition image visible to the user. For example, the original image is a visible image in RGB or YUV format.

S202. Obtain an operation instruction.

The operation instruction may be an instruction corresponding to the user's sliding operation.

For example, when the terminal device is a smart phone, the sliding operation may be a user's sliding operation on the touch screen of the terminal device.

Optionally, the sliding operation can be continuous or paused.

S203. Determine the image to be decoded in the encoded image according to the operation instruction and the encoding information.

The image to be decoded is the coded image corresponding to the target original image in the marked area of the original image.

Optionally, the size of the image to be decoded may be the same as or different from the size of the marked area.

When the size of the image to be decoded is different from the size of the marked area, the size of the image to be decoded is larger than the size of the marked area. For example, the height of the image to be decoded is equal to the first height of the marked area, and the width of the image to be decoded is greater than or equal to the first width of the marked area. For example, the height of the image to be decoded is greater than the first height of the marked area, and the width of the image to be decoded is greater than or equal to the first width of the marked area.

Optionally, S203 may be implemented in the following two manners (mode B1 and manner B2).

Mode B1, judging whether the operation instruction includes the initial position of the calibration area, the first height and the first width are all preset values;

If so, analyze and process the SPS in the coded information to obtain the decoded information; include the second height and the second width of the coded image in the decoded information; and according to the starting position, the first height, the first width, the second height and The second width is to determine the image to be decoded in the encoded image;

Otherwise, perform full-frame decoding to obtain the original image.

Wherein, the preset value may be 0 or other values, and the value of the preset value is not limited here.

Optionally, when the starting position is within the encoded image, the first height is less than or equal to the second height, and the first width is less than or equal to the second width, then determine the image to be decoded in the encoded image.

The first height being less than or equal to the second height indicates that the Coding Tree Unit (Coding Tree Unit, CTU) row corresponding to the first height in the marked area is included in the CTU row corresponding to the second height in the coded image.

The first width being less than or equal to the second width indicates that the CTU line corresponding to the first width in the marked area is included in the CTU line corresponding to the second width in the coded image.

Specifically, for the method of performing full-frame decoding to obtain the original image, reference may be made to the prior art, which will not be repeated here.

Mode B2, the operation instruction is an instruction corresponding to the sliding operation; S203 specifically includes: detecting the pause time corresponding to the sliding operation; when the pause time exceeds a preset value, determining the calibration area in the current display interface; according to the calibration area and coding information, in the The image to be decoded corresponding to the marked area is determined in the coded image.

Further, the method B1 can be used to implement the method B2 in which the image to be decoded corresponding to the marked area is determined in the coded image according to the marked area and the coding information.

S204. Decode the image to be decoded to obtain the target original image, and display the target original image in the marked area.

Optionally, when the size of the image to be decoded is the same as the size of the marked area, the image to be decoded is decoded to obtain the target original image. Optionally, when the size of the image to be decoded is different from the size of the marked area, the image to be decoded is decoded to obtain a decoded image, and the decoded image is clipped according to the size of the marked area to obtain a target original image.

The image processing method provided by the embodiment in Fig. 2 includes: according to the operation instruction and the encoding information, determine the image to be decoded in the encoded image, decode the image to be decoded, obtain the target original image, and display the target original image in the marked area, which can realize The terminal device only decodes part of the coded images (that is, the images to be decoded) in the coded images to obtain the target original image that the user wants to browse, thereby saving the memory space of the terminal device and avoiding the freeze or crash of the information application program. The performance of the information application program in the terminal device, thereby improving the application experience of the user.

On the basis of the above-mentioned embodiment, in conjunction with Fig. 3 below, taking the size of the image to be decoded larger than the size of the marked area as an example, according to the starting position, first height, first width, second height and second width, in the encoding The specific execution process of determining the image to be decoded in the image will be described.

FIG. 3 is a first flowchart of a method for determining an image to be decoded provided by an embodiment of the present application. As shown in Figure 3, on the basis of the above S203, the method includes:

S301. Acquire a starting position, a first height, a first width, a second height, and a second width.

Optionally, after the operation instruction is acquired, the initial position, first height and first width of the marked area may be stored.

Optionally, after parsing and processing the sequence parameter set included in the coding information to obtain the second height and the second width of the coded image, the second height and the second width may be stored.

When S301 is executed, acquiring the stored starting position, first height, first width, second height and second width may be performed.

Optionally, after determining that the initial position, the first height, and the first width are not preset values, directly

S302. According to the starting position, the first height, the first width, the second height, and the second width, determine whether the marked area is located in the coded image.

If yes, execute S303, otherwise execute S304.

Optionally, if the starting position is located in the coded image, the first height is smaller than the second height, and the first width is smaller than the second width, the marked area is located in the coded image;

Otherwise, the labeled region is not in the coded image.

S303. Determine an image to be decoded in the encoded image according to the starting position, the first height, and the second height.

Optionally, the specific execution method of S303 may include the following two ways (way A1 and way A2).

Mode A1, the starting position includes the height starting position; determine the minimum boundary value according to the height starting position; determine the maximum boundary value according to the first height, the second height and the height starting position; encode the minimum boundary value in the image to The CTU row with the maximum boundary value is determined as the image to be decoded.

Optionally, the following method can be used to determine the minimum boundary value:

Move the height start position to the right by the third preset value; the third preset value is the preset value corresponding to the encoding method corresponding to the encoded image; the first preset value and the height start position after the right shift are the same as the second The minimum value among the differences of the preset values is determined as the minimum boundary value.

That is, the minimum boundary value can be obtained by the following formula 1:

Min_crop_ctu_row＝max(F1,(Y0>>C)-F2) (1);

Among them, Min_crop_ctu_row is the minimum boundary value, max is the operation of taking a large value, F1 is the first preset value, Y0 is the height starting position, >> is the right shift symbol, - is the minus sign, C is the third preset value, F2 is the second preset value. Optionally, F1 may be 0 or other positive integers, and F2 may be 2 or other positive integers.

Optionally, the following method can be used to determine the maximum boundary value:

Move the third preset value to the left by the tenth preset value; determine the sum of the height starting position, the first height, and the third preset value after the left shift, and the difference between the tenth preset value; The third preset value is shifted to the right; the minimum value of the difference between the second height and the right shifted value is determined as the maximum boundary value.

That is, the maximum boundary value can be obtained by the following formula 2:

Max_crop_ctu_row＝min(H2,(Y0+H1+(F3<<C)-F3)>>C) (2);

Among them, Max_crop_ctu_row is the maximum boundary value, min is the operation of taking a small value, H2 is the second height, H1 is the first height, F3 is the tenth preset value, << is the left shift symbol. Optionally, F3 may be 0 or other positive integers.

It should be noted that, in the above formula 2, F3<<C means shifting the third preset value to the left by the tenth preset value, Y0+H1+(F3<<C) means the above sum value, Y0+H1+(F3< <C)-F3 is the above-mentioned difference, (Y0+H1+(F3<<C)-F3)>>C means shifting the above-mentioned difference to the right by a third preset value.

FIG. 4 is a schematic diagram of determining an image to be decoded according to an embodiment of the present application. As shown in FIG. 4 , it includes: a coded image 41 , a marked area 42 and an image to be decoded 43 . Wherein, the minimum boundary value above the calibration area 42 is obtained by using Formula 1, and the maximum boundary value below the calibration area 42 is obtained by using Formula 2. In FIG. 4 , the image 43 to be decoded includes: a CTU line where the minimum boundary value is located, a CTU line where the maximum boundary value is located, and a CTU line between the minimum boundary value and the maximum boundary value.

Way A2, the starting position includes the starting position of the height; the coded picture that satisfies the first preset condition in the coded picture is determined as the picture to be decoded, wherein the first preset condition is the first height, the first preset height and The sum of the second preset heights is less than the second height.

FIG. 5 is another schematic diagram of determining an image to be decoded according to an embodiment of the present application. As shown in FIG. 5 , it includes: a coded image 51 , a marked area 52 and an image to be decoded 53 . Wherein, the first preset height is located above the first height, and the second preset height is located below the first height. In FIG. 5 , the image to be decoded 53 includes: the first preset height, the sum of the heights of the first height and the second preset height, and the CTU rows spanned.

S304. Perform full-frame decoding to obtain an original image.

In the embodiment in Fig. 3, in the process of determining the image to be decoded in the coded image according to the starting position, the first height and the second height, the CTU line from the minimum boundary value to the maximum boundary value in the coded image can be determined For the image to be decoded, the image to be decoded includes a coded image corresponding to the target original image, thereby improving the accuracy of the target original image obtained by decoding the image to be decoded.

On the basis of the above-mentioned embodiments, the specific execution process of S204 above will be described below in conjunction with FIG. 6 , where the image to be decoded is taken as an example of the CTU behavior from the minimum boundary value to the maximum boundary value in the coded image.

FIG. 6 is a flowchart of a method for obtaining an original image of a target provided by an embodiment of the present application. As shown in Figure 6, the method includes:

S601. Acquire decoding reference information.

Optionally, the decoding information includes the switch identification of the filter processing tool; S601 specifically includes: judging whether the filter processing tool is enabled according to the switch identification; row and the fifth preset value CTU row located below the image to be decoded are determined as the first reference decoded image, and the decoding reference information is determined according to the first reference decoded image; otherwise, the sixth preset value in the coded image The CTU lines above the picture to be decoded and the seventh preset CTU lines below the picture to be decoded are determined as the second reference decoded picture, and the decoding reference information is determined according to the second reference decoded picture.

Optionally, the filter processing tool may be any at least one of an adaptive loop filter tool (adaptive loop filter), a deblocking filter tool (deblocking filter), and a sample adaptive compensation tool (SAO).

Optionally, when the filtering processing tool includes a deblocking filtering tool and a sample adaptive compensation tool, the decoding information includes a first switch identifier corresponding to the deblocking filtering tool and a second switch identifier corresponding to the sample adaptive compensation tool.

Further, according to the first switch identification, it is judged whether the filtering tool for removing blockiness is enabled, and according to the second switch identification, it is determined whether the sample adaptive compensation tool is enabled.

For example, if the first switch flag is an on flag, it is determined that the deblocking filtering tool is turned on, and if the first switch flag is an off flag, then it is determined that the deblocking filtering tool is off.

Optionally, the fourth preset value can be 2, the fifth preset value can be 1, the sixth preset value can be 1, and the seventh preset value can be 1. Of course, the fourth preset value, the fifth preset value The preset value, the sixth preset value, and the seventh preset value can also be other positive integers.

Optionally, when the fourth preset value is 2 and the fifth preset value is 1, the fourth preset CTU rows above the image to be decoded include the first CTU row and the second CTU row, and the fifth preset Set the CTU row below the image to be decoded to include the third CTU row; according to the first reference decoded image, determine the decoding reference information, including:

Decoding the first CTU line, the second CTU line and the third CTU line respectively to obtain the first original pixel set, the second original pixel set and the third original pixel set;

performing in-loop filtering processing on the first original pixel set and the second original pixel set respectively by using a filtering processing tool to obtain a first processed pixel set and a second processed pixel set;

Decoding reference information is determined according to the first processed pixel set, the second processed pixel set and the third original pixel set.

Wherein, the first original pixel set is obtained by decoding the first CTU row, the second original pixel set is obtained by decoding the second CTU row, and the third original pixel set is obtained by decoding the third CTU row.

Wherein, the first processed pixel set is obtained by performing in-loop filtering on the first original pixel set, and the second processed pixel set is obtained by performing in-loop filtering on the second original pixel set.

Optionally, when the filtering processing tool includes a blocking removal filtering tool and a sample adaptive compensation tool, the de-filtering processing tool is used to perform in-loop filtering processing on the first original pixel set and the second original pixel set respectively, to obtain the first processing A pixel set and a second processed pixel set comprising:

performing in-loop filtering on the first original set of pixels and the second set of original pixels respectively by using a filtering tool for removing block effects, to obtain a first set of processed pixels and a second set of processed pixels; or,

Using a sample adaptive compensation tool, respectively performing in-loop filtering on the first original pixel set and the second original pixel set to obtain the first processed pixel set and the second processed pixel set; or,

The in-loop filtering process is performed on the first original pixel set by using the block effect removal filter tool to obtain the first processed pixel set, and the in-loop filter process is performed on the second original pixel set by the sample adaptive compensation tool to obtain the second processed pixel set.

Optionally, the first processed pixel set and the second processed pixel set respectively include N rows of pixel data, and N is an integer greater than or equal to 2; according to the first processed pixel set, the second processed pixel set and the third original pixel set, Determine decoding reference information, including:

Using the last M rows of pixel data in the N rows of pixel data in the second processed pixel set to replace the last M rows of pixel data in the N rows of pixel data in the first processed pixel set to obtain a third processed pixel set corresponding to the first processed pixel set; M is a positive integer less than N;

The second processed pixel set, the third processed pixel set and the third original pixel set are determined as decoding reference information.

Optionally, N can be 64, 32, 16, 4, etc.

The block diagram of determining the decoding reference information according to the first reference decoded image in this application will be described below with reference to FIG. 7 .

FIG. 7 is a schematic diagram of determining decoding reference information provided by an embodiment of the present application. As shown in FIG. 7 , the first CTU line, the second CTU line and the third CTU line are respectively decoded to obtain the first original pixel set, the second original pixel set and the third original pixel set.

Loop filtering is performed on the first original pixel set and the second original pixel set respectively to obtain the first processed pixel set and the second processed pixel set.

According to the first processed pixel set and the second processed pixel set, a third processed pixel set is obtained.

Further, the third processed pixel set, the second processed pixel set, and the third original pixel set are determined as decoding reference information.

Optionally, when both the sixth preset value and the seventh preset value are 1, the sixth preset value of the CTU row above the image to be decoded includes the first CTU row, and the seventh preset value of the CTU row above the image to be decoded includes the first CTU row. The CTU row below the image includes the second CTU row;

Determining decoding reference information according to the second reference decoding image, including:

Decoding the first CTU line and the second CTU line respectively to obtain a first original pixel set and a second original pixel set; the first original pixel set and the second original pixel set respectively include N rows of original pixel data;

Using the last M rows of raw pixel data of the N rows of raw pixel data in the first raw pixel set to replace the first M rows of raw pixel data of the N rows of raw pixel data in the first raw pixel set to obtain a second pixel set;

The second pixel set and the second original pixel set are determined as decoding reference information.

The block diagram of determining the decoding reference information according to the second reference decoded image in this application will be described below with reference to FIG. 8 . FIG. 8 is a schematic diagram of another method for determining decoding reference information provided by the embodiment of the present application. As shown in FIG. 8 , the first CTU row, the second CTU row, and the third CTU row are decoded respectively to obtain the first original pixel set and the second original pixel. According to the first original pixel set, a second pixel set is obtained. The second pixel set and the second original pixel are determined as decoding reference information.

S602. Decode the image to be decoded according to the decoding reference information to obtain a target original image.

Optionally, S602 specifically includes: decoding the image to be decoded according to the decoding reference information to obtain a reconstructed pixel value; storing the reconstructed pixel value in a preset area to obtain a reconstructed image; the size of the preset area is the same as that of the image to be decoded; The starting position, the first height, and the first width determine the target original image in the reconstructed image.

Optionally, according to the initial position, the first height, and the first width, the target original image is determined in the reconstructed image, including: the initial position of the height, and the left shift value after the third preset value is shifted to the left by the minimum boundary value The difference is determined as the top offset;

The value after the third preset value is shifted to the left by the maximum boundary value and the sum of the ninth preset value, and the minimum value of the difference between the second height and the height initial position and the first height sum value is determined as the bottom deviation displacement;

Determine the width starting position of the starting position as the left offset;

The difference between the second width and the sum of the left offset and the first width is determined as the right offset; the top offset, bottom offset, left offset, and right offset indicate the image to be deleted in the reconstructed image;

Delete the image to be deleted in the reconstructed image to obtain the target original image.

Optionally, the top offset can be determined by the following formula 3:

luma_offset_top=Y0-(Min_crop_ctu_row<<C) Formula 3;

Among them, luma_offset_top is the top offset.

Optionally, the bottom offset can be determined by the following formula 4:

luma_offset_bottom＝min(((Max_crop_ctu_row+1)<<C),H2-(Y0+H1)) (4);

Among them, luma_offset_bottom is the bottom offset, + is the plus sign.

Optionally, the left offset can be determined by the following formula 5:

luma_offset_left = X0 (5);

Among them, luma_offset_left is the left offset, and X0 is the starting position of the width.

Optionally, the right offset can be determined by the following formula 6:

luma_offset_right＝src->width-(luma_offset_left+W1) (6);

Among them, luma_offset_right is the right offset, and src->width indicates the second width.

FIG. 9 is a schematic diagram of obtaining a reconstructed image provided by an embodiment of the present application. On the basis of Figure 7, as shown in Figure 9, the third processed pixel set and the second processed pixel set are sequentially stored above the preset area, and the third original pixel set is stored below the preset area, according to The decoding reference information determined by the third processed pixel set, the second processed pixel set and the third original pixel, and the reconstructed pixel values obtained by decoding the image to be decoded are stored in a preset area to obtain a reconstructed image.

Wherein, the reconstructed image includes reconstructed pixel values.

FIG. 10 is a schematic diagram of determining a target original image in a reconstructed image according to an embodiment of the present application. As shown in FIG. 10 , it includes: a reconstructed image 1001 , a target original image 1002 , and an image to be deleted 1003 .

Wherein, image 1003 indicates images to be deleted in the reconstructed image for top offset, bottom offset, left offset and right offset. Further, after the image 1003 is deleted from the reconstructed image, the target original image 1002 is obtained.

In this application, it is also possible to determine the memory address of the target original image. For example, in Figure 10, the memory address of the upper left corner of the reconstructed image is p_src0, and the pixel memory interval is stride, then the memory address of the upper left corner of the target original image is p_src0+luma_offset_top*stride+luma_offset_left.

On the basis of the above embodiments, the present application also provides another method for determining the region to be decoded according to the starting position, the first height and the second height. The following description will be made in conjunction with FIG. 11 .

FIG. 11 is a second flow chart of a method for determining an image to be decoded provided by an embodiment of the present application. As shown in Figure 10, the method includes:

S111, analyze and process the image parameter set included in the encoding information to obtain the location area and the number of image slices included in the encoded image.

Optionally, the image slices may be image slices or image tiles.

It should be noted that the slice is to divide the coded image according to the strip shape, and the tile is to divide the coded image according to the rectangle.

S112. Determine whether the number of fragments is equal to a preset value.

If yes, execute S113, otherwise execute S114.

S113. Determine the area to be decoded according to the starting position, the first height and the second height.

It should be noted that, the execution process of S113 may refer to S303 in FIG. 3 , which will not be repeated here.

S114. Delete image slices that do not overlap with the marked area in the coded image, and determine a region to be decoded in the coded image after the image slice is deleted according to the starting position, the first height, and the second height.

S114 is exemplarily described below with reference to FIG. 12 , taking tiles as an example of an image slice.

FIG. 12 is a schematic diagram of a coded image provided by an embodiment of the present application including multiple tiles. As shown in FIG. 12, a coded image includes multiple tiles. For example, multiple tiles include T1-T10. Among them, the image slices T1-T4 and T7-T10 are 8 image slices that do not overlap with the marked area in the coded image.

The image slices T1-T4 and T7-T10 that do not overlap with the marked area can be deleted in the coded image to obtain the remaining coded image, and then according to the starting position, the first height and the second height, determine the remaining coded image to be decoded area.

It should be noted that, according to the starting position, the first height and the second height, the method of determining the area to be decoded in the remaining coded image is similar to the above S303, and will not be repeated here.

S114 will be exemplarily described below with reference to FIG. 13 , taking the image slice as slices as an example.

FIG. 13 is a schematic diagram of a coded image provided by an embodiment of the present application including multiple slices. As shown in FIG. 12 , a coded image includes multiple slices. For example, multiple slices include S1-S5. Among them, the image slices S1, S2 and S5 are three image slices that do not overlap with the marked area in the coded image.

The image slices S1, S2, and S5 that do not overlap with the marked area can be deleted in the coded image to obtain the remaining coded image, and then according to the starting position, the first height and the second height, determine the area to be decoded in the remaining coded image.

According to the method design of the present application, for the image to be decoded, it is necessary to decode the completed CTU lines included in the image to be decoded, that is, the first width in the present application has no influence on the decoding speed and memory. Below in conjunction with Table 1, compared to full-frame decoding (i.e. full-frame decoding of the encoded image (with height H2)), the image to be decoded with H2/2, H2/4, H2/8 height (i.e. the first height) Perform decoding to obtain the decoding time and occupied memory size, as shown in Table 1 below.

Table 1

解析高度resolution height	解码时长(毫秒，百分比)Decoding time (milliseconds, percentage)	内存(兆字节，百分比)memory (megabytes, percent)
1080(H2)1080(H2)	82.679，100％82.679, 100%	6.09，100％6.09, 100%
540(H2/2)540(H2/2)	51.000，61.68％51.000, 61.68%	4.15，68.14％4.15, 68.14%
270(H2/4)270(H2/4)	31.008，37.50％31.008, 37.50%	3.03，49.75％3.03, 49.75%
135(H2/8)135(H2/8)	21.383，25.86％21.383, 25.86%	2.46，40.39％2.46, 40.39%

It should be noted that Table 1 is an example of the test results obtained based on the fact that the initial height position and width initial position of the test data are both 0, and the first width is 1920.

In Table 1, 1080 (H2) is used to indicate full-frame decoding. When performing full-frame decoding, the decoding time is 82.679 milliseconds (considered as a reference time percentage of 100%), and the size of the memory occupation is 6.09 megabytes (considered as a reference memory occupation percentage of 100%).

For example, when the image to be decoded whose first height is 540 (that is, H2/2 height) is decoded, the decoding time length is 51.000 milliseconds (61.68% of 82.679 milliseconds), and the size of the memory occupation is 4.15 megabytes (6.09 megabytes 68.14% of bytes).

Further, it can be seen from Table 1 that the smaller the first height is, the shorter the decoding duration is, and the smaller the occupied memory is.

It should be noted that the codec standards to which the image processing method provided in this application can be applied include but are not limited to the heif codec standard or a certain codec standard.

FIG. 14 is a schematic structural diagram of an image processing device provided by an embodiment of the present application. As shown in Figure 14, the image processing device 10 includes:

The receiving module 11 is configured to receive the encoded information sent by the server; the encoded information includes the encoded image corresponding to the original image;

An acquisition module 12, configured to acquire an operation instruction;

The determination module 13 is used to determine the image to be decoded in the encoded image according to the operation instruction and the encoding information; the image to be decoded is the encoded image corresponding to the target original image in the marked area in the original image;

The processing module 14 is used to decode the image to be decoded to obtain the target original image;

The display module 15 is used for displaying the target original image in the marked area.

The image processing device provided in the embodiment of the present application can execute the above-mentioned image processing method, and its implementation principles and beneficial effects are similar, and will not be repeated here.

Optionally, the operation instruction includes the starting position, the first height and the first width of the marking area, and the encoding information includes a sequence parameter set; the determination module 13 is specifically used to: determine whether the starting position, the first height and the first width are are default values;

Optionally, the determining module 13 is specifically configured to: judge whether the marked area is located in the coded image according to the starting position, the first height, the first width, the second height and the second width;

Optionally, the determining module 13 is specifically configured to:

Otherwise, the labeled region is not in the coded image.

Optionally, the determination module 13 is specifically configured to: determine the area to be decoded according to the starting position, the first height and the second height; determine the image in the area to be decoded in the coded image as the image to be decoded.

Optionally, the starting position includes a height starting position; the determination module 13 is specifically used to: determine the minimum boundary value according to the height starting position; determine the maximum boundary value according to the first height, the second height and the height starting position; Determine the CTU line from the minimum boundary value to the maximum boundary value in the coded image as the image to be decoded.

Optionally, the determining module 13 is specifically configured to: move the height starting position to the right by a third preset value; the third preset value is a preset value corresponding to the encoding method corresponding to the encoded image; set the first preset value, and The maximum value of the difference between the height starting position after the right shift and the second preset value is determined as the minimum boundary value.

Optionally, the determining module 13 is specifically configured to: shift the third preset value to the left by the tenth preset value; determine the sum of the height starting position, the first height, and the third preset value after the left shift, and the third preset value after the left shift; The difference between ten preset values; the difference is shifted to the third preset value to the right; the minimum value of the difference between the second height and the right shift is determined as the maximum boundary value.

Optionally, the processing module 14 is specifically configured to: acquire decoding reference information; decode the image to be decoded according to the decoding reference information to obtain the target original image.

Optionally, the decoding information includes a switch identification of the filter processing tool; the processing module 14 is specifically configured to: judge that the filter processing tool is turned on according to the switch identification;

Optionally, the fourth preset number of CTU rows above the image to be decoded includes the first CTU row and the second CTU row, and the fifth preset number of CTU rows below the image to be decoded includes the third CTU row; Module 14 is specifically used to: respectively decode the first CTU line, the second CTU line and the third CTU line to obtain the first original pixel set, the second original pixel set and the third original pixel set; In-loop filtering is performed on the first original pixel set and the second original pixel set to obtain the first processed pixel set and the second processed pixel set; according to the first processed pixel set, the second processed pixel set and the third original pixel set, determine Decode reference information.

Optionally, the first processing pixel set and the second processing pixel set respectively include N rows of pixel data, and N is an integer greater than or equal to 2; the processing module 14 is specifically configured to: use The last M rows of pixel data are replaced with the last M rows of pixel data in the N rows of pixel data in the first processing pixel set to obtain the third processing pixel set corresponding to the first processing pixel set; M is a positive integer less than N; the second processing The pixel set, the third processed pixel set and the third original pixel set are determined as decoding reference information.

Optionally, the sixth preset number of CTU rows above the image to be decoded includes the first CTU row, and the seventh preset number of CTU rows below the image to be decoded includes the second CTU row; the processing module 14 is specifically used to : Decode the first CTU line and the second CTU line respectively to obtain the first original pixel set and the second original pixel set; the first original pixel set includes N lines of original pixel data; adopt N lines of original pixels in the first original pixel set The last M lines of original pixel data of the data are replaced with the first M lines of original pixel data of the N lines of original pixel data in the first original pixel set to obtain a second pixel set; the second pixel set and the second original pixel set are determined as decoding references information.

Optionally, the processing module 14 is specifically configured to: decode the image to be decoded according to the decoding reference information to obtain a reconstructed pixel value; store the reconstructed pixel value in a preset area to obtain a reconstructed image; the preset area and the image to be decoded The same size; determine the target original image in the reconstructed image according to the starting position, the first height, and the first width.

Optionally, the processing module 14 is specifically configured to: determine the difference between the initial height position and the left shift value after the third preset value is shifted to the left by the minimum boundary value as the top offset; determine the maximum boundary value and the first boundary value The sum of the nine preset values, the value after the third preset value is shifted to the left, and the minimum value of the difference between the second height and the initial position of the height and the first height, is determined as the bottom offset; The width starting position of the starting position is determined as the left offset; the difference between the second width and the sum of the left offset and the first width is determined as the right offset; the top offset, The bottom offset, the left offset and the right offset indicate the image to be deleted in the reconstructed image; the image to be deleted is deleted in the reconstructed image to obtain the target original image.

Optionally, the determining module 13 is specifically configured to: analyze and process the image parameter set included in the encoding information to obtain the location area and number of image slices included in the encoded image; if the number of slices is equal to a preset value, Then according to the starting position, the first height and the second height, determine the area to be decoded; if the number of slices is not equal to the preset value, delete the image slices that do not overlap with the marked area in the coded image, and according to the starting position , the first height and the second height, the area to be decoded is determined in the coded image after the image slice is deleted.

Optionally, the operation instruction is an instruction corresponding to the sliding operation; the determination module 13 is specifically used to: detect the pause time corresponding to the sliding operation; when the pause time exceeds a preset value, determine the marked area in the current display interface; according to the marked area and The encoding information is used to determine the image to be decoded corresponding to the marked area in the encoded image.

FIG. 15 is a schematic hardware diagram of a terminal device provided by an embodiment of the present application. As shown in FIG. 15 , the terminal device 20 may include: a transceiver 21 , a memory 22 , and a processor 23 .

The transceiver 21 may include: a transmitter and/or a receiver. The transmitter may also be called a transmitter, a transmitter, a sending port, or a sending interface, and similar descriptions, and the receiver may also be called a receiver, a receiver, a receiving port, or a receiving interface, or similar descriptions. Exemplarily, parts of the transceiver 21 , the memory 22 , and the processor 23 are connected to each other through a bus 404 .

The memory 22 is used to store computer-executable instructions;

The processor 23 is configured to execute the computer-executed instructions stored in the memory 22, so that the processor 23 executes the above image processing method.

An embodiment of the present application provides a computer-readable storage medium, wherein computer-executable instructions are stored in the computer-readable storage medium, and when the computer-executable instructions are executed by a processor, the foregoing image processing method is realized.

An embodiment of the present application further provides a computer program product, including a computer program, and when the computer program is executed by a processor, the above image processing method can be realized.

An embodiment of the present application further provides a computer program, which can implement the above image processing method when the computer program is executed by a processor.

All or part of the steps for implementing the above method embodiments can be completed by program instructions and related hardware. The aforementioned program can be stored in a readable memory. When the program is executed, it executes the steps comprising the above-mentioned method embodiments; and the aforementioned memory (storage medium) includes: read-only memory (read-only memory, ROM for short), RAM, flash memory, hard disk, solid-state hard disk, Magnetic tape, floppy disk, optical disc, and any combination thereof.

Embodiments of the present application are described with reference to flowcharts and/or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each procedure and/or block in the flowchart and/or block diagram, and a combination of procedures and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processing unit of other programmable data processing equipment to produce a machine such that the instructions executed by the processing unit of the computer or other programmable data processing equipment produce a An apparatus for realizing the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions The device realizes the function specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby The instructions provide steps for implementing the functions specified in the flow chart or blocks of the flowchart and/or the block or blocks of the block diagrams.

Apparently, those skilled in the art can make various changes and modifications to the embodiments of the present application without departing from the spirit and scope of the present application. In this way, if the modifications and variations of the embodiments of the present application fall within the scope of the claims of the present application and equivalent technologies, the present application also intends to include these modifications and variations.

In this application, the term "include" and its variants may mean non-limiting inclusion; the term "or" and its variants may mean "and/or". The terms "first", "second", etc. in this application are used to distinguish similar objects, and not necessarily used to describe a specific order or sequence. In the present application, "plurality" means two or more. "And/or" describes the association relationship of associated objects, indicating that there may be three types of relationships, for example, A and/or B may indicate: A exists alone, A and B exist simultaneously, and B exists independently. The character "/" generally indicates that the contextual objects are an "or" relationship.

Other embodiments of the present application will be readily apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any modification, use or adaptation of the application, these modifications, uses or adaptations follow the general principles of the application and include common knowledge or conventional technical means in the technical field not disclosed in the application . The specification and examples are to be considered exemplary only, with a true scope and spirit of the application indicated by the following claims.

It should be understood that the present application is not limited to the precise constructions which have been described above and shown in the accompanying drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims

An image processing method, characterized in that it is applied to a terminal device, the method comprising:

receiving the encoding information sent by the server; the encoding information includes an encoding image corresponding to the original image;

Obtain operation instructions;

According to the operation instruction and the encoding information, determine the image to be decoded in the encoded image; the image to be decoded is an encoded image corresponding to the target original image in the marked area in the original image;

Decoding the image to be decoded to obtain the target original image, and displaying the target original image in the marked area.
The method according to claim 1, wherein the operation instruction includes the starting position, the first height and the first width of the marked area, and the encoded information includes a sequence parameter set;

The determining the image to be decoded in the encoded image according to the operation instruction and the encoding information includes:

judging whether the initial position, the first height and the first width are all preset values;

If so, analyze and process the sequence parameter set to obtain decoding information, the decoding information includes the second height and the second width of the coded image; and according to the starting position, the first height, The first width, the second height and the second width determine the image to be decoded in the coded image.
The method according to claim 2, wherein, according to the starting position, the first height, the first width, the second height and the second width, in the encoding The image to be decoded is determined in the image, including:

judging whether the marked area is located in the coded image according to the starting position, the first height, the first width, the second height, and the second width;

If yes, then determine the image to be decoded in the coded image according to the starting position, the first height and the second height.
The method according to claim 3, characterized in that, according to the initial position, the first height, the first width, the second height and the second width, it is judged that the calibration Whether a region is located in the coded image, including:

If the starting position is located in the encoded image, the first height is smaller than the second height, and the first width is smaller than the second width, the marked area is located in the encoded image;

Otherwise, the marked area is not in the coded picture.
The method according to any one of claims 2-4, wherein, according to the starting position, the first height and the second height, the to-be-decoded images, including:

determining an area to be decoded according to the starting position, the first height, and the second height;

Determining an image in the area to be decoded in the coded image as the image to be decoded.
The method according to claim 5, wherein the starting position comprises a height starting position;

The determining the area to be decoded according to the starting position, the first height and the second height includes:

Determine the minimum boundary value according to the height starting position;

determining a maximum boundary value according to the first height, the second height, and the height starting position;

Determining the CTU lines from the minimum boundary value to the maximum boundary value in the coded image as the image to be decoded.
The method according to claim 6, wherein said determining the minimum boundary value according to the height starting position comprises:

Move the starting position of the height to the right by a third preset value; the third preset value is a preset value corresponding to the encoding method corresponding to the encoded image;

The maximum value of the first preset value and the difference between the height starting position after the right shift and the second preset value is determined as the minimum boundary value.
The method according to claim 7, wherein said determining a maximum boundary value according to said first height, said second height and said height starting position comprises:

shifting the third preset value to the left by a tenth preset value;

Determine the difference between the height starting position, the first height, the sum of the third preset value after the left shift, and the tenth preset value;

shifting the difference to the right by the third preset value;

The minimum value of the difference between the second height and the right-shifted value is determined as the maximum boundary value.
The method according to any one of claims 1-8, wherein the decoding the image to be decoded to obtain the target original image comprises:

Obtain decoding reference information;

Decoding the image to be decoded according to the decoding reference information to obtain the target original image.
The method according to claim 9, wherein the decoding information includes a switch identification of a filtering processing tool;

The acquisition of decoding reference information includes:

According to the switch identification, it is judged whether the filter processing tool is turned on;

If so, the fourth preset value of CTU lines located above the image to be decoded and the fifth preset value of CTU lines located below the image to be decoded in the coded image are determined as the first reference decoded image , and determine decoding reference information according to the first reference decoded image;

Otherwise, the sixth preset number of CTU lines located above the image to be decoded and the seventh preset number of CTU lines located below the image to be decoded in the coded image are determined as the second reference decoded image , and determine decoding reference information according to the second reference decoded image.
The method according to claim 10, wherein the fourth preset number of CTU lines above the image to be decoded includes the first CTU line and the second CTU line, and the fifth preset number of CTU lines is located above the image to be decoded. The CTU row below the image to be decoded includes the third CTU row;

The determining the decoding reference information according to the first reference decoded image includes:

Decoding the first CTU row, the second CTU row, and the third CTU row respectively to obtain a first original pixel set, a second original pixel set, and a third original pixel set;

performing in-loop filtering processing on the first original pixel set and the second original pixel set by using the filtering processing tool to obtain a first processed pixel set and a second processed pixel set;

Decoding reference information is determined according to the first set of processed pixels, the second set of processed pixels, and the third set of original pixels.
The method according to claim 11, wherein the first processing pixel set and the second processing pixel set respectively include N rows of pixel data, and N is an integer greater than or equal to 2;

The determining the decoding reference information according to the first processed pixel set, the second processed pixel set and the third original pixel set includes:

Using the last M rows of pixel data in the N rows of pixel data in the second processing pixel set to replace the last M rows of pixel data in the N rows of pixel data in the first processing pixel set, to obtain the pixel data corresponding to the first processing pixel set The third processing pixel set; M is a positive integer less than N;

The second processed pixel set, the third processed pixel set, and the third original pixel set are determined as the decoding reference information.
The method according to claim 10, wherein the sixth preset value of the CTU lines above the image to be decoded includes the first CTU line, and the seventh preset value of CTU lines located above the image to be decoded The CTU row below the image includes the second CTU row;

The determining the decoding reference information according to the second reference decoded image includes:

Decoding the first CTU line and the second CTU line respectively to obtain a first original pixel set and a second original pixel set; the first original pixel set includes N rows of original pixel data;

Using the last M lines of original pixel data of N lines of original pixel data in the first original pixel set to replace the first M lines of original pixel data of N lines of original pixel data in the first original pixel set to obtain a second pixel set;

The second pixel set and the second original pixel set are determined as decoding reference information.
The method according to any one of claims 9-13, wherein decoding the image to be decoded according to the decoding reference information to obtain the target original image includes:

Decoding the image to be decoded according to the decoding reference information to obtain reconstructed pixel values;

and storing the reconstructed pixel values in a preset area to obtain a reconstructed image; the size of the preset area is the same as that of the image to be decoded;

The target original image is determined in the reconstructed image according to the starting position, the first height, and the first width.
The method according to claim 14, wherein the determining the target original image in the reconstructed image according to the initial position, the first height, and the first width includes:

The difference between the initial position of the height and the left shift value after the third preset value is shifted to the left by the minimum boundary value is determined as the top offset;

Determine the sum of the maximum boundary value and the ninth preset value, shift the third preset value to the left after the sum, and the difference between the second height and the initial position of the height and the first height The minimum value of is determined as the bottom offset;

Determining the width starting position of the starting position as the left offset;

Determine the difference between the second width and the sum of the left offset and the first width as the right offset; the top offset, bottom offset, left offset The amount and the right offset indicate the image to be deleted in the reconstructed image;

Deleting the image to be deleted from the reconstructed image to obtain a target original image.
The method according to claim 5, wherein the determining the area to be decoded according to the starting position, the first height and the second height comprises:

Analyzing the image parameter set included in the encoding information to obtain the location area and number of image slices included in the encoded image;

If the number of slices is equal to a preset value, then determine the region to be decoded according to the starting position, the first height and the second height;

If the number of slices is not equal to the preset value, delete image slices that do not overlap with the marked area in the coded image, and according to the starting position, the first height and the second height, The area to be decoded is determined in the coded image after the image slice is deleted.
The method according to any one of claims 1-16, wherein the operation instruction is an instruction corresponding to a sliding operation;

The determining the image to be decoded in the encoded image according to the operation instruction and the encoding information includes:

Detecting the pause time corresponding to the sliding operation; when the pause time exceeds a preset value, determine the calibration area in the current display interface;

An image to be decoded corresponding to the marked area is determined in the coded image according to the marked area and the encoding information.
An image processing device, characterized in that it is applied to a terminal device, and the device includes:

The receiving module is used to receive the encoded information sent by the server; the encoded information includes the encoded image corresponding to the original image;

an acquisition module, configured to acquire operation instructions;

A determining module, configured to determine an image to be decoded in the encoded image according to the operation instruction and the encoding information; the image to be decoded is an encoded image corresponding to the target original image in the marked area in the original image;

A processing module, configured to decode the image to be decoded to obtain the target original image;

A display module, configured to display the target original image in the marked area.
A terminal device, characterized by comprising: a processor, and a memory communicatively connected to the processor;

the memory stores computer-executable instructions;

The processor executes the computer-implemented instructions stored in the memory to implement the method according to any one of claims 1-17.
A computer-readable storage medium, characterized in that, computer-executable instructions are stored in the computer-readable storage medium, and the computer-executable instructions are used to implement any one of claims 1-17 when executed by a processor Methods.
A computer program product, characterized by comprising: a computer program; when the computer program is executed by a processor, the method according to any one of claims 1-17 is implemented.
A computer program, characterized in that, when the computer program is executed by a processor, the method according to any one of claims 1-17 is implemented.