CN114007076A

CN114007076A - Image processing method, apparatus, device, storage medium, and program product

Info

Publication number: CN114007076A
Application number: CN202111267729.1A
Authority: CN
Inventors: 黎凌宇; 张莉; 王悦
Original assignee: Beijing ByteDance Technology Co Ltd
Current assignee: Beijing Douyin Information Service Co Ltd
Priority date: 2021-10-29
Filing date: 2021-10-29
Publication date: 2022-02-01
Also published as: WO2023071584A1

Abstract

An embodiment of the application provides an image processing method, an image processing device, an image processing apparatus, a storage medium and a program product, wherein the method comprises the following steps: receiving coding information sent by a server; the coding information comprises a coding image corresponding to the original image; acquiring an operation instruction; determining an image to be decoded in the coded image according to the operation instruction and the coding information; the image to be decoded is a coded image corresponding to a target original image in a specified area in the original image; and decoding the image to be decoded to obtain a target original image, and displaying the target original image in the calibration area. The image processing method, the image processing device, the image processing equipment, the image processing storage medium and the image processing program product are used for saving the memory space of the terminal equipment and avoiding the jamming or breakdown of the information application program and the like.

Description

Image processing method, apparatus, device, storage medium, and program product

Technical Field

The present application relates to the field of image processing technologies, and in particular, to an image processing method, an image processing apparatus, an image processing device, a storage medium, and a program product.

Background

Currently, the information application can include long pictures with rich graphic contents, and the terminal equipment can show the long pictures included in the information application to the user.

In the related art, the terminal device corresponding to the information application program may send the encoded information corresponding to the long image to the terminal device, and when the user browses the information application program through the terminal device, the terminal device may decode the encoded information to obtain the long image, and then display the long image to the user.

In practical applications, a user usually only browses a part of an image (e.g., a beginning part of an image) in a long image, so that if a terminal device decodes encoded information to obtain the long image, the terminal device wastes more memory space, and an information application program is stuck or crashed.

Disclosure of Invention

The embodiment of the application provides an image processing method, an image processing device, image processing equipment, a storage medium and a program product, which are used for solving the problems that more memory space is wasted by terminal equipment, and further an information application program is jammed or crashed.

In a first aspect, an embodiment of the present application provides an image processing method, which is applied to a terminal device, and the method includes:

receiving coding information sent by a server; the coding information comprises a coding image corresponding to the original image;

acquiring an operation instruction;

determining an image to be decoded in the coded image according to the operation instruction and the coding information; the image to be decoded is a coded image corresponding to a target original image in a specified area in the original image;

and decoding the image to be decoded to obtain a target original image, and displaying the target original image in the calibration area.

Optionally, the operation instruction includes a start position, a first height, and a first width of the calibration region, and the encoding information includes a sequence parameter set; determining an image to be decoded in the coded image according to the operation instruction and the coding information, wherein the method comprises the following steps: judging whether the initial position, the first height and the first width are preset values;

if so, analyzing the sequence parameter set to obtain decoding information, wherein the decoding information comprises a second height and a second width of the coded image; and determining an image to be decoded in the encoded image according to the starting position, the first height, the first width, the second height and the second width.

Optionally, determining an image to be decoded in the encoded image according to the start position, the first height, the first width, the second height, and the second width includes: judging whether the calibration area is positioned in the coded image or not according to the initial position, the first height, the first width, the second height and the second width;

and if so, determining the image to be decoded in the coded image according to the initial position, the first height and the second height.

Optionally, judging whether the calibration region is located in the encoded image according to the start position, the first height, the first width, the second height, and the second width includes:

if the initial position is located in the coded image, the first height is smaller than the second height, and the first width is smaller than the second width, the calibration area is located in the coded image;

otherwise, the calibration region is not in the encoded image.

Optionally, determining an image to be decoded in the encoded image according to the start position, the first height and the second height includes: determining a region to be decoded according to the initial position, the first height and the second height; and determining the image in the region to be decoded in the coded image as the image to be decoded.

Optionally, the starting position comprises a height starting position; determining a region to be decoded according to the starting position, the first height and the second height, including: determining a minimum boundary value according to the height initial position; determining a maximum boundary value according to the first height, the second height and the height initial position; and determining the CTU row from the minimum boundary value to the maximum boundary value in the coded image as an image to be decoded.

Optionally, determining a minimum boundary value from the height starting position comprises: shifting the height starting position to the right by a third preset value; the third preset value is a preset value corresponding to a coding mode corresponding to the coded image; and determining the maximum value of the first preset value and the difference value between the height starting position after the right shift and the second preset value as the minimum boundary value.

Optionally, determining a maximum boundary value according to the first height, the second height and the height starting position comprises: shifting the third preset value to the left by a tenth preset value; determining the difference value between the sum of the height starting position, the first height and the third preset value after left movement and the tenth preset value; shifting the difference value to the right by a third preset value; the minimum value of the second height and the difference value after the right shift is determined as the maximum boundary value.

Optionally, decoding the image to be decoded to obtain the target original image, including: acquiring decoding reference information; and decoding the image to be decoded according to the decoding reference information to obtain a target original image.

Optionally, the decoding information includes a switch identifier of the filtering processing tool; acquiring decoding reference information, including: judging whether the filtering processing tool is started or not according to the switch identifier;

if so, determining a fourth preset value of CTU rows above the image to be decoded and a fifth preset value of CTU rows below the image to be decoded in the coded image as a first reference decoded image, and determining decoding reference information according to the first reference decoded image;

otherwise, determining the sixth preset value of CTU lines above the image to be decoded and the seventh preset value of CTU lines below the image to be decoded in the coded image as second reference decoded images, and determining decoding reference information according to the second reference decoded images.

Optionally, the CTU rows above the image to be decoded include a first CTU row and a second CTU row, and the CTU rows below the image to be decoded include a third CTU row; determining decoding reference information from a first reference decoded picture, comprising: respectively decoding the first CTU line, the second CTU line and the third CTU line to obtain a first original pixel set, a second original pixel set and a third original pixel set; performing in-loop filtering processing on the first original pixel set and the second original pixel set respectively by using a filtering processing tool to obtain a first processing pixel set and a second processing pixel set; and determining decoding reference information according to the first processing pixel set, the second processing pixel set and the third original pixel set.

Optionally, the first processing pixel set and the second processing pixel set respectively include N lines of pixel data, where N is an integer greater than or equal to 2; determining decoding reference information according to the first processing pixel set, the second processing pixel set and the third original pixel set, wherein the determining comprises: replacing the last M rows of pixel data in the N rows of pixel data in the first processing pixel set by the last M rows of pixel data in the N rows of pixel data in the second processing pixel set to obtain a third processing pixel set corresponding to the first processing pixel set; m is a positive integer less than N; and determining the second processing pixel set, the third processing pixel set and the third original pixel set as decoding reference information.

Optionally, the sixth preset value CTU row above the image to be decoded includes a first CTU row, and the seventh preset value CTU row below the image to be decoded includes a second CTU row; determining decoding reference information according to a second reference decoded picture, including: respectively decoding the first CTU line and the second CTU line to obtain a first original pixel set and a second original pixel set; the first original pixel set comprises N lines of original pixel data; replacing the first M rows of original pixel data of the N rows of original pixel data in the first original pixel set by the last M rows of original pixel data of the N rows of original pixel data in the first original pixel set to obtain a second pixel set; and determining the second pixel set and the second original pixel set as decoding reference information.

Optionally, decoding the image to be decoded according to the decoding reference information to obtain the target original image, including: decoding the image to be decoded according to the decoding reference information to obtain a reconstructed pixel value; storing the reconstructed pixel value to a preset area to obtain a reconstructed image; the preset area and the image to be decoded have the same size; and determining a target original image in the reconstructed image according to the starting position, the first height and the first width.

Optionally, determining the target original image in the reconstructed image according to the starting position, the first height, and the first width includes: determining a difference value between the height initial position and a left shift value obtained by left-shifting the third preset value by a minimum boundary value as a top offset; determining a sum of the maximum boundary value and a ninth preset value, and determining a minimum value of a value obtained after the third preset value is shifted to the left by the sum and a difference value between the second height and the height initial position and the first height as a bottom offset; determining the width starting position of the starting position as the left offset; determining the difference between the second width and the sum of the left offset and the first width as a right offset; the top offset, the bottom offset, the left offset, and the right offset indicate images to be deleted in the reconstructed image; and deleting the image to be deleted in the reconstructed image to obtain the target original image.

Optionally, determining the region to be decoded according to the starting position, the first height and the second height includes: analyzing the image parameter set included in the coding information to obtain the position area and the fragment number of the image fragments included in the coded image; if the number of the fragments is equal to a preset value, determining a region to be decoded according to the initial position, the first height and the second height; if the number of the fragments is not equal to the preset value, deleting the image fragments which are not overlapped with the calibration area in the coded image, and determining the area to be decoded in the coded image after deleting the image fragments according to the initial position, the first height and the second height.

Optionally, the operation instruction is an instruction corresponding to a sliding operation; determining an image to be decoded in the coded image according to the operation instruction and the coding information, wherein the method comprises the following steps: detecting the pause time corresponding to the sliding operation; when the pause time exceeds a preset value, determining a calibration area in the current display interface; and determining an image to be decoded corresponding to the calibration area in the coded image according to the calibration area and the coding information.

In a second aspect, an embodiment of the present application provides an image processing apparatus, which is applied to a terminal device, and includes:

the receiving module is used for receiving the coding information sent by the server; the coding information comprises a coding image corresponding to the original image;

the acquisition module is used for acquiring an operation instruction;

the determining module is used for determining an image to be decoded in the coded image according to the operation instruction and the coding information; the image to be decoded is a coded image corresponding to a target original image in a specified area in the original image;

the processing module is used for decoding the image to be decoded to obtain a target original image;

and the display module is used for displaying the target original image in the calibration area.

Optionally, the operation instruction includes a start position, a first height, and a first width of the calibration region, and the encoding information includes a sequence parameter set; the determination module is specifically configured to: judging whether the initial position, the first height and the first width are preset values;

Optionally, the determining module is specifically configured to: judging whether the calibration area is positioned in the coded image or not according to the initial position, the first height, the first width, the second height and the second width;

Optionally, the determining module is specifically configured to:

otherwise, the calibration region is not in the encoded image.

Optionally, the determining module is specifically configured to: determining a region to be decoded according to the initial position, the first height and the second height; and determining the image in the region to be decoded in the coded image as the image to be decoded.

Optionally, the starting position comprises a height starting position; the determination module is specifically configured to: determining a minimum boundary value according to the height initial position; determining a maximum boundary value according to the first height, the second height and the height initial position; and determining the CTU row from the minimum boundary value to the maximum boundary value in the coded image as an image to be decoded.

Optionally, the determining module is specifically configured to: shifting the height starting position to the right by a third preset value; the third preset value is a preset value corresponding to a coding mode corresponding to the coded image; and determining the maximum value of the first preset value and the difference value between the height starting position after the right shift and the second preset value as the minimum boundary value.

Optionally, the determining module is specifically configured to: shifting the third preset value to the left by a tenth preset value; determining the difference value between the sum of the height starting position, the first height and the third preset value after left movement and the tenth preset value; shifting the difference value to the right by a third preset value; the minimum value of the second height and the difference value after the right shift is determined as the maximum boundary value.

Optionally, the processing module is specifically configured to: acquiring decoding reference information; and decoding the image to be decoded according to the decoding reference information to obtain a target original image.

Optionally, the decoding information includes a switch identifier of the filtering processing tool; the processing module is specifically configured to: judging whether the filtering processing tool is started or not according to the switch identifier;

Optionally, the CTU rows above the image to be decoded include a first CTU row and a second CTU row, and the CTU rows below the image to be decoded include a third CTU row; the processing module is specifically configured to: respectively decoding the first CTU line, the second CTU line and the third CTU line to obtain a first original pixel set, a second original pixel set and a third original pixel set; performing in-loop filtering processing on the first original pixel set and the second original pixel set respectively by using a filtering processing tool to obtain a first processing pixel set and a second processing pixel set; and determining decoding reference information according to the first processing pixel set, the second processing pixel set and the third original pixel set.

Optionally, the first processing pixel set and the second processing pixel set respectively include N lines of pixel data, where N is an integer greater than or equal to 2; the processing module is specifically configured to: replacing the last M rows of pixel data in the N rows of pixel data in the first processing pixel set by the last M rows of pixel data in the N rows of pixel data in the second processing pixel set to obtain a third processing pixel set corresponding to the first processing pixel set; m is a positive integer less than N; and determining the second processing pixel set, the third processing pixel set and the third original pixel set as decoding reference information.

Optionally, the sixth preset value CTU row above the image to be decoded includes a first CTU row, and the seventh preset value CTU row below the image to be decoded includes a second CTU row; the processing module is specifically configured to: respectively decoding the first CTU line and the second CTU line to obtain a first original pixel set and a second original pixel set; the first original pixel set comprises N lines of original pixel data; replacing the first M rows of original pixel data of the N rows of original pixel data in the first original pixel set by the last M rows of original pixel data of the N rows of original pixel data in the first original pixel set to obtain a second pixel set; and determining the second pixel set and the second original pixel set as decoding reference information.

Optionally, the processing module is specifically configured to: decoding the image to be decoded according to the decoding reference information to obtain a reconstructed pixel value; storing the reconstructed pixel value to a preset area to obtain a reconstructed image; the preset area and the image to be decoded have the same size; and determining a target original image in the reconstructed image according to the starting position, the first height and the first width.

Optionally, the processing module is specifically configured to: determining a difference value between the height initial position and a left shift value obtained by left-shifting the third preset value by a minimum boundary value as a top offset; determining a sum of the maximum boundary value and a ninth preset value, and determining a minimum value of a value obtained after the third preset value is shifted to the left by the sum and a difference value between the second height and the height initial position and the first height as a bottom offset; determining the width starting position of the starting position as the left offset; determining the difference between the second width and the sum of the left offset and the first width as a right offset; the top offset, the bottom offset, the left offset, and the right offset indicate images to be deleted in the reconstructed image; and deleting the image to be deleted in the reconstructed image to obtain the target original image.

Optionally, the determining module is specifically configured to: analyzing the image parameter set included in the coding information to obtain the position area and the fragment number of the image fragments included in the coded image; if the number of the fragments is equal to a preset value, determining a region to be decoded according to the initial position, the first height and the second height; if the number of the fragments is not equal to the preset value, deleting the image fragments which are not overlapped with the calibration area in the coded image, and determining the area to be decoded in the coded image after deleting the image fragments according to the initial position, the first height and the second height.

Optionally, the operation instruction is an instruction corresponding to a sliding operation; the determination module is specifically configured to: detecting the pause time corresponding to the sliding operation; when the pause time exceeds a preset value, determining a calibration area in the current display interface; and determining an image to be decoded corresponding to the calibration area in the coded image according to the calibration area and the coding information.

In a third aspect, an embodiment of the present application further provides a terminal device, including: a processor, and a memory communicatively coupled to the processor;

the memory stores computer-executable instructions;

the processor executes computer-executable instructions stored by the memory to implement the method of any one of the first aspect.

In a fourth aspect, embodiments of the present application provide a computer-readable storage medium, in which computer-executable instructions are stored, and when the computer-executable instructions are executed by a processor, the method of any one of the first aspect is implemented.

In a fifth aspect, an embodiment of the present application provides a computer program product, including: a computer program; the computer program, when executed by a processor, implements the method of any of the first aspects.

An embodiment of the application provides an image processing method, an image processing apparatus, a storage medium and a program product, wherein the image processing method comprises the following steps: receiving coding information sent by a server; the coding information comprises a coding image corresponding to the original image; acquiring an operation instruction; determining an image to be decoded in the coded image according to the operation instruction and the coding information; the image to be decoded is a coded image corresponding to a target original image in a specified area in the original image; and decoding the image to be decoded to obtain a target original image, and displaying the target original image in the calibration area. In the method, the image to be decoded is determined in the coded image according to the operation instruction and the coded information, the image to be decoded is decoded to obtain the target original image, and the target original image is displayed in the calibration area, so that the terminal equipment can decode only part of the coded image (namely the image to be decoded) in the coded image to obtain the target original image which the user wants to browse, the memory space of the terminal equipment is saved, and the information application program is prevented from being blocked or crashed and the like.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.

Fig. 1 is a schematic view of a scenario provided in an embodiment of the present application;

fig. 2 is a flowchart of an image processing method according to an embodiment of the present application;

FIG. 3 is a flowchart illustrating a method for determining an image to be decoded according to an embodiment of the present disclosure;

FIG. 4 is a diagram illustrating an embodiment of determining an image to be decoded;

FIG. 5 is another diagram for determining an image to be decoded according to an embodiment of the present disclosure;

FIG. 6 is a flowchart of a method for obtaining an original target image according to an embodiment of the present disclosure;

fig. 7 is a schematic diagram of determining decoding reference information according to an embodiment of the present application;

fig. 8 is a schematic diagram of another method for determining decoding reference information according to an embodiment of the present application;

fig. 9 is a schematic diagram of obtaining a reconstructed image according to an embodiment of the present application;

FIG. 10 is a schematic diagram illustrating an example of determining an original target image in a reconstructed image according to an embodiment of the present disclosure;

fig. 11 is a flowchart of a second method for determining an image to be decoded according to an embodiment of the present application;

FIG. 12 is a schematic diagram of an encoded image including multiple tiles according to an embodiment of the present disclosure;

fig. 13 is a schematic diagram of an encoded image including multiple slices according to an embodiment of the present application;

fig. 14 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application;

fig. 15 is a hardware schematic diagram of a terminal device according to an embodiment of the present application.

With the above figures, there are shown specific embodiments of the present application, which will be described in more detail below. These drawings and written description are not intended to limit the scope of the inventive concepts in any manner, but rather to illustrate the inventive concepts to those skilled in the art by reference to specific embodiments.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

The prior art will first be briefly described.

In the prior art, when a user browses a long image (or a high-definition video) included in an information application program through a terminal device, the terminal device decodes encoded information corresponding to the long image (or the high-definition video) issued by the terminal device to obtain a completed long image (or the high-definition video), and displays the completed long image (or the high-definition video).

In practice, research and research on browsing a long image (or a high-definition video) by a user finds that the user usually browses only the beginning part or the middle part of the long image (or the high-definition video) with a high probability, and therefore if a method for decoding encoded information by a terminal device to obtain a finished long image (or a high-definition video) in the prior art is adopted, the terminal device wastes a large amount of memory space, and an information application program is blocked or crashed.

In the present application, in order to save memory space of the terminal device and further avoid the information application program from being stuck or broken down, the inventor thinks that: in the browsing process of the user, the terminal device only decodes the image to be coded to obtain the partial image which the user wants to browse in the long image (or high-definition video), so that the memory space of the terminal device is saved, and the information application program is prevented from being blocked or crashed.

An application scenario of the image processing method provided in the embodiment of the present application is described below with reference to fig. 1.

Fig. 1 is a schematic view of a scene according to an embodiment of the present application. As shown in fig. 1, includes: a terminal device and at least one terminal device.

The terminal equipment can be installed with the information application program, and the terminal equipment is the terminal equipment corresponding to the information application program.

In order to save the transmission traffic of the terminal device sending the original image to the terminal device, the terminal device usually encodes the visible original image to obtain the encoded information corresponding to the original image, and sends the encoded information to the terminal device.

And after receiving the coding information, the terminal equipment determines an image to be decoded according to the operation instruction and the coding information, decodes the image to be decoded to obtain a target original image, and displays the target original image. The target original image is a partial image which is wanted to be browsed by a user in the original image.

In the process, the terminal equipment determines the image to be decoded according to the operation instruction and the coding information, and then decodes the image to be decoded to obtain the target original image, so that the memory space of the terminal equipment can be saved, and the information application program is prevented from being blocked or broken down and the like.

The following describes the technical solutions of the present application and how to solve the above technical problems with specific embodiments. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present application will be described below with reference to the accompanying drawings.

Fig. 2 is a flowchart of an image processing method according to an embodiment of the present application. As shown in fig. 2, the method includes:

s201, receiving coding information sent by a server; the coding information includes a coded image corresponding to the original image.

Optionally, the execution main body of the embodiment of the present application may be a terminal device, or may also be an image processing apparatus provided in the terminal device, where the image processing apparatus may be implemented by a combination of software and/or hardware.

The terminal device may be a wireless terminal or a wired terminal. A wireless terminal may refer to a device that provides voice and/or other traffic data connectivity to a user, a handheld device having wireless connection capability, or other processing device connected to a wireless modem. A wireless terminal, which may be a mobile terminal such as a mobile phone (or referred to as a "cellular" phone or as a smart phone) and a computer having a mobile terminal, may communicate with one or more core Network devices via a Radio Access Network (RAN). For example, the Wireless terminal may also be a Personal Communication Service (PCS) phone, a cordless phone, a Session Initiation Protocol (SIP) phone, a Wireless Local Loop (WLL) station, a Personal Digital Assistant (PDA), or the like. The wireless Terminal may also be referred to as a system, a User Terminal (User Terminal), or a User Device or User Equipment (User Equipment), but is not limited thereto. Optionally, the terminal device may also be a tablet computer or the like.

The coding information may include a coded picture corresponding to the original picture, a Sequence Parameter Set (SPS), and a Picture Parameter Set (PPS).

The original image is a long image visible to a user or a high-definition image. For example, the original image is a visual image in RGB or YUV format.

S202, acquiring an operation instruction.

The operation instruction may be an instruction corresponding to a sliding operation by a user.

For example, when the terminal device is a smartphone, the sliding operation may be a sliding operation of a touch screen on the terminal device by a user.

Alternatively, the sliding operation may be continuous or may be paused.

S203, determining an image to be decoded in the coded image according to the operation instruction and the coding information.

And the image to be decoded is a coded image corresponding to the target original image in the specified area in the original image.

Alternatively, the size of the image to be decoded may be the same as or different from the size of the calibration area.

And when the size of the image to be decoded is different from the size of the calibration area, the size of the image to be decoded is larger than the size of the calibration area. For example, the height of the image to be decoded is equal to the first height of the calibration area, and the width of the image to be decoded is greater than or equal to the first width of the calibration area. For example, the height of the image to be decoded is greater than the first height of the calibration area, and the width of the image to be decoded is greater than or equal to the first width of the calibration area.

Alternatively, S203 may be implemented in the following two ways (way B1 and way B2).

In the mode B1, whether the initial position, the first height and the first width of the calibration area included in the operation instruction are preset values is judged;

if so, analyzing the SPS in the coded information to obtain decoded information; the decoding information comprises a second height and a second width of the coded image; determining an image to be decoded in the coded image according to the initial position, the first height, the first width, the second height and the second width;

otherwise, carrying out full-frame decoding to obtain an original image.

The preset value may be 0 or other values, and the value of the preset value is not limited here.

Optionally, when the starting position is located within the encoded image, the first height is less than or equal to the second height, and the first width is less than or equal to the second width, the image to be decoded in the encoded image is determined.

The first height is less than or equal to the second height, and the Coding Tree Unit (CTU) row corresponding to the first height in the calibration area is included in the CTU row corresponding to the second height in the coded image.

The first width is smaller than or equal to the second width, and the CTU rows corresponding to the first width in the calibration area are contained in the CTU rows corresponding to the second width in the coded image.

Specifically, the method for performing full-frame decoding to obtain the original image may refer to the prior art, and is not described herein again.

In the mode B2, the operation command is a command corresponding to a slide operation; s203 specifically includes: detecting the pause time corresponding to the sliding operation; when the pause time exceeds a preset value, determining a calibration area in the current display interface; and determining an image to be decoded corresponding to the calibration area in the coded image according to the calibration area and the coding information.

Further, the image to be decoded corresponding to the calibration region may be determined in the encoded image according to the calibration region and the encoding information in the implementation B2 by the above-described implementation B1.

And S204, decoding the image to be decoded to obtain a target original image, and displaying the target original image in the calibration area.

Optionally, when the size of the image to be decoded is the same as the size of the calibration area, the image to be decoded is decoded to obtain a target original image. Optionally, when the size of the image to be decoded is different from the size of the calibration region, the image to be decoded is decoded to obtain a decoded image, and the decoded image is clipped according to the size of the calibration region to obtain the target original image.

The image processing method provided by the embodiment of FIG. 2 comprises the following steps: according to the operation instruction and the coding information, the image to be decoded is determined in the coding image, the image to be decoded is decoded to obtain the target original image, the target original image is displayed in the calibration area, the terminal equipment can decode only part of the coding image (namely the image to be decoded) in the coding image to obtain the target original image which the user wants to browse, so that the memory space of the terminal equipment is saved, the phenomenon that an information application program is blocked or crashed is avoided, the performance of the information application program in the terminal equipment is improved, and the application experience of the user is improved.

On the basis of the above embodiment, a specific implementation process for determining an image to be decoded in an encoded image according to a starting position, a first height, a first width, a second height, and a second width is described below with reference to fig. 3 as an example where the size of the image to be decoded is larger than the size of a calibration area.

Fig. 3 is a first flowchart of a method for determining an image to be decoded according to an embodiment of the present disclosure. As shown in fig. 3, on the basis of the above S203, the method includes:

s301, acquiring a starting position, a first height, a first width, a second height and a second width.

Alternatively, after the operation instruction is acquired, the start position, the first height, and the first width of the calibration area may be stored.

Optionally, after parsing the sequence parameter set included in the coding information to obtain a second height and a second width of the coded image, the second height and the second width may be stored.

When S301 is performed, acquiring the start position, the first height, the first width, the second height, and the second width that have been stored may be performed.

Alternatively, after determining that the starting position, the first height and the first width are not all preset values, the method can be directly carried out

S302, judging whether the calibration area is located in the coded image according to the initial position, the first height, the first width, the second height and the second width.

If so, perform S303, otherwise perform S304.

Optionally, if the starting position is located in the encoded image, the first height is smaller than the second height, and the first width is smaller than the second width, the calibration region is located in the encoded image;

otherwise, the calibration region is not in the encoded image.

S303, determining an image to be decoded in the coded image according to the initial position, the first height and the second height.

Alternatively, the specific execution method of S303 may include the following two ways (way a1 and way a 2).

Mode a1, the starting position includes a height starting position; determining a minimum boundary value according to the height initial position; determining a maximum boundary value according to the first height, the second height and the height initial position; and determining the CTU row from the minimum boundary value to the maximum boundary value in the coded image as an image to be decoded.

Alternatively, the minimum boundary value may be determined by:

shifting the height starting position to the right by a third preset value; the third preset value is a preset value corresponding to a coding mode corresponding to the coded image; and determining the minimum value of the first preset value and the difference value between the height starting position after the right shift and the second preset value as a minimum boundary value.

That is, the minimum boundary value can be obtained by the following equation 1:

Min_crop_ctu_row＝max(F1,(Y0>>C)-F2) (1)；

wherein Min _ crop _ ctu _ row is the minimum boundary value, max is the operation of taking a large value, F1 is the first preset value, Y0 is the height starting position, > > is the right shift sign, -is the minus sign, C is the third preset value, and F2 is the second preset value. Alternatively, F1 can be 0 or other positive integer, and F2 can be 2 or other positive integer.

Alternatively, the maximum boundary value may be determined as follows:

shifting the third preset value to the left by a tenth preset value; determining the difference value between the sum of the height starting position, the first height and the third preset value after left movement and the tenth preset value; shifting the difference value to the right by a third preset value; the minimum value of the second height and the difference value after the right shift is determined as the maximum boundary value.

That is, the maximum boundary value can be obtained by the following equation 2:

Max_crop_ctu_row＝min(H2,(Y0+H1+(F3<<C)-F3)>>C) (2)；

max _ crop _ ctu _ row is the maximum boundary value, min is the operation of taking a small value, H2 is the second height, H1 is the first height, F3 is the tenth preset value, and < < is the left-shifted symbol. Alternatively, F3 may be 0 or other positive integer.

It should be noted that, in the above formula 2, F3< < C indicates that the third preset value is shifted to the left by a tenth preset value, Y0+ H1+ (F3< < C) indicates the above sum, Y0+ H1+ (F3< < C) -F3 is the above difference, and (Y0+ H1+ (F3< < C) -F3) > > C indicates that the above difference is shifted to the right by the third preset value.

Fig. 4 is a schematic diagram of determining an image to be decoded according to an embodiment of the present application. As shown in fig. 4, includes: an encoded image 41, a calibration area 42, and an image to be decoded 43. The minimum boundary value above the calibration region 42 is obtained by using formula 1, and the maximum boundary value below the calibration region 42 is obtained by using formula 2. In fig. 4, the image to be decoded 43 includes: the CTU row at which the minimum boundary value is located, the CTU row at which the maximum boundary value is located, and the CTU rows between the minimum boundary value and the maximum boundary value.

Mode a2, the starting position includes a height starting position; determining the coded image meeting a first preset condition in the coded image as an image to be decoded, wherein the first preset condition is that the sum of the first height, the first preset height and the second preset height is less than the second height.

Fig. 5 provides another schematic diagram for determining an image to be decoded according to an embodiment of the present application. As shown in fig. 5, includes: an encoded image 51, a calibration area 52, and an image to be decoded 53. The first preset height is located above the first height, and the second preset height is located below the first height. In fig. 5, an image to be decoded 53 includes: and the CTU row spanned by the sum of the first preset height, the first height and the second preset height.

S304, carrying out full-frame decoding to obtain an original image.

In the embodiment of fig. 3, in the process of determining the image to be decoded in the encoded image according to the start position, the first height, and the second height, the CTU row from the minimum boundary value to the maximum boundary value in the encoded image may be determined as the image to be decoded, so that the image to be decoded includes the encoded image corresponding to the target original image, and thus the accuracy of the target original image obtained by decoding the image to be decoded is improved.

Based on the above embodiment, the following describes a specific implementation procedure of S204 with reference to fig. 6 as an example of CTU behavior in which the image to be decoded is the minimum boundary value to the maximum boundary value in the encoded image.

Fig. 6 is a flowchart of a method for obtaining an original target image according to an embodiment of the present disclosure. As shown in fig. 6, the method includes:

s601, acquiring decoding reference information.

Optionally, the decoding information includes a switch identifier of the filtering processing tool; s601 specifically includes: judging whether the filtering processing tool is started or not according to the switch identifier; if so, determining a fourth preset value of CTU rows above the image to be decoded and a fifth preset value of CTU rows below the image to be decoded in the coded image as a first reference decoded image, and determining decoding reference information according to the first reference decoded image; otherwise, determining the sixth preset value of CTU lines above the image to be decoded and the seventh preset value of CTU lines below the image to be decoded in the coded image as second reference decoded images, and determining decoding reference information according to the second reference decoded images.

Alternatively, the filter processing tool may be any at least one of an adaptive loop filter tool (adaptive loop filter), a deblocking filter tool (deblocking filter), and a sample adaptive compensation tool (SAO).

Optionally, when the filtering processing tool includes a deblocking filtering tool and a sample adaptive compensation tool, the decoding information includes a first switch identifier corresponding to the deblocking filtering tool and a second switch identifier corresponding to the sample adaptive compensation tool.

And further, judging whether the block effect removing filtering tool is started or not according to the first switch identifier, and judging whether the sample self-adaptive compensation tool is started or not according to the second switch identifier.

For example, if the first switch identifier is an on identifier, it is determined that the deblocking filter tool is on, and if the first switch identifier is an off identifier, it is determined that the deblocking filter tool is off.

Optionally, the fourth preset value may be 2, the fifth preset value may be 1, the sixth preset value may be 1, and the seventh preset value may be 1, and of course, the fourth preset value, the fifth preset value, the sixth preset value, and the seventh preset value may also be other positive integers.

Optionally, when the fourth preset value is 2 and the fifth preset value is 1, the fourth preset value CTU row above the image to be decoded includes a first CTU row and a second CTU row, and the fifth preset value CTU row below the image to be decoded includes a third CTU row; determining decoding reference information from a first reference decoded picture, comprising:

respectively decoding the first CTU line, the second CTU line and the third CTU line to obtain a first original pixel set, a second original pixel set and a third original pixel set;

performing in-loop filtering processing on the first original pixel set and the second original pixel set respectively by using a filtering processing tool to obtain a first processing pixel set and a second processing pixel set;

and determining decoding reference information according to the first processing pixel set, the second processing pixel set and the third original pixel set.

The first original pixel set is obtained by decoding a first CTU line, the second original pixel set is obtained by decoding a second CTU line, and the third original pixel set is obtained by decoding a third CTU line.

The first processing pixel set is obtained by performing in-loop filtering processing on the first original pixel set, and the second processing pixel set is obtained by performing in-loop filtering processing on the second original pixel set.

Optionally, when the filtering processing tool includes a deblocking filtering tool and a sample adaptive compensation tool, performing in-loop filtering processing on the first original pixel set and the second original pixel set respectively by using the deblocking filtering tool to obtain a first processed pixel set and a second processed pixel set, including:

performing in-loop filtering processing on the first original pixel set and the second original pixel set respectively by adopting a blocking effect removing filtering tool to obtain a first processing pixel set and a second processing pixel set; alternatively, the first and second electrodes may be,

respectively carrying out in-loop filtering processing on the first original pixel set and the second original pixel set by adopting a sample self-adaptive compensation tool to obtain a first processing pixel set and a second processing pixel set; alternatively, the first and second electrodes may be,

and performing in-loop filtering processing on the first original pixel set by adopting a blocking effect removing filtering tool to obtain a first processed pixel set, and performing in-loop filtering processing on the second original pixel set by adopting a sample self-adaptive compensation tool to obtain a second processed pixel set.

Optionally, the first processing pixel set and the second processing pixel set respectively include N lines of pixel data, where N is an integer greater than or equal to 2; determining decoding reference information according to the first processing pixel set, the second processing pixel set and the third original pixel set, wherein the determining comprises:

replacing the last M rows of pixel data in the N rows of pixel data in the first processing pixel set by the last M rows of pixel data in the N rows of pixel data in the second processing pixel set to obtain a third processing pixel set corresponding to the first processing pixel set; m is a positive integer less than N;

and determining the second processing pixel set, the third processing pixel set and the third original pixel set as decoding reference information.

Alternatively, N may be 64, 32, 16, 4, etc.

A block diagram for determining decoding reference information from a first reference decoded picture in the present application will be described with reference to fig. 7.

Fig. 7 is a schematic diagram of determining decoding reference information according to an embodiment of the present application. As shown in fig. 7, the first CTU row, the second CTU row, and the third CTU row are decoded respectively to obtain a first original pixel set, a second original pixel set, and a third original pixel set.

And respectively carrying out loop filtering on the first original pixel set and the second original pixel set to obtain a first processing pixel set and a second processing pixel set.

And obtaining a third processing pixel set according to the first processing pixel set and the second processing pixel set.

And determining the third processing pixel set, the second processing pixel set and the third original pixel set as decoding reference information.

Optionally, when the sixth preset value and the seventh preset value are both 1, the sixth preset value CTU row above the image to be decoded includes a first CTU row, and the seventh preset value CTU row below the image to be decoded includes a second CTU row;

determining decoding reference information according to a second reference decoded picture, including:

respectively decoding the first CTU line and the second CTU line to obtain a first original pixel set and a second original pixel set; the first original pixel set and the second original pixel set respectively comprise N lines of original pixel data;

replacing the first M rows of original pixel data of the N rows of original pixel data in the first original pixel set by the last M rows of original pixel data of the N rows of original pixel data in the first original pixel set to obtain a second pixel set;

and determining the second pixel set and the second original pixel set as decoding reference information.

A block diagram for determining decoding reference information from a second reference decoded picture in the present application will be described with reference to fig. 8. Fig. 8 is a schematic diagram of another method for determining decoding reference information according to an embodiment of the present application. As shown in fig. 8, the first CTU line and the second CTU line and the third CTU line are decoded respectively to obtain a first original pixel set and a second original pixel. And obtaining a second pixel set according to the first original pixel set. And determining the second pixel set and the second original pixel as decoding reference information.

And S602, decoding the image to be decoded according to the decoding reference information to obtain a target original image.

Optionally, S602 specifically includes: decoding the image to be decoded according to the decoding reference information to obtain a reconstructed pixel value; storing the reconstructed pixel value to a preset area to obtain a reconstructed image; the preset area and the image to be decoded have the same size; and determining a target original image in the reconstructed image according to the starting position, the first height and the first width.

Optionally, determining the target original image in the reconstructed image according to the starting position, the first height, and the first width includes: determining a difference value between the height initial position and a left shift value obtained by left-shifting the third preset value by a minimum boundary value as a top offset;

determining the minimum value of the value obtained by shifting the third preset value left by the sum of the maximum boundary value and the ninth preset value and the difference value between the second height and the height initial position and the first height sum as the bottom offset;

determining the width starting position of the starting position as the left offset;

determining the difference between the second width and the sum of the left offset and the first width as a right offset; the top offset, the bottom offset, the left offset, and the right offset indicate images to be deleted in the reconstructed image;

and deleting the image to be deleted in the reconstructed image to obtain the target original image.

Alternatively, the top offset amount may be determined by the following equation 3:

luma _ offset _ top ═ Y0- (Min _ crop _ ctu _ row < < C) equation 3;

where luma _ offset _ top is the top offset.

Alternatively, the bottom offset may be determined by equation 4 as follows:

luma_offset_bottom＝min(((Max_crop_ctu_row+1)<<C),H2-(Y0+H1)) (4)；

where luma _ offset _ bottom is the bottom offset and + is the plus sign.

Alternatively, the left offset amount may be determined by the following equation 5:

luma_offset_left＝X0 (5)；

here, luma _ offset _ left is a left offset amount, and X0 is a width start position.

Alternatively, the right offset amount may be determined by the following equation 6:

luma_offset_right＝src->width-(luma_offset_left+W1) (6)；

where luma _ offset _ right is the right offset and src- > width indicates the second width.

Fig. 9 is a schematic diagram of obtaining a reconstructed image according to an embodiment of the present application. On the basis of fig. 7, as shown in fig. 9, the third processing pixel set and the second processing pixel set are sequentially stored above the preset region, the third original pixel set is stored below the preset region, and the reconstructed pixel value obtained by decoding the image to be decoded according to the decoding reference information determined by the third processing pixel set, the second processing pixel set and the third original pixel set is stored in the preset region, so as to obtain the reconstructed image.

Wherein the reconstructed image comprises reconstructed pixel values.

Fig. 10 is a schematic diagram of determining an original target image in a reconstructed image according to an embodiment of the present disclosure. As shown in fig. 10, includes: a reconstructed image 1001, a target original image 1002, an image to be deleted 1003.

Where the image 1003 is a top offset, a bottom offset, a left offset, and a right offset indicating an image to be deleted in the reconstructed image. Further, after the image 1003 is deleted in the reconstructed image, the target original image 1002 is obtained.

In the application, the memory address of the target original image can be determined. For example, in fig. 10, the memory address of the top left corner of the reconstructed image is p _ src0, and the memory interval of the pixels is stride, so the memory address of the top left corner of the target original image is p _ src0+ luma _ offset _ top × stride + luma _ offset _ left.

On the basis of the above embodiments, the present application further provides another method for determining a region to be decoded according to the starting position, the first height and the second height. This will be explained with reference to fig. 11.

Fig. 11 is a flowchart of a method for determining an image to be decoded according to an embodiment of the present application. As shown in fig. 10, the method includes:

and S111, analyzing the image parameter set included in the coding information to obtain the position area and the number of the image slices included in the coded image.

Alternatively, the image slice may be an image slice (slice) or an image tile (tile).

Slice is a division of the encoded image into stripes, and tile is a division of the encoded image into rectangles.

And S112, judging whether the number of the fragments is equal to a preset value or not.

If so, S113 is performed, otherwise S114 is performed.

S113, determining a region to be decoded according to the starting position, the first height and the second height.

It should be noted that the execution process of S113 may refer to S303 in fig. 3, and is not described herein again.

S114, deleting the image slices which are not overlapped with the calibration area in the coded image, and determining the area to be decoded in the coded image after the image slices are deleted according to the initial position, the first height and the second height.

S114 is exemplarily described below with reference to fig. 12, taking the image slice as tile as an example.

Fig. 12 is a schematic diagram of an encoded image including multiple tiles according to an embodiment of the present disclosure. As shown in fig. 12, the encoded image includes a plurality of tiles. For example, multiple tiles include T1-T10. The image slices T1-T4 and T7-T10 are 8 image slices which do not overlap with the calibration region in the encoded image.

The image slices T1-T4 and T7-T10 that do not overlap with the nominal region may be deleted from the encoded image to obtain a remaining encoded image, and the region to be decoded may be determined in the remaining encoded image according to the start position, the first height, and the second height.

It should be noted that the method for determining the region to be decoded in the remaining encoded image according to the starting position, the first height and the second height is similar to that of S303 described above, and is not repeated here.

S114 is exemplarily described below with reference to fig. 13, taking the image slice as an example.

Fig. 13 is a schematic diagram of an encoded image including multiple slices according to an embodiment of the present application. As shown in fig. 12, the encoded image includes a plurality of slices. For example, the plurality of slices includes S1-S5. The image slices S1, S2, and S5 are 3 image slices that do not overlap with the calibration region in the encoded image.

The image slices S1, S2, and S5 that do not overlap with the calibration area may be deleted from the encoded image to obtain a remaining encoded image, and then the area to be decoded may be determined in the remaining encoded image according to the start position, the first height, and the second height.

According to the method design of the application, for the image to be decoded, the completed CTU line included in the image to be decoded needs to be decoded, that is, the first width in the application has no influence on the decoding speed and the memory. In comparison with the full-frame decoding (i.e., full-frame decoding of the encoded image (having the height H2)), the decoding of the image to be decoded having the heights H2/2, H2/4 and H2/8 (i.e., the first height) with reference to table 1 below results in the decoding time and the memory size, as shown in table 1 below.

TABLE 1

Resolved height	Decoding duration (millisecond, percentage)	EMS memory (megabyte, percentage)
			1080(H2)	82.679，100％	6.09，100％
540(H2/2)	51.000，61.68％	4.15，68.14％
			270(H2/4)	31.008，37.50％	3.03，49.75％
135(H2/8)	21.383，25.86％	2.46，40.39％

In table 1, the test results obtained according to the test data with the height starting position and the width starting position both being 0 and the first width being 1920 are shown as an example.

In table 1, 1080(H2) is used to indicate that full frame decoding is performed. When full frame decoding is performed, the decoding duration is 82.679 milliseconds (considered as the reference duration percentage 100%) and the memory footprint size is 6.09 megabytes (considered as the reference memory footprint percentage 100%).

For example, when decoding an image to be decoded with a first height of 540 (i.e., height H2/2), the decoding duration is 51.000 ms (61.68% of 82.679 ms and the memory size is 4.15 mbytes (68.14% of 6.09 mbytes).

Further, it can be seen from table 1 that the smaller the first height, the shorter the decoding duration is, and the smaller the occupied memory is.

It should be noted that the codec standards to which the image processing method provided in the present application can be applied include, but are not limited to, the heif codec standard or a certain codec standard.

Fig. 14 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application. As shown in fig. 14, the image processing apparatus 10 includes:

a receiving module 11, configured to receive coding information sent by a server; the coding information comprises a coding image corresponding to the original image;

an obtaining module 12, configured to obtain an operation instruction;

a determining module 13, configured to determine, according to the operation instruction and the encoding information, an image to be decoded in the encoded image; the image to be decoded is a coded image corresponding to a target original image in a specified area in the original image;

the processing module 14 is configured to decode an image to be decoded to obtain a target original image;

and the display module 15 is used for displaying the target original image in the calibration area.

The image processing apparatus provided in the embodiment of the present application can execute the image processing method, and the implementation principle and the beneficial effect thereof are similar, and are not described herein again.

Optionally, the operation instruction includes a start position, a first height, and a first width of the calibration region, and the encoding information includes a sequence parameter set; the determining module 13 is specifically configured to: judging whether the initial position, the first height and the first width are preset values;

Optionally, the determining module 13 is specifically configured to: judging whether the calibration area is positioned in the coded image or not according to the initial position, the first height, the first width, the second height and the second width;

Optionally, the determining module 13 is specifically configured to:

otherwise, the calibration region is not in the encoded image.

Optionally, the determining module 13 is specifically configured to: determining a region to be decoded according to the initial position, the first height and the second height; and determining the image in the region to be decoded in the coded image as the image to be decoded.

Optionally, the starting position comprises a height starting position; the determining module 13 is specifically configured to: determining a minimum boundary value according to the height initial position; determining a maximum boundary value according to the first height, the second height and the height initial position; and determining the CTU row from the minimum boundary value to the maximum boundary value in the coded image as an image to be decoded.

Optionally, the determining module 13 is specifically configured to: shifting the height starting position to the right by a third preset value; the third preset value is a preset value corresponding to a coding mode corresponding to the coded image; and determining the maximum value of the first preset value and the difference value between the height starting position after the right shift and the second preset value as the minimum boundary value.

Optionally, the determining module 13 is specifically configured to: shifting the third preset value to the left by a tenth preset value; determining the difference value between the sum of the height starting position, the first height and the third preset value after left movement and the tenth preset value; shifting the difference value to the right by a third preset value; the minimum value of the second height and the difference value after the right shift is determined as the maximum boundary value.

Optionally, the processing module 14 is specifically configured to: acquiring decoding reference information; and decoding the image to be decoded according to the decoding reference information to obtain a target original image.

Optionally, the decoding information includes a switch identifier of the filtering processing tool; the processing module 14 is specifically configured to: judging the opening of the filtering processing tool according to the switch identifier;

Optionally, the CTU rows above the image to be decoded include a first CTU row and a second CTU row, and the CTU rows below the image to be decoded include a third CTU row; the processing module 14 is specifically configured to: respectively decoding the first CTU line, the second CTU line and the third CTU line to obtain a first original pixel set, a second original pixel set and a third original pixel set; performing in-loop filtering processing on the first original pixel set and the second original pixel set respectively by using a filtering processing tool to obtain a first processing pixel set and a second processing pixel set; and determining decoding reference information according to the first processing pixel set, the second processing pixel set and the third original pixel set.

Optionally, the first processing pixel set and the second processing pixel set respectively include N lines of pixel data, where N is an integer greater than or equal to 2; the processing module 14 is specifically configured to: replacing the last M rows of pixel data in the N rows of pixel data in the first processing pixel set by the last M rows of pixel data in the N rows of pixel data in the second processing pixel set to obtain a third processing pixel set corresponding to the first processing pixel set; m is a positive integer less than N; and determining the second processing pixel set, the third processing pixel set and the third original pixel set as decoding reference information.

Optionally, the sixth preset value CTU row above the image to be decoded includes a first CTU row, and the seventh preset value CTU row below the image to be decoded includes a second CTU row; the processing module 14 is specifically configured to: respectively decoding the first CTU line and the second CTU line to obtain a first original pixel set and a second original pixel set; the first original pixel set comprises N lines of original pixel data; replacing the first M rows of original pixel data of the N rows of original pixel data in the first original pixel set by the last M rows of original pixel data of the N rows of original pixel data in the first original pixel set to obtain a second pixel set; and determining the second pixel set and the second original pixel set as decoding reference information.

Optionally, the processing module 14 is specifically configured to: decoding the image to be decoded according to the decoding reference information to obtain a reconstructed pixel value; storing the reconstructed pixel value to a preset area to obtain a reconstructed image; the preset area and the image to be decoded have the same size; and determining a target original image in the reconstructed image according to the starting position, the first height and the first width.

Optionally, the processing module 14 is specifically configured to: determining a difference value between the height initial position and a left shift value obtained by left-shifting the third preset value by a minimum boundary value as a top offset; determining a sum of the maximum boundary value and a ninth preset value, and determining a minimum value of a value obtained after the third preset value is shifted to the left by the sum and a difference value between the second height and the height initial position and the first height as a bottom offset; determining the width starting position of the starting position as the left offset; determining the difference between the second width and the sum of the left offset and the first width as a right offset; the top offset, the bottom offset, the left offset, and the right offset indicate images to be deleted in the reconstructed image; and deleting the image to be deleted in the reconstructed image to obtain the target original image.

Optionally, the determining module 13 is specifically configured to: analyzing the image parameter set included in the coding information to obtain the position area and the fragment number of the image fragments included in the coded image; if the number of the fragments is equal to a preset value, determining a region to be decoded according to the initial position, the first height and the second height; if the number of the fragments is not equal to the preset value, deleting the image fragments which are not overlapped with the calibration area in the coded image, and determining the area to be decoded in the coded image after deleting the image fragments according to the initial position, the first height and the second height.

Optionally, the operation instruction is an instruction corresponding to a sliding operation; the determining module 13 is specifically configured to: detecting the pause time corresponding to the sliding operation; when the pause time exceeds a preset value, determining a calibration area in the current display interface; and determining an image to be decoded corresponding to the calibration area in the coded image according to the calibration area and the coding information.

Fig. 15 is a hardware schematic diagram of a terminal device according to an embodiment of the present application. As shown in fig. 15, the terminal device 20 may include: a transceiver 21, a memory 22, a processor 23.

The transceiver 21 may include: a transmitter and/or a receiver. The transmitter may also be referred to as a sender, a transmitter, a sending port or a sending interface, and the like, and the receiver may also be referred to as a receiver, a receiving port or a receiving interface, and the like. The transceiver 21, memory 22, and processor 23 are illustratively interconnected via a bus 404.

The memory 22 is used for storing computer execution instructions;

the processor 23 is configured to execute computer-executable instructions stored in the memory 22, so that the processor 23 executes the image processing method described above.

The embodiment of the application provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the computer-executable instructions are executed by a processor, the image processing method is implemented.

An embodiment of the present application further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the image processing method can be implemented.

All or a portion of the steps of implementing the above-described method embodiments may be performed by hardware associated with program instructions. The aforementioned program may be stored in a readable memory. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned memory (storage medium) includes: read-only memory (ROM), RAM, flash memory, hard disk, solid state disk, magnetic tape (magnetic tape), floppy disk (optical disk), and any combination thereof.

Embodiments of the present application are described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processing unit of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

It will be apparent to those skilled in the art that various changes and modifications may be made in the embodiments of the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the embodiments of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to encompass such modifications and variations.

In the present application, the terms "include" and variations thereof may refer to non-limiting inclusions; the term "or" and variations thereof may mean "and/or". The terms "first," "second," and the like in this application are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. In the present application, "a plurality" means two or more. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.

Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.

It will be understood that the present application is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims

1. An image processing method is applied to a terminal device, and the method comprises the following steps:

acquiring an operation instruction;

and decoding the image to be decoded to obtain the target original image, and displaying the target original image in the calibration area.

2. The method according to claim 1, wherein the operation instruction includes a start position, a first height and a first width of the calibration region, and the encoded information includes a sequence parameter set;

the determining, according to the operation instruction and the encoding information, an image to be decoded in the encoded image includes:

judging whether the initial position, the first height and the first width are preset values;

if so, analyzing the sequence parameter set to obtain decoding information, wherein the decoding information comprises a second height and a second width of the coded image; and determining the image to be decoded in the encoded image according to the starting position, the first height, the first width, the second height and the second width.

3. The method of claim 2, wherein the determining the image to be decoded in the encoded image according to the start position, the first height, the first width, the second height, and the second width comprises:

judging whether the calibration area is positioned in the coded image or not according to the initial position, the first height, the first width, the second height and the second width;

and if so, determining the image to be decoded in the coded image according to the starting position, the first height and the second height.

4. The method of claim 3, wherein said determining whether the calibration region is located in the encoded image according to the start position, the first height, the first width, the second height, and the second width comprises:

if the starting position is located in the coded image, the first height is smaller than the second height, and the first width is smaller than the second width, the calibration region is located in the coded image;

otherwise, the calibration region is not in the encoded image.

5. The method according to any of claims 2-4, wherein said determining the image to be decoded in the encoded image according to the starting position, the first height and the second height comprises:

determining a region to be decoded according to the starting position, the first height and the second height;

and determining the image in the region to be decoded in the coded image as the image to be decoded.

6. The method of claim 5, wherein the starting position comprises a height starting position;

determining a region to be decoded according to the starting position, the first height and the second height comprises:

determining a minimum boundary value according to the height initial position;

determining a maximum boundary value according to the first height, the second height and the height starting position;

and determining the CTU row from the minimum boundary value to the maximum boundary value in the coded image as the image to be decoded.

7. The method of claim 6, wherein determining a minimum boundary value based on the height starting position comprises:

shifting the height starting position to the right by a third preset value; the third preset value is a preset value corresponding to a coding mode corresponding to the coded image;

and determining the maximum value of the first preset value and the difference value between the height starting position after the right shift and the second preset value as the minimum boundary value.

8. The method of claim 7, wherein determining a maximum boundary value based on the first height, the second height, and the height starting location comprises:

shifting the third preset value to the left by a tenth preset value;

determining the difference value between the sum of the height starting position, the first height and a third preset value after left movement and the tenth preset value;

shifting the difference value to the right by the third preset value;

determining a minimum value of the second height and the difference value after the right shift as the maximum boundary value.

9. The method according to claim 5, wherein the decoding the image to be decoded to obtain the target original image comprises:

acquiring decoding reference information;

and decoding the image to be decoded according to the decoding reference information to obtain the target original image.

10. The method of claim 9, wherein the decoded information includes a switch identification of the filtering processing tool;

the acquiring of the decoding reference information includes:

judging whether the filtering processing tool is started or not according to the switch identifier;

if so, determining a CTU line with a fourth preset value positioned above the image to be decoded and a CTU line with a fifth preset value positioned below the image to be decoded in the coded image as a first reference decoded image, and determining decoding reference information according to the first reference decoded image;

otherwise, determining the CTU line with the sixth preset value positioned above the image to be decoded and the CTU line with the seventh preset value positioned below the image to be decoded in the coded image as a second reference decoded image, and determining decoding reference information according to the second reference decoded image.

11. The method according to claim 10, wherein the fourth preset number of rows of CTUs above the picture to be decoded comprises a first row of CTUs and a second row of CTUs, and the fifth preset number of rows of CTUs below the picture to be decoded comprises a third row of CTUs;

the determining the decoding reference information according to the first reference decoded picture includes:

decoding the first CTU line, the second CTU line and the third CTU line respectively to obtain a first original pixel set, a second original pixel set and a third original pixel set;

performing in-loop filtering processing on the first original pixel set and the second original pixel set by using the filtering processing tool to obtain a first processing pixel set and a second processing pixel set;

12. The method of claim 11, wherein the first set of processed pixels and the second set of processed pixels each include N rows of pixel data, N being an integer greater than or equal to 2;

determining, by the determining according to the first processed pixel set, the second processed pixel set, and the third original pixel set, decoding reference information, including:

determining the second set of processed pixels, the third set of processed pixels, and the third set of original pixels as the decoding reference information.

13. The method according to claim 10, wherein the sixth preset number of rows of CTUs above the picture to be decoded comprises a first row of CTUs, and the seventh preset number of rows of CTUs below the picture to be decoded comprises a second row of CTUs;

the determining the decoding reference information according to the second reference decoded picture includes:

respectively decoding the first CTU line and the second CTU line to obtain a first original pixel set and a second original pixel set; the first original pixel set comprises N lines of original pixel data;

14. The method according to claim 9, wherein decoding the image to be decoded according to the decoding reference information to obtain the target original image comprises:

decoding the image to be decoded according to the decoding reference information to obtain a reconstructed pixel value;

storing the reconstructed pixel value to a preset area to obtain a reconstructed image; the preset area and the image to be decoded have the same size;

and determining the target original image in the reconstructed image according to the starting position, the first height and the first width.

15. The method of claim 14, wherein determining the original target image in the reconstructed image according to the starting position, the first height, and the first width comprises:

determining the difference between the initial height position and a left shift value obtained by shifting a third preset value to the left by a minimum boundary value as a top offset;

determining a sum of a maximum boundary value and a ninth preset value, and determining a minimum value of a value obtained by left-shifting the sum by the third preset value and a difference value between the second height and the initial position and the first height of the height as a bottom offset;

determining the width starting position of the starting position as a left offset;

determining a difference between the second width and a sum of the left offset and the first width as a right offset; the top offset, bottom offset, left offset, and right offset indicate an image to be deleted in the reconstructed image;

and deleting the image to be deleted in the reconstructed image to obtain a target original image.

16. The method of claim 5, wherein determining the region to be decoded according to the starting position, the first height, and the second height comprises:

analyzing the image parameter set included in the coding information to obtain the position area and the fragment number of the image fragments included in the coded image;

if the number of the fragments is equal to a preset value, determining the area to be decoded according to the initial position, the first height and the second height;

if the number of the slices is not equal to the preset value, deleting the image slices which are not overlapped with the calibration area in the coded image, and determining the area to be decoded in the coded image after deleting the image slices according to the initial position, the first height and the second height.

17. The method according to claim 1, wherein the operation instruction is an instruction corresponding to a slide operation;

detecting the pause time corresponding to the sliding operation; when the pause time exceeds a preset value, determining a calibration area in the current display interface;

and determining an image to be decoded corresponding to the calibration region in the coded image according to the calibration region and the coding information.

18. An image processing apparatus, applied to a terminal device, the apparatus comprising:

the acquisition module is used for acquiring an operation instruction;

the determining module is used for determining an image to be decoded in the coded image according to the operation instruction and the coded information; the image to be decoded is a coded image corresponding to a target original image in a specified area in the original image;

the processing module is used for decoding the image to be decoded to obtain the target original image;

19. A terminal device, comprising: a processor, and a memory communicatively coupled to the processor;

the memory stores computer-executable instructions;

the processor executes computer-executable instructions stored by the memory to implement the method of any of claims 1-17.

20. A computer-readable storage medium having computer-executable instructions stored thereon, which when executed by a processor, perform the method of any one of claims 1-17.

21. A computer program product, comprising: a computer program; the computer program, when executed by a processor, implements the method of any of claims 1-17.