CN113810717B

CN113810717B - Image processing method and device

Info

Publication number: CN113810717B
Application number: CN202010531794.XA
Authority: CN
Inventors: 马祥; 杨海涛; 宋楠; 张恋
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2020-06-11
Filing date: 2020-06-11
Publication date: 2024-05-17
Anticipated expiration: 2040-06-11
Also published as: CN113810717A

Abstract

The application discloses an image processing method and device, and relates to the field of image processing. The method is applied to an encoding device. The method comprises the following steps: determining bits to be encoded; estimating probability distribution of the bits to be encoded based on the context of the bits to be encoded; wherein the context of the bits to be encoded comprises: bits in a preset range corresponding to the bits to be encoded in the first bit string and/or the second bit string; wherein the second bit string is used to represent another coefficient in the first subband; and encoding the bits to be encoded according to the probability distribution of the bits to be encoded so as to obtain the encoding information of the image to be processed.

Description

Image processing method and device

Technical Field

The present application relates to the field of image processing, and in particular, to an image processing method and apparatus.

Background

With the rapid development of internet technology and the increasing abundance of physical and mental culture of people, the application requirements for images and videos in the internet are increasing, so that the data volume of high-definition images and high-definition videos is increasing. In addition, the requirements of people on image and video quality are also increasing. For example, the spatial resolution of images is increasing, and the frame rate is increasing in addition to the spatial resolution of video.

After compression encoding these images and videos by the prior art, a large amount of storage space is still required at the user side or the server side to store these compression encoded images and videos. And, after compression encoding is performed on high-quality images and videos by the existing method, the efficiency is still low when the high-quality images and videos are transmitted in the internet with limited bandwidth. Therefore, how to improve the coding efficiency (or called compression efficiency) of the image and the video, so as to relieve the pressure of the image and the video in storage and transmission is a technical problem to be solved.

Disclosure of Invention

The application provides an image processing method and device, which improves the coding efficiency of images (or videos), thereby saving the storage resources for storing the images (or videos) and saving the transmission bandwidth for transmitting the images (or videos).

In order to achieve the above purpose, the present application provides the following technical scheme:

In a first aspect, the present application provides an image processing method, which is applied to an image processing apparatus. The method comprises the following steps: determining bits to be encoded; the bit to be encoded is any bit in a first bit string, the first bit string is any bit in a plurality of binary bit strings included in a first bit matrix, the first bit string is used for representing one coefficient in a first sub-band, the first bit matrix is any bit matrix in a plurality of bit matrices obtained after the image to be processed is processed, and the first bit matrix is a three-dimensional bit matrix. Estimating probability distribution of the bits to be encoded based on the determined context of the bits to be encoded; here, the context of the bits to be encoded includes: bits in a preset range corresponding to the bits to be encoded in the first bit string and/or the second bit string; wherein the second bit string is used to represent another coefficient in the first subband. And encoding the bits to be encoded according to the probability distribution of the bits to be encoded so as to obtain the encoding information of the image to be processed.

According to the method provided by the application, the image processing device determines the context of the bit to be encoded in a plurality of binarized bit matrixes obtained by processing the image to be processed, and estimates the probability distribution of the bit to be encoded based on the context. Wherein the context comprises bits in a bit string (i.e. a second bit string) belonging to the same subband as the bit string (i.e. the first bit string) in which the bits to be encoded are located. By this method, the accuracy of the probability distribution of the bits to be encoded estimated by the image processing apparatus can be improved. Further, the image processing device encodes the bits to be encoded based on the estimated probability distribution of the bits to be encoded with higher accuracy than the prior art, thereby improving the encoding efficiency.

In one possible implementation, if the plurality of binary bit strings are used to represent coefficients in different subbands, respectively, the second bit string is a binary bit string in a second bit matrix, where the second bit matrix is a bit matrix of the plurality of bit matrices that is located within a preset range of the first bit matrix. Or if the plurality of binary bit strings are used for representing coefficients in the first sub-band, the second bit string is a binary bit string located within a preset range of the first bit string in the first bit matrix.

With this possible implementation, if the plurality of bit matrices obtained after the image to be processed is an aggregated bit matrix, that is, the plurality of binary bit strings in the first bit matrix are all used to represent coefficients in the first subband. In this way, the speed at which the image processing apparatus acquires the context of the bit to be encoded is increased, thereby increasing the encoding speed.

In another possible implementation manner, the context of the bits to be encoded further includes: and in the third bit string and/or the fourth bit string, bits in a preset range corresponding to the bits to be encoded are provided. The third bit string is used to represent one coefficient in the second subband and the fourth bit string is used to represent another coefficient in the second subband. Wherein,

If a plurality of binary bit strings in the first bit matrix are used to represent coefficients in different subbands, respectively, the third bit string is a binary bit string in the first bit matrix and the fourth bit string is a binary bit string in the second bit matrix. The second bit matrix is a bit matrix of the plurality of bit matrices that is within a preset range of the first bit matrix. The second sub-band is a sub-band corresponding to a binary bit string located within a preset range of the first bit string. Or alternatively

If a plurality of binary bit strings in the first bit matrix are each used to represent coefficients in the first sub-band, i.e. the plurality of bit matrices are in one-to-one correspondence with the plurality of sub-bands, then the third bit string and the fourth bit string are each a binary bit string in the third bit matrix. The third bit matrix is a bit matrix of the plurality of bit matrices that is within a preset range of the first bit matrix. The second subband is the subband corresponding to the third bit matrix.

In another possible implementation manner, before the "determining the bits to be encoded", the method further includes: and acquiring a plurality of two-dimensional coefficient blocks corresponding to the image to be processed. Each two-dimensional coefficient block of the plurality of two-dimensional coefficient blocks includes a plurality of coefficients, the plurality of coefficients being coefficients in different subbands, respectively. Each coefficient of the plurality of two-dimensional coefficient blocks is binarized to obtain the plurality of bit matrices.

In another possible implementation, the binarizing each coefficient of the plurality of two-dimensional coefficient blocks includes: and binarizing each coefficient in the two-dimensional coefficient blocks according to a binarization mode of fixed length.

By means of the two possible implementations, the image processing device can obtain a plurality of binarized bit matrixes corresponding to the image to be processed. In this way, the image processing apparatus can acquire the context of the bits to be encoded in the binarized bit matrix, and estimate the probability distribution of the bits to be encoded with higher accuracy than in the prior art based on the context. Thereby improving coding efficiency.

In another possible implementation manner, before the "acquiring the plurality of two-dimensional coefficient blocks corresponding to the image to be processed", the method further includes: and acquiring an image to be processed, wherein the image to be processed comprises an image frame in a picture or a video. The above-mentioned "obtaining a plurality of two-dimensional coefficient blocks corresponding to an image to be processed" includes: the image to be processed is segmented and transformed to obtain a plurality of two-dimensional coefficient blocks.

In another possible implementation manner, the determining the bits to be encoded includes: and determining the bits to be encoded in the plurality of bit matrixes according to a preset encoding sequence.

In another possible implementation manner, the estimating the probability distribution of the bits to be encoded based on the context of the bits to be encoded includes: based on the context of the bits to be encoded, the probability distribution of the bits to be encoded is estimated by a probability estimation network.

In another possible implementation manner, the "encoding the bits to be encoded according to the probability distribution of the bits to be encoded" includes: and carrying out entropy coding on the bits to be coded according to the probability distribution of the bits to be coded.

In a second aspect, the present application provides an image processing method applied to an image processing apparatus. The method comprises the following steps: the position of the bit to be decoded is determined. Wherein the bit to be decoded is any one bit of a first bit string, the first bit string is any one binary bit string of a plurality of binary bit strings included in a first bit matrix, the first bit string is used for representing one coefficient in a first subband, the first bit matrix is any one bit matrix of a plurality of bit matrices corresponding to the image to be processed, and the first bit matrix is a three-dimensional bit matrix. Based on the context of the bits to be decoded, a probability distribution of the bits to be decoded is estimated. Here, the context of the bits to be decoded includes: and in the first bit string and/or the second bit string, the bits in the preset range corresponding to the bits to be decoded are selected. Wherein the second bit string is used to represent another coefficient in the first subband. Decoding the bits to be decoded based on the positions of the bits to be decoded, the probability distribution of the bits to be decoded and the coding information of the image to be processed, so as to obtain a plurality of bit matrixes corresponding to the image to be processed.

In one possible implementation, if the plurality of binary bit strings are used to represent coefficients in different subbands, respectively, the second bit string is a binary bit string in a second bit matrix. Here, the second bit matrix is a bit matrix located within a preset range of the first bit matrix among the plurality of bit matrices. Or if the plurality of binary bit strings are used for representing coefficients in the first sub-band, the second bit string is a binary bit string located within a preset range of the first bit string in the first bit matrix.

In another possible implementation manner, the context of the bits to be decoded further includes: and in the third bit string and/or the fourth bit string, the bits in the preset range corresponding to the bits to be decoded are obtained. The third bit string is used to represent one coefficient in the second subband and the fourth bit string is used to represent another coefficient in the second subband. Wherein,

If the above-mentioned plurality of binary bit strings are used to represent coefficients in different sub-bands, respectively, the third bit string is a binary bit string in the first bit matrix and the fourth bit string is a binary bit string in the second bit matrix. The second bit matrix is a bit matrix of the plurality of bit matrices that is within a preset range of the first bit matrix. The second sub-band is a sub-band corresponding to a binary bit string located within a preset range of the first bit string. Or alternatively

If the above-mentioned plurality of binary bit strings are each used to represent coefficients in the first sub-band, i.e. the plurality of bit matrices are in one-to-one correspondence with the plurality of sub-bands, then the third bit string and the fourth bit string are each a binary bit string in the third bit matrix. The third bit matrix is a bit matrix of the plurality of bit matrices that is within a preset range of the first bit matrix. The second subband is the subband corresponding to the third bit matrix.

In another possible implementation manner, the method further includes: and inversely binarizing the binary bit strings in the plurality of bit matrixes obtained by decoding to obtain a plurality of two-dimensional coefficient blocks corresponding to the plurality of bit matrixes. Wherein each two-dimensional coefficient block of the plurality of two-dimensional coefficient blocks includes a plurality of coefficients, the plurality of coefficients being coefficients in different subbands, respectively. And carrying out inverse transformation on the two-dimensional coefficient blocks to obtain an image to be processed. Wherein the image to be processed comprises an image frame in a picture or video.

In another possible implementation manner, the "inverse binarizing the binary bit strings in the plurality of bit matrices obtained by decoding" includes: and according to the fixed-length inverse binarization mode, inversely binarizing the binary bit strings in the plurality of bit matrixes obtained by decoding.

In another possible implementation manner, the determining the position of the bit to be decoded includes: and determining the positions of the bits to be decoded in a plurality of bit matrixes corresponding to the images to be processed according to a preset decoding sequence.

In another possible implementation manner, the estimating the probability distribution of the bits to be decoded based on the context of the bits to be decoded includes: based on the context of the bits to be decoded, the probability distribution of the bits to be decoded is estimated by a probability estimation network.

In another possible implementation manner, the decoding the bits to be decoded based on the positions of the bits to be decoded, the probability distribution of the bits to be decoded, and the encoded information of the image to be processed includes: and carrying out entropy decoding on the bits to be decoded based on the positions of the bits to be decoded, the probability distribution of the bits to be decoded and the coding information of the image to be processed.

The advantages of the second aspect and any possible implementation manner thereof may refer to the description of the advantages of the first aspect and any possible implementation manner thereof, and are not repeated here.

In a third aspect, the present application provides an image processing apparatus.

In a possible design, the image processing apparatus is configured to perform any of the methods provided in the first aspect or the second aspect. The present application may divide the image processing apparatus into functional modules according to any of the methods provided in the first or second aspects. For example, each functional module may be divided corresponding to each function, or two or more functions may be integrated in one processing module. The present application may divide the image processing apparatus into a determination unit, an estimation unit, an encoding unit, and the like, or the present application may divide the image processing apparatus into a determination unit, an estimation unit, a decoding unit, and the like, as a function, for example. The description of possible technical solutions and beneficial effects executed by the above-divided functional modules may refer to the technical solutions provided by the above-mentioned first aspect or the corresponding possible designs thereof, or may refer to the technical solutions provided by the above-mentioned second aspect or the corresponding possible designs thereof, which are not repeated herein.

In another possible design, the image processing apparatus includes: the memory is coupled to the one or more processors. The memory is for storing computer instructions that the processor is for invoking to perform any of the methods as provided in the first aspect and any of its possible designs, or to perform any of the methods as provided in the second aspect and any of its possible designs.

In a fourth aspect, the present application provides a computer-readable storage medium, such as a computer-non-transitory readable storage medium. On which a computer program (or instructions) is stored which, when run on an image processing apparatus, causes the image processing apparatus to perform any one of the methods provided in any one of the possible implementations of the first or second aspects described above.

In a fifth aspect, the present application provides a computer readable storage medium having stored thereon encoded information obtained according to any one of the image processing methods provided in any one of the possible implementations of the first aspect.

In a sixth aspect, the application provides a computer program product which, when run on an image processing apparatus, causes any one of the methods provided in any one of the possible implementations of the first or second aspects to be performed.

In a seventh aspect, the present application provides a chip system, comprising: a processor for calling from a memory and running a computer program stored in the memory, performing any one of the methods provided by the implementation manner in the first aspect or in the second aspect.

It should be appreciated that any of the apparatus, computer storage medium, computer program product, or chip system provided above may be applied to the corresponding method provided above, and thus, the benefits achieved by the apparatus, computer storage medium, computer program product, or chip system may refer to the benefits in the corresponding method, which are not described herein.

In the present application, the names of the above-described image processing apparatuses do not constitute limitations on the devices or function modules themselves, and in actual implementations, these devices or function modules may appear under other names. Insofar as the function of each device or function module is similar to that of the present application, it falls within the scope of the claims of the present application and the equivalents thereof.

These and other aspects of the application will be more readily apparent from the following description.

Drawings

FIG. 1 is a schematic diagram of an image processing system according to an embodiment of the present application;

Fig. 2 is a schematic diagram of an image encoding device and an image decoding device applied to a terminal device according to an embodiment of the present application;

FIG. 3 is a hardware configuration diagram of an image processing apparatus according to an embodiment of the present application;

fig. 4 is a flowchart of an image processing method according to an embodiment of the present application;

FIG. 5 is a schematic diagram of an image partition according to an embodiment of the present application;

FIG. 6 is a schematic diagram of a two-dimensional coefficient block according to an embodiment of the present application;

FIG. 7 is a schematic diagram of a three-dimensional bit matrix according to an embodiment of the present application;

FIG. 8 is a schematic diagram of a coding sequence of a plurality of bit matrices according to an embodiment of the present application;

FIG. 9 is a schematic diagram of a binary bit string provided by an embodiment of the present application;

FIG. 10 is a schematic diagram illustrating a preset range of a first bit matrix according to an embodiment of the present application;

FIG. 11 is a schematic diagram of a context model according to an embodiment of the present application;

FIG. 12 is a schematic diagram of a preset range of a first bit string according to an embodiment of the present application;

FIG. 13a is a schematic diagram II of a context model according to an embodiment of the present application;

fig. 13b is a schematic diagram of estimating probability distribution of bits to be encoded according to an embodiment of the present application;

fig. 13c is a schematic diagram II of estimating probability distribution of bits to be encoded according to an embodiment of the present application;

FIG. 14 is a schematic diagram of a sub-belt polymerization method according to an embodiment of the present application;

Fig. 15 is a second flowchart of an image processing method according to an embodiment of the present application;

fig. 16 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application;

Fig. 17 is a schematic diagram of a second structure of an image processing apparatus according to an embodiment of the present application;

FIG. 18 is a schematic diagram of a chip system according to an embodiment of the present application;

fig. 19 is a schematic structural diagram of a computer program product according to an embodiment of the present application.

Detailed Description

In embodiments of the application, words such as "exemplary" or "such as" are used to mean serving as an example, instance, or illustration. Any embodiment or design described herein as "exemplary" or "e.g." in an embodiment should not be taken as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "such as" is intended to present related concepts in a concrete fashion.

In embodiments of the application, the terms "first", "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature. In the description of the present application, unless otherwise indicated, the meaning of "a plurality" is two or more.

The term "at least one" in the present application means one or more, and the term "plurality" in the present application means two or more, for example, a plurality of second messages means two or more second messages. The terms "system" and "network" are often used interchangeably herein.

It is to be understood that the terminology used in the description of the various examples described herein is for the purpose of describing particular examples only and is not intended to be limiting. As used in the description of the various described examples and in the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

It will also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. The term "and/or" is an association relationship describing an associated object, and means that there may be three relationships, for example, a and/or B, and may mean: a exists alone, A and B exist together, and B exists alone. In the present application, the character "/" generally indicates that the front and rear related objects are an or relationship.

It should also be understood that, in the embodiments of the present application, the sequence number of each process does not mean that the execution sequence of each process should be determined by the function and the internal logic, and should not limit the implementation process of the embodiments of the present application.

It should be appreciated that determining B from a does not mean determining B from a alone, but may also determine B from a and/or other information.

It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It should also be understood that the term "if" may be interpreted to mean "when" ("white" or "upon") or "in response to a determination" or "in response to detection". Similarly, the phrase "if a [ stated condition or event ] is detected" may be interpreted to mean "upon a determination" or "in response to a determination" or "upon a detection of a [ stated condition or event ] or" in response to a detection of a [ stated condition or event ], depending on the context.

It should be appreciated that reference throughout this specification to "one embodiment," "an embodiment," "one possible implementation" means that a particular feature, structure, or characteristic described in connection with the embodiment or implementation is included in at least one embodiment of the present application. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment," "one possible implementation" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

The embodiment of the application provides an image processing method which can be used for improving the coding efficiency of an image, thereby reducing the storage resources consumed for storing the image (or video) and reducing the transmission bandwidth consumed for transmitting the image (or video). The image processing method may include an image encoding method and an image decoding method, among others.

The embodiment of the application also provides an image processing device which can comprise an image encoding device and an image decoding device. The image encoding device may be used to perform the image encoding method described above, and the image decoding device may be used to perform the image decoding method described above.

Referring to fig. 1, an embodiment of the present application also provides an image processing system 10. As shown in fig. 1, the image processing system 10 includes an image encoding device 101 and an image decoding device 102. The image encoding device 101 is configured to encode a bit matrix corresponding to an image to be processed to obtain an encoded code stream (may also be referred to as a compressed code stream or a bit stream). The encoded code stream may be transmitted from the image encoding apparatus 101 to the image decoding apparatus 102 by a wired or wireless transmission method. The image decoding device 102 is configured to decode the received encoded code stream to obtain a bit matrix corresponding to the image to be processed, and restore the image to be processed based on the bit matrix. For a description of the bit matrix corresponding to the image to be processed, refer to the following, and the description is omitted here.

The image encoding device and the image decoding device described above may be applied to a terminal device, which may be a portable device such as a mobile phone, a tablet PC, or a wearable electronic device, may be a device such as a vehicle-mounted device or an intelligent robot, or may be a computing device such as a personal computer (personal computer, PC), a Personal Digital Assistant (PDA), a netbook, or a server, which is not limited thereto.

The image encoding device and the image decoding device described above can be applied to wireless devices, core network devices, and the like that have an image transcoding requirement.

The image encoding apparatus provided by the embodiment of the present application may be an image encoder in the terminal device, the wireless device, or the core network device. The image decoding apparatus provided by the embodiment of the present application may be an image decoder in the above terminal device, the wireless device, or the core network device. This is not limited thereto.

In the embodiment of the present application, a terminal device, a wireless device, or a core network device in which an image encoding apparatus is disposed is referred to as an encoding end device. A terminal device, a wireless device, or a core network device in which the image decoding apparatus is disposed is referred to as a decoding end device. It is to be understood that the encoding end device and the decoding end device may be the same device or may be different devices, which is not limited.

It should be understood that the encoding end device may further include a channel encoding device (e.g., a channel encoder), and the decoding end device may further include a channel decoding device (e.g., a channel decoder), which is not limited thereto. Wherein the channel encoding means may be adapted to channel encode the image signal and the channel decoding means may be adapted to channel decode the image signal.

Referring to fig. 2, fig. 2 is a schematic diagram of an image encoding apparatus and an image decoding apparatus applied to a terminal device according to an embodiment of the present application. As shown in fig. 2, the first terminal device 20 may include: an image encoding device 201 and a channel encoding device 202. The second terminal device 21 may include: an image decoding device 211 and a channel decoding device 212. The first terminal device 20 is connected to a first network communication device 22, which is wireless or wired, and the first network communication device 22 is connected to a second network communication device 23, which is wireless or wired, through a digital channel, and the second terminal device 21 is connected to the second network communication device 23, which is wireless or wired. The above-mentioned wireless or wired network communication device may refer to a signal transmission device, such as a communication base station, a data exchange device, etc., which is not particularly limited.

Referring to fig. 3, an embodiment of the present application provides a hardware configuration diagram of an image processing apparatus 30. The image processing apparatus 30 may be the image encoding apparatus described above, or may be the image decoding apparatus described above. As shown in fig. 3, the image processing apparatus 30 includes a processor 301, a memory 302, a communication interface 303, and a bus 304. The processor 301, the memory 302 and the communication interface 303 may be connected via a bus 304.

The processor 301 is a control center of the image processing apparatus 30, and may be a general-purpose central processing unit (central processing unit, CPU), another general-purpose processor, or the like. Wherein the general purpose processor may be a microprocessor or any conventional processor or the like.

As an example, processor 301 may include one or more CPUs, such as CPU 0 and CPU 1 shown in fig. 3.

Memory 302 may be, but is not limited to, read-only memory (ROM) or other type of static storage device that can store static information and instructions, random access memory (random access memory, RAM) or other type of dynamic storage device that can store information and instructions, or electrically erasable programmable read-only memory (EEPROM), magnetic disk storage or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer.

In one possible implementation, the memory 302 may exist separately from the processor 301. The memory 302 may be connected to the processor 301 through a bus 304 for storing data, instructions or program code. The image processing method provided by the embodiment of the present application can be implemented when the processor 301 calls and executes instructions or program codes stored in the memory 302.

In another possible implementation, the memory 302 may also be integrated with the processor 301.

A communication interface 303 for connecting the image processing apparatus 30 with other devices (such as a wireless or limited first network communication device in fig. 2, etc.) through a communication network, which may be an ethernet, a radio access network (radio access network, RAN), a wireless local area network (wireless local area networks, WLAN), etc. The communication interface 303 may include a receiving unit for receiving data and a transmitting unit for transmitting data.

Bus 304 may be an industry standard architecture (industry standard architecture, ISA) bus, an external device interconnect (PERIPHERAL COMPONENT INTERCONNECT, PCI) bus, or an extended industry standard architecture (extended industry standard architecture, EISA) bus, among others. The bus may be classified as an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in fig. 3, but not only one bus or one type of bus.

It should be noted that the structure shown in fig. 3 does not constitute a limitation of the image processing apparatus, and the image processing apparatus 30 may include more or less components than those shown in fig. 3, or may combine some components, or may be a different arrangement of components.

The image processing method provided by the embodiment of the application is described below with reference to the accompanying drawings.

Referring to fig. 4, fig. 4 is a flowchart illustrating an image processing method according to an embodiment of the present application, which can be applied to an image encoding device in an image processing device. For convenience of description, the image encoding apparatus will hereinafter be abbreviated as encoding apparatus. The method may include:

S101, acquiring an image to be processed.

The image to be processed may be an entire original image, or may be a rectangular area in the entire original image, or may be an area with other shapes in the entire original image, which is not limited.

Alternatively, the encoding device may acquire a picture from a local or a network, and take the acquired picture as the original picture, or the encoding device acquires a video from the local or the network, and take an image frame in the video as the original image, which is not limited.

It can be seen that when the encoding means encodes image frames in the video consecutively, the encoding of the video is achieved.

S102 (optional), dividing the image to be processed into a plurality of image blocks.

The encoding device may divide the image to be processed into a plurality of image blocks according to a preset image blocking rule.

Optionally, the encoding device may divide the image to be processed into a plurality of image blocks according to a preset size. For example, the encoding device divides the image to be processed into t×s image blocks with a size m×n. Wherein M, N, T, S are positive integers, and the values of M and N can be the same or different. It can be seen that when the values of T and S are both 1, the size of the image to be processed is m×n, i.e. the image to be processed does not need to be partitioned.

As shown in fig. 5, an example is given in which T is 2, s is 4, and m and N are both 4. If the image 50 in fig. 5 is the image to be processed, the size of the image 50 is 8×16. The decoding means may divide the image 50 equally into 2 x 4 image blocks of size 4*4, any of the 2 x 4 image blocks 501 being 4*4 in size.

It may be understood that the image block in the embodiment of the present application may be obtained by the encoding device by partitioning the image to be processed, or may be obtained by dividing the image to be processed into image blocks by other image processing devices (or other image processing apparatuses) and transmitting the image blocks to the encoding device, which is not limited. Wherein an image block may also be referred to as a sub-image, or sub-image block, etc.

S103, acquiring a plurality of two-dimensional coefficient blocks corresponding to the image blocks.

Wherein any one of the plurality of two-dimensional coefficient blocks includes a plurality of coefficients. Each position in one two-dimensional coefficient block may represent a different subband (hereinafter, the "subband" will be simply referred to as "subband" in the embodiments of the present application). For a two-dimensional coefficient block, the coefficients included in the two-dimensional coefficient block are coefficients in different subbands, i.e., the coefficients and the subbands are in one-to-one correspondence. For example, if a two-dimensional coefficient block includes d×k coefficients, the d×k coefficients are coefficients in d×k different subbands, and d×k coefficients and d×k subbands are in one-to-one correspondence. Wherein D and K are positive integers. It will be appreciated that if the first subband is any one of the D x K subbands, then coefficients in the first subband are included in each of the plurality of two-dimensional coefficient blocks.

Referring to fig. 6 for an example, fig. 6 shows a schematic diagram of a two-dimensional coefficient block 60. Wherein the two-dimensional coefficient block 60 includes 4*4 coefficients, the 4*4 coefficients may be coefficients in 4*4 subbands (16 subbands total). For example, coefficient y (0, 0) may be a coefficient in subband f1, coefficient y (0, 3) may be a coefficient in subband f4, coefficient y (3, 0) may be a coefficient in subband f13, coefficient y (3, 3) may be a coefficient in subband f16, and so on. Of the 4*4 coefficients, each coefficient may correspond to one subband, i.e., 4*4 coefficients correspond to 4*4 subbands.

Specifically, in one possible implementation manner, the encoding device may transform the plurality of image blocks respectively, so as to obtain a plurality of two-dimensional coefficient blocks. Wherein, for any image block in a plurality of image blocks, the encoding device transforms the image block to obtain a two-dimensional coefficient block. I.e. one image block corresponds to one two-dimensional coefficient block.

In another possible implementation manner, the encoding device may transform residual blocks corresponding to the plurality of image blocks, so as to obtain a plurality of two-dimensional coefficient blocks.

Specifically, for any one of the plurality of image blocks to obtain an image block, the encoding apparatus may predict the image block to obtain a prediction block corresponding to the image block. Then, the encoding apparatus determines a residual block corresponding to the image block by calculating a difference between the image block and the prediction block. That is, one image block corresponds to one residual block, and the specific process of obtaining a corresponding prediction block according to the plurality of image blocks and determining the residual block according to the image blocks and the prediction block by the encoding device in the embodiment of the present application is not described herein.

Then, the encoding device transforms the determined plurality of residual blocks, thereby obtaining a plurality of two-dimensional coefficient blocks. Wherein, for any residual block in the plurality of residual blocks, the encoding device transforms the residual block to obtain a two-dimensional coefficient block. I.e. one residual block corresponds to one two-dimensional coefficient block.

As an example, a process of obtaining a plurality of two-dimensional coefficient blocks by performing discrete cosine transform (discrete cosine transform, DCT) transform on each of the plurality of image blocks by an encoding apparatus will be described below.

Specifically, the encoding device may perform DCT transform of a corresponding size on each of the plurality of image blocks according to the size of the image block, so as to obtain a plurality of two-dimensional coefficient blocks of a corresponding size.

For example, if the size of each of the plurality of image blocks is 4*4, the encoding apparatus may perform 4*4 DCT transform on each of the plurality of image blocks, thereby obtaining a plurality of 4*4-sized two-dimensional coefficient blocks.

Illustratively, the encoding apparatus may perform 4*4 DCT on a 4*4-sized image block X according to the following formula (1), thereby obtaining a 4*4-sized transform coefficient matrix Y, i.e., the above-mentioned two-dimensional coefficient block.

Formula (1):

Wherein a and a ^T are coefficients of the DCT transform, and a ^T are inverse matrices to each other, and a, b, c are coefficients in a and a ^T, respectively. Y _(0,0) to Y _(3,3) denote 16 coefficients in the transform coefficient matrix Y, the 16 coefficients being coefficients in 16 subbands, i.e. one subband for each coefficient, the 16 coefficients corresponding to 16 different subbands.

It can be seen that a 4*4-sized image block, after transformation, results in a two-dimensional coefficient block comprising 4*4 coefficients. Therefore, if the size of each of the plurality of image blocks corresponding to the image to be processed is m×n, the encoding device transforms the plurality of image blocks, and each of the plurality of two-dimensional coefficient blocks obtained after transforming the plurality of image blocks includes m×n coefficients.

It can be seen that the encoding apparatus transforms the information of the image block or the information of the residual block from the spatial domain to the frequency domain by transforming the image block or the residual block, and represents the information of the image block or the information of the residual block by the coefficients in different sub-bands and different sub-bands in the frequency domain.

It should be understood that, in the embodiment of the present application, the plurality of two-dimensional coefficient blocks may be obtained by transforming an image block or a residual block by the encoding device, or may be obtained by transforming an image block or a residual block by another image processing device (or another image processing apparatus), and then transmitted to the encoding device, which is not limited.

S104, binarizing each coefficient in the acquired two-dimensional coefficient blocks to obtain a plurality of bit matrixes.

Specifically, the encoding device may binarize each coefficient in the obtained plurality of two-dimensional coefficient blocks according to a preset binarization manner, so as to obtain a plurality of bit matrices.

Alternatively, the encoding means may binarize each coefficient of the acquired plurality of two-dimensional coefficient blocks in a fixed-length binarization manner to obtain a plurality of bit matrices.

Wherein any one of the plurality of bit matrices (for example, a first bit matrix in the embodiment of the present application) is a three-dimensional bit matrix, and a plurality of binary bit strings for representing a plurality of coefficients in a two-dimensional coefficient block are included in the first bit matrix (for simplicity of description, "binary bit strings" will be simply referred to as "bit strings").

As an example, the encoding apparatus may binarize each coefficient of the acquired plurality of two-dimensional coefficient blocks into a bit string including 8 bits. Referring to fig. 6, if the value of the coefficient y (0, 0) is 1, the coefficient y (0, 0) may be binarized into a 0000 0001 bit string, that is, the coefficient y (0, 0) is represented by the bit string 0000 0001. If the value of the coefficient y (0, 3) is 128, the coefficient y (0, 3) can be binarized into a 1000 0000 bit string, that is, the coefficient y (0, 3) is represented by the bit string 1000 0000. If the value of the coefficient y (3, 3) is 255, the coefficient y (3, 3) can be binarized into a 1111 1111 bit string, that is, the coefficient y (3, 3) is represented by the bit string 1111 1111.

For a clearer understanding of the bit matrix in the embodiment of the present application, an example of a bit matrix obtained by binarizing each coefficient in a two-dimensional coefficient block including 4*4 coefficients by an encoding apparatus will be described. Referring to fig. 7, fig. 7 shows a schematic diagram of the resulting bit matrix 72 after 8-bit fixed length binarization for each coefficient in the two-dimensional coefficient block 60 of fig. 6 comprising 4*4 coefficients.

As shown in fig. 7, the encoding apparatus binarizes each coefficient in the two-dimensional coefficient block 60 at 8-bit fixed length, that is, represents one coefficient with a bit string including 8 bits (bits represented by black dots in fig. 7). As an example, the coefficient y (3, 0) in the two-dimensional coefficient block 60 may be represented by 8 bits in the bit string 73, and the coefficient y (0, 3) in the two-dimensional coefficient block 60 may be represented by 8 bits in the bit string 74. Thus, 16 coefficients in the two-dimensional coefficient block 60 may correspond to 16 bit strings, which 16 bit strings constitute a bit matrix 72 as shown in fig. 7, the bit matrix 72 comprising 4 x 8 (i.e., 128) bits.

It can be seen that if the two-dimensional coefficient block includes m×n coefficients, when the encoding apparatus binarizes the two-dimensional coefficient block with P bits of fixed length, the resulting bit matrix includes bit strings of length P and number m×n. Wherein P is a positive integer.

In the embodiment of the present application, the same position in the plurality of bit strings is referred to as a bit plane (bit plane). Thus, the bits in the same position in the plurality of bit strings are bits in the same bit plane. For example, if each of the plurality of bit strings includes P bits, then the P-th bit in each of the plurality of bit strings is a bit of the same bit plane. Wherein P is a positive integer, and P is 1-P.

Illustratively, as shown in FIG. 7, the planes 75 schematically illustrate one bit plane 75 in the bit matrix 72. The bit plane 75 includes the third bit of the 16 bit string in the bit matrix 72. For example, bit 731 in the third bit in bit string 73, and bit 741 in the third bit in bit string 74.

It should be noted that, as is clear from the above description, for any one of the two-dimensional coefficient blocks, the two-dimensional coefficient blocks include coefficients in different subbands, that is, the coefficients are in one-to-one correspondence with the subbands, and each of the two-dimensional coefficient blocks includes a coefficient in a first subband.

Thus, in one case, the encoding device may directly binarize the plurality of two-dimensional coefficient blocks to obtain a plurality of bit matrices. For any one of the plurality of bit matrices (e.g., the first bit matrix), the first bit matrix includes a plurality of bit strings representing coefficients in different subbands. I.e. for the first bit matrix, a plurality of binary bit strings comprised by the first bit matrix are used to represent coefficients in different sub-bands, respectively. For example, if the two-dimensional coefficient block for obtaining the first bit matrix includes m×n coefficients, the m×n coefficients are coefficients in m×n subbands, respectively, that is, the m×n coefficients are in one-to-one correspondence with the m×n subbands. The first bit matrix includes m×n bit strings, where the m×n bit strings are used to represent coefficients in m×n subbands, respectively. For example, referring to fig. 7, bit string 73 may be used to represent coefficient y (3, 0) in subband f13 and bit string 74 may be used to represent coefficient y (0, 3) in subband f 4. That is, a plurality of bit strings in the first bit matrix are in one-to-one correspondence with a plurality of subbands. Further, each of the plurality of bit matrices including the first bit matrix includes a bit string for representing coefficients in the first subband.

In another case, the encoding apparatus may perform subband aggregation on the plurality of two-dimensional coefficient blocks before binarizing the plurality of two-dimensional coefficient blocks such that a plurality of coefficients included in one two-dimensional coefficient block are each a coefficient in one subband, and coefficients in different two-dimensional coefficient blocks are coefficients in different subbands. Then, the encoding device binarizes the plurality of two-dimensional coefficient blocks subjected to the aggregation processing to obtain a plurality of bit matrices. For any one of the plurality of bit matrices (e.g., the first bit matrix), the first bit matrix includes a plurality of bit strings representing coefficients in a same subband (e.g., the first subband), and bit strings in different bit matrices represent coefficients in different subbands. That is, one bit matrix corresponds to one subband, and a plurality of bit matrices correspond to a plurality of subbands one by one.

Or the encoding device binarizes the two-dimensional coefficient blocks to obtain a plurality of initial bit matrixes. The encoding means may then sub-band aggregate the plurality of initial bit matrices such that one bit matrix (e.g. the first bit matrix) after aggregation comprises a plurality of bit strings representing coefficients in the same sub-band (e.g. the first sub-band). Bit strings in different bit matrices are used to represent coefficients in different sub-bands. That is, one bit matrix corresponds to one subband, and a plurality of bit matrices correspond to a plurality of subbands one to one.

Specifically, the encoding device performs subband aggregation on the plurality of two-dimensional coefficient blocks, and the specific process of performing subband aggregation on the plurality of initial bit matrices may be referred to below, which is not described herein.

S105, determining bits to be encoded in the plurality of bit matrixes according to a preset encoding sequence.

Specifically, for a plurality of bit matrices, the coding order of the plurality of bit matrices may be: the plurality of bit matrices are encoded in a zig-zag (zig-zag) order, or in other orders, such as row-by-row, column-by-column, diagonal, horizontal reverse, vertical reverse, etc.

Referring to fig. 8, fig. 8 illustrates a schematic diagram of an encoding sequence of a plurality of bit matrices. As shown in fig. 8, fig. 8 shows a top view of 4*4 bit matrices, where each small square represents a top view of one bit matrix. As shown in fig. 8 (a), the encoding apparatus may sequentially encode the 4*4 bit matrices according to the bolded-black-line-schematic word track shown in fig. 8 (a), starting from the bit matrix 81 or the bit matrix 82 shown in fig. 8 (a). Alternatively the encoding means may start with bit matrix 81 or bit matrix 82 shown in fig. 8 (b) and encode the 4*4 bit matrices sequentially according to a row-by-row trajectory indicated by the bolded black line shown in fig. 8 (a).

For different bit strings, the coding order of the different bit strings may be: different bit strings are encoded according to the frequency of the sub-band corresponding to the coefficient represented by each bit string. For example, the bit string for representing the coefficients in the low frequency sub-band is encoded first, the bit string for representing the coefficients in the second lowest frequency sub-band is encoded again, and the like, without limitation.

For a single bit string, the encoding order of the single bit string may be: the bits in a bit string are encoded according to where each bit in the bit string is located. For example, the lowest order bit in the bit string is encoded first, the next lowest order bit in the bit string is encoded, and so on. Or first encoding the most significant bit of the bit string, then encoding the next most significant bit of the bit string, etc. The embodiment of the present application is not limited thereto.

Referring to fig. 9, the bit string 90 shown in fig. 9 includes 8 bits. Wherein the lowest order bit in the bit string 90 is bit 0, the next lowest order bit in the bit string 90 is bit 1, and so on, and the highest order bit in the bit string 90 is bit 7. Therefore, when the encoding device encodes the bit string 90, the bit 0 of the lowest order may be encoded first, and the bits in the bit string 90 may be encoded one by one sequentially from the lower order to the higher order. Of course, the encoding device may encode the bit 7 of the highest order first, and sequentially encode the bits in the bit string 90 one by one from the high order to the low order. This is not limited thereto.

Based on the above described coding order of the plurality of bit matrices, the coding order of the different bit strings, and the coding order of the single bit string, the following exemplary list of several embodiments of the present application is used to code the preset coding order of each bit in the plurality of bit matrices.

A possible preset coding sequence is: among the plurality of bit matrices, the encoding apparatus starts from a first bit matrix (e.g., bit matrix 1), determines a bit string 11 corresponding to a lowest frequency subband among the bit matrices 1, and encodes the lowest bit 11 among the bit strings 11. Next, the encoding apparatus determines a bit matrix 2 to be encoded next in accordance with the encoding order (e.g., sub-encoding, line-by-line encoding, column-by-column encoding, etc.) of the plurality of bit matrices, determines a bit string 21 corresponding to the sub-band having the lowest frequency among the bit matrices 2, and encodes the bit 21 of the lowest bit among the bit strings 21. Similarly, after the encoding device encodes the bit of the lowest order bit in the bit string corresponding to the lowest frequency subband in all the bit matrices, the encoding device encodes the bit 12 of the next lowest order bit in the bit string 11 corresponding to the lowest frequency subband in the bit matrix 1, and then encodes the bit 22 of the next lowest order bit in the bit string 11 corresponding to the lowest frequency subband in the bit matrix 2. Thus, the encoding device encodes the bits of the next lower order bit string corresponding to the lowest frequency subband among all the bit matrices one by one. Similarly, when the encoding device finishes encoding all bits in the bit string corresponding to the lowest-frequency sub-band in all bit matrices, the encoding device starts encoding bits in the bit string 12 corresponding to the next lowest-frequency sub-band in the bit matrix 1. In this way, the encoding device finishes encoding all bits in the bit string corresponding to all subbands in all bit matrices.

Another possible preset coding sequence is: the encoding device starts with a first bit matrix (e.g. bit matrix 1), determines a bit string 11 corresponding to the lowest frequency subband in the bit matrix 1, encodes the lowest bit 11 in the bit string 11, and then encodes the next lowest bit 12 in the bit string 11. Similarly, after the encoding device encodes all the bits in the bit string 11, the encoding device determines the next bit matrix 2 to be encoded according to the encoding sequence (e.g., sub-encoding, line-by-line encoding, etc.) of the plurality of bit matrices, and determines the bit string 21 corresponding to the sub-band with the lowest frequency in the bit matrix 2, encodes the bit 21 with the lowest bit in the bit string 21, and then encodes the bit 22 with the next lowest bit in the bit string. And so on until the encoding means has encoded all the bits in the bit string 21. In this way, the encoding device encodes all bits in the bit string corresponding to the lowest frequency subband in all bit matrices one by one. Similarly, after the encoding device encodes all bits in the bit string corresponding to the lowest-frequency subband in all the bit matrices, the encoding device encodes bits in the bit string corresponding to the next lowest-frequency subband in the bit matrix 1, then encodes bits in the bit string corresponding to the next lowest-frequency subband in the bit matrix 2, and so on, until the encoding device encodes all bits in the bit string corresponding to all the subbands in all the bit matrices.

Yet another possible preset coding sequence is: the encoding device starts with a first bit matrix (e.g. bit matrix 1), determines the bit string 11 corresponding to the lowest frequency subband in the bit matrix 1, encodes the lowest bit 11 in the bit string 11, and then encodes the next lowest bit 12 in the bit string 11 until the encoding device has encoded all the bits in the bit string 11. Next, the encoding device determines the bit string 12 corresponding to the sub-band with the next lowest frequency in the bit matrix 1, and encodes the bit 11 of the lowest bit in the bit string 12, and then encodes the bit 12 of the next lowest bit in the bit string 12 until the encoding device finishes encoding all the bits in the bit string 12. And so on, after the encoding device encodes all bits in the bit matrix 1, determining the next bit matrix 2 to be encoded according to the encoding sequence (such as sub-encoding, line-by-line encoding, column-by-column encoding, etc.) of the plurality of bit matrices, and encoding all bits in the bit matrix 2 until the encoding device encodes all bits in the bit string corresponding to all subbands in all bit matrices.

It should be noted that the foregoing several possible preset encoding sequences are only exemplary, and do not constitute any limitation on the preset encoding sequences in the embodiments of the present application.

Based on the preset coding sequence, the coding device determines the current bit to be coded in the multiple bit matrixes determined in S104. For convenience of description, the bits to be encoded, which are determined by the encoding device, hereinafter are any one of the first bit strings in the first bit matrix, which represents one coefficient in the first subband, and the first bit matrix is any one of the plurality of bit matrices.

S106, acquiring the context of the bit to be encoded.

Specifically, the context of the bits to be encoded may include: and in the first bit string and/or the second bit string, bits in a preset range corresponding to the bits to be encoded are included. Here, the second bit string is used to represent another coefficient in the first subband.

It is understood that the context of the bits to be encoded may be all bits in the predetermined range corresponding to the bits to be encoded in the first bit string and/or the second bit string, or may be part of bits in the predetermined range corresponding to the bits to be encoded in the first bit string and/or the second bit string, which is not limited.

If the context of the bits to be encoded is a part of bits in the first bit string and/or the second bit string within a preset range corresponding to the bits to be encoded, the encoding device may directly determine the position of each bit included in the context of the bits to be encoded, without determining the position of each bit in the context through the preset range corresponding to the bits to be encoded.

For the first bit string, the preset range corresponding to the bit to be encoded may be a range spaced from the position where the bit to be encoded is located by a preset distance. For the second bit string, the preset range corresponding to the bit to be encoded may be a range based on a bit position of one bit plane co-located with the bit to be encoded and a preset distance from the position.

Alternatively, the above-mentioned preset distance may be measured by the number of bits. As an example, the position located a predetermined distance from the position where the bit to be encoded is located may be a position located j bits away from the position where the bit to be encoded is located. Thus, a range of a predetermined distance from the position of the bit to be encoded means a range between the position of j bits from the position of the bit to be encoded and the position of the bit to be encoded. Where j is a positive integer. Thus, bits within a preset range corresponding to the bits to be encoded, that is, bits included between positions separated by j bits from the positions where the bits to be encoded are located and the positions where the bits to be encoded are located.

In the following, a predetermined range corresponding to a bit to be encoded in the first bit string is exemplified as an example.

The preset range corresponding to the bit to be encoded in the first bit string may be a range in which the position of the bit to be encoded is taken as the center and two sides are respectively spaced apart by a preset distance. Or the preset range corresponding to the bit to be encoded in the first bit string may be a range taking the position of the bit to be encoded as an endpoint and having a preset distance from the position of the bit to be encoded, which is not limited. On the premise that the preset range does not exceed one bit matrix, the value of the preset distance is not particularly limited in the embodiment of the application.

Referring to fig. 9, fig. 9 (a) shows a schematic diagram of a preset range corresponding to a bit to be encoded in a first bit string, wherein the range is centered on a position where the bit to be encoded is located, and two sides are separated by a preset distance. As shown in fig. 9 (a), the bit string 90 is a first bit string, the bits to be encoded are bit 4 in the bit string 90, and the preset range corresponding to the bits to be encoded may be a range centered on the position where the bit 4 is located, separated upward from the position where the bit 4 is located by L1, and separated downward from the position where the bit 4 is located by L2. As shown in (a) of fig. 9, in this range, bit 3 and bit 5 are included. Bit 3 and bit 5 are the context bits of bit 4 to be encoded in the bit string 90. It is to be understood that the values of L1 and L2 may be the same or different, and are not limited thereto.

Referring to fig. 9, fig. 9 (b) shows a schematic diagram of a preset range corresponding to a bit to be encoded in the first bit string, where the bit to be encoded is located as an endpoint and a preset distance from the location of the bit to be encoded. As shown in fig. 9 (b), the bit string 90 is a first bit string, the bits to be encoded are bit 4 in the bit string 90, and the preset range corresponding to the bits to be encoded may be a range in which the position of the bit 4 is taken as an end point and is separated downward from the position of the bit 4 by L3. As shown in (b) of fig. 9, in this range, bit 5 and bit 6 are included. Bits 5 and 6 are the context bits of bit 4 to be encoded in the bit string 90. Of course, the preset range corresponding to the bit to be encoded may also be a range with the position of bit 4 as an end point and the distance L3 between the position of bit 4 and the end point, which is not limited.

In the first case, if a plurality of bit strings in a first bit matrix are used to represent coefficients in different subbands, respectively, the second bit string is a bit string in a second bit matrix, and the second bit matrix is a bit matrix located within a preset range of the first bit matrix among the plurality of bit matrices.

Among the plurality of bit matrices, a bit matrix located within a predetermined range of the first bit matrix may include a bit matrix adjacent to the first bit matrix, or may include a bit matrix having a distance from the first bit matrix smaller than a predetermined distance, or may include a bit matrix within a predetermined area having the first bit matrix as a base point, which is not limited. It is easy to understand that the number of bit matrices located within the first bit matrix preset range is at least one. Therefore, the second bit matrix may be one or a plurality of. Accordingly, the second bit string may be one bit string or a plurality of bit strings.

Alternatively, the distance between the first bit matrices described above may be measured in terms of the number of bit matrices. As an example, the bit matrix spaced apart from the first bit matrix by a predetermined distance may be a bit matrix spaced apart from the first bit matrix by q bit matrices. Thus, a bit matrix having a distance from the first bit matrix smaller than the predetermined distance, that is, a bit matrix having a predetermined distance from the first bit matrix, and q bit matrices spaced apart from the first bit matrix. Where q is a positive integer.

Referring to fig. 10, fig. 10 shows a schematic diagram in which a preset range of the first bit matrix is a preset area based on the first bit matrix. As shown in fig. 10, fig. 10 shows a top view of 4*4 bit matrices, where each small square is used to represent a top view of one bit matrix. As shown in fig. 10 (a), if the bit matrix 101 is a first bit matrix, a dashed box region 1001 represents a preset range of the first bit matrix among the 4*4 bit matrices. It can be seen that the first bit matrix includes a bit matrix 102, a bit matrix 103, a bit matrix 104, a bit matrix 105, a bit matrix 106, a bit matrix 107, a bit matrix 108, and a bit matrix 109 within a preset range, for a total of 8bit matrices. Or as shown in fig. 10 (b), if the bit matrix 101 is the first bit matrix, the dashed box area 1002 represents a preset range of the first bit matrix among the 4*4 bit matrices, and in this case, the preset range of the first bit matrix includes 11 bit matrices in total, namely, the bit matrix 102, the bit matrix 103, the bit matrix 104, the bit matrix 105, the bit matrix 106, the bit matrix 107, the bit matrix 108, the bit matrix 109, the bit matrix 110, and the bit matrix 111. Thus, the second bit matrix may be any one of the 8bit matrices or the 11 bit matrices, or the 8bit matrices or the 11 bit matrices are all second bit matrices.

It will be appreciated that when bits in the predetermined range corresponding to the bits to be encoded in the bit string representing the first subband are extracted from the bit matrix within the predetermined range of the first bit matrix, a context model of the bits to be encoded may be constructed.

Referring to fig. 11, fig. 11 shows a schematic diagram of a context model 110 of bits 1111 to be encoded. Wherein the bit string 111 represents a bit string in the first bit string that is within a preset range corresponding to the bits to be encoded. The bit strings in the context model 110 except the bit string 111 are all bit strings in the second bit string, which are located in a preset range corresponding to the bits to be encoded. It can be seen that the context model 110 comprises 8 second bit strings, it can be seen that the number of second bit matrices is also 8.

In the second case, if a plurality of binary bit strings in the first bit matrix are each used to represent coefficients in the first subband, the above-mentioned second bit string is one bit string in the first bit matrix that is different from the first bit string. The second bit string is a bit string located within a predetermined range of the first bit string in the first bit matrix.

In the first bit matrix, the bit strings located in the preset range of the first bit string may include bit strings adjacent to the first bit string, or may include bit strings having a distance from the first bit string smaller than a preset distance, or may include bit strings in a preset area having the first bit string as a base point, which is not limited. It is easy to understand that the number of bit strings located within the first bit string preset range is at least one. Thus, the second bit string may be one bit string or a plurality of bit strings.

Alternatively, the distance between the first bit strings described above may be measured in terms of the number of bit strings. As an example, the bit string spaced apart from the first bit string by a preset distance may be a bit string spaced apart from the first bit string by r bit strings. Thus, a bit string having a distance from the first bit string smaller than the preset distance, that is, a bit string having a preset distance from the first bit string, and r bit strings spaced apart from the first bit string, are represented. Where r is a positive integer.

As an example, referring to fig. 12, fig. 12 shows a schematic diagram in which a preset range of a first bit string is a preset region based on the first bit string in a first bit matrix. As shown in fig. 12, fig. 12 shows a top view of a first bit matrix 120 comprising 4*4 bit strings, wherein each dot represents a top view of one bit string. As shown in fig. 12 (a), if the bit string 1201 (shown as open dots) is a first bit string, a dashed box region 121 represents a preset range of the first bit string in the first bit matrix 120. It can be seen that 8 bit strings (shown as solid dots) other than the first bit string are included within the preset range of the first bit string. Or as shown in (b) of fig. 12, if the bit string 1201 is the first bit string, the dashed box area 122 represents a preset range of the first bit string in the first bit matrix 120, and at this time, 11 bit matrices other than the first bit string are included in the preset range of the first bit string. Thus, the second bit string may be any one of the 8 bit strings or the 11 bit strings, or the 8 bit strings or the 11 bit strings are each the second bit string.

It can be seen that, in the first bit matrix, bits in the bit string within the preset range corresponding to the bits to be encoded in the preset range of the first bit string may form a context model of the bits to be encoded.

Referring to fig. 13a, fig. 13a shows a schematic diagram of a context model 131 of bits 1311 to be encoded in a first bit matrix 130. The bit string 1322 includes bits in a predetermined range corresponding to the bits to be encoded in the first bit string where the bits to be encoded 1311 are located. The context model 131 further includes bits located within a preset range corresponding to the bits to be encoded among 8 second bit strings located within a preset range of the first bit string, as shown in fig. 13 a. It can be seen that the length, width, and height of the context model 131 may be less than or equal to the length, width, and height of the first bit matrix.

Optionally, the context of the bits to be encoded may include, in addition to the context described above: and in the third bit string and/or the fourth bit string, bits in a preset range corresponding to the bits to be encoded are provided. Wherein the third bit string is used to represent one coefficient in the second subband and the fourth bit string is used to represent another coefficient in the second subband. Here, the second subband is one subband different from the first subband.

In the third bit string and/or the fourth bit string, the description of the bits in the preset range corresponding to the bits to be encoded may refer to the description of the bits in the preset range corresponding to the bits to be encoded in the first bit string and/or the second bit string, which is not repeated herein.

In the first case, if a plurality of binary bit strings in a first bit matrix are used to represent coefficients in different subbands, respectively, the third bit string is a bit string in the first bit matrix and the fourth bit string is a bit string in the second bit matrix. Wherein the second bit matrix is a bit matrix located within a preset range of the first bit matrix among the plurality of bit matrices. The second sub-band is a sub-band corresponding to a bit string located within a preset range of the first bit string.

The second bit matrix is a bit matrix located in a preset range of the first bit matrix in the above multiple bit matrices, and may be referred to above, where the second bit matrix is a description related to a bit matrix located in a preset range of the first bit matrix in the above multiple bit matrices, and is not repeated herein.

For the description of the bit strings within the preset range of the first bit string, reference may be made to the above, where the second bit string is a related description of the bit strings within the preset range of the first bit string in the first bit matrix, and details are not repeated here.

In the second case, if a plurality of binary bit strings in the first bit matrix are each used to represent coefficients in the first subband, i.e. the first bit matrix corresponds to the first subband, the above-mentioned third bit string and the above-mentioned fourth bit string are each bit strings in the third bit matrix. The third bit matrix is a bit matrix located in a preset range of the first bit matrix among the plurality of bit matrices, and the second sub-band is a sub-band corresponding to the third bit matrix.

The third bit matrix is a description of a bit matrix located in the preset range of the first bit matrix in the above multiple bit matrices, and reference may be made to the above, where the second bit matrix is a description of a bit matrix located in the preset range of the first bit matrix in the above multiple bit matrices, which is not repeated here.

S107, estimating probability distribution of the bits to be encoded based on the context of the bits to be encoded.

After the encoding device obtains the context of the bit to be encoded, based on the context, the probability distribution of the bit to be encoded is estimated through a probability estimation network (or a probability estimation model, a probability estimation module, etc.).

In the above context, encoded bits may be included, and also unencoded bits may be included. If the context includes uncoded bits, the encoding means sets default values for the uncoded bits, i.e. the encoding means performs a process of setting default values for the uncoded bits. The default value may be, for example, "0", or "1", which is not limited thereto.

As an example, referring to fig. 11, in the context model 110 shown in fig. 11, a hollow ellipse may represent encoded bits, a solid perfect circle may represent uncoded bits, and for 4 uncoded bits shown in fig. 11, the encoding apparatus may set all of the 4 uncoded bits to "0".

Then, the encoding device inputs the context of the bits to be encoded of the processed uncoded bits into a probability estimation network (or a probability estimation model, a probability estimation module, etc.) to estimate the probability distribution of the current bits to be encoded. Wherein the context of the bits to be encoded for estimating the probability distribution of the bits to be encoded comprises the values of the encoded bits and the default values set by the encoding means for the uncoded bits. As an example, the probability distribution of the current bit to be encoded estimated by the encoding apparatus may be: the probability of the bit to be encoded being encoded as "0" is 0.6 and the probability of the bit to be encoded as "1" is 0.4".

The probability estimation method used in the probability estimation network may be a probability distribution estimation method based on a deep learning network. The deep learning network may be a recurrent neural network (recurrent neural network, RNN), a convolutional neural network (convolutional neural network, CNN), or the like, which is not limited thereto.

Referring to fig. 13b, fig. 13b is a schematic diagram showing the probability distribution of estimating bits to be encoded by a probability estimation network. As shown in fig. 13b, the encoding device may input the context of the bits to be encoded into the probability estimation network 130b, thereby obtaining a probability distribution of the bits to be encoded. Wherein the context of the bits to be encoded, including the values of the encoded bits, and the default values set by the encoding device for the uncoded bits, is input into the probability estimation network 130 b.

As an example, if the probability estimation method used by the probability estimation network 130b shown in fig. 13b is a probability distribution estimation method based on a deep learning network, that is, the probability estimation network 130b shown in fig. 13b is implemented by the deep learning network. Then, as shown in fig. 13c, the encoding apparatus may input the context of the bits to be encoded into the deep learning network 130c, thereby obtaining a probability distribution of the bits to be encoded. Wherein the context of the bits to be encoded, including the values of the encoded bits, input into the deep learning network 130c, and the default values set by the encoding device for the uncoded bits.

It should be noted that, the embodiment of the present application is not limited to the specific implementation of the probability estimation network or the probability estimation model. It should be understood that, all probability estimation networks or probability estimation models capable of estimating probability distribution of bits to be encoded based on the context of the bits to be encoded in the present application are included in the embodiments of the present application.

In practical application, for convenient operation, the encoding device may set default processing for the bits to be encoded. Then, the encoding device inputs the context of the processed uncoded bit and the bit to be encoded with the default value into a probability estimation network or a probability estimation model for estimating the probability distribution of the current bit to be encoded. That is, the encoding device inputs all the context model information including the bits to be encoded into a probability estimation network or probability estimation model for estimating the probability distribution of the current bits to be encoded.

It should be noted that, by the image processing method provided by the embodiment of the present application, the accuracy of estimating the probability distribution of the bit to be encoded according to the context of the bit to be encoded obtained in the binary bit matrix is higher than that of estimating the probability distribution of the bit to be encoded in the prior art.

S108, encoding the bits to be encoded according to probability distribution of the bits to be encoded so as to obtain encoding information of the images to be processed corresponding to the bit matrixes.

The encoding device performs entropy encoding on the current bit to be encoded based on the estimated probability distribution of the current bit to be encoded. For example, the encoding device arithmetically encodes (ARITHMETIC CODING) the current bit to be encoded.

When the encoding device performs entropy encoding on each bit of the plurality of bit matrices according to the image processing method provided by the embodiment of the present application, the encoding information (or referred to as an encoding code stream or a compressed code stream or a bit stream) of the image to be processed corresponding to the plurality of bit matrices is obtained.

So far, the encoding device finishes encoding a plurality of bit matrixes corresponding to the image to be processed. By the method provided by the embodiment of the application, the probability distribution of the bit to be coded estimated by the coding device has high accuracy, so that the efficiency of entropy coding is improved.

Next, a specific process of the encoding apparatus performing subband aggregation on the plurality of two-dimensional coefficient blocks will be described.

Specifically, the encoding device aggregates coefficients in the same subband in each two-dimensional coefficient block among the plurality of two-dimensional coefficient blocks acquired in S103, so as to obtain a two-dimensional coefficient block corresponding to the subband. That is, the plurality of coefficients included in the two-dimensional coefficient block corresponding to the subband are all coefficients in the subband, that is, the two-dimensional coefficient block corresponds to the subband.

As an example, if the number of the obtained two-dimensional coefficient blocks is t×s, each two-dimensional coefficient block includes m×n coefficients, and the m×n coefficients are coefficients in m×n subbands, respectively, that is, the m×n coefficients are in one-to-one correspondence with the m×n subbands. The encoding means may then extract the coefficients in the T x S first sub-bands from the T x S two-dimensional coefficient blocks to obtain a two-dimensional coefficient block corresponding to the first sub-band. Here, the first subband is any one of m×n subbands. The two-dimensional coefficient block corresponding to the first sub-band is an aggregated two-dimensional coefficient block. In this way, the encoding device may obtain m×n aggregated two-dimensional coefficient blocks, where the m×n aggregated two-dimensional coefficient blocks are in one-to-one correspondence with the m×n subbands. And, each two-dimensional coefficient block in the m×n two-dimensional coefficient blocks after aggregation includes t×s coefficients, where the t×s coefficients are coefficients in the same subband.

Referring to fig. 14, taking T as 2, s as 4, m and N as 4. As shown in fig. 14, fig. 14 (a) shows 2×4 two-dimensional coefficient blocks, and any one of the 2×4 two-dimensional coefficient blocks is Y1. Fig. 14 (b) shows a schematic diagram of a two-dimensional coefficient block Y1, the two-dimensional coefficient block Y1 including 4*4 coefficients, corresponding to 4*4 subbands. For example, coefficient y1 (0, 0) may be a coefficient in the first subband and coefficient y1 (0, 1) may be a coefficient in the second subband.

The encoding apparatus aggregates coefficients in each subband in 2×4 two-dimensional coefficient blocks shown in fig. 14 (a), and can obtain 4*4 aggregated two-dimensional coefficient blocks shown in fig. 14 (c). The 4*4 aggregated two-dimensional coefficient blocks correspond to 4*4 subbands. Of the 4*4 aggregated two-dimensional coefficient blocks, the two-dimensional coefficient block Y2 may be a two-dimensional coefficient block corresponding to the first subband. Fig. 14 (d) shows a schematic diagram of the two-dimensional coefficient block Y2. The two-dimensional coefficient block Y2 includes 2×4 coefficients, and the 2×4 coefficients are coefficients in the first subband.

In this way, the encoding apparatus performs subband aggregation on the plurality of two-dimensional coefficient blocks acquired in S103, so that one two-dimensional coefficient block includes coefficients corresponding to the same subband. In this way, the encoding device binarizes each coefficient in the aggregated two-dimensional coefficient blocks, and a plurality of bit strings in one bit matrix can be used to represent coefficients in one subband among the obtained bit matrices. Thus, when the encoding device acquires the context of the bit to be encoded, the speed of acquiring the context can be increased, and the speed of encoding the bit matrix by the encoding device can be further increased.

Next, a specific procedure of the encoding apparatus for subband-aggregating the plurality of initial bit matrices described in S104 will be described.

Specifically, the encoding apparatus binarizes each coefficient in the plurality of two-dimensional coefficient blocks acquired in S103, and then obtains a plurality of initial bit matrices. Wherein an initial bit matrix comprises a plurality of bit strings respectively representing coefficients in different sub-bands.

Then, the encoding device may aggregate, in each of the plurality of initial bit matrices, bit strings representing coefficients in the same subband to obtain a bit matrix corresponding to the subband. That is, a plurality of bit strings included in a bit matrix corresponding to the subband are used to represent coefficients in the subband, i.e., the bit matrix corresponds to the subband.

As an example, if the number of the initial bit matrices is t×s, each initial bit matrix includes m×n bit strings, where the m×n bit strings are respectively used to represent coefficients in m×n subbands, and the m×n bit strings are in one-to-one correspondence with the m×n subbands. The encoding means may then extract T x S bit strings representing coefficients in the first sub-band from the T x S initial bit matrices to obtain a bit matrix corresponding to the first sub-band. Here, the first subband is any one of m×n subbands. The bit matrix corresponding to the first sub-band is an aggregated bit matrix. In this way, the encoding device may obtain m×n aggregated bit matrices, where the m×n aggregated bit matrices are in one-to-one correspondence with the m×n subbands. And, each of the m×n aggregated bit matrices includes t×s bit strings, where the t×s bit strings are used to represent coefficients in the same subband.

Considering (a) in fig. 14 as a top view of 2×4 initial bit matrices, a small square indicates one initial bit matrix, and any initial bit matrix of the 2×4 initial bit matrices is Y1. Fig. 14 (b) shows a top view of the initial bit matrix Y1, and a small square represents a top view of a bit string. As shown in (b) of fig. 14, the initial bit matrix Y1 includes 4*4 bit strings for representing coefficients in 4*4 subbands, respectively. For example, bit string y1 (0, 0) may represent coefficients in a first subband and bit string y1 (0, 1) may represent coefficients in a second subband.

The encoding device aggregates a plurality of bit strings in 2×4 initial bit matrices shown in fig. 14 (a), and can obtain 4*4 aggregated bit matrices shown in fig. 14 (c), and a small square shown in fig. 14 (c) represents a top view of one aggregated bit matrix. In the 4*4 aggregated bit matrices, a plurality of bit strings included in any one of the aggregated bit matrices are used to represent coefficients in the same subband. I.e. an aggregated bit matrix, corresponding to one subband. Of the 4*4 aggregated bit matrices, bit matrix Y2 may be a bit matrix corresponding to the first subband. Fig. 14 (d) shows a schematic diagram of the bit matrix Y2, and a small square shown in fig. 14 (d) represents a top view of a bit string. The bit matrix Y2 comprises 2 x 4 bit strings, each of which 2 x 4 bit strings is used to represent coefficients in the first subband.

In this way, the encoding apparatus binarizes the plurality of two-dimensional coefficient blocks acquired in S103 and aggregates them, so that a plurality of bit strings in one bit matrix can be used to represent coefficients in the same subband. In this way, the encoding device can increase the speed of acquiring the context when acquiring the context of the bit to be encoded, thereby increasing the speed of encoding the bit matrix by the encoding device.

The above is an image processing method applied to an encoding apparatus provided by an embodiment of the present application, and an image processing method applied to a decoding apparatus is described below.

Referring to fig. 15, fig. 15 shows a flowchart of an image processing method according to an embodiment of the present application, which can be applied to an image decoding apparatus in an image processing apparatus. For convenience of description, an image decoding apparatus that performs the image processing method provided by the embodiment of the present application is hereinafter abbreviated as a decoding apparatus.

The method may include:

s201, determining the position of a bit to be decoded in a plurality of bit matrixes corresponding to the image to be processed.

Specifically, the decoding device may determine the position of the bit to be decoded in a plurality of bit matrixes corresponding to the image to be processed according to a preset decoding sequence.

The decoding device may be preset with a preset decoding order, which is the same as the encoding order adopted in the encoding device. The decoding device may preset information such as the sizes of a plurality of image blocks obtained by dividing the image to be processed. In this way, the decoding device can determine the size of the bit matrix corresponding to the plurality of image blocks based on the sizes of the plurality of image blocks obtained by dividing the image to be processed. It will be appreciated that the bit matrix determined by the decoding device based on the size of the image blocks obtained by partitioning the image to be processed may be a subband aggregate bit matrix (i.e. one bit matrix corresponds to one subband, i.e. a plurality of bit strings in one bit matrix are each used to represent coefficients in the same subband), or a subband non-aggregate bit matrix (i.e. one bit matrix corresponds to a plurality of subbands, i.e. a plurality of bit strings in one bit matrix are each used to represent coefficients in different subbands). Here, description of one bit matrix corresponding to one subband and one bit matrix corresponding to a plurality of subbands may be referred to above, and will not be repeated here.

It can be understood that the decoding device may further obtain size information of a plurality of image blocks obtained by dividing the image to be processed from the encoded information of the image to be processed. In this way, the decoding apparatus can determine the size of the bit matrix corresponding to the plurality of image blocks based on the size information of the plurality of image blocks acquired from the encoding information of the image to be processed.

The encoding information of the image to be processed may be encoding information of the image to be processed acquired by the decoding apparatus from the local in advance, or the encoding information of the image to be processed may be encoding information of the image to be processed transmitted by the decoding apparatus to receive other devices (for example, the wireless or wired second network communication device 23 shown in fig. 2) in real time. The embodiment of the application does not limit the time and the mode of the decoding device for acquiring the coding information of the image to be processed.

The local encoding information of the image to be processed may be encoding information of an image/video encoded by the local device through the methods of S101 to S108, or encoding information of an image to be processed transmitted by another device (for example, the wireless or wired second network communication device 23 shown in fig. 2) that is received and stored in advance by the local device, which is not limited.

Then, the decoding device may determine the position of the current bit to be decoded in the determined plurality of bit matrices according to the preset decoding order.

The process of determining, by the decoding device, the position of the current bit to be decoded in the plurality of bit matrices according to the decoding order may refer to the above step S105, where the description of the bit to be encoded is determined in the plurality of bit matrices according to the preset encoding order, which is not described herein again.

For convenience of description, the following description will be given by taking, as an example, a case where a bit to be decoded is a bit at any one position in a first bit string representing one coefficient in a first subband in a first bit matrix, which is any one of the plurality of bit matrices.

S202, acquiring the context of the bit to be decoded.

The context of the bits to be decoded may include: and in the first bit string and/or the second bit string, bits in a preset range corresponding to the bits to be decoded are included. Wherein the second bit string is used to represent another coefficient in the first subband.

It is understood that the context of the bits to be decoded may be all bits in the preset range corresponding to the bits to be decoded in the first bit string and/or the second bit string, or may be part of bits in the preset range corresponding to the bits to be decoded in the first bit string and/or the second bit string, which is not limited.

If the context of the bits to be decoded is a part of bits in the first bit string and/or the second bit string within a preset range corresponding to the bits to be decoded, the decoding device may directly output the position of each bit included in the context of the bits to be decoded, without determining the position of each bit in the context through the preset range corresponding to the bits to be decoded.

For the first bit string, the preset range corresponding to the bits to be decoded may be a range spaced apart from the position where the bits to be decoded are located by a preset distance. For the second bit string, the predetermined range corresponding to the bits to be decoded may be a range based on a bit position of one bit plane co-located with the bits to be decoded and a predetermined distance from the position.

For the description of the bits in the preset range corresponding to the bits to be decoded in the first bit string and/or the second bit string, reference may be made to the description related to the bits in the preset range corresponding to the bits to be encoded in the first bit string and/or the second bit string in S106, which is not repeated here.

Optionally, the context of the bits to be decoded may include, in addition to the context described above: and in the third bit string and/or the fourth bit string, the bits in the preset range corresponding to the bits to be decoded are obtained. Wherein the third bit string is used to represent one coefficient in the second subband and the fourth bit string is used to represent another coefficient in the second subband. Here, the second subband is one subband different from the first subband.

In the first case, if a plurality of binary bit strings in a first bit matrix are used to represent coefficients in different subbands, respectively, the third bit string is a bit string in the first bit matrix and the fourth bit string is a bit string in the second bit matrix. Or the fourth bit string is a bit string in the first bit matrix, and the third bit string is a bit string in the second bit matrix. Wherein the second bit matrix is a bit matrix located within a preset range of the first bit matrix among the plurality of bit matrices. The second sub-band is a sub-band corresponding to a bit string located within a preset range of the first bit string.

In the second case, if a plurality of binary bit strings in the first bit matrix are each used to represent coefficients in the first sub-band, i.e. the first bit matrix corresponds to the first sub-band, then the third bit string and the fourth bit string are each bit strings in the third bit matrix. The third bit matrix is a bit matrix located in a preset range of the first bit matrix among the plurality of bit matrices, and the second sub-band is a sub-band corresponding to the third bit matrix.

Specifically, for the description of the context of the bit to be decoded obtained by the decoding device, reference may be made to the description of the context of the bit to be encoded obtained by the encoding device in S106, which is not repeated here.

S203, estimating probability distribution of the bits to be decoded based on the context of the bits to be decoded.

After the decoding device obtains the context of the bits to be decoded, the probability distribution of the bits to be decoded is estimated through a probability estimation network (or a probability estimation model, a probability estimation module, etc.) based on the context.

Further, in the above context, decoded bits may be included, as well as non-decoded bits. If the context includes the undecoded bits, the decoding apparatus sets default values for the undecoded bits, i.e., the decoding apparatus performs a process of setting default values for the undecoded bits. The default value may be, for example, "0", or "1", which is not limited thereto.

As an example, referring to fig. 11, in the context model 110 shown in fig. 11, a hollow ellipse may represent decoded bits, a solid perfect circle may represent non-decoded bits, and for 4 non-decoded bits shown in fig. 11, the decoding apparatus may set all of the 4 non-decoded bits to "0".

Specifically, the decoding device may refer to the above step S107 for estimating the probability distribution of the bits to be decoded through the probability estimation network (or the probability estimation model, the probability estimation module, etc.) based on the context of the bits to be decoded, and the description of the probability distribution of the bits to be encoded based on the context of the bits to be encoded is not repeated here. It should be noted that, the context of the bits to be decoded, which is used to estimate the probability distribution of the bits to be decoded, includes the values of the decoded bits, and the default values set for the un-decoded bits by the decoding apparatus.

In practical application, for convenient operation, the decoding device may set default processing for the bits to be decoded. Then, the decoding device inputs the context of the processed un-decoded bits and the bits to be decoded with default values into a probability estimation network or a probability estimation model for estimating the probability distribution of the current bits to be decoded. That is, the decoding apparatus inputs all the context model information including the bits to be decoded into the probability estimation network or the probability estimation model for estimating the probability distribution of the current bits to be decoded.

S204, decoding the bits to be decoded according to the positions of the bits to be decoded, the probability distribution of the bits to be decoded and the coding information of the image to be processed, so as to obtain the multiple bit matrixes.

The decoding device may entropy decode the current bit to be decoded based on the position of the bit to be decoded, the estimated probability distribution of the current bit to be decoded, and the encoding information of the image to be processed. For example, the decoding apparatus arithmetically decodes (ARITHMETIC DECODING) the current bit to be decoded.

When the decoding device decodes each bit entropy in the plurality of bit matrixes based on the image processing method provided by the embodiment of the application, the plurality of bit matrixes are obtained.

S205, performing inverse binarization on the plurality of bit matrixes obtained by decoding to obtain a plurality of two-dimensional coefficient blocks.

Wherein the decoding means is preset with an inverse binarization scheme corresponding to the binarization scheme used by the encoding means in encoding. For example, in S104, the encoding apparatus binarizes each coefficient in the two-dimensional coefficient block in a binarized manner of a fixed length. Then at the decoding end, the decoding device performs inverse binarization on the bit matrix obtained by decoding in a manner inverse to the binarization manner of the fixed length.

As an example, the encoding device binarizes each coefficient in the two-dimensional coefficient block by using a binarization method with a fixed length of 8 bits, and then the decoding device restores the continuous 8 bits in the plurality of bit matrices obtained by decoding to one coefficient. For example, the decoding apparatus restores consecutive bits 0000 0001 in the bit matrix to a coefficient having a value of 1, and restores consecutive bits 1000 0000 in the bit matrix to a coefficient having a value of 128. Thus, a plurality of two-dimensional coefficient blocks corresponding to the plurality of bit matrices can be obtained.

For example, referring to fig. 7, after the bit matrix 72 in fig. 7 is inverse binarized with 8-bit fixed length, the two-dimensional coefficient block 60 shown in fig. 6 can be obtained.

In S205, any one of the two-dimensional coefficient blocks of the plurality of two-dimensional coefficient blocks obtained finally includes a plurality of coefficients in different subbands.

It should be noted that, if the decoding device determines that one of the plurality of bit matrices to be decoded corresponds to one subband, that is, the plurality of bit strings in the one bit matrix are all used to represent coefficients in the same subband. For convenience of description, the embodiment of the present application refers to a plurality of bit matrices decoded in this case as an initial bit matrix. In order to make the plurality of coefficients included in any one of the plurality of two-dimensional coefficient blocks obtained in S205 be coefficients in different subbands, respectively. The decoding apparatus also performs the following operations:

In one possible implementation, the decoding apparatus may perform inverse aggregation on the decoded multiple initial bit matrices to obtain multiple target bit matrices. For any one of the target bit matrices, the target bit matrix includes a plurality of bit strings for representing coefficients in different subbands, respectively. Then, the decoding apparatus performs inverse binarization on the plurality of target bit matrices, thereby obtaining a plurality of two-dimensional coefficient blocks as described above. The plurality of coefficients included in any one of the plurality of two-dimensional coefficient blocks are coefficients in different subbands, respectively.

Specifically, the decoding device may extract one bit string from each of the plurality of initial bit matrices, to obtain a plurality of bit strings, and use the obtained plurality of bit strings as a target bit matrix. In this way, the decoding apparatus can extract one bit string from each of the plurality of initial bit matrices by a plurality of times, thereby obtaining a plurality of target bit matrices. The plurality of bit strings included in one target bit matrix are used to represent coefficients in different subbands, respectively. The decoding device extracts the number of times of bit strings from each of a plurality of initial bit matrixes, wherein the number of times of bit strings extracted from each of the plurality of initial bit matrixes is equal to the number of bit strings included in one initial bit matrix in the plurality of initial bit matrixes obtained by decoding.

The decoding means differ each time a bit string is extracted from the same initial bit matrix. In this way, when the decoding apparatus inversely binarizes the plurality of target bit matrices obtained by the inverse aggregation process, a plurality of coefficients included in any one of the obtained plurality of two-dimensional coefficient blocks are coefficients in different subbands, respectively.

As an example, if the number of the initial bit matrices is t×s, the initial bit matrices correspond to t×s subbands. Any one of the t×s initial bit matrices includes m×n bit strings, where the m×n bit strings are used to represent m×n coefficients in the same subband. Then, the decoding device may extract a bit string from each of the t×s initial bit matrices to obtain a target bit matrix. The target bit matrix includes t×s bit strings, where the t×s bit strings are used to represent coefficients in t×s subbands, respectively. Since each initial bit matrix includes m×n bit strings, the decoding apparatus needs to perform bit string extraction m×n times, so as to obtain m×n target bit matrices. Wherein, the t×s bit strings included in any one of the m×n target bit matrices are used to represent coefficients in the t×s subbands, respectively. It will be appreciated that the bit string extracted from the initial bit matrix by the decoding means is different each time for the same initial bit matrix.

In another possible implementation manner, the decoding device may perform inverse binarization on the decoded multiple initial bit matrices to obtain multiple initial two-dimensional coefficient blocks. Then, the decoding device performs inverse aggregation on the plurality of initial two-dimensional coefficient blocks, thereby obtaining a plurality of two-dimensional coefficient blocks. The plurality of coefficients included in any one of the plurality of two-dimensional coefficient blocks are coefficients in different subbands, respectively.

Specifically, the decoding apparatus may extract one coefficient from each of the plurality of initial two-dimensional coefficient blocks, and take the extracted plurality of coefficients as one target two-dimensional coefficient block. In this way, the decoding apparatus can obtain a plurality of target two-dimensional coefficient blocks by extracting one coefficient from each of the plurality of initial two-dimensional coefficient blocks a plurality of times. Thus, the plurality of coefficients included in one target two-dimensional coefficient block are coefficients in different subbands, respectively. The decoding device extracts the number of times of coefficients from each of the plurality of initial two-dimensional coefficient blocks, wherein the number of times of extracting the coefficients from each of the plurality of initial two-dimensional coefficient blocks is equal to the number of coefficients included in one of the plurality of initial two-dimensional coefficient blocks.

The decoding device extracts coefficients from the initial two-dimensional coefficient block each time, for the same initial two-dimensional coefficient block. In this way, the decoding apparatus inversely aggregates the plurality of initial two-dimensional coefficient blocks, and the plurality of coefficients included in any one of the plurality of obtained two-dimensional coefficient blocks are coefficients in different subbands.

As an example, if the number of the initial two-dimensional coefficient blocks is t×s, the initial two-dimensional coefficient blocks correspond to t×s subbands. Any one of the t×s initial two-dimensional coefficient blocks includes m×n coefficients, where the m×n coefficients are m×n coefficients in the same subband. Then, the decoding device may extract a coefficient from each of the t×s initial two-dimensional coefficient blocks to obtain a target two-dimensional coefficient block. The target two-dimensional coefficient block includes t×s coefficients, where the t×s coefficients are coefficients in t×s subbands, respectively. Since each initial two-dimensional coefficient block includes m×n coefficients, the decoding apparatus needs to perform coefficient extraction m×n times, so as to obtain m×n target two-dimensional coefficient blocks. Wherein, the t×s coefficients included in any one of the m×n target two-dimensional coefficient blocks are coefficients in t×s subbands, respectively. It will be appreciated that the coefficients extracted from the initial two-dimensional coefficient block by the decoding device are different each time for the same initial two-dimensional coefficient block.

S206, inversely transforming the two-dimensional coefficient blocks to obtain a plurality of image blocks, and restoring the image to be processed based on the image blocks.

The image blocks are obtained by dividing the image to be processed.

The decoding device may preset an inverse transformation scheme that is inverse to the transformation scheme of the encoding device, and inverse-transform the plurality of two-dimensional coefficient blocks by the inverse transformation scheme to obtain a plurality of image blocks. Of course, if the encoded information acquired in S201 is encoded information obtained by encoding a bit matrix corresponding to the residual block, the decoding apparatus performs inverse transform on the plurality of two-dimensional coefficient blocks in S206, and then obtains a plurality of residual blocks. Here, the residual block is a plurality of residual blocks corresponding to a plurality of image blocks obtained by dividing the image to be processed. Further, the decoding apparatus restores the plurality of image blocks based on the plurality of residual blocks.

Then, the decoding device determines the number of image blocks corresponding to the image to be processed according to the size of the image to be processed, which is obtained from the encoding information of the image to be processed in advance, and the sizes of a plurality of image blocks obtained by dividing the image to be processed. In this way, the decoding device can restore the image to be processed according to the size of the image to be processed and the number of image blocks corresponding to the image to be processed.

The image processing method provided by the embodiment of the application realizes the decoding of the encoded information obtained by S101-108 encoding.

In summary, according to the image processing method provided by the embodiment of the application, the corresponding plurality of three-dimensional bit matrixes are obtained by binarizing the plurality of two-dimensional coefficient blocks corresponding to the image block or the residual block. The encoding device then obtains the context of the current bit to be encoded from the plurality of three-dimensional bit matrices and estimates the probability distribution of the bit to be encoded based on the context. By the method provided by the embodiment of the application, the accuracy of estimating the probability distribution of the current bit to be coded is improved. Therefore, the coding device carries out entropy coding on the current bit to be coded based on the probability distribution estimated by the bit to be coded and having higher accuracy than the prior art, so that the coding efficiency of the image (or video) can be improved, further the storage resource for storing the image (or video) is saved, and the transmission bandwidth for transmitting the image (or video) is saved.

The foregoing description of the solution provided by the embodiments of the present application has been mainly presented in terms of a method. To achieve the above functions, it includes corresponding hardware structures and/or software modules that perform the respective functions. Those of skill in the art will readily appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as hardware or combinations of hardware and computer software. Whether a function is implemented as hardware or computer software driven hardware depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

The embodiment of the present application may divide the functional modules of the image processing apparatus according to the above-described method example, for example, each functional module may be divided corresponding to each function, or two or more functions may be integrated in one processing module. The integrated modules may be implemented in hardware or in software functional modules. It should be noted that, in the embodiment of the present application, the division of the modules is schematic, which is merely a logic function division, and other division manners may be implemented in actual implementation.

As shown in fig. 16, fig. 16 shows a schematic structural diagram of an image processing apparatus 160 according to an embodiment of the present application. The image processing apparatus 160 may be used to perform the image encoding method in the image processing method described above, for example, to perform the method shown in fig. 4. The image processing apparatus 160 may include a determination unit 161, an estimation unit 162, and an encoding unit 163, among others.

A determining unit 161, configured to determine bits to be encoded. Wherein the bit to be encoded is any one bit of a first bit string, the first bit string is any one binary bit string of a plurality of binary bit strings included in a first bit matrix, the first bit string is used for representing one coefficient in a first subband, the first bit matrix is any one bit matrix of a plurality of bit matrices obtained by processing an image to be processed, and the first bit matrix is a three-dimensional bit matrix. An estimating unit 162, configured to estimate a probability distribution of the bits to be encoded based on the context of the bits to be encoded. Here, the context of the bits to be encoded includes: and in the first bit string and/or the second bit string, bits in a preset range corresponding to the bits to be encoded are provided. Wherein the second bit string is used to represent another coefficient in the first subband. The encoding unit 163 is configured to encode the bits to be encoded according to the probability distribution of the bits to be encoded, so as to obtain encoding information of the image to be processed.

As an example, in connection with fig. 4, the determining unit 161 may be used to perform S105, the estimating unit 162 may be used to perform S107, and the encoding unit 163 may be used to perform S108.

Alternatively, if the plurality of binary bit strings are used to represent coefficients in different subbands, respectively, the second bit string is a binary bit string in a second bit matrix. Here, the second bit matrix is a bit matrix located within a preset range of the first bit matrix among the plurality of bit matrices. Or if the plurality of binary bit strings are used for representing coefficients in the first sub-band, the second bit string is a binary bit string located within a preset range of the first bit string in the first bit matrix.

Optionally, the context of the bits to be encoded further includes: and in the third bit string and/or the fourth bit string, bits in a preset range corresponding to the bits to be encoded are provided. The third bit string is used to represent one coefficient in the second subband and the fourth bit string is used to represent another coefficient in the second subband. Wherein,

If the above-mentioned plurality of binary bit strings are each used to represent coefficients in the first sub-band, i.e. the plurality of bit matrices are in one-to-one correspondence with the plurality of sub-bands, then the third bit string and the fourth bit string are each a binary bit string in the third bit matrix. The third bit matrix is a bit matrix of the plurality of bit matrices within a preset range of the first bit matrix, and the second sub-band is a sub-band corresponding to the third bit matrix.

Optionally, the image processing apparatus 160 further includes: an obtaining unit 164, configured to obtain a plurality of two-dimensional coefficient blocks corresponding to the image to be processed before the determining unit 161 determines the bit to be encoded. Wherein each of the plurality of two-dimensional coefficient blocks includes a plurality of coefficients, the plurality of coefficients being coefficients in different subbands, respectively. A binarizing unit 165 for binarizing each coefficient of the plurality of two-dimensional coefficient blocks to obtain the plurality of bit matrices.

As an example, in connection with fig. 4, the acquisition unit 164 may be used to perform S103 and the binarization unit 165 may be used to perform S104.

Optionally, the binarizing unit 165 is specifically configured to binarize each coefficient in the plurality of two-dimensional coefficient blocks according to a binarization manner of a fixed length.

As an example, in connection with fig. 4, the binarization unit 165 may be used to perform S104.

Optionally, the acquiring unit 164 is further configured to acquire the image to be processed before acquiring the plurality of two-dimensional coefficient blocks corresponding to the image to be processed. Wherein the image to be processed comprises an image frame in a picture or video. The obtaining unit 164 is further specifically configured to block and transform the image to be processed, so as to obtain a plurality of two-dimensional coefficient blocks.

As an example, in connection with fig. 4, the acquisition unit 164 may be used to perform S101 and S103.

Optionally, the determining unit 161 is specifically configured to determine the bits to be encoded in the plurality of bit matrices according to a preset encoding sequence.

As an example, in connection with fig. 4, the determination unit 161 may be used to perform S105.

Optionally, the estimating unit 162 is specifically configured to estimate, based on the context of the bits to be encoded, a probability distribution of the bits to be encoded through a probability estimation network.

As an example, in connection with fig. 4, the estimation unit 162 may be used to perform S107.

Optionally, the encoding unit 163 is specifically configured to entropy encode the bits to be encoded according to probability distribution of the bits to be encoded.

As an example, in connection with fig. 4, the encoding unit 163 may be used to perform S108.

For a specific description of the above alternative modes, reference may be made to the foregoing method embodiments, and details are not repeated here. In addition, any explanation and description of the beneficial effects of the image processing apparatus 160 provided above may refer to the corresponding method embodiments described above, and will not be repeated.

As an example, in connection with fig. 3, the determining unit 161, estimating unit 162, encoding unit 163, acquiring unit 164, and binarizing unit 165 in the image processing apparatus 160 may be realized by the processor 301 in fig. 3 executing the program codes in the memory 302 in fig. 3.

As shown in fig. 17, fig. 17 is a schematic diagram showing the structure of an image processing apparatus 170 according to an embodiment of the present application. The image processing apparatus 170 may be used to perform an image decoding method among the above-described image processing methods, for example, to perform the method shown in fig. 15. The image processing apparatus 170 may include a determination unit 171, an estimation unit 172, and a decoding unit 173, among others.

A determining unit 171 for determining the position of the bit to be decoded. Wherein the bit to be decoded is any one bit of a first bit string, the first bit string is any one binary bit string of a plurality of binary bit strings included in a first bit matrix, the first bit string is used for representing one coefficient in a first subband, the first bit matrix is any one bit matrix of a plurality of bit matrices corresponding to the image to be processed, and the first bit matrix is a three-dimensional bit matrix. An estimating unit 172, configured to estimate a probability distribution of the bits to be decoded based on the context of the bits to be decoded. Here, the context of the bits to be decoded includes: and in the first bit string and/or the second bit string, the bits in the preset range corresponding to the bits to be decoded are selected. Wherein the second bit string is used to represent another coefficient in the first subband. The decoding unit 173 is configured to decode the bits to be decoded based on the positions of the bits to be decoded, the probability distribution of the bits to be decoded, and the encoding information of the image to be processed, so as to obtain a plurality of bit matrices.

As an example, in connection with fig. 15, the determining unit 171 may be used to perform S201, the estimating unit 172 may be used to perform S203, and the decoding unit 173 may be used to perform S204.

Optionally, the context of the bits to be decoded further includes: and in the third bit string and/or the fourth bit string, the bits in the preset range corresponding to the bits to be decoded are obtained. The third bit string is used to represent one coefficient in the second subband and the fourth bit string is used to represent another coefficient in the second subband. Wherein,

If the above-mentioned plurality of binary bit strings are used to represent coefficients in different sub-bands, respectively, the third bit string is a binary bit string in the first bit matrix and the fourth bit string is a binary bit string in the second bit matrix. The second bit matrix is a bit matrix in a preset range of the first bit matrix in the plurality of bit matrices, and the second sub-band is a sub-band corresponding to a binary bit string in the preset range of the first bit string. Or alternatively

Optionally, the image processing apparatus 170 further includes: an inverse binarization unit 174, configured to inverse binarize the binary bit strings in the plurality of bit matrices obtained by decoding, so as to obtain a plurality of two-dimensional coefficient blocks corresponding to the plurality of bit matrices. Wherein each two-dimensional coefficient block of the plurality of two-dimensional coefficient blocks includes a plurality of coefficients, the plurality of coefficients being coefficients in different subbands, respectively. An inverse transform unit 175 for inversely transforming the plurality of two-dimensional coefficient blocks to obtain an image to be processed. Wherein the image to be processed comprises an image frame in a picture or video.

As an example, in connection with fig. 15, the inverse binarization unit 174 may be used to perform S205, and the inverse transformation unit 175 may be used to perform S206.

Optionally, the inverse binarization unit 174 is specifically configured to inverse binarize the binary bit strings in the plurality of bit matrices obtained by decoding according to a fixed-length inverse binarization manner.

As an example, in connection with fig. 15, the inverse binarization unit 174 may be used to perform S205.

Optionally, the determining unit 171 is specifically configured to determine the position of the bit to be decoded in the plurality of bit matrices according to a preset decoding order.

As an example, in connection with fig. 15, the determination unit 171 may be used to perform S201.

Optionally, the estimating unit 172 is specifically configured to estimate, based on the context of the bits to be decoded, a probability distribution of the bits to be decoded through the probability estimating network.

As an example, in connection with fig. 15, the estimation unit 172 may be used to perform S203.

Optionally, the decoding unit 173 is specifically configured to entropy decode the bits to be decoded based on the positions of the bits to be decoded, the probability distribution of the bits to be decoded, and the encoding information of the image to be processed.

As an example, in connection with fig. 15, the decoding unit 173 may be used to perform S204.

For a specific description of the above alternative modes, reference may be made to the foregoing method embodiments, and details are not repeated here. In addition, any explanation and description of the beneficial effects of the image processing apparatus 170 provided above may refer to the corresponding method embodiments described above, and will not be repeated.

As an example, in connection with fig. 3, the determining unit 171, the estimating unit 172, the decoding unit 173, the inverse binarizing unit 174, and the inverse transforming unit 175 in the image processing apparatus 170 may be implemented by the processor 301 in fig. 3 executing the program codes in the memory 302 in fig. 3.

Embodiments of the present application also provide a chip system 180, as shown in fig. 18, the chip system 180 including at least one processor and at least one interface circuit. By way of example, when the chip system 180 includes one processor and one interface circuit, the one processor may be the processor 181 shown in the solid line box (or the processor 181 shown in the broken line box) in fig. 18, and the one interface circuit may be the interface circuit 182 shown in the solid line box (or the interface circuit 182 shown in the broken line box) in fig. 18. When the chip system 180 includes two processors including a processor 181 shown in a solid line box and a processor 181 shown in a broken line box in fig. 18, and two interface circuits including an interface circuit 182 shown in a solid line box and an interface circuit 182 shown in a broken line box in fig. 18. This is not limited thereto.

The processor 181 and the interface circuit 182 may be interconnected by wires. For example, the interface circuit 182 may be used to receive signals (e.g., acquire images to be processed, etc.). For another example, the interface circuit 182 may be used to send signals to other devices, such as the processor 181. The interface circuit 182 may, for example, read instructions stored in a memory and send the instructions to the processor 181. The instructions, when executed by the processor 181, may cause the image processing apparatus to perform the steps of the embodiments described above. Of course, the system-on-chip 180 may also include other discrete devices, which are not particularly limited in accordance with embodiments of the present application.

Another embodiment of the present application also provides a computer readable storage medium having stored therein instructions which, when executed on an image processing apparatus, perform the steps performed by the image processing apparatus in the method flow shown in the above-described method embodiment.

In some embodiments, the disclosed methods may be implemented as computer program instructions encoded on a computer-readable storage medium in a machine-readable format or encoded on other non-transitory media or articles of manufacture.

Fig. 19 schematically illustrates a conceptual partial view of a computer program product comprising a computer program for executing a computer process on a computing device, provided by an embodiment of the application.

In one embodiment, the computer program product is provided using a signal bearing medium 190. The signal bearing medium 190 may include one or more program instructions that when executed by one or more processors may provide the functionality or portions of the functionality described above with respect to fig. 4 or 15. Thus, for example, reference to S101-S108 in FIG. 4, or reference to one or more features of S201-S206 in FIG. 15, may be undertaken by one or more instructions associated with the signal bearing medium 190. Further, the program instructions in fig. 19 also describe example instructions.

In some examples, signal bearing medium 190 may comprise a computer readable medium 191 such as, but not limited to, a hard disk drive, a Compact Disk (CD), a Digital Video Disk (DVD), a digital magnetic tape, memory, read-only memory (ROM), or random access memory (random access memory, RAM), among others.

In some implementations, the signal bearing medium 190 may comprise a computer recordable medium 192 such as, but not limited to, memory, read/write (R/W) CD, R/W DVD, and the like.

In some implementations, the signal bearing medium 190 may include a communication medium 193 such as, but not limited to, a digital and/or analog communication medium (e.g., fiber optic cable, waveguide, wired communications link, wireless communications link, etc.).

The signal bearing medium 190 may be conveyed by a communication medium 193 in wireless form (e.g., a wireless communication medium conforming to the IEEE 1902.11 standard or other transmission protocol). The one or more program instructions may be, for example, computer-executable instructions or logic-implemented instructions.

In some examples, an image processing apparatus such as described with respect to fig. 4 or 15 may be configured to provide various operations, functions, or actions in response to program instructions through one or more of computer readable medium 191, computer recordable medium 192, and/or communication medium 193.

It should be understood that the arrangement described herein is for illustrative purposes only. Thus, those skilled in the art will appreciate that other arrangements and other elements (e.g., machines, interfaces, functions, orders, and groupings of functions, etc.) can be used instead, and some elements may be omitted altogether depending on the desired results. In addition, many of the elements described are functional entities that may be implemented as discrete or distributed components, or in any suitable combination and location in conjunction with other components.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using a software program, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. The processes or functions in accordance with embodiments of the present application are produced in whole or in part on and when the computer-executable instructions are executed by a computer. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, a website, computer, server, or data center via a wired (e.g., coaxial cable, fiber optic, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. Computer readable storage media can be any available media that can be accessed by a computer or data storage devices including one or more servers, data centers, etc. that can be integrated with the media. Usable media may be magnetic media (e.g., floppy disks, hard disks, magnetic tape), optical media (e.g., DVD), or semiconductor media (e.g., solid State Disk (SSD)) or the like.

The foregoing is merely illustrative of the present invention, and the present invention is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. An image processing method, comprising:

Determining bits to be encoded; wherein the bit to be encoded is any bit in a first bit string, the first bit string is any bit in a plurality of binary bit strings included in a first bit matrix, the first bit string is used for representing one coefficient in a first subband, the first bit matrix is any bit matrix in a plurality of bit matrices obtained by processing an image to be processed, and the first bit matrix is a three-dimensional bit matrix;

estimating probability distribution of the bits to be encoded based on the context of the bits to be encoded; the context includes: bits in a preset range corresponding to the bits to be encoded in the first bit string and/or the second bit string; wherein the second bit string is used to represent another coefficient in the first subband;

And encoding the bits to be encoded according to the probability distribution of the bits to be encoded so as to obtain the encoding information of the image to be processed.

2. The method of claim 1, wherein the step of determining the position of the substrate comprises,

The second bit string is a binary bit string in a second bit matrix if the plurality of binary bit strings are each used to represent coefficients in a different sub-band; the second bit matrix is a bit matrix of the plurality of bit matrices located within a preset range of the first bit matrix; or alternatively

If the plurality of binary bit strings are each used to represent a coefficient in the first subband, the second bit string is a binary bit string within the first bit string preset range in the first bit matrix.

3. The method of claim 1 or 2, wherein the context further comprises: a third bit string and/or a fourth bit string, wherein the third bit string is used for representing one coefficient in a second sub-band, and the fourth bit string is used for representing the other coefficient in the second sub-band, and the bits are in a preset range and correspond to the bits to be coded; wherein,

If the plurality of binary bit strings are used to represent coefficients in different subbands, respectively, the third bit string is a binary bit string in the first bit matrix, the fourth bit string is a binary bit string in a second bit matrix, and the second bit matrix is a bit matrix of the plurality of bit matrices that is within a preset range of the first bit matrix; the second sub-band is a sub-band corresponding to a binary bit string located in the preset range of the first bit string; or alternatively

If the plurality of binary bit strings are each used to represent a coefficient in the first subband, the third bit string and the fourth bit string are each a binary bit string in a third bit matrix, the third bit matrix being a bit matrix of the plurality of bit matrices that is within a preset range of the first bit matrix; the second sub-band is a sub-band corresponding to the third bit matrix, and the plurality of bit matrices are in one-to-one correspondence with the plurality of sub-bands.

4. The method according to claim 1 or 2, characterized in that before said determining the bits to be encoded, the method further comprises:

Acquiring a plurality of two-dimensional coefficient blocks corresponding to the image to be processed, wherein each two-dimensional coefficient block in the plurality of two-dimensional coefficient blocks comprises a plurality of coefficients which are coefficients in different sub-bands respectively;

Binarizing each coefficient of the plurality of two-dimensional coefficient blocks to obtain the plurality of bit matrices.

5. The method of claim 4, wherein binarizing each coefficient of the plurality of two-dimensional coefficient blocks comprises:

And binarizing each coefficient in the two-dimensional coefficient blocks according to a binarization mode of fixed length.

6. The method of claim 4, wherein prior to said acquiring the corresponding plurality of two-dimensional coefficient blocks of the image to be processed, the method further comprises:

Acquiring the image to be processed, wherein the image to be processed comprises an image frame in a picture or a video;

The obtaining a plurality of two-dimensional coefficient blocks corresponding to the image to be processed comprises the following steps:

and performing blocking and transformation on the image to be processed to obtain the plurality of two-dimensional coefficient blocks.

7. The method according to claim 1 or 2, wherein said determining the bits to be encoded comprises:

And determining the bits to be coded in the plurality of bit matrixes according to a preset coding sequence.

8. The method according to claim 1 or 2, wherein said estimating a probability distribution of said bits to be encoded based on a context of said bits to be encoded comprises:

And estimating the probability distribution of the bits to be encoded through a probability estimation network based on the context of the bits to be encoded.

9. The method according to claim 1 or 2, wherein said encoding said bits to be encoded according to a probability distribution of said bits to be encoded comprises:

And carrying out entropy coding on the bits to be coded according to the probability distribution of the bits to be coded.

10. An image processing method, comprising:

Determining the position of a bit to be decoded; wherein the bits to be decoded are any one bit of a first bit string, the first bit string is any one bit string of a plurality of binary bit strings included in a first bit matrix, the first bit string is used for representing one coefficient in a first subband, the first bit matrix is any one bit matrix of a plurality of bit matrices corresponding to an image to be processed, and the first bit matrix is a three-dimensional bit matrix;

Estimating probability distribution of the bits to be decoded based on the context of the bits to be decoded; the context includes: bits in a preset range corresponding to the bits to be decoded in the first bit string and/or the second bit string; wherein the second bit string is used to represent another coefficient in the first subband;

and decoding the bits to be decoded based on the positions of the bits to be decoded, the probability distribution of the bits to be decoded and the coding information of the image to be processed, so as to obtain the bit matrixes.

11. The method of claim 10, wherein the step of determining the position of the first electrode is performed,

12. The method of claim 10 or 11, wherein the context further comprises: a third bit string and/or a fourth bit string, wherein the third bit string is used for representing one coefficient in a second sub-band, and the fourth bit string is used for representing the other coefficient in the second sub-band, and the bits are in a preset range and correspond to the bits to be decoded; wherein,

13. The method according to claim 10 or 11, characterized in that the method further comprises:

Inversely binarizing binary bit strings in the plurality of bit matrixes obtained by decoding to obtain a plurality of two-dimensional coefficient blocks corresponding to the plurality of bit matrixes; wherein each two-dimensional coefficient block of the plurality of two-dimensional coefficient blocks includes a plurality of coefficients, the plurality of coefficients being coefficients in different subbands, respectively;

And carrying out inverse transformation on the two-dimensional coefficient blocks to obtain the image to be processed, wherein the image to be processed comprises an image frame in a picture or a video.

14. The method of claim 13, wherein said inverse binarizing the decoded binary bit strings in the plurality of bit matrices comprises:

And according to the fixed-length inverse binarization mode, inversely binarizing the binary bit strings in the plurality of bit matrixes obtained by decoding.

15. The method according to claim 10 or 11, wherein said determining the position of the bits to be decoded comprises:

And determining the position of the bit to be decoded in the plurality of bit matrixes according to a preset decoding sequence.

16. The method according to claim 10 or 11, wherein said estimating a probability distribution of said bits to be decoded based on a context of said bits to be decoded comprises:

And estimating the probability distribution of the bits to be decoded through a probability estimation network based on the context of the bits to be decoded.

17. The method according to claim 10 or 11, wherein the decoding the bits to be decoded based on the positions of the bits to be decoded, the probability distribution of the bits to be decoded, and the encoding information of the image to be processed, comprises:

And carrying out entropy decoding on the bits to be decoded based on the positions of the bits to be decoded, the probability distribution of the bits to be decoded and the coding information of the image to be processed.

18. An image processing apparatus, characterized in that the apparatus comprises:

a determining unit, configured to determine a bit to be encoded; wherein the bit to be encoded is any bit in a first bit string, the first bit string is any bit in a plurality of binary bit strings included in a first bit matrix, the first bit string is used for representing one coefficient in a first subband, the first bit matrix is any bit matrix in a plurality of bit matrices obtained by processing an image to be processed, and the first bit matrix is a three-dimensional bit matrix;

an estimating unit, configured to estimate a probability distribution of the bits to be encoded based on a context of the bits to be encoded; the context includes: bits in a preset range corresponding to the bits to be encoded in the first bit string and/or the second bit string; wherein the second bit string is used to represent another coefficient in the first subband;

And the encoding unit is used for encoding the bits to be encoded according to the probability distribution of the bits to be encoded so as to obtain the encoding information of the image to be processed.

19. The apparatus of claim 18, wherein the device comprises a plurality of sensors,

20. The apparatus of claim 18 or 19, wherein the context further comprises: a third bit string and/or a fourth bit string, wherein the third bit string is used for representing one coefficient in a second sub-band, and the fourth bit string is used for representing the other coefficient in the second sub-band, and the bits are in a preset range and correspond to the bits to be coded; wherein,

21. The apparatus according to claim 18 or 19, characterized in that the apparatus further comprises:

An obtaining unit, configured to obtain a plurality of two-dimensional coefficient blocks corresponding to the to-be-processed image before the determining unit determines the to-be-encoded bit, where each two-dimensional coefficient block in the plurality of two-dimensional coefficient blocks includes a plurality of coefficients, and the plurality of coefficients are coefficients in different subbands respectively;

and a binarization unit for binarizing each coefficient of the plurality of two-dimensional coefficient blocks to obtain the plurality of bit matrices.

22. The apparatus of claim 21, wherein the device comprises a plurality of sensors,

The binarization unit is specifically configured to binarize each coefficient in the plurality of two-dimensional coefficient blocks according to a binarization manner of a fixed length.

23. The apparatus of claim 21, wherein the device comprises a plurality of sensors,

The obtaining unit is further configured to obtain the image to be processed before obtaining a plurality of two-dimensional coefficient blocks corresponding to the image to be processed, where the image to be processed includes an image frame in a picture or a video;

The obtaining unit is further specifically configured to block and transform the image to be processed, so as to obtain the plurality of two-dimensional coefficient blocks.

24. The device according to claim 18 or 19, wherein,

The determining unit is specifically configured to determine the bits to be encoded in the plurality of bit matrices according to a preset encoding sequence.

25. The device according to claim 18 or 19, wherein,

The estimation unit is specifically configured to estimate, based on the context of the bit to be encoded, a probability distribution of the bit to be encoded through a probability estimation network.

26. The device according to claim 18 or 19, wherein,

The coding unit is specifically configured to entropy code the bit to be coded according to probability distribution of the bit to be coded.

27. An image processing apparatus, characterized in that the apparatus comprises:

A determining unit configured to determine a position of a bit to be decoded; wherein the bits to be decoded are any one bit of a first bit string, the first bit string is any one bit string of a plurality of binary bit strings included in a first bit matrix, the first bit string is used for representing one coefficient in a first subband, the first bit matrix is any one bit matrix of a plurality of bit matrices corresponding to an image to be processed, and the first bit matrix is a three-dimensional bit matrix;

An estimating unit, configured to estimate a probability distribution of the bits to be decoded based on a context of the bits to be decoded; the context includes: bits in a preset range corresponding to the bits to be decoded in the first bit string and/or the second bit string; wherein the second bit string is used to represent another coefficient in the first subband;

And the decoding unit is used for decoding the bits to be decoded based on the positions of the bits to be decoded, the probability distribution of the bits to be decoded and the coding information of the image to be processed so as to obtain the bit matrixes.

28. The apparatus of claim 27, wherein the device comprises a plurality of sensors,

29. The apparatus of claim 27 or 28, wherein the context further comprises: a third bit string and/or a fourth bit string, wherein the third bit string is used for representing one coefficient in a second sub-band, and the fourth bit string is used for representing the other coefficient in the second sub-band, and the bits are in a preset range and correspond to the bits to be decoded; wherein,

30. The apparatus according to claim 27 or 28, characterized in that the apparatus further comprises:

The inverse binarization unit is used for inversely binarizing binary bit strings in the plurality of bit matrixes obtained by decoding to obtain a plurality of two-dimensional coefficient blocks corresponding to the plurality of bit matrixes; wherein each two-dimensional coefficient block of the plurality of two-dimensional coefficient blocks includes a plurality of coefficients, the plurality of coefficients being coefficients in different subbands, respectively;

And the inverse transformation unit is used for inversely transforming the two-dimensional coefficient blocks to obtain the image to be processed, wherein the image to be processed comprises an image frame in a picture or a video.

31. The apparatus of claim 30, wherein the device comprises a plurality of sensors,

The inverse binarization unit is specifically configured to inverse binarize the binary bit strings in the plurality of bit matrices obtained by decoding according to an inverse binarization mode of fixed length.

32. The apparatus of claim 27 or 28, wherein the device comprises a plurality of sensors,

The determining unit is specifically configured to determine, according to a preset decoding order, a position of the bit to be decoded in the plurality of bit matrices.

33. The apparatus of claim 27 or 28, wherein the device comprises a plurality of sensors,

The estimation unit is specifically configured to estimate, based on the context of the bit to be decoded, a probability distribution of the bit to be decoded through a probability estimation network.

34. The apparatus of claim 27 or 28, wherein the device comprises a plurality of sensors,

The decoding unit is specifically configured to entropy decode the bits to be decoded based on the positions of the bits to be decoded, probability distribution of the bits to be decoded, and encoding information of the image to be processed.

35. An image processing apparatus, characterized in that the apparatus comprises: a memory for storing computer instructions and one or more processors for invoking the computer instructions to perform the method of any of claims 1 to 17.

36. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a computer program which, when run on a computer, causes the computer to perform the method of any of claims 1 to 17.