WO2020168501A1

WO2020168501A1 - Image encoding method and decoding method, and device and system to which said methods are applicable

Info

Publication number: WO2020168501A1
Application number: PCT/CN2019/075642
Authority: WO
Inventors: 钮旋; 周新生; 阮俊瑾; 朱怀安; 李翔
Original assignee: 上海极清慧视科技有限公司
Priority date: 2019-02-21
Filing date: 2019-02-21
Publication date: 2020-08-27
Also published as: CN111316644A; CN111316644B

Abstract

The present application provides an image encoding method and decoding method, and a device and a system to which said methods are applicable. Said encoding method comprises: dividing acquired image data into a plurality of bit plane matrix data according to binary data bits; performing serialization processing on bit plane matrix data of at least a part of bit planes on the basis of a preset serialization period to obtain bit plane serialized data, the serialization period being a period set by serializing a preset m*n matrix according to neighboring data; and encoding the obtained bit plane serialized data, and generating encoded image data of the image data. The present application uses a serialization period to implement serialization processing on bit plane matrix data, and is beneficial in improving the cohesiveness of an original image, particularly the cohesiveness of high definition images of 4K or more.

Description

Image coding method, decoding method and applicable equipment and system

Technical field

This application relates to the field of image processing technology, and in particular to an image encoding method, decoding method, and applicable equipment and systems.

Background technique

High-definition images have a wide range of applications, such as urban security, medical imaging, and event broadcasting. For this reason, front-end equipment, such as infrared cameras, array cameras, etc., are usually equipped with lenses and image chips that can capture 4K or even above 4K. However, high-definition video files acquired by front-end equipment pose a challenge to data storage and network transmission due to the huge amount of data. For this reason, the front-end equipment usually uses an image encoding method to encode and compress the acquired high-definition images, in the hope of reducing the data volume of the original image data.

However, the existing image coding standards, such as H.264, etc., are all lossy compression, which makes it a pair of contradictory requirements that both want to obtain high-definition images and have to lose part of the high-definition images for transmission and storage.

Summary of the invention

In view of the above-mentioned shortcomings of the prior art, the purpose of this application is to provide an image encoding method, decoding method, and applicable equipment and systems, which are used to solve the problem of the large amount of high-definition image data in the prior art that is difficult to save. And transmission problems.

In order to achieve the above and other related purposes, the first aspect of this application provides an image encoding method, which includes: dividing the acquired image data into multiple bit-plane matrix data according to binary data bits; and based on preset serialization Cycle, serialize at least part of the bit-plane matrix data of the bit-plane to obtain bit-plane sequence data; wherein, the serialization cycle is a cycle set by the preset m*n matrix according to the serialization of adjacent data ; Perform encoding processing on each of the obtained bit-plane sequence data, and generate encoded image data of the image data.

In some implementation manners of the first aspect, the step of dividing the acquired image data into a plurality of bit-plane matrix data according to binary data bits includes: performing frequency domain conversion on the acquired image data, and according to a preset binary The data bits divide the converted frequency domain image data into multiple bit plane matrix data.

In some implementations of the first aspect, the encoding method further includes a step of dividing the acquired original image into multiple channels of image data according to colors; so as to perform frequency domain conversion processing on each channel of the image data.

In some implementations of the first aspect, the encoding method further includes a step of performing block processing on the obtained bit-plane matrix data; correspondingly, based on the preset serialization period, at least part of the bit-plane The step of serializing the bit-plane matrix data of the bit-plane includes: serializing each matrix data block in the bit-plane matrix data of at least part of the bit-plane based on a preset serialization cycle to obtain each serial data block; and According to the position of each matrix data block in the bit plane matrix data, the sequence data blocks are connected into bit plane sequence data.

In some implementation manners of the first aspect, the step of serializing at least part of the obtained bit-plane matrix data based on a preset serialization period includes: following a sequence set based on the preset bit-plane In the conversion cycle, the bit-plane matrix data of the corresponding bit-plane is serialized.

In some implementation manners of the first aspect, the number of serialization periods set based on a preset bit plane is multiple, and the number of the sequence segments described in the serialization period set corresponding to a higher bit plane The length is greater than the length of the sequence segment described in the serialization period set corresponding to the lower bit plane.

In some implementation manners of the first aspect, the step of serializing at least part of the obtained bit-plane matrix data based on a preset serialization period includes: according to the serialization period, the corresponding bit-plane The matrix data is serialized into a plurality of sequence segments; according to the start data and the end data of the sequence segment described in the serialization period, the sequence segments of the corresponding bit planes are connected to obtain the corresponding bit plane sequence data.

In some implementation manners of the first aspect, the step of encoding each bit plane sequence data includes: performing encoding processing on each bit plane sequence data according to a preset encoding method corresponding to each bit plane.

In some implementations of the first aspect, the step of encoding each bit-plane sequence data includes: encoding the corresponding bit-plane sequence data with a coding unit set based on the serialization period.

In some implementation manners of the first aspect, the step of encoding each bit plane sequence data includes: using an entropy coding method to encode each bit plane sequence data.

In some embodiments of the first aspect, the frequency domain conversion method includes wavelet transform.

In some implementations of the first aspect, the serialization period is obtained based on the Hilbert polyline algorithm.

In some embodiments of the first aspect, the image data includes 4K and above image data.

A second aspect of the present application provides an image decoding method, including: decoding the acquired encoded image data to extract bit-plane sequence data used to describe multiple bit-planes of the image data; based on a preset serialization period, Convert the bit-plane sequence data of the corresponding bit-plane into bit-plane matrix data; wherein, the serialization period is a period set by the preset m*n matrix according to the serialization of adjacent data; according to the binary data bits of each bit plane , Merge all the obtained bit-plane matrix data into the described image data.

In some implementation manners of the second aspect, the decoding method further includes: combining the obtained multiple image data into an original image according to colors.

In some implementation manners of the second aspect, the step of decoding the acquired encoded image data includes: setting an encoding unit based on the serialization period, and processing the encoded bit plane sequence data in the encoded image data Decoding processing.

In some implementation manners of the second aspect, the step of decoding the acquired encoded image data includes: using an entropy decoding method to decode the acquired encoded image data.

In some implementation manners of the second aspect, the step of converting the bit-plane sequence data of the corresponding bit-plane into bit-plane matrix data based on a preset serialization period includes: performing bit-plane sequence data of the corresponding bit-plane Block processing to obtain multiple sequence data blocks; based on the preset serialization cycle, convert each sequence data block of the corresponding bit plane into a matrix data block; based on the position of each sequence data block in the corresponding bit plane sequence data, Combine the blocks into bit-plane matrix data.

In some implementations of the second aspect, the step of converting the bit-plane sequence data of the corresponding bit-plane into bit-plane matrix data based on a preset serialization period includes: according to a sequence set based on the preset bit-plane The conversion cycle converts the bit-plane sequence data of the corresponding bit-plane into bit-plane matrix data.

In some implementations of the second aspect, the number of serialization periods set based on the preset bit plane is multiple, and the length of the sequence segment described in the serialization period set corresponding to the higher bit plane It is greater than the length of the sequence segment described in the serialization period set corresponding to the lower bit plane.

In some implementation manners of the second aspect, the step of converting the bit-plane sequence data of the corresponding bit-plane into bit-plane matrix data based on a preset serialization period includes: according to the serialization period, converting the corresponding bit-plane The plane sequence data is serialized into multiple sequence segments; according to the start data and end data of the sequence segment described in the serialization cycle, each sequence segment of the corresponding bit plane is converted into a matrix form, and the data in each matrix form is merged Into bit plane matrix data.

In some implementation manners of the second aspect, the step of decoding the acquired coded image data includes: according to a decoding method corresponding to a preset bit plane, the coded bit plane sequence data in the coded image data Perform decoding processing.

In some implementation manners of the second aspect, the step of combining all the obtained bit-plane matrix data into the described image data according to the binary data bits of each bit plane includes: according to the binary data bits of each bit plane, combining All the obtained bit-plane matrix data undergoes frequency domain inverse transformation to obtain the described image data.

In some embodiments of the second aspect, the frequency domain inverse transformation method includes wavelet transform inverse transformation.

In some embodiments of the second aspect, the serialization period is configured based on the Hilbert polyline algorithm.

In some embodiments of the second aspect, the image data obtained after decoding includes 8K image data.

A third aspect of the present application provides an image encoding device, including: an image acquisition interface for acquiring the image data; a storage device for storing at least one program and image data to be encoded; and a processing device for calling and The program is executed to perform encoding processing on the image data according to the image encoding method described in any one of the first aspect.

A fourth aspect of the present application provides a camera device, including: a capturing device for acquiring an original image, wherein the original image is composed of multiple image data set based on color; and a storage device for storing at least one A program and image data to be encoded; a processing device for calling and executing the program to perform encoding processing on the image data according to the image encoding method according to any one of the first aspects.

A fifth aspect of the present application provides an image decoding device, which includes: a storage device for storing at least one program and encoded image data to be decoded; a processing device for calling and executing the program to perform as described in the second aspect The image decoding method described in any one of the image decoding methods decodes the encoded image data to obtain image data that can be displayed.

A sixth aspect of the present application provides an image playback device, including: a storage device for storing at least one program and encoded image data to be decoded; and a processing device for calling and executing the program to perform as described in the second aspect The image decoding method described in any one of the image decoding methods decodes the encoded image data; the interface device is used to transmit the decoded image data to the connected display screen.

A seventh aspect of the present application provides an image transmission system, including: an image acquisition interface for acquiring the image data; a storage device for storing at least one program, image data to be encoded, and encoded image data to be decoded; processing A device for calling and executing the program to perform encoding processing on the image data according to the image encoding method as described in any one of the first aspect; and/or according to any one of the second aspect The image decoding method performs encoding processing on the encoded image data.

An eighth aspect of the present application provides an image transmission system, including: the image encoding device as shown in the third aspect, or the imaging device as shown in the fourth aspect; and the decoding device as shown in the fifth aspect, or The playback device shown in the sixth aspect.

A ninth aspect of the present application provides a computer storage medium, which is characterized by including: storing at least one program; when called, the at least one program executes the image encoding method according to any one of the first aspects; or , When the at least one program is called, the decoding method as described in any one of the second aspect is executed.

As mentioned above, the image encoding method, decoding method, and applicable equipment and system of this application have the following beneficial effects: this application uses a serialization period to implement serialization of bit-plane matrix data, which is beneficial to improve the original image The cohesion, especially the cohesion of high-definition images of 4K and above. In addition, the use of different serialization cycles based on binary data bits effectively improves the image compression rate.

Description of the drawings

Figure 1 shows a flowchart of an embodiment of the coding method of this application.

FIG. 2 shows a schematic diagram of a bit plane of a channel of image data divided by color in this application.

Figure 3 shows a schematic diagram of the spectrum distribution of the image data of this application after three-level wavelet transformation.

FIG. 4 shows a schematic diagram of the serialization rule of the 4*4 matrix in this application as the serialization cycle.

FIG. 5 shows a schematic diagram of the serialization rule of the 8*8 matrix in this application as the serialization cycle.

FIG. 6 shows a schematic diagram of serializing bit-plane matrix data (8*16) according to the serialization cycle shown in FIG. 4.

FIG. 7 shows a flowchart of an embodiment of the decoding method of this application.

FIG. 8 shows a schematic structural diagram of an image transmission system of this application in an embodiment.

FIG. 9 shows a schematic diagram of the structure of the image transmission system of this application in another embodiment.

detailed description

The following specific examples illustrate the implementation of this application. Those familiar with this technology can easily understand other advantages and effects of this application from the content disclosed in this specification.

As used herein, the singular forms "a", "an" and "the" are intended to also include the plural forms, unless the context dictates to the contrary. It should be further understood that the terms "comprising" and "including" indicate the existence of the described features, steps, operations, elements, components, items, types, and/or groups, but do not exclude one or more other features, steps, operations, The existence, appearance or addition of elements, components, items, categories, and/or groups. The terms "or" and "and/or" used herein are interpreted as inclusive, or mean any one or any combination. Therefore, "A, B or C" or "A, B and/or C" means "any of the following: A; B; C; A and B; A and C; B and C; A, B and C" . An exception to this definition will only occur when the combination of elements, functions, steps or operations is inherently mutually exclusive in some way.

The contradictory demands of obtaining high-definition images and using lossy compression to save and transmit high-definition images have prompted people to conduct in-depth research on lossless encoding of images. In some practical applications, the Shannon information entropy calculation method can be used for lossless compression of SD images, and the compression efficiency depends on the probability distribution of each information in the image. Therefore, for high-definition images containing rich details, the compression efficiency is not satisfactory.

To this end, the present application provides an image encoding method, which aims to achieve the encoding purpose of not only retaining as much image information as possible, but also effectively reducing the amount of image data. The encoding method mainly re-serializes the image data according to the binary data bits in the image data, and encodes the serialized data after the serialization processing, and realizes the image information concentration by using a more cohesive serialization method To achieve the above purpose. Wherein, the image data may come from the original image. Wherein: the original image includes but is not limited to: high-definition images (such as 2K images), standard-definition images (such as 720*576 images), ultra-high-definition images (such as 4K images or 8K images), and compressed and decompressed Images etc. For example, the original image is a high-definition image from an original video captured by a high-definition camera. For another example, the original image is a high-definition image transmitted through a dedicated data channel. For another example, the original image is an image that comes from the Internet and needs to be re-encoded. Wherein, the image data may be an original image, or the image data is obtained by dividing the original image into multiple paths according to colors. For example, the original image is divided into three channels of image data according to RGB, and each channel of image data is encoded using the encoding method. For another example, the original image is divided into three channels of image data according to YUN, and each channel of image data is encoded using the encoding method.

The encoding method is mainly executed by an image encoding device. Wherein, the encoding device may be a terminal device or a server.

Wherein, the terminal equipment includes but is not limited to camera equipment, personal electronic terminal equipment, etc. The camera equipment includes a camera device, a storage device, a processing device, and may also include an interface device. Wherein, the camera device is used to obtain an original image, wherein the original image is composed of multi-channel image data set based on colors. The imaging device includes at least a lens composed of a lens group, a light sensing device, etc., where the light sensing device includes, for example, a CCD device, a CMOS device, and the like. The storage device may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. The storage device also includes a memory controller, which can control access to the memory by other components of the device, such as a CPU and a peripheral interface. The storage device is used to store at least one program and image data to be encoded. The program stored in the storage device includes an operating system, a communication module (or instruction set), a graphics module (or instruction set), a text input module (or instruction set), and an application (or instruction set). The program in the storage device further includes an instruction set for performing an encoding operation on the image data in a time sequence based on the technical solution provided by the encoding method. The processing device 13 includes but is not limited to: CPU, GPU, FPGA (Field-Programmable Gate Array), ISP (Image Signal Processing image processing chip), or other memory devices that are dedicated to processing At least one program processing chip (such as AI dedicated chip), etc. The processing device calls and executes at least one program stored in the storage device to perform encoding processing on the stored original image or image data in the original image according to the encoding method. Among them, the use of processing devices such as FPGA that can process matrix data in parallel is more suitable for efficient and real-time encoding of the acquired image data. The interface device includes, but is not limited to: a data line interface and a network interface; among them, examples of the data line interface include at least one of the following: serial interfaces such as USB, and parallel interfaces such as bus interfaces. Examples of network interfaces include at least one of the following: short-range wireless network interfaces such as Bluetooth-based network interfaces and WiFi network interfaces, such as wireless network interfaces of mobile networks based on 3G, 4G, or 5G protocols, such as wired network interfaces that include network cards Wait. In some scenarios, the camera device is set on a pan-tilt above the road to monitor vehicle violations, such as speeding, red light running, etc. In other scenarios, the camera device is configured on a minimally invasive medical device, and the camera device is set at the front end of the hose through an optical fiber or other dedicated data cable. In other scenes, the camera device is configured on a high-speed moving track of a stadium to capture high-definition pictures of competitive games.

The electronic terminal equipment for personal use includes desktop computers, notebook computers, tablet computers, and editing equipment dedicated to the production of TV programs, movies, TV series, and the like. The electronic terminal equipment includes a storage device and a processing device. Wherein, the storage device and the processing device may be the same or similar to the corresponding devices in the aforementioned camera equipment, and will not be described in detail here. The electronic terminal equipment may also include a camera device for capturing original images. Here, in some examples, the hardware and software modules of the camera device may be the same as or similar to the corresponding device in the aforementioned camera device, and will not be repeated here. In still other examples, the electronic terminal device may further include an image acquisition interface for acquiring image data derived from the original image or the original image. The image acquisition interface may be a network interface, a data line interface, or a program interface. Wherein, the network interface and the data line interface can be the same or similar to the corresponding devices in the aforementioned camera equipment, and will not be described in detail here. For example, through the network interface, the processing device of the electronic terminal equipment downloads the original image from the Internet. For another example, through the program interface, the processing device of the electronic terminal device obtains the original image or image data displayed by the drawing software on the display screen. Wherein, the drawing software is, for example, PS software, or screenshot software, etc. For another example, through the data line interface, the processing device of the electronic terminal equipment obtains an original frame of the unedited high-definition video from the storage device.

The server includes but is not limited to a single server, a server cluster, a distributed server, a server based on cloud technology, and the like. Wherein, the server includes a storage device, a processing device, an image acquisition interface, and the like. The storage device and the processing device may be configured in the same physical server device, or be configured in multiple physical server devices according to the division of labor of each physical server device. The image acquisition interface may be a network interface or a data line interface. The storage device, processing device, image acquisition interface, etc. included in the server may be the same as the corresponding devices mentioned in the aforementioned terminal equipment; or specifically set for the server based on the server's throughput, processing capacity, and storage requirements The corresponding devices. For example, the storage device may also include a solid state drive or the like. For example, the processing device may also include a CPU dedicated to a server or the like. The image acquisition interface in the server acquires image data and encoding instructions from the Internet, and the processing device executes the encoding method described in this application on the acquired image data based on the encoding instructions.

Based on the requirements for encoding image data generated in any of the foregoing scenarios, the present application provides an encoding method. Please refer to FIG. 1, which shows a flowchart of the encoding method in an embodiment. The processing device in any of the encoding device examples mentioned above executes at least one program and schedules the hardware in the encoding device to perform the following steps.

In step S110, the acquired image data is divided into a plurality of bit-plane matrix data according to binary data bits. Wherein, the image data is a data matrix arranged based on pixel positions. According to the color depth used by the image data, the color depth of the data at each pixel position in the corresponding data matrix can be expressed by multi-bit binary data. For example, if the color depth of the image data is 256, the data at each pixel position in the data matrix is expressed by an 8-bit binary number. Based on each channel of image data divided by color, the same type of color from the high bit to the low bit of the multi-bit binary data can be regarded as set according to the binary data bit from high to low. Please refer to Figure 2, which shows an example of a bit plane of image data divided by color, where the red component of each pixel is represented by an 8-bit binary number {b7, b6,..., b0}, adjacent pixels The relative frequency of the b7 bit in the red component of the red component is lower than the b0 bit. It can be seen that the bit-plane matrix data composed of the b7 bit of each pixel in the image data can be regarded as the higher binary data bit in the image data. The bit-plane matrix data of the image data, and the bit-plane matrix data composed of the b0 bits of each pixel in the image data can be regarded as the bit-plane matrix data of the lower binary data bits in the image data.

It should be noted that, according to the requirements of the image color depth, the color value of each pixel in each channel of image data may also be represented by a value of more than 8 bits or a value of less than 8 bits. For example, for 4K and above 4K images, the pixel bit of each channel of image data is represented by a 10-bit value.

According to the above example of bit-plane matrix data divided based on binary data bits, it can be seen that in some examples, this step can be divided into multiple bit-plane matrix data by the position of each data bit in the binary data expressing the color in the image data. .

In some applications, the frequency domain conversion method can be used to obtain the bit-plane matrix data based on the frequency spectrum more accurately. The step S110 includes: performing frequency domain conversion on the acquired image data, and dividing the converted frequency domain image data into a plurality of bit plane matrix data according to preset binary data bits.

Here, the encoding device performs frequency domain conversion on the image data to obtain the distribution of each frequency spectrum of the image data in the frequency domain, divides the frequency domain image data into multiple pixel data blocks based on the frequency spectrum, and divides the frequency domain image data into a plurality of pixel data blocks according to the constituent pixel data blocks. The binary data bits of the pixel data are divided into multiple bit plane matrix data for each pixel data block.

Wherein, the frequency domain conversion method includes Fourier transform, cosine transform, etc., for example. For example, the frequency domain transform may adopt Discrete Fourier Transform (DFT). For example, the frequency domain transform may adopt a discrete cosine transform (Discrete Fourier Transform, DCT). The frequency domain transform may also adopt wavelet transform (Wavelet Transform, WT).

Please refer to FIG. 3, which shows a schematic diagram of the spectrum distribution of image data after three-level wavelet transform. Among them, LLi in the figure represents the low frequency subband; LHi represents the horizontal detail component, which belongs to the high frequency subband; HLi represents the vertical detail component, which belongs to the high frequency subband; HHi represents the diagonal detail component, which also belongs to the high frequency subband. Among them, i=1, 2, 3. After an image data is converted to the frequency domain through a three-level wavelet transform, it can be divided into pixel image blocks distributed in ten subbands, as shown in Figure 3, LL3, HL3, HL2, HL1, LH3, LH2, LH1, HH3, HH2 , HH1 ten sub-band pixel image blocks, wherein the pixel value in each pixel image block is represented by 8-bit or 10-bit binary for example.

The encoding device performs bit plane division on the pixel data in each obtained pixel data block. For example, if the color value in the image data is represented by 10-bit binary, the encoding device divides the pixel data block located in HL3 into 10 bit-plane matrix data. By analogy, the encoding device can divide the pixel data blocks in at least HL3, HL2, HL1, LH3, LH2, LH1, HH3, HH2, HH1 subbands into 10 bit-plane matrices each. Among them, for the low-frequency subband LL3, bit-plane matrix data can be processed or processed separately according to encoding methods such as H.264 and JPEG.

For the original image containing multiple colors, this step can divide the original image into multiple channels of image data according to the color, and operate the frequency domain conversion processing as mentioned in any of the above examples on each channel of image data, thereby obtaining the The bit-plane matrix data of each channel of image data. After the bit-plane matrix data is obtained, step S130 may be performed to serialize the bit-plane matrix data.

In some embodiments, in order to improve the efficiency of hardware processing serialization in the encoding device, the encoding method further includes: step S120, which is to perform block processing on the obtained bit-plane matrix data.

Here, the block processing aims to block the bit-plane matrix data according to the preset number of rows and columns to obtain a plurality of matrix data blocks, wherein each adjacent block has no overlapping data. For example, the block processing method adopts block processing of bit-plane matrix data according to a preset number of rows or columns. For another example, the block processing method adopts the block processing of the bit-plane matrix data according to the maximum M*N matrix data amount.

In some examples, the block processing performs block processing on the obtained bit-plane matrix data according to a preset serialization cycle for serialization processing. Wherein, the serialization cycle includes an m*n matrix used to describe serialization processing rules, m and n are both natural numbers greater than 1, and m and n may be the same or different. For example, m is an integer multiple of n. To this end, the block processing method includes: performing block processing on the obtained bit-plane matrix data according to the number of rows/columns of the matrix described by the serialization period. For example, the block processing method is that a*m acts as a block unit to block the bit-plane matrix data, where a is a coefficient, and a≥1. For another example, the block processing method is to block the bit-plane matrix data according to b*n as the block unit, where b is a coefficient, and b≥1. For another example, the block processing method is to block the bit-plane matrix data according to the (a*m, b*n) matrix as the block unit, where a and b are coefficients, and both a and b are greater than An integer equal to 1, a and b can be equal or unequal. When using any of the above examples for block processing, there may be cases where the number of rows or columns is less than one block unit, and the block processing method also includes the remaining data of less than one block unit in the bit-plane matrix data Into a separate matrix data block.

After the matrix data block obtained according to any of the above examples, step S130 may be performed to perform serialization processing on each matrix data block in each bit-plane matrix data to obtain a sequence data block, and according to each matrix data block The position in the bit-plane matrix data connects each sequence data block into bit-plane sequence data.

In step S130, based on a preset serialization period, at least part of the bit-plane matrix data of the bit-plane is serialized to obtain bit-plane sequence data.

Wherein, the serialization period is a period set by the preset m*n matrix according to the serialization of adjacent data. Here, the serialization period describes the rule of serialized data obtained by traversing adjacent data in a preset m*n matrix. The rules include: starting data and ending data in the matrix determined for serialization, and in the matrix determined for serialization from the position of the starting data to the position of the ending data, the data in the matrix are determined according to The sequence relationship of each position when the sequence is arranged. The adjacent position relationship means that the adjacent data after serialization are adjacent in the matrix, that is, after the data in the matrix is serialized into sequence segments, the adjacent data corresponds to the matrix having adjacent data in the same row or column. Positional relationship. In some examples, the data in the matrix constructs a serialization cycle based on the area enclosed by adjacent data in the same row and the same column, forming a sequence segment with start data and end data. Among them, in some specific examples, the start data and end data of a sequence segment are located in the same row or column in the corresponding matrix data. For example, please refer to Figure 4, which shows a schematic diagram of the serialization rule of the 4*4 matrix as the serialization cycle, where the area enclosed by every 2 adjacent data in the 4*4 matrix is serialized according to the arrow. a11 and a12 are adjacent data, a12 and a22 are adjacent data; the serialization cycle starts with a11 and a14 as the end point to serialize all the data in the 4*4 matrix into a row of 16-bit sequence segments: a11 , A12, a22, a21, a31, a41, a42, a32, a33, a43, a44, a34, a24, a23, a13, a14. For another example, please refer to Figure 5, which shows a schematic diagram of the serialization rule of an 8*8 matrix as the serialization period. The area enclosed by every 4 adjacent data in the 8*8 matrix is called Wi. The data at d0 and d1, d1 and d2, and d2 and d3 in the area Wi are all adjacent data; the serialization cycle starts from the data at the d0 position of W0 and takes the d3 position in the area WF As the end point, all data in the 8*8 matrix is serialized into a row of 64-bit sequence segments, that is, the sequence segments connected by arrows as shown in Figure 5.

Here, the serialization period may be a predetermined fixed period based on the statistics of the frequency spectrum of the sample image. Wherein, the statistical method may determine the serialization period by pre-setting the cohesion condition of the image information; or may determine the serialization period by statistically calculating the data volume change ratio before and after encoding. In some examples, the serialization period may be obtained by using the Hilbert polyline algorithm. For example, the fourth-order Hilbert polyline algorithm is used to generate multiple candidate serialization periods, and the coded image data obtained by serializing each sample image in each candidate serialization period is counted to compare the phases of the coded image data. Compared with the data volume change ratio of the original sample image, the serialization period is selected and configured as the serialization period used when the encoding method is executed. It should be noted that the serialization period can also be obtained by other broken line algorithms set based on the aforementioned broken line principle, which will not be described in detail here.

In some embodiments, the serialization period is set based on a bit plane. In some examples, a serialization period that can be universally applied to each bit plane can be preset. For example, each bit plane uses the same serialization cycle for serialization. In another example, different serialization periods are set corresponding to different bit planes. For example, each bit plane corresponds to a serialization period. For another example, the bit plane is divided into multiple groups in the order of binary bits, and each group of bit planes corresponds to a serialization period. In other examples, the bit planes belonging to each subband use the same one or more serialization cycles. For example, the serialization period is set according to the frequency spectrum segment where the subbands are located. For another example, each bit plane in each subband corresponds to a serialization period separately. For another example, each subband separately divides the bit plane into multiple groups in the order of binary bits, and each group of bit planes individually corresponds to a serialization period. For another example, each sub-band is divided into multiple groups according to the spectrum section where the sub-bands are located. Each bit plane in each sub-band can have a one-to-one correspondence with multiple serialization periods, or divide the sub-bands in each group. The bit planes continue to be grouped, and the bit plane groups obtained after multiple groupings have a one-to-one correspondence with multiple serialization periods.

According to the encoding setting, the step S130 includes: serializing the bit plane matrix data of the corresponding bit plane according to the serialization period set based on the preset bit plane.

Among them, taking the bit-plane matrix data obtained based on the bit-plane directly divided by the image data, an 8-bit binary number is used to represent a color value (b7, b6,..., b1, b0) of a pixel position in the image data as an example. The bit-plane matrix data obtained by the b7-b6 bits can be divided into a group, the bit-plane matrix data obtained by the b5-b2 bits can be divided into a group, and the bit-plane matrix data obtained by the b1-b0 bits can be divided into a group. One group; each group corresponds to a serialization cycle. The encoding device can at least perform the steps of serializing the bit-plane matrix data of the b7-b6 bits according to the preset serialization cycle corresponding to the b7-b6 bits, and according to the preset b5-b2 bits corresponding The serialization cycle is a step of serializing the bit-plane matrix data of the b5-b2 bits.

Take the bit-plane matrix data obtained based on each pixel data block as an example. As shown in FIG. 3, in the 10 pixel data blocks obtained by wavelet transform, the pixel data blocks are distributed in LL3, HL3, LH3, HH3, HL2, LH2, HH2, H1, HL1, and HH1 spectrum sub-bands; among them, the binary data in each pixel data block is divided into 10 bit planes according to the high to low bits, and the corresponding The bit plane matrix data of the 10 bit planes are distributed in each pixel data block. The encoding device can perform at least the bit-plane matrix data of each sub-band: the step of serializing the bit-plane matrix data of the b9-b6 bits according to the preset serialization cycle corresponding to the b9-b6 bits, and the step It is assumed that the serialization period corresponding to the b5-b0 bits is a step of serializing the bit-plane matrix data of the b5-b0 bits.

It should be noted that the above examples are only examples, and are not limitations on the application. According to the actual coding design, the coding method can be used in combination with the existing serialization processing method. In some specific examples, some bit-plane matrix data divided into low frequencies can be serialized according to the existing serialization method, for example, The bit plane matrix data of the pixel data block in the LL3 subband can be processed by Zigzag serialization. For another example, the bit-plane matrix data corresponding to the bit-plane located in the lower bits (such as bits b1-b0) can be processed in a Zigzag serialization manner. The encoding method can also be used in combination with existing encoding methods. In some specific examples, some bit-plane matrix data divided into low frequencies can be encoded according to the existing encoding methods. For example, for the pixel data block in the LL3 subband, step S130 may not be executed and step S140 may be executed directly. For another example, the bit-plane matrix data corresponding to the bit-plane located in the lower bits (such as bits b1-b0) may not perform step S130 but directly perform step S140.

Because the higher the bit of the binary data, the more concentrated the amount of information. Therefore, the serialization period of the high-order interval in the binary data bit can be obtained by serializing the matrix with a larger amount of data. The serialization of the low-order interval in the binary data bit The period can be obtained by serializing a matrix with a small amount of data. In other words, the length of the sequence segment described in the serialization period set at a higher position is greater than the length of the sequence segment described in the serialization period set at a lower position. Take the serialization period T1 obtained by the eighth-order Hilbert polyline algorithm shown in Fig. 5 and the serialization period T2 obtained by the fourth-order Hilbert polyline algorithm shown in Fig. 4 as examples, based on the serialization period T1 serializes the bit plane matrix data in the pixel data blocks HL3, LH3, and HH3, and serializes the bit plane matrix data in the pixel data blocks HL2, LH2, and HH2 based on the serialization period T2. After obtaining the bit plane sequence data, the encoding device executes step S140. Among them, the pixel data in the pixel data block LL3 can be sequenced using the existing serialization processing method; or the serialization period T1 can be used for the serialization processing; or the encoding processing can be directly performed.

Wherein, when the number of serialization periods set based on the preset bit plane is multiple, the length of the sequence segment described in the serialization period set for the higher bit plane is greater than that of the lower bit plane. Set the length of the sequence segment described in the serialization period. Take the serialization period shown in Figure 4 and Figure 5, and the bit planes determined by binary bits from low to high, including: bit plane 0, bit plane 1,..., bit plane 9 as examples, including 8*8 matrix The serialization period is configured to serialize the bit-plane matrix data of bit planes 6-9; the serialization period including the 4*4 matrix is configured to serialize the bit-plane matrix data of at least bit planes 2-5. It should be noted that the foregoing correspondence between the serialization period and the bit plane is only an example, and is not a limitation of the present application. In fact, according to actual coding needs, low-order bit-plane data, such as the bit-plane 0-1 bit-plane matrix data in the above example, can also be directly encoded or sequenced using a serialization cycle containing a 4*4 matrix. Or use other existing serialization processing methods for serialization processing. Among them, existing serialization processing methods include but are not limited to Zigzag polyline processing methods.

Based on the serialization period determined in any of the foregoing examples, in some examples, the step S130 includes: directly serializing the obtained bit-plane matrix data according to a preset serialization period to obtain bit-plane sequence data . In other words, there is no need to block the bit-plane matrix data, and the obtained bit-plane matrix data of at least a part of the bit-plane is directly serialized according to the preset serialization cycle. According to the description of the above example, this step may perform serialization processing on the bit plane matrix data in the higher bit plane of the acquired image data according to the preset serialization cycle. For example, according to a preset serialization cycle, the bit plane matrix data of each bit plane except bit planes 0 and 1 is serialized. Alternatively, in this step, serialization processing is performed on each bit plane matrix data in all bit planes corresponding to the image data according to a preset serialization cycle. For example, according to a preset serialization cycle, the bit-plane matrix data of all bit-planes including 10 bit-planes are serialized.

Here, the serialization processing method includes: serializing the corresponding bit-plane matrix data into a plurality of sequence segments according to the serialization period; according to the start data and end data of the sequence segment described in the serialization period , Connect the sequence segments of the corresponding bit plane to obtain the corresponding bit plane sequence data.

Please refer to Figure 6, which shows a schematic diagram of serializing bit-plane matrix data (8*16) according to the serialization cycle shown in Figure 4, where the number of rows and columns of the matrix described in the serialization cycle The number is the traversal window. Using the number of rows of the matrix as the step size, the traversal window is traversed in the bit-plane matrix data, and the matrix data in the traversal window is in the sequence described by the serialization period each time it moves The serialization rule performs serialization processing to obtain the corresponding sequence segment, which includes the sequence segment described by the arrow (x _i-1,0 ,x _i-1,1 ,x _i,1 ,x _i,0 ,... ,x _i-1,3 ), sequence segment (x _i-1,4 ,x _i-1,5 ,x _i,4 ,x _i+1,4 ,...,x _i-1,7 ), etc. Etc.; According to the start data and end data of each sequence segment, the sequence segments obtained by the traversal are connected to the end to obtain the bit-plane sequence data. Among them, x _{j, k} in FIG. 6 are data at the (j, k)th position in the bit plane matrix data.

It can be seen from the above example that the direction traversed by the traversal window is related to the row/column of the start data and the end data in the serialization cycle. Therefore, the above-mentioned method of moving the traversal window in the row direction of the bit-plane matrix data It is only an example, not a limitation of this application.

In some other examples, when processing the bit-plane matrix data after block processing in step S120, the step S130 includes: based on a preset serialization period, at least part of the bit-plane bit-plane matrix data Each matrix data block is serialized to obtain each sequence data block; and according to the position of each matrix data block in the bit plane matrix data, the sequence data blocks are connected into bit plane sequence data.

Here, for multiple matrix data blocks divided into the same bit plane, similar to the aforementioned serialization process, the number of rows and columns of the matrix described in the serialization cycle is used as the traversal window, and the matrix The number of rows (or the number of columns) is the step size, and each matrix data block is serialized, and the obtained serialized data is called a sequence data block, which will not be described in detail here. Then, according to the preset position sequence of each matrix data block in the bit plane matrix data, and the start data and end data of each sequence data block, the sequence data blocks are connected end to end to obtain the bit plane sequence data.

In step S140, encoding processing is performed on each of the obtained bit-plane sequence data, and encoded image data of the image data is generated. Wherein, the encoding process aims to convert each of the bit-plane sequence data into a code stream described by encoding symbols at the cost of a minimum amount of information loss. Wherein, the encoding processing method is an example of a lossless encoding processing method. In some examples, the lossless encoding processing method is an entropy encoding method. Correspondingly, the step S140 includes using an entropy encoding method to encode each bit plane sequence data. Wherein, the entropy coding method includes but is not limited to: Shannon coding and Huffman coding. In other examples, the lossless encoding processing method is an improved encoding method based on entropy coding, for example, an entropy encoding method based on run length is adopted.

Optionally, the encoding process is also based on the adjacent relationship between the divided bit planes, the color type expressed by the image data, the additional information of the original image, etc., to add header information to the code stream. In addition, the encoded image data obtained according to the encoding process also includes a file of code streams and header information obtained by encoding multiple channels of image data separately.

On the basis of the coding method in any of the above-mentioned examples, the step S140 includes: encoding the bit-plane sequence data according to the coding method corresponding to the preset bit-plane. Here, the bit planes divided according to any method in step S130 are correspondingly set with different encoding methods, and the bit plane sequence data is encoded. For example, the bit plane sequence data corresponding to the b9-b2th bit plane is coded according to entropy coding, and other data to be coded is coded according to the byte coding method.

On the basis of the encoding method described in any of the above-mentioned examples, the step S140 further includes encoding the corresponding bit-plane sequence data with an encoding unit set based on the serialization period.

Here, the bit-plane matrix data divided according to the serialization cycle has better cohesion, so according to the code word or byte, the bit-plane sequence data in each serialization cycle is encoded. Taking the serialization cycle shown in Figure 4 and encoding according to the byte encoding as an example, the sequence segment corresponding to each serialization cycle in the bit-plane sequence data is encoded with a set of 4 bits of binary to obtain a 4-bit encoding The symbol represents a sequence segment to describe the coding symbol of the bit-plane sequence data.

In addition, because the higher the bit in the binary data, the more concentrated the amount of information. Therefore, corresponding to the serialization period of different bit planes, different coding units can be set. For example, using an 8*8 matrix serialization cycle T1 corresponding word encoding method, the pixel data blocks HL3, LH3, HH3, HL2, LH2, HH2, and HH1 in the upper 4 bit plane sequence data are serialized to obtain The coding symbol composed of 8 bits represents a sequence segment to describe the coding symbol of each bit plane sequence data; and adopts the byte coding method corresponding to the serialization period of the 4*4 matrix, and the bit plane sequence of the lower 6 bits in each pixel data block is used. The data is serialized, and a coding symbol composed of 4 bits is obtained to represent a sequence segment to describe the coding symbol of each bit plane sequence data.

Here, some coding methods for each bit plane sequence data corresponding to low data bit planes (such as b1 and b0 bit planes), and coding methods for each pixel data of pixel data blocks corresponding to low frequency spectrum subbands (such as LL3 subband) , Entropy-based coding can be used to encode each bit plane sequence data and pixel data; or, an existing encoding method can be used for encoding.

It should be noted that any of the above examples are not mutually exclusive, but can combine multiple examples based on coding requirements to obtain encoded image data.

Based on the encoding methods described in the above examples, this application uses a serialization cycle to serialize bit-plane matrix data, which is beneficial to improve the cohesion of the original image, especially the cohesion of 4K and above HD images Sex. In addition, different serialization cycles are adopted based on the frequency spectrum, which effectively improves the compression rate of the high frequency spectrum segment.

The encoded image data encoded by the technical ideas provided by the above encoding method can be transmitted between devices or within devices through transmission media such as data lines and the Internet. For example, in a camcorder with integrated recording and playback, the hardware constituting the encoding device encodes the captured original image into corresponding encoded image data under the instruction of the software, and saves it in the storage device. When the user operates the camcorder to play the encoded image data, each hardware constituting the decoding device decodes the encoded image data under the instruction scheduling of the software and plays it (or called display). For another example, a camera device that can perform the encoding method encodes the captured original image into corresponding encoded image data (such as an encoded file or code stream), and transmits the encoded image data to a server using the Internet or a dedicated network cable, The decoding device provided in the server decodes the encoded image data and plays it (or called display).

This application also provides an image decoding method for decoding encoded image data encoded based on the foregoing encoding method. The decoding method is mainly executed by a decoding device. Wherein, the decoding device may be a terminal device or a server.

Wherein, the terminal equipment includes, but is not limited to, playback equipment, personal electronic terminal equipment, and the like. The playback device includes a storage device, a processing device, and may also include an interface device. Wherein, the storage device may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. The storage device also includes a memory controller, which can control access to the memory by other components of the device, such as a CPU and a peripheral interface. The storage device is used to store at least one program and image data to be decoded. The program stored in the storage device includes an operating system, a communication module (or instruction set), a graphics module (or instruction set), a text input module (or instruction set), and an application (or instruction set). The program in the storage device also includes an instruction set for performing a decoding operation on the image data in time sequence based on the technical solution provided by the decoding method. The processing device includes, but is not limited to: CPU, GPU, FPGA (Field-Programmable Gate Array), ISP (Image Signal Processing image processing chip), or other including at least data stored in a storage device dedicated to processing A program processing chip (such as AI dedicated chip), etc. The processing device calls and executes at least one program stored in the storage device to decode the stored original image or image data in the original image according to the decoding method. Among them, the use of processing devices such as FPGA that can process matrix data in parallel is more suitable for efficient and real-time decoding processing of the acquired image data. The interface device includes, but is not limited to: a data line interface and a network interface; examples of the data line interface include: display interfaces such as VGA interface and HDMI interface, serial interfaces such as USB, and parallel interfaces such as data bus. Examples of network interfaces include at least one of the following: short-range wireless network interfaces such as Bluetooth-based network interfaces and WiFi network interfaces, such as wireless network interfaces of mobile networks based on 3G, 4G, or 5G protocols, such as wired network interfaces that include network cards Wait. The playback device also includes a display device for displaying the decoded image data, wherein the image data is one of multiple channels of image data set based on the color of an original image. The display device at least includes a display screen, a display screen controller, etc., where the display screen includes, for example, a liquid crystal display screen, a curved display screen, a touch screen, and the like. The display screen controller includes, for example, a processor dedicated to the display device, a processor integrated with the processor in the processing device, and the like. In some scenarios, the playback device is set up with a traffic command center for decoding and displaying the encoded image data transmitted from the camera device. In other scenarios, the playback device is configured on a computer device that is communicatively connected with the minimally invasive medical device, which is connected to the minimally invasive medical device through an optical fiber or other dedicated data line, and connects the current minimally invasive medical device The encoded image data is decoded and played. In other scenarios, the playback device is configured in the computer room of the TV forwarding center, and is used to decode and play the encoded image data transmitted by the camera set on the stadium for video editing. In other scenarios, the playback device is a set-top box, which is used to decode the code stream in the corresponding TV channel in the TV signal and output it to the TV for display.

The electronic terminal equipment for personal use includes desktop computers, notebook computers, tablet computers, and editing equipment dedicated to the production of TV programs, movies, TV series, and the like. The electronic terminal equipment includes a storage device and a processing device. Wherein, the storage device and the processing device may be the same or similar to the corresponding devices in the aforementioned camera equipment, and will not be described in detail here. The electronic terminal equipment may also include a display device for displaying the decoded image data. Here, in some examples, the hardware and software modules of the electronic terminal may be the same as or similar to the corresponding devices in the aforementioned playback device, and will not be repeated here. In still other examples, the electronic terminal device may further include an image acquisition interface for acquiring encoded image data derived from the encoding. The image acquisition interface may be a network interface, a data line interface, or a program interface. Wherein, the network interface and the data line interface can be the same or similar to the corresponding devices in the aforementioned playback device, and will not be described in detail here. For example, through the network interface, the processing device of the electronic terminal device downloads encoded image data from the Internet. For another example, through the data line interface, the processing device of the electronic terminal device obtains the edited file from the storage device.

The server includes but is not limited to a single server, a server cluster, a distributed server, a server based on cloud technology, and the like. Wherein, the server includes a storage device, a processing device, an image acquisition interface, and the like. The storage device and the processing device may be configured in the same physical server device, or be configured in multiple physical server devices according to the division of labor of each physical server device. The image acquisition interface may be a network interface or a data line interface. The storage device, processing device, image acquisition interface, etc. included in the server may be the same as the corresponding devices mentioned in the aforementioned terminal equipment; or specifically set for the server based on the server's throughput, processing capacity, and storage requirements The corresponding devices. For example, the storage device may also include a solid state drive or the like. For example, the processing device may also include a CPU dedicated to a server or the like. The image acquisition interface in the server acquires coded image data and playback instructions from the Internet, and the processing device executes the decoding method described in this application on the acquired encoded image data based on the playback instructions.

Please refer to FIG. 7, which shows a flowchart of the decoding method in an embodiment. In step S210, the acquired encoded image data is decoded to extract bit-plane sequence data describing multiple bit-planes of the image data. Here, the encoded image data includes the aforementioned encoded file and code stream. For example, the acquired encoded image data is a file in a complete format that is downloaded and stored locally. For another example, the acquired encoded image data is a video stream transmitted in real time using a streaming protocol.

Wherein, the encoded image data may contain the following header information: used to describe the encoding method of the encoded image data, the starting data position of the image data obtained by dividing the original image based on color, and the bit plane sequence data in each image data The starting position and so on. According to the header information, the method of performing decoding processing on the acquired encoded image data in this step is the inverse processing of the encoding processing method in the foregoing step S140.

In some examples, the encoding processing manner is a lossless encoding processing manner. In some examples, the lossless encoding processing method is an entropy encoding method. Correspondingly, the step S210 includes adopting an entropy decoding method to de-encode the encoded image data. Wherein, the entropy decoding method includes, but is not limited to: Shannon decoding and Huffman decoding. In other examples, the lossless encoding processing method is an improved encoding method based on entropy encoding, for example, an entropy encoding method based on run length is adopted. Correspondingly, the step S210 includes adopting an entropy decoding method based on run length to The encoded image data is decoded.

On the basis of the encoding method in any of the above-mentioned examples, the encoding processing method includes: encoding each bit plane sequence data according to the encoding method corresponding to the preset frequency spectrum segment. Correspondingly, in the step S210, different decoding modes are set correspondingly based on the bit plane, and the encoded image file is decoded. For example, the bit plane sequence data corresponding to the b9-b2th bit plane is decoded according to the decoding scheme corresponding to the entropy encoding, and other data to be decoded is encoded and decoded according to the decoding method corresponding to the byte encoding.

On the basis of the encoding method in any of the aforementioned examples, the encoding processing method further includes encoding the corresponding bit-plane sequence data with an encoding unit set based on the serialization period. Correspondingly, the step S210 includes: setting a decoding unit based on the serialization period to decode the encoded bit plane sequence data in the encoded image data.

Here, the bit-plane matrix data divided according to the serialization period has better cohesion. Therefore, each encoded bit-plane sequence data is decoded according to words or bytes. Taking the serialization cycle shown in Figure 4 and encoding according to byte encoding as an example, the sequence segment corresponding to each serialization cycle in the encoded bit-plane sequence data is represented by a set of 4bit encoding symbols, and then according to the encoding symbols The corresponding 4-bit set of binary data is decoded to obtain the bit-plane sequence data represented by four sets of 4-bit binary data and described by multiple sequence sections.

In addition, because the higher the bit in the binary data, the more concentrated the amount of information. Therefore, according to the serialization period of different bit planes provided by the encoding device, different decoding units are set. For example, using an 8*8 matrix serialization cycle T1 corresponding word encoding method to decode the encoded bit-plane sequence data of the upper 6-bit bit-plane to obtain a sequence segment represented by 4 groups of 8-bit binary data, and The above-mentioned bit-plane sequence data described by multiple sequence segments; and using the corresponding word encoding method of the serialization period T2 containing a 4*4 matrix, the coded bit-plane sequence data of the lower 4 bit-planes is decoded to obtain a 4 A group of 4bit binary data represents a sequence segment, and the above-mentioned bit plane sequence data described by multiple sequence segments.

Here, some decoding methods for each bit plane sequence data corresponding to low data bit planes (such as b1 and b0 bit planes), and decoding methods for each pixel data of pixel data blocks corresponding to low frequency spectrum subbands (such as LL3 subband) , The encoded bit plane sequence data and pixel data can be decoded by a decoding scheme based on entropy coding; or, the existing decoding method corresponding to the encoding method can be used for the decoding processing.

It should be noted that any of the above examples are not mutually exclusive, but can be decoded based on a decoding rule set correspondingly including an encoding rule set using a serialization period to obtain multiple bit-plane sequence data.

Here, according to the starting position of the bit-plane sequence data described in the header information, by performing step S220, the plane sequence data of the multiple bit planes will be matrixed. Among them, according to the bit plane set by the encoding and the corresponding serialization period, the corresponding bit plane serialized data is selected to execute the following step S220.

In step S220, based on a preset serialization period, the bit-plane sequence data of the corresponding bit-plane is converted into bit-plane matrix data; wherein, the serialization period is to convert the preset m*n matrix according to the sequence of adjacent data The cycle set by Wherein, the serialization period is consistent with the serialization period mentioned in the foregoing encoding method, and will not be described in detail here.

In some embodiments, the serialization period is set based on a bit plane. According to the bit plane corresponding to the obtained bit-plane sequence data, each plane sequence data is converted into bit-plane matrix data according to the serialization cycle. Take the serialization period shown in Figures 4 and 5, and the bit planes determined from low to high according to binary data, including: bit plane 0, bit plane 1,..., bit plane 9 as an example, using 8*8 The serialization cycle of the matrix converts the bit-plane sequence data of bit planes 6-9 into corresponding bit-plane matrix data; the serialization cycle containing the 4*4 matrix converts the bit-plane sequence data of bit planes 2-5 into the corresponding bit planes. Plane matrix data.

It should be noted that the foregoing correspondence between the serialization period and the bit plane is only an example, and is not a limitation of the present application. In fact, according to actual coding needs, low-frequency bit-plane data, such as the bit-plane 0-1 bit-plane matrix data in the above example, can also be converted directly according to the matrix information provided in the packet header to obtain the bit-plane matrix after decoding. For the data, either a serialization cycle including a 4*4 matrix is used for the conversion process, or an existing conversion process in the corresponding coding scheme is used for the conversion process. Among them, the existing conversion processing methods include, but are not limited to, Zigzag polyline processing methods.

According to the serialization period determined by any of the examples mentioned in the encoding method, in some examples, the step S220 includes: directly performing the conversion processing on the obtained bit-plane sequence data according to the preset serialization period , To get the bit plane matrix data. In other words, there is no need to perform the inverse processing of the block processing in the corresponding coding scheme on the bit-plane sequence data, but directly convert the corresponding bit-plane sequence data into bit-plane matrix data according to the preset serialization cycle. According to the description of the above example, this step can convert the acquired bit-plane sequence data in the higher bit plane according to the preset serialization cycle; or, in this step, the bit-plane sequence data in all bit planes can be converted according to The preset serialization cycle performs conversion processing.

Here, the conversion processing method includes: serializing the corresponding bit-plane sequence data into a plurality of sequence segments according to the serialization period; according to the start data and end data of the sequence segment described in the serialization period, Convert each sequence segment of the corresponding bit plane into a matrix form, and merge the data in each matrix form into bit plane matrix data according to the position of each sequence segment in the plane sequence data.

As shown in Figure 4, the corresponding bit-plane sequence data is divided by the length of the sequence segment described in the serialization cycle, and each segment of the segmented sequence is converted according to the matrix form described in the serialization cycle to obtain 4 *4 Data in matrix form. According to the position of the start data and end data of each sequence segment in the corresponding matrix, the data in the form of each matrix is merged into bit-plane matrix data. Among them, x _{j, k} in FIG. 4 is the data at the (j, k)th position in the bit plane matrix data.

It can be seen from the above example that the position of adjacent sequence segments in the bit-plane matrix data is related to the row/column of the matrix where the start data and the end data of the serialization period are located. The way to merge the directions is only an example, not a limitation of the application. The decoded image data is obtained by performing step S230 on the obtained bit-plane matrix data.

In other examples, according to the block operation of the bit-plane matrix data used in the encoding process, correspondingly, the step S220 includes: block the bit-plane sequence data of the corresponding bit-plane to obtain multiple sequences Data block; based on a preset serialization cycle, convert each sequence data block of the corresponding bit plane into a matrix data block; and based on the position of each sequence data block in the corresponding bit plane sequence data, merge each of the blocks into Bit plane matrix data.

According to the encoding method including step S120, the extracted bit-plane sequence data is composed of a plurality of sequence data blocks, and the decoding device converts each sequence data block into a matrix data block according to the conversion method provided in the above example, wherein The conversion method is the same as or similar to the conversion method in any of the aforementioned examples, and will not be described in detail here. According to the sequence and position of the sequence data blocks in the same bit-plane sequence data, the corresponding matrix data blocks are merged into bit-plane matrix data, and step S230 is executed.

In step S230, according to the binary data bits of each bit plane, all the obtained bit-plane matrix data are merged into the described image data.

Take the frequency spectrum and its bit plane determined by the wavelet transform used in the aforementioned encoding as an example. As shown in Figure 3, fill in the bit plane matrix data corresponding to the bit plane location described by the bit plane matrix data to obtain Full-spectrum matrix data corresponding to one image data in the frequency domain; performing inverse wavelet transformation on the full-spectrum matrix data to obtain image data.

Taking the binary number used in the foregoing encoding as an example to determine the bit plane, as shown in FIG. 2, according to the binary data bits described by the bit plane matrix data, the bit plane matrix data is correspondingly filled in to obtain the image data.

The image data provided based on any of the above examples can be that the acquired original image is divided into one of the multiple channels of image data according to color, and the multiple channels of image data corresponding to the same original image are combined according to the decoding header information, etc., Obtain the decoded original image, and output the original image to the display screen, which is the image content displayed on the display screen.

Based on any one of the encoding equipment and decoding equipment provided above, in some applications, an image transmission system is composed of at least the encoding equipment and decoding equipment. Please refer to FIG. 8, which shows a schematic structural diagram of an image transmission system in an embodiment. Taking the transmission of a TV program as an example, the encoding device can be configured in a computer device used to produce a TV program, and the original image (such as the main frame image) in the original video of the TV program is encoded based on the foregoing encoding method. And through the video coding technology to make the encoded image data into a video file. The video file can be transmitted to the decoding device via the Internet, a dedicated channel for television signals, and the like. The decoding device may be configured in a set-top box or a TV set for playing television programs, and the decoding device decodes the received video file based on a decoding method set corresponding to the encoding method, and decodes the original image after the decoding method. The video is displayed on the TV screen.

It should be noted that the television can be regarded as a specific example of a playback device. In other words, the image transmission system shown in FIG. 8 may include an encoding device and a playback device.

In other applications, the encoding device shown in FIG. 8 can be implemented by a camera device, and the decoding device can be implemented by a playback device including a display screen. Taking the transmission of high-definition pictures of road monitoring as an example, the encoding device can be configured in a camera device for producing and acquiring road monitoring images, and each original image captured is encoded according to the above-mentioned encoding method to obtain an image code Stream, the image code stream can be transmitted to the playback device via the Internet, a specially constructed data line (such as optical fiber, etc.). The playback device can be configured in a computer room for monitoring roads, and is equipped with at least one display screen. The playback device obtains the image code stream provided by the designated camera device based on the user's operation, decodes the received image code stream according to the decoding method set corresponding to the encoding method, and displays the decoded original image on the display On the screen.

It should be noted that the unplayed graphics stream will be saved in the form of video files, and when you need to read them, the saved video files will be decoded, and the decoded original images will be displayed on the decoding device one by one. On the connected display. It can be seen that the image transmission device shown in FIG. 8 may also include a camera device and a decoding device. I will not give examples one by one here.

Based on any of the encoding and decoding methods provided above, this application also provides an image transmission system. Please refer to FIG. 9 which shows a schematic structural diagram of the image transmission system in another embodiment. The encoding equipment and decoding equipment included in the image transmission system may at least partially share hardware devices. Examples of the image transmission system are a recording and playback camera, an electronic terminal including a display screen, and the like.

Here, the image transmission system includes an image acquisition interface, a storage device, and a processing device. Wherein, the image acquisition interface may include a network interface, a data line interface, or a program interface. During encoding, the original image in the camera device or the Internet is acquired through the graphic acquisition interface, and the processing device executes the encoding operation by calling the program stored in the storage device to encode the acquired original image into encoded image data, And stored in the storage device. When the image transmission system displays the original image based on a user operation, the processing device executes the decoding operation by calling the program in the storage device, and displays the original image obtained after decoding on the display screen. Wherein, the encoding and decoding operations in the image transmission system can be performed based on the corresponding methods provided in this application, and will not be repeated here.

It should be noted that through the description of the above implementation manners, those skilled in the art can clearly understand that part or all of this application can be implemented by means of software in combination with a necessary general hardware platform. If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can also be stored in a computer readable storage medium. Based on this understanding, this application also provides a computer-readable storage medium that stores at least one program that, when executed, implements any of the foregoing encoding methods or decoding methods, such as The foregoing corresponds to the method described in FIG. 1 or FIG. 7.

Based on this understanding, the technical solution of the present application essentially or the part that contributes to the prior art can be embodied in the form of a software product. The computer software product can include one or more machine executable instructions stored thereon. A machine-readable medium, when these instructions are executed by one or more machines, such as a computer, a computer network, or other electronic devices, can cause the one or more machines to perform operations according to the embodiments of the present application. For example, the steps in the encoding method or the decoding method. Machine-readable media may include, but are not limited to, floppy disks, optical disks, CD-ROM (compact disk-read only memory), magneto-optical disks, ROM (read only memory), RAM (random access memory), EPROM (erasable Except programmable read-only memory), EEPROM (electrically erasable programmable read-only memory), magnetic or optical cards, flash memory, or other types of media/machine-readable media suitable for storing machine-executable instructions.

In addition, any connection is properly termed a computer-readable medium. For example, if the instruction is sent from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technology such as infrared, radio, and microwave, the Coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of the medium. However, it should be understood that computer readable and writable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but are intended for non-transitory, tangible storage media. For example, the magnetic disks and optical disks used in the application include compact disks (CD), laser disks, optical disks, digital versatile disks (DVD), floppy disks and Blu-ray disks. Among them, disks usually copy data magnetically, while optical disks use lasers for optical Copy data locally.

It should be understood that in the various embodiments of the present application, the size of the sequence numbers of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, rather than corresponding to the embodiments of the present application. The implementation process constitutes any limitation.

In the several embodiments provided in this application, it should be understood that the disclosed system, device, and method may be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components can be combined or It can be integrated into another system, or some features can be ignored or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

The foregoing embodiments only exemplarily illustrate the principles and effects of the present application, and are not used to limit the present application. Anyone familiar with this technology can modify or change the above-mentioned embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or changes made by persons with ordinary knowledge in the technical field without departing from the spirit and technical ideas disclosed in this application should still be covered by the claims of this application.

Claims

An image coding method, characterized in that it comprises:

Dividing the acquired image data into multiple bit plane matrix data according to binary data bits;

Based on the preset serialization period, serialize at least part of the bit-plane matrix data of the bit-plane to obtain bit-plane serial data; wherein, the serialization period is based on the preset m*n matrix according to adjacent data The period set for serialization;

Perform encoding processing on each of the obtained bit-plane sequence data, and generate encoded image data of the image data.
The image encoding method according to claim 1, wherein the step of dividing the acquired image data into a plurality of bit-plane matrix data according to binary data bits comprises: performing frequency domain conversion on the acquired image data, And divide the converted frequency domain image data into a plurality of bit-plane matrix data according to the preset binary data bits.
The image encoding method according to claim 1, further comprising a step of dividing the acquired original image into multiple channels of image data according to colors; so as to perform frequency domain conversion processing on each channel of the image data.
The image encoding method according to claim 1, further comprising a step of performing block processing on the obtained bit-plane matrix data;

Correspondingly, the step of serializing at least part of the bit-plane matrix data based on the preset serialization period includes:

Based on a preset serialization cycle, serialize each matrix data block in the bit-plane matrix data of at least a part of the bit-plane to obtain each serial data block; and

According to the position of each matrix data block in the bit plane matrix data, the sequence data blocks are connected into bit plane sequence data.
The image encoding method according to claim 1 or 2, wherein the step of serializing at least part of the obtained bit-plane matrix data based on a preset serialization period comprises:

According to the serialization period set based on the preset bit plane, the bit plane matrix data of the corresponding bit plane is serialized.
The image encoding method according to claim 5, wherein the number of serialization periods set based on a preset bit plane is multiple, as described in the serialization period set corresponding to a higher bit plane The length of the sequence segment is greater than the length of the sequence segment described in the serialization period set corresponding to the lower bit plane.
The image encoding method according to claim 1, wherein the step of serializing at least part of the obtained bit-plane matrix data based on a preset serialization period comprises:

Serialize the corresponding bit-plane matrix data into multiple sequence segments according to the serialization cycle;

According to the start data and the end data of the sequence segment described in the serialization period, the sequence segments of the corresponding bit plane are connected to obtain the corresponding bit plane sequence data.
The image encoding method according to claim 1, wherein the step of encoding each bit plane sequence data comprises: encoding each bit plane sequence data according to a preset encoding method corresponding to each bit plane .
The image encoding method according to claim 1 or 8, wherein the step of encoding each bit plane sequence data comprises: using a coding unit set based on the serialization period to convert the corresponding bit plane sequence The data is encoded.
The image encoding method according to claim 1 or 8, wherein the step of encoding each bit plane sequence data includes: using an entropy coding method to encode each bit plane sequence data.
The image encoding method according to claim 2, wherein the frequency domain conversion method includes wavelet transform.
The image encoding method according to claim 1, wherein the serialization period is obtained based on the Hilbert polyline algorithm.
The image encoding method according to claim 1, wherein the image data includes 4K and above image data.
An image decoding method, characterized in that it comprises:

Decoding the acquired encoded image data to extract bit-plane sequence data used to describe multiple bit-planes of the image data;

Convert the bit-plane sequence data of the corresponding bit-plane into bit-plane matrix data based on a preset serialization period; wherein, the serialization period is a period set by the preset m*n matrix according to the serialization of adjacent data ；

According to the binary data bits of each bit plane, all the obtained bit plane matrix data are merged into the described image data.
The image decoding method according to claim 14, further comprising: combining the obtained multiple image data into one original image according to the color.
The image decoding method according to claim 14, wherein the step of decoding the acquired encoded image data comprises: setting an encoding unit based on the serialization period, and transforming the encoded image data into Each bit plane sequence data is decoded.
The image decoding method according to claim 14, wherein the step of decoding the acquired encoded image data comprises: using an entropy decoding method to decode the acquired encoded image data.
The image decoding method according to claim 14, wherein the step of converting the bit-plane sequence data of the corresponding bit-plane into bit-plane matrix data based on a preset serialization period comprises:

Block the bit-plane sequence data of the corresponding bit-plane to obtain multiple sequence data blocks;

Based on the preset serialization period, convert each sequence data block of the corresponding bit plane into a matrix data block;

Based on the position of each sequence data block in the corresponding bit-plane sequence data, each of the blocks is merged into bit-plane matrix data.
The image decoding method according to claim 14, wherein the step of converting the bit-plane sequence data of the corresponding bit-plane into bit-plane matrix data based on a preset serialization period comprises:

According to the serialization period set based on the preset bit plane, the bit plane sequence data of the corresponding bit plane is converted into bit plane matrix data.
The image decoding method according to claim 19, wherein the number of serialization periods set based on a preset bit plane is multiple, as described in the serialization period set corresponding to a higher bit plane The length of the sequence segment is greater than the length of the sequence segment described in the serialization period set corresponding to the lower bit plane.
The image decoding method according to claim 14, wherein the step of converting the bit-plane sequence data of the corresponding bit-plane into bit-plane matrix data based on a preset serialization period comprises:

According to the serialization cycle, serialize the corresponding bit-plane sequence data into multiple sequence segments;

According to the start data and end data of the sequence segment described by the serialization period, each sequence segment of the corresponding bit plane is converted into a matrix form, and the data in each matrix form is merged into bit plane matrix data.
The image decoding method according to claim 21, wherein the step of decoding the acquired coded image data comprises: according to a decoding method corresponding to a preset bit plane, the coded image data is coded Each bit plane sequence data is decoded.
The image decoding method according to claim 14, wherein the step of combining all the obtained bit-plane matrix data into the described image data according to the binary data bits of each bit plane comprises: For binary data bits, all the obtained bit-plane matrix data is subjected to frequency domain inverse transformation to obtain the described image data.
The image decoding method according to claim 23, wherein the frequency domain inverse transformation method includes wavelet transform inverse transformation.
The image decoding method according to claim 14, wherein the serialization period is configured based on the Hilbert polyline algorithm.
The image decoding method according to claim 14, wherein the image data obtained after decoding includes 8K image data.
An image encoding device, characterized in that it comprises:

An image acquisition interface for acquiring the image data;

A storage device for storing at least one program and image data to be encoded;

The processing device is configured to call and execute the program to perform encoding processing on the image data according to the image encoding method according to any one of claims 1-14.
A camera device, characterized in that it comprises:

A capturing device for acquiring an original image, wherein the original image is composed of multiple image data set based on color;

A storage device for storing at least one program and image data to be encoded;

The processing device is configured to call and execute the program to perform encoding processing on the image data according to the image encoding method according to any one of claims 1-13.
An image decoding device, characterized in that it comprises:

A storage device for storing at least one program and encoded image data to be decoded;

The processing device is used to call and execute the program to decode the encoded image data according to the image decoding method according to any one of claims 14-26 to obtain image data that can be displayed.
An image playback device, characterized in that it comprises:

A storage device for storing at least one program and encoded image data to be decoded;

A processing device, configured to call and execute the program to decode the encoded image data according to the image decoding method according to any one of claims 14-26;

The interface device is used to transmit the decoded image data to the connected display screen.
An image transmission system, characterized by comprising:

An image acquisition interface for acquiring the image data;

A storage device for storing at least one program, image data to be encoded and encoded image data to be decoded;

A processing device for calling and executing the program to perform encoding processing on the image data according to the image encoding method according to any one of claims 1-13; and/or according to claims 14-26 Any one of the image decoding methods performs encoding processing on the encoded image data.
An image transmission system, characterized by comprising:

The image encoding device as shown in claim 27, or the imaging device as shown in claim 28; and

The decoding device shown in claim 29, or the playback device shown in claim 30.
A computer storage medium, comprising: storing at least one program; when called, the at least one program executes the image encoding method according to any one of claims 1-13; or, the at least When a program is called, it executes the decoding method according to any one of claims 14-26.