WO2020244296A1

WO2020244296A1 - System and method for multi-layer representation of depth map during intra-frame coding

Info

Publication number: WO2020244296A1
Application number: PCT/CN2020/082464
Authority: WO
Inventors: 陈成就; 覃泓胨
Original assignee: 万维数码有限公司
Priority date: 2019-06-04
Filing date: 2020-03-31
Publication date: 2020-12-10
Also published as: CN112040245A; CN112040245B

Abstract

The present invention relates to a system and method for processing a depth map, and in particular, to a system and method for multi-layer representation of a depth map during intra-frame coding. In the present invention, depth map data is blocked, progressive quantification and stopping conditions are set for the blocked depth map data, data encapsulation is performed on a multi-layer depth map for which iteration is stopped, the bit stream obtained after the encapsulation is outputted to a decoder, the pixels in the same smooth region are reduced to simple representative values, and resources are invested in recorded pixel grouping to achieve the smoothness of the depth map. The quality target can make the reconstruction and output of the method better serve the overall quality and bit rate control of a video encoder. One direct application of the present invention is 3D video content compression for online video broadcasting. Another application is 3D video format conversion.

Description

System and method for multi-layer representation of intra-frame coding depth map

Technical field

The present invention relates to a system and method for processing a depth map, and more particularly to a system and method for multi-layer representation of an intra-frame coding depth map.

Background technique

With the continuous improvement of image transmission requirements, in order to obtain efficient storage and transmission of images and videos, effective compression methods for images and videos are required. The depth map is an image or video that records the distance between the observable scene point and the camera light point. It provides additional information for related color pixels in color images or videos shot at the same location by specifying its depth in the scene. Therefore, the depth map is a key component of the 3D multimedia experience. The display device has 3D structure information from which to recover the scene depicted in the image or video with the depth map. The compression and encoding of the depth map is a process to reduce the amount of depth map data and provide standards for different terminal devices of the network to understand the transmitted data. Depth map coding can also be regarded as a part of the entire 3D video data compression process.

The most common depth map coding or general video coding schemes follow the same "hybrid" video coding framework. The frame is divided into blocks, taking advantage of the spatial dependence between adjacent blocks and frames. The data is predicted and coded from previously coded blocks and frames. Intra coding is the basic step of depth map coding. It is part of the encoding process where data is only predicted from previous data from the same frame. The intra-frame coding method only performs operations with respect to the information included in the current frame, and does not perform operations with respect to the information included in any other frames in the video sequence.

The most advanced standard 3D high-efficiency video coding ("3D-High Efficiency Video Coding") in the prior art adopts the following intra-frame coding methods, including: 1. Direct current (DC) and planar prediction; 2. Based on wedgelet ("wedgelet") The depth modeling; 3. The depth modeling of the contour segment. Only smooth data can be processed by direction prediction. Wedge-based depth modeling and contour segment depth modeling can handle drastic changes, but there are only two layers in the blocks that the frame is divided into. In addition, the depth modeling of contour line segments often obtains sharp changes from the corresponding texture video. The limitation of these methods is that the segmentation is performed only once. Therefore, if the segmentation quality is poor, you must rely on other coding methods to improve the reconstruction quality.

Therefore, in the prior art, there is a need for a system and method for compressing a depth map that improves compression efficiency and better reconstruction quality.

Summary of the invention

The method of the present invention to adaptively quantize the image and divide the image into constant-valued layers provides a system and method for compressing depth map data in a relatively complex scene.

The present invention provides a system for multi-layer representation of an intra-frame coded depth map, including the following devices: a block module to block the depth map data; a progressive quantization module to set progressive quantization and stop conditions for the block depth map data.

One aspect of the present invention also includes the following devices: a data encapsulation module, which performs data encapsulation on the multi-layer depth map that stops iteration; and a data output module, which outputs the encapsulated bit stream to the decoder end.

In one aspect of the present invention, the progressive quantization module for setting progressive quantization and its stopping conditions for the divided depth map data further includes: a layering module, which decomposes the depth block into multiple layers, and each layer contains the depth block A subset of pixels that are mutually exclusive with other layers; a multi-layer representation module that represents the multiple layers in a non-parametric way, one of which can represent any subset of all pixels in the depth block; an iterative module, which uses an iterative method to Encode multiple layers and continuously monitor reconstruction residuals and remaining areas with high priority.

In one aspect of the present invention, the progressive quantization module for setting progressive quantization and its stopping conditions for the divided depth map data includes: a new layer creation module, which initializes an empty layer list with an encoder, in the layer list Create a new layer, and then classify all pixels in the block as a unique layer; the encoder repeats the following modules until the end of the iteration process: calculation module, for each layer in the layer list, use the encryptor to calculate the graph The average value and variance of the depth value of all pixels in the layer, and each average value is attached to the corresponding layer; the depth value reconstruction module, the encoder finds the maximum value of all the calculated variances, and identifies the layer with the largest variance , Call it as the maximum variance layer; and create a reconstructed block by assigning a depth value to each pixel using the average value of the layer to which each pixel belongs; calculate the sum of squared errors between the reconstructed block and the original depth block; new layer The creation module uses the encoder to create a new layer at the end of the layer list, selects all pixels in the maximum variance layer and whose depth value is greater than the average value of the maximum variance layer, deletes these pixels from the maximum variance layer and assigns them to the new Layer; when the sum of the square errors is less than the preset threshold, the iterative process ends.

In another aspect of the present invention, the predetermined threshold is the required reconstruction quality in terms of the sum of squared errors.

In another aspect of the present invention, the data encapsulation module further includes: an average value addition module, which uses an encoder to calculate the average value and variance of the depth values of all pixels in the layer for all layers in the layer list; Each average value is attached to the corresponding layer; the layer processing module, the encoder reorders the layers by sorting the area or the number of pixels of each layer in descending order, so that the layer with the most pixels is processed first; binary Mapping module, the encoder uses the layer that has not been encoded and has the largest number of pixels to form a binary map, mark all pixels in the layer as "1", mark other pixels as "0", and use context adaptive binary arithmetic coding Method to encode this binary mapping; after completing the mapping, the encoder continues to the next largest layer and repeats until one layer is left; the last layer does not require binary mapping because it will automatically fill in all remaining pixels.

In another aspect of the present invention, the final output data in the data encapsulation module is composed of the following content: an integer representing the number of layers in the depth block; a series of bits containing the binary mapping of all layers; a value representing the depth value of each layer Series integer.

The present invention also provides a method for multi-layer representation of an intra-frame coding depth map, which includes the following steps: dividing the depth map data into blocks; setting the progressive quantization and stopping conditions for the divided depth map data.

The present invention also provides an encoder for implementing the method of the present invention. The stepwise quantization and stopping conditions are set for the block depth map data, which includes: a new layer creation module, which uses the encoder to initialize an empty layer List, create a new layer in the layer list, and then classify all pixels in the block as a unique layer; the encoder repeats the following modules until the end of the iteration process: calculation module, for the layer list For each layer of, use the encryptor to calculate the mean value and variance of the depth value of all pixels in the layer, and attach each mean value to the corresponding layer; depth value reconstruction module, the encoder finds all the calculated variances The maximum value in, identify the layer with the largest variance, call it the largest variance layer; and create a reconstruction block by assigning a depth value to each pixel using the average value of the layer to which each pixel belongs; calculate the reconstruction block and the original depth block The sum of squared errors between; the new layer creation module uses the encoder to create a new layer at the end of the layer list, selects all pixels in the maximum variance layer and whose depth value is greater than the average value of the maximum variance layer, starting from the maximum variance Delete these pixels from the layer and assign them to a new layer; when the sum of the square errors is less than a preset threshold, the iterative process ends.

The method described in the present invention processes depth map data in a manner that mimics the properties of depth data. Depth maps usually contain large and smooth areas with a clear boundary between the two. The present invention realizes the smoothness of the depth map by reducing the pixels in the same smooth area to simple representative values and at the same time investing resources in the recording pixel grouping. The quality target can make the reconstruction output of this method better serve the overall quality and bit rate control of the video encoder. One immediate application of the present invention is 3D video content compression for online video broadcasting. Another application is 3D video format conversion.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings in the following description are only some examples of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work.

Fig. 1 is a schematic diagram of a method for multi-layer representation of an intra-frame coded depth map according to the present invention.

Figures 2a-2e are examples of the steps of setting progressive quantization and its stopping conditions in the method for multi-layer representation of intra-coded depth maps according to the present invention.

Fig. 3 schematically shows a block diagram of a server for executing the method according to the present invention; and

Fig. 4 schematically shows a storage unit for holding or carrying program codes for implementing the method according to the present invention.

Detailed ways

What is set forth below is what is currently considered to be the preferred embodiment or best representative example of the claimed invention. After careful consideration of the future and present representations or modifications to the embodiments and preferred embodiments, any changes or modifications that make substantial changes in function, purpose, structure or results are intended to be covered by the claims of this patent. The preferred embodiments of the present invention will now be described by way of example only with reference to the accompanying drawings.

The present invention aims to provide an effective method for processing complex depth map data. According to the method of the present invention, the depth map of the present invention is actually decomposed into several "shapes", which are combined into all the pixels in the depth block to be processed, and each of these shapes One is appended with a depth value for use as a decoding result. Further, the method uses the standard derived based on variance to identify regions with poor reconstruction quality in an iterative manner to repeatedly improve the reconstruction quality.

Fig. 1 is a schematic diagram of a method for multi-layer representation of an intra-frame coded depth map according to the present invention. In step 101, the depth block is decomposed into multiple layers, and each layer contains a subset of pixels in the depth block that are mutually exclusive with other layers. Refer to step B. Progressive quantization and its stopping conditions below for specific content. In step 102, the multiple layers are represented in a non-parametric manner, and each layer can represent any subset of all pixels in the depth block. Refer to the detailed content of step B. Progressive quantification and sub-step d. of its stopping conditions below. It adapts to complex environments and can assign depth values in the scene arbitrarily. In step 103, the encoding process adopts an iterative method. See step B. Progressive quantification and sub-step c of its stopping conditions below. Continuously monitor the reconstruction residuals (a measure of the difference between the original and decoded depth maps) and the remaining areas with high priority. This step 103 is different from the contour line segment depth modeling in the prior art, and the contour line segment depth modeling only performs one segmentation.

The method of the present invention is further described in detail, and the method includes the following steps:

A. Divide the depth map data into blocks;

B. Set the progressive quantization and its stopping conditions for the divided depth map data;

C. Data encapsulation of the multi-layer depth map that stops iteration;

D. Output the encapsulated bit stream. among them,

Step A. Divide the depth data into blocks

In this step, starting with "blocks" in the depth map, the picture is divided into smaller units. The block width and height are usually powers of 2. For example, 2×2, 4×4, 8×8...

Step B. Set progressive quantization and its stopping conditions

The step of setting the progressive quantization step and its stopping condition includes multiple stages. Among them, the encoder first initializes an empty "layer list", creates a new layer in the list, and then classifies all pixels in the block as a unique layer. After initialization, the encoder repeats the following process until the stop condition set during the process is met.

a. For each layer in the layer list, the encryptor calculates the mean and variance of the depth values of all pixels in the layer. Each average value is attached to the corresponding layer.

b. The encoder creates a "reconstruction block" by assigning a depth value to each pixel using the average value of the layer to which each pixel belongs. Calculate the sum of squared errors (SSE) between the reconstructed block and the original depth block. If the SSE is less than a preset threshold, for example, the predetermined threshold is the required reconstruction quality in terms of SSE, then this iterative process ends.

c. The encoder finds the maximum value among all the variances calculated in step a, identifies the layer with the largest variance, and calls it the maximum variance layer ("LLV").

d. The encoder creates a new layer at the end of the layer list, selects all pixels in the maximum variance layer and whose depth value is greater than the average value of the maximum variance layer, deletes these pixels from the maximum variance layer and assigns them to the new layer . Go back to step a.

Figures 2a-2e are an example of the step of setting progressive quantization and its stopping conditions in the method for multi-layer representation of an intra-coded depth map according to the present invention.

Figure 2a-1 is an example of a depth block, which is colored with different shades of gray to indicate different values. Among them, the white part 201 represents the part with the depth value of "30"; the second light gray part 202 represents the part with the depth value of "25"; the darker gray part 203 represents the part with the depth value of "20" and "12"; The dark gray portion 204 represents a portion with a depth value of "10".

Figure 2a-2 shows that for the initial layer list, all pixels are located in the "0 layer".

Figure 2b-1 shows that after the first iteration, the entire block is divided into two layers. The dark gray part 205 in Figure 2b-1 is the part whose value is lower than 15 in the depth map of Figure 2a-1; the light gray part 206 in Figure 2b-1 is the value higher than 20 in the depth map of Figure 2a-1 part.

Figure 2b-2 shows that according to Figure 2b-1, the block is divided into two layers, and a new layer "1 layer" is generated on the basis of "0 layer".

Figure 2c-1 shows that the depth map is reconstructed from two average values. Among them, the dark gray part 205 in Fig. 2b-1 is taken as the depth map average value "10"; the light gray part 206 in Fig. 2b-2 is taken as the depth map average value "25".

Figure 2c-2 shows that the level list after one iteration is divided into "level 1" and "level 0", where the part with the value "10" is regarded as "level 0"; the part with the value "25" is regarded as "level 1" .

Figure 2d-1 shows that the second iteration is performed: the "layer 1" is split into two layers. Among them, the original depth value of the first layer is restored, and the original depth value of Figure 2a-1 is divided into two parts less than or equal to "25" in the "1 layer" part and two parts equal to "30". That is, the white part 207 and the darker gray part 208 in Fig. 2d-1.

Figure 2d-2 shows that the part of the depth value equal to "30" is regarded as a new layer "2 layer".

Figure 2e-1 shows that the depth map is reconstructed from three average values. In the dark gray part 205 in Figure 2b-1, the part with the average depth map "10" remains unchanged; the white part 207 in Figure 2d-1 with the depth value less than or equal to "25" is retaken The average value is "22"; the part with the depth value of "30" in the darker gray part 208 in Figure 2d-1 is kept unchanged; the average value is no longer taken.

Figure 2e-2 shows the level list after the second iteration: divided into "2 levels", "1 level" and "0 level", where the depth value is "10" as "0 level"; the depth value is "22" The part with "" is regarded as "1 layer"; the part with a depth value of "30" is regarded as "2 layer".

Step C. Data encapsulation

By completing the previous stage, the encoder now has a layer list of one or more layers, each layer has an average depth value and contains some pixels in the depth block. Further, the data encapsulation is completed through the following two sub-steps; in order to output the data to the decoder side:

a. Calculation steps: The encoder calculates the mean and variance of the depth values of all pixels in the layer for all layers in the layer list. Append each average to the corresponding layer. The encoder then reorders the layers by sorting the area or number of pixels of each layer in descending order, so that the layer with the most pixels is processed first.

b. Binary mapping step: the encoder uses the layer that has not been coded and has the largest number of pixels to form a binary mapping, marking all pixels in the layer as "1", marking other pixels as "0", and using context adaptation Binary arithmetic coding method to encode this binary map. After completing the mapping, the encoder continues to the next largest layer and repeats until one layer remains. The last layer does not require binary mapping because it will automatically fill in all remaining pixels.

Step D. Bitstream output

After the data encapsulation step is completed, the decoder will receive the final output data. The final output data is composed of three parts: an integer representing the number of layers in the depth block in step B. Progressive quantization and its stopping conditions. A series of bits containing the binary mapping of all layers from step C. data encapsulation; and a series of integers corresponding to the depth value of each layer in step B. progressive quantization and its stopping conditions.

The various component embodiments of the present invention may be implemented by hardware, or by software modules running on one or more processors, or by their combination. Those skilled in the art should understand that a microprocessor or a digital signal processor (DSP) can be used in practice to implement the method for improving video resolution and quality and the video encoder and the decoder of the display terminal according to the embodiments of the present invention. Some or all of the functions of some or all of the components. The present invention can also be implemented as a device or device program (for example, a computer program and a computer program product) for executing part or all of the methods described herein. Such a program for realizing the present invention may be stored on a computer-readable medium, or may have the form of one or more signals. Such signals can be downloaded from Internet websites, or provided on carrier signals, or provided in any other form.

For example, Figure 3 shows a server, such as an application server, that can implement the invention. The server traditionally includes a processor 1010 and a computer program product in the form of a memory 1020 or a computer readable medium. The memory 1020 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), EPROM, hard disk, or ROM. The memory 1020 has a storage space 1030 for executing the program code 1031 of any method step in the above method. For example, the storage space 1030 for program codes may include various program codes 1031 for implementing various steps in the above method. These program codes can be read from or written into one or more computer program products. These computer program products include program code carriers such as hard disks, compact disks (CDs), memory cards or floppy disks. Such computer program products are usually portable or fixed storage units as described with reference to FIG. 4. The storage unit may have storage segments, storage spaces, etc. arranged similarly to the storage 1020 in the server of FIG. 3. The program code can be compressed in an appropriate form, for example. Generally, the storage unit includes computer-readable codes 1031', that is, codes that can be read by, for example, a processor such as 1010, which, when run by a server, causes the server to perform the steps in the method described above.

The term "one embodiment", "an embodiment" or "one or more embodiments" referred to herein means that a specific feature, structure or characteristic described in conjunction with the embodiment is included in at least one embodiment of the present invention. In addition, please note that the word examples "in one embodiment" herein do not necessarily all refer to the same embodiment.

The above description is not intended to limit the meaning or scope of the words used in the following claims that define the present invention. Rather, descriptions and instructions are provided to help understand the various embodiments. It is expected that future changes in structure, function, or results will exist without substantial changes, and all these insubstantial changes in the claims are intended to be covered by the claims. Therefore, although the preferred embodiments of the present invention have been illustrated and described, those skilled in the art will understand that many changes and modifications can be made without departing from the claimed invention. In addition, although the term "claimed invention" or "present invention" is sometimes used herein in the singular, it will be understood that there are multiple inventions as described and claimed.

Claims

A method for multi-layer representation of an intra-frame coded depth map includes the following steps:

Divide the depth map data into blocks;

Set the progressive quantization and its stopping conditions for the divided depth map data.
The method according to claim 1, further comprising the following steps:

Data encapsulation of multi-layer depth maps that stop iteration;

The encapsulated bit stream is output to the decoder side.
The method according to claim 1-2, wherein:

For the block depth map data, the steps to set the progressive quantization and its stopping conditions include:

Decompose the depth block into multiple layers, and each layer contains a subset of pixels in the depth block that are mutually exclusive with other layers;

Representing the multiple layers in a non-parametric manner, one of which can represent any subset of all pixels in the depth block;

An iterative method is used to encode the multiple layers, and the reconstruction residuals and remaining areas with high priority are continuously monitored.
2. The method according to claim 1-2, the step of setting the progressive quantization and its stopping conditions for the divided depth map data includes:

Use the encoder to initialize an empty layer list, create a new layer in the layer list, and then classify all pixels in the block as a unique layer;

The encoder repeats the following steps until the end of the iteration process:

For each layer in the layer list, use an encryptor to calculate the mean and variance of the depth values of all pixels in the layer, and attach each average to the corresponding layer;

The encoder finds the largest value among all the calculated variances, identifies the layer with the largest variance, and calls it as the largest variance layer; and creates a reconstruction block by assigning a depth value to each pixel using the average value of the layer to which each pixel belongs ; Calculate the sum of squared errors between the reconstructed block and the original depth block;

Use the encoder to create a new layer at the end of the layer list, select all pixels in the maximum variance layer and whose depth value is greater than the average value of the maximum variance layer, delete these pixels from the maximum variance layer and assign them to the new layer;

When the sum of the square errors is less than the preset threshold, the iterative process ends.
The method of claim 4, wherein the predetermined threshold is the required reconstruction quality in terms of the sum of squared errors.
3. The method of claim 2, wherein the step of performing data encapsulation on the multi-layer depth map with stopped iteration comprises the following steps:

Use the encoder to calculate the mean and variance of the depth values of all pixels in the layer for all the layers in the layer list; attach each average to the corresponding layer;

The encoder reorders the layers by sorting the area or the number of pixels of each layer in descending order, so that the layer with the most pixels is processed first;

The encoder uses the layer that has not been encoded and has the largest number of pixels to form a binary map, marks all pixels in the layer as "1" and other pixels as "0", and uses the context-adaptive binary arithmetic coding method to encode This binary mapping;

After completing the mapping, the encoder continues to the next largest layer and repeats until one layer remains;

The last layer does not require binary mapping because it will automatically fill in all remaining pixels.
The method according to claim 2 or 6, wherein the final output data in the step of outputting the encapsulated bit stream consists of the following content:

An integer representing the number of layers in the depth block;

A series of bits containing the binary mapping of all layers;

A series of integers representing the depth value of each layer.
A system for multi-layer representation of an intra-frame coded depth map includes the following devices:

Blocking module to block the depth map data;

The progressive quantization module sets the progressive quantization and its stopping conditions for the block depth map data.
The system according to claim 1, further comprising the following devices:

Data encapsulation module, which encapsulates the multi-layer depth map that stops iteration;

The data output module outputs the encapsulated bit stream to the decoder side.
The system of claims 8-9, wherein:

The progressive quantization module for setting progressive quantization and its stopping conditions for the divided depth map data also includes:

The layering module decomposes the depth block into multiple layers, and each layer contains a subset of pixels in the depth block that are mutually exclusive with other layers;

A multi-layer representation module, which represents the multiple layers in a non-parametric manner, one of which can represent any subset of all pixels in the depth block;

The iterative module uses an iterative method to encode the multiple layers and continuously monitors reconstruction residuals and remaining areas with high priority.
The system according to claims 8-9, the progressive quantization module for setting progressive quantization and its stopping conditions for the divided depth map data includes:

The new layer creation module uses the encoder to initialize an empty layer list, creates a new layer in the layer list, and then classifies all pixels in the block as a unique layer;

The encoder repeatedly executes the following modules until the end of the iteration process:

Calculation module, for each layer in the layer list, use the encryptor to calculate the mean and variance of the depth values of all pixels in the layer, and attach each average to the corresponding layer;

Depth value reconstruction module, the encoder finds the maximum value of all the calculated variances, identifies the layer with the largest variance, calls it as the maximum variance layer; and assigns depth to each pixel by using the average value of the layer to which each pixel belongs Value to create a reconstructed block; calculate the sum of squared errors between the reconstructed block and the original depth block;

New layer creation module, use the encoder to create a new layer at the end of the layer list, select all pixels in the maximum variance layer and whose depth value is greater than the average value of the maximum variance layer, delete these pixels from the maximum variance layer and remove them Assign to the new layer;

When the sum of the square errors is less than the preset threshold, the iterative process ends.
The system of claim 11, wherein the predetermined threshold is the required reconstruction quality in terms of the sum of squared errors.
The system according to claim 9, wherein the data encapsulation module further comprises:

The average value appending module uses the encoder to calculate the average value and variance of the depth value of all pixels in the layer for all layers in the layer list; attach each average value to the corresponding layer;

Layer processing module, the encoder reorders the layers by sorting the area or the number of pixels of each layer in descending order, so that the layer with the most pixels is processed first;

Binary mapping module, the encoder uses the layer that has not been coded and has the largest number of pixels to form a binary mapping, marking all pixels in the layer as "1" and other pixels as "0", and uses context adaptive binary arithmetic The encoding method is used to encode this binary mapping; after the mapping is completed, the encoder continues to the next largest layer and repeats until one layer is left; the last layer does not need a binary mapping because it will automatically fill all remaining pixels.
The system as claimed in claim 9 or 13, wherein the final output data in the data encapsulation module consists of the following content:

An integer representing the number of layers in the depth block;

A series of bits containing the binary mapping of all layers;

A series of integers representing the depth value of each layer.
An encoder for implementing the method as claimed in claims 1-7, which sets progressive quantization and its stopping conditions for the block depth map data, which includes:

The new layer creation module uses the encoder to initialize an empty layer list, creates a new layer in the layer list, and then classifies all pixels in the block as a unique layer;

The encoder repeatedly executes the following modules until the iteration process ends:

Calculation module, for each layer in the layer list, use the encryptor to calculate the mean and variance of the depth values of all pixels in the layer, and attach each average to the corresponding layer;

Depth value reconstruction module, the encoder finds the maximum value of all the calculated variances, identifies the layer with the largest variance, calls it as the maximum variance layer; and assigns depth to each pixel by using the average value of the layer to which each pixel belongs Value to create a reconstructed block; calculate the sum of squared errors between the reconstructed block and the original depth block;

New layer creation module, use the encoder to create a new layer at the end of the layer list, select all pixels in the maximum variance layer and whose depth value is greater than the average value of the maximum variance layer, delete these pixels from the maximum variance layer and remove them Assign to the new layer;

When the sum of the square errors is less than the preset threshold, the iterative process ends.
An encoder for implementing the method according to claims 1-7, which performs data encapsulation on the multi-layer depth map that stops iteration, which comprises:

The average value appending module uses the encoder to calculate the average value and variance of the depth value of all pixels in the layer for all layers in the layer list; attach each average value to the corresponding layer;

Layer processing module, the encoder reorders the layers by sorting the area or the number of pixels of each layer in descending order, so that the layer with the most pixels is processed first;

Binary mapping module, the encoder uses the layer that has not been coded and has the largest number of pixels to form a binary mapping, marking all pixels in the layer as "1" and other pixels as "0", and uses context adaptive binary arithmetic The encoding method is used to encode this binary mapping; after the mapping is completed, the encoder continues to the next largest layer and repeats until one layer is left; the last layer does not need a binary mapping because it will automatically fill all remaining pixels.