WO2024077611A1

WO2024077611A1 - Decoding method, encoding method, decoder, and encoder

Info

Publication number: WO2024077611A1
Application number: PCT/CN2022/125476
Authority: WO
Inventors: 杨铀; 刘琼; 蒋小广; 黎振行
Original assignee: Oppo广东移动通信有限公司
Priority date: 2022-10-14
Filing date: 2022-10-14
Publication date: 2024-04-18

Abstract

Embodiments of the present application provide a decoding method, an encoding method, a decoder, and an encoder. The decoding method comprises: decoding a code stream, and determining a current sub-image block in a current image; determining whether the current sub-image block is a sub-image block needing illumination transformation; and when the current sub-image block is a sub-image block needing illumination transformation, performing illumination transformation on the current sub-image block, and obtaining a sub-image block after illumination transformation. The decoding method provided by the present application can improve decoding performance.

Description

Decoding method, encoding method, decoder and encoder

Technical Field

The embodiments of the present application relate to the field of coding and decoding, and more specifically, to a decoding method, an encoding method, a decoder and an encoder.

Background technique

For the Moving Picture Experts Group Immersive Video (MIV) encoding process, a limited number of viewpoints can be selected as basic viewpoints in the reference viewpoints, and the selected basic viewpoints can express the visible range of the scene as much as possible. Among them, the image under the basic viewpoint (also called the basic view) can be transmitted as a complete image; then, by removing the redundant pixels between the basic view and the image under the non-basic viewpoint (also called the additional view), that is, only retaining the effective information of non-repeated expression, and then extracting the effective information into sub-blocks and reorganizing them with the image under the basic viewpoint to form a larger rectangular image (also called a mosaic or mosaic image), which can be used to generate video data so that the encoder can encode the video data to obtain a bitstream. For example, the pixel pruning module can be used to detect repeated pixels between the basic view and the additional view, and prune the repeated pixels in the additional view. However, since the detection of repeated pixels between the base view and the additional view is based on depth difference values and brightness difference values, the retained pixels may include pixels that are greatly affected by lighting (such as highlight pixels). The information of highlight pixels will be degraded after video encoding and decoding, such as the loss of texture information of sub-blocks, thereby reducing the decoding performance.

Summary of the invention

The embodiments of the present application provide a decoding method, an encoding method, a decoder and an encoder, which can improve decoding performance.

In a first aspect, an embodiment of the present application provides a decoding method, including:

Decode the code stream and determine the current sub-block in the current image;

Determining whether to perform illumination transformation on the current sub-block;

When it is determined to perform illumination transformation on the current sub-image block, illumination transformation is performed on the current sub-image block to obtain a sub-image block after illumination transformation.

In a second aspect, an embodiment of the present application provides an encoding method, including:

Determine a current sub-block in a current image;

When it is determined to perform illumination transformation on the current sub-image block, perform illumination transformation on the current sub-image block to obtain a sub-image block after illumination transformation;

The sub-image block after the illumination transformation is encoded to obtain a bit stream.

In a third aspect, an embodiment of the present application provides a decoder, including:

A decoding unit, used for decoding the code stream and determining a current sub-block in a current image;

A determination unit, used to determine whether to perform illumination transformation on the current sub-block;

The transformation unit is used to perform the illumination transformation on the current sub-image block when it is determined to perform the illumination transformation on the current sub-image block, so as to obtain the sub-image block after the illumination transformation.

In a fourth aspect, an embodiment of the present application provides an encoder, including:

A first determining unit, configured to determine a current sub-block in a current image;

A second determining unit, used to determine whether to perform illumination transformation on the current sub-block;

A transformation unit, configured to perform a lighting transformation on the current sub-image block to obtain a sub-image block after the lighting transformation when it is determined to perform a lighting transformation on the current sub-image block;

The encoding unit is used to encode the sub-image block after the illumination transformation to obtain a code stream.

In a fifth aspect, an embodiment of the present application provides a decoder, including:

a processor adapted to implement computer instructions; and,

A computer-readable storage medium stores computer instructions, wherein the computer instructions are suitable for being loaded by a processor and executing the decoding method in the first aspect or its various implementations involved above.

In one implementation, the number of the processor is one or more, and the number of the memory is one or more.

In one implementation, the computer-readable storage medium may be integrated with the processor, or the computer-readable storage medium may be disposed separately from the processor.

In a sixth aspect, an embodiment of the present application provides an encoder, including:

a processor adapted to implement computer instructions; and,

A computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are suitable for being loaded by a processor and executing the encoding method in the second aspect or its various implementation methods involved above.

In the seventh aspect, an embodiment of the present application provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are read and executed by a processor of a computer device, the computer device executes the decoding method involved in the first aspect mentioned above or the encoding method involved in the second aspect mentioned above.

In an eighth aspect, an embodiment of the present application provides a computer program product or a computer program, the computer program product or the computer program including a computer instruction, the computer instruction being stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the decoding method involved in the first aspect mentioned above or the encoding method involved in the second aspect mentioned above.

In a ninth aspect, an embodiment of the present application provides a code stream, which is a code stream as described in the method described in the first aspect above or a code stream generated by the method described in the second aspect above.

Based on the above technical scheme, by introducing illumination transformation for the current sub-block that needs illumination transformation on the decoder side, it is beneficial for the encoder to enhance the quality of the current sub-block by illumination transformation of the current sub-block. For the decoder, the illumination transformation of the current sub-block can not only restore the illumination effect of the current sub-block, but also ensure that the sub-block decoded by the decoder is the sub-block after quality enhancement by the encoder, thereby alleviating the problem of quality degradation of its pixel information after video encoding and decoding, for example, it can alleviate the problem of texture information loss of the sub-block, thereby improving the decoding performance.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an example of a MIV encoding and decoding system provided in an embodiment of the present application.

FIG. 2 is an example of a preprocessing process provided in an embodiment of the present application.

FIG. 3 is an example of a depth map provided by an embodiment of the present application.

FIG. 4 is an example of a texture map provided in an embodiment of the present application.

FIG. 5 is an example of an occupancy map provided by an embodiment of the present application.

FIG. 6 is an example of a process for generating a mosaic graph provided in an embodiment of the present application.

FIG. 7 is a schematic flowchart of a decoding method provided in an embodiment of the present application.

FIG8 is a schematic flowchart of the encoding method provided in an embodiment of the present application.

FIG. 9 is an example of a mapping relationship between pixels in a base view and a pixel block where pixels in a current sub-block are located, provided in an embodiment of the present application.

FIG. 10 is an example of a similar pixel pair determination process provided in an embodiment of the present application.

FIG. 11 is an example of incorporating illumination transformation into the determination process of similar pixel pairs provided in an embodiment of the present application.

FIG12 is a schematic block diagram of a decoder provided in an embodiment of the present application.

FIG13 is a schematic block diagram of an encoder provided in an embodiment of the present application.

FIG. 14 is an example of an electronic device provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings.

FIG. 1 is an example of a MIV encoding and decoding system 10 provided in an embodiment of the present application.

As shown in FIG. 1 , the MIV encoding and decoding system 10 can be roughly divided into the following modules according to the task line: a data acquisition module 11 , a pre-processing module 12 , a video encoding framework 13 , a video decoding framework 14 and a post-processing module 15 .

The data acquisition module 11 is used to collect data. The data collected by the data acquisition module 11 may include not only texture information, but also depth information corresponding to the texture information. The texture information may be a three-channel color image, and the depth information may be a depth map. Optionally, the pixel value of the depth map may be used to reflect the distance measurement from the corresponding point to the acquisition device, that is, the pixel value of the depth map may be used to reflect the geometric information of the corresponding point.

The preprocessing module 12 is used to organize and express the data collected by the data acquisition module 11 to obtain the video to be encoded. The input source format of the preprocessing module 12 is a multi-viewpoint texture plus depth video, and the format of the video can be a perspective-projected plane video or a panoramic video. Specifically, after the data acquisition module 11 collects the corresponding video sequence through cameras at different positions, the preprocessing module 12 can first perform lighting balance and color correction on the images in the video sequence, and can also perform camera calibration and image calibration. If the video format collected by the data acquisition module 11 is a panoramic video, the preprocessing module 12 can also use image stitching technology to stitch the images in the panoramic video, and map the stitched stitching image into a two-dimensional plane video.

As shown in Figure 2, the reference viewpoint is the viewpoint used by the data acquisition module 11 to collect data. The reference viewpoint can be associated through viewpoint parameters, texture data and geometric data. The preprocessing module 12 can select a limited number of viewpoints from the reference viewpoint as basic viewpoints, and can enable the selected basic viewpoints to express the visible range of the scene as much as possible.

Among them, the image under the basic viewpoint (also called basic view) can be transmitted as a complete image. Furthermore, the redundant pixels between the remaining non-basic viewpoints (also called additional views) and the basic viewpoint can be removed, that is, only the valid information of non-repetitive expression is retained, and then the valid information is extracted into sub-blocks and reorganized with the image under the basic viewpoint to form a larger rectangular image (also called a mosaic or mosaic image).

Further, the spliced image can be used to generate video data, and the video data may include texture video data and geometry video data. The video data may be used as an input image of the video encoding framework 13, and the output of the video encoding framework 13 is a code stream obtained by encoding the input image. The code stream output by the video encoding framework 13 may include a geometry code stream and an attribute code stream. The geometry code stream is a code stream generated by encoding a geometry depth map (such as the depth map shown in FIG. 3), which is used to represent geometry information; the attribute code stream is a code stream generated by encoding a texture map (such as the texture map shown in FIG. 4), which is used to represent attribute information. Optionally, the code stream output by the video encoding framework 13 may also include an occupancy code stream, which is a code stream generated by encoding an occupancy map (such as the occupancy map shown in FIG. 5), which is used to indicate valid areas in the depth map and the texture map; these three types of images are encoded and decoded using a video encoder. The code stream output by the video encoding framework 13 may be used as an input code stream of the video decoding framework 14.

Of course, for the stitched image, there may also be auxiliary data related to the stitching of sub-blocks.

As shown in FIG. 2 , auxiliary data related to the stitching information of the sub-blocks can also be written into the bitstream, thereby reducing the transmission pixel rate while retaining scene information as much as possible, thereby ensuring that there is sufficient information for the post-processing module 15 to render the reconstructed image to obtain the final view.

As shown in Figure 6, the reference viewpoint may include viewpoint 0, viewpoint 1 and viewpoint 2. The reference viewpoint image collected at viewpoint 0 can be used to extract sub-block 2 and sub-block 4, the reference viewpoint image collected at viewpoint 1 can be used to extract sub-block 8, and the reference viewpoint image collected at viewpoint 2 can be used to extract sub-block 3 and sub-block 7. Furthermore, mosaic image 0 can be obtained based on sub-block 2, sub-block 4 and sub-block 8, and mosaic image 1 can be obtained based on sub-block 3 and sub-block 7; then mosaic image 0 and mosaic image 1 are spliced to the image under the basic viewpoint to obtain the final mosaic image, and the final mosaic image can be used as the input image of the video encoding framework 13 to encode the input image to obtain a bitstream.

The video encoding framework 13 is used to encode and compress the video, and the video decoding framework 14 is used to decode and reconstruct the video.

It should be noted that the present application does not limit the specific implementation of the video encoding framework 13 and the video decoding framework 14. For example, a codec framework that applies three-dimensional video encoding technologies such as MV-HEVC and 3D-HEVC to multi-viewpoint video can be used, and the encoding and decoding efficiency of this codec framework is higher than that of HEVC. Of course, a codec framework of traditional planar video hybrid encoding technologies such as HEVC and VVC can be used.

The post-processing module 15 is used to synthesize and render the image decoded by the video decoding framework 14 to obtain the final view. For example, the target view at the user's viewpoint position can be synthesized and rendered based on the decoded and reconstructed image and the user's current posture information.

FIG7 is a schematic flow chart of a decoding method 200 provided in an embodiment of the present application. It should be understood that the decoding method 200 can be executed by a decoder. For example, it is applied to the decoding framework 14 shown in FIG1. For ease of description, the following description is made by taking a decoder as an example.

As shown in FIG. 7 , the decoding method 200 may include part or all of the following:

S210, decoding the code stream to determine a current sub-block in the current image;

S220, determining whether to perform illumination transformation on the current sub-block;

S230: When it is determined to perform illumination transformation on the current sub-image block, perform illumination transformation on the current sub-image block to obtain a sub-image block after illumination transformation.

Exemplarily, the current image may have an additional view.

Exemplarily, the current sub-image block may be an image block obtained by the encoder processing the additional view. For example, the encoder may remove redundant pixels between the base view and the additional view, that is, retain only valid pixels (also referred to as valid pixel clusters) that are not repeatedly expressed, and then divide and pack the valid pixel clusters to form image blocks (i.e., sub-image blocks) of regular shapes, and then reorganize the base view and one or more sub-image blocks obtained based on the additional view (for example, splicing the base view and the sub-image blocks obtained based on the additional view) to form a larger rectangular image (also referred to as a spliced image or a spliced image), which may be used to generate video data so that the encoder can encode the video data to obtain a bitstream.

Accordingly, the decoder may restore or decode the additional view based on the current sub-image block.

Specifically, the decoder may determine, based on the current sub-image block, a valid pixel cluster retained by the encoder after clipping redundant pixels in the additional view, and then restore the additional view based on the valid pixel cluster. For example, the decoder may determine the position of the current sub-image block in the additional view based on the clipping mask of the additional view, and based on the position of the decoded valid pixel cluster, use pixels in the base view to restore the clipped redundant pixels in the additional view, and then restore the additional view based on the pixels of the current sub-image block and the clipped points in the additional view.

Exemplarily, the current sub-image block may be a rectangular image block.

In this embodiment, by introducing illumination transformation for the current sub-block that needs illumination transformation on the decoder side, it is beneficial for the encoder to enhance the quality of the current sub-block by illumination transformation of the current sub-block. For the decoder, the illumination transformation of the current sub-block can not only restore the illumination effect of the current sub-block, but also ensure that the sub-block decoded by the decoder is the sub-block whose quality has been enhanced by the encoder, thereby alleviating the problem of quality degradation of its pixel information after video encoding and decoding, for example, it can alleviate the problem of texture information loss of the sub-block, thereby improving the decoding performance.

In some embodiments, the S220 may include:

Decoding the bitstream to determine a sub-image block identifier of the current sub-image block; the sub-image block identifier of the current sub-image block indicates whether to perform illumination transformation on the current sub-image block;

Based on the sub-image block identifier of the current sub-image block, it is determined whether to perform illumination transformation on the current sub-image block.

Exemplarily, if the sub-block identifier of the current sub-block indicates that the current sub-block is to be illuminated by a transformation, the decoder determines that the current sub-block is to be illuminated by a transformation; if the sub-block identifier of the current sub-block indicates that the current sub-block is not to be illuminated by a transformation, the decoder determines that the current sub-block is not to be illuminated by a transformation.

Of course, in other alternative embodiments, whether to perform illumination transformation on the current sub-tile may also be determined in other ways.

In a possible implementation, the decoder may determine whether to perform illumination transformation on the current sub-image block based on whether the bitstream carries the identifier of the current sub-image block. For example, if the bitstream carries the identifier of the current sub-image block, the decoder determines to perform illumination transformation on the current sub-image block; if the bitstream does not carry the identifier of the current sub-image block, the decoder determines not to perform illumination transformation on the current sub-image block.

In another possible implementation, the decoder may also perform a pixel-level illumination transformation or a view-level illumination transformation. For example, the decoder may also determine whether to perform an illumination transformation on the current pixel based on an identifier corresponding to the current pixel in the current sub-image block, where the identifier corresponding to the current pixel indicates whether to perform an illumination transformation on the current pixel. For another example, the decoder may also determine whether to perform an illumination transformation on a sub-image block in an additional view based on an identifier corresponding to an additional view to which the current sub-image block belongs; wherein the identifier corresponding to the additional illustration indicates whether to perform illumination changes with the sub-image block in the additional illustration.

In some embodiments, the decoder may also decode the bitstream to determine an enable flag of the sub-picture block identifier of the current sub-picture block; the enable flag indicates whether the bitstream carries the sub-picture block identifier of the current sub-picture block.

Exemplarily, the enable flag may be carried in an Atlas sequence parameter set MIV extension syntax in the code stream.

Exemplarily, the Atlas sequence parameter set MIV extended syntax may be as shown in Table 1:

Table 1

As shown in Table 1, asme_ic_enabled_flag is an enable flag.

Where asme_ic_enabled_flag is 1, which means that the syntax element pdu_ic_flag[tileID][p] exists in the syntax structure pdu_miv_extension(). asme_ic_enabled_flag is 0, which means that the syntax element pdu_ic_flag[tileID][p] does not exist in the syntax structure pdu_miv_extension(). When asme_ic_enabled_flag does not exist, its value defaults to 0. (asme_ic_enabled_flag equal to 1 specifies that the pdu_ic_flag[tileID][p]syntax elements are present in the pdu_miv_extension()syntax structure. asme_ic_enabled_flag equal to 0 specifies that the pdu_ic_flag[tileID][p]syntax elements are not present in the pdu_miv_extension()syntax structure. When not present, the value of asme_ic_enabled_flag is inferred to be equal to 0).

Exemplarily, the sub-tile identifier of the current sub-tile may be carried in a Patch data unit MIV extension syntax (Patch data unit MIV extension syntax) in the code stream.

Exemplarily, the sub-block data unit MIV extended syntax may be as shown in Table 2:

Table 2

As shown in Table 2, pdu_ic_flag[tileID][p] is the sub-tile identifier.

Among them, pdu_ic_flag[tileID][p] is 1, which means that the sub-tile performs lighting transformation. pdu_ic_flag[tileID][p] is 0, which means that the sub-tile does not perform lighting transformation. When pdu_ic_flag[tileID][p] does not exist, its value defaults to 0. (pdu_ic_flag[tileID][p]equal to 1 specifies that the patch is an IC patch. When pdu_ic_flag[tileID][p]is equal to 0, then the patch is not an IC patch. When not present, the value of pdu_ic_flag[tileID][p] is inferred to be equal to 0.)

In some embodiments, the S230 may include:

Determine a target transformation mode used by the current sub-block;

Determining target transformation parameters used by the current sub-block;

Based on the target transformation mode and the target transformation parameters, the current sub-image block is subjected to illumination transformation to obtain the sub-image block after illumination transformation.

Exemplarily, when the decoder determines to perform illumination transformation on the current sub-image block, it first determines the target transformation method and the target transformation parameters, and then performs illumination transformation on the current sub-image block using the target transformation method based on the target transformation parameters to obtain the sub-image block after illumination transformation.

In some embodiments, the target transformation mode is a default transformation mode.

Exemplarily, the default transformation method may be a logarithmic transformation method, a gamma transformation method, or other transformation methods, and even the default transformation method may be a transformation method based on machine learning, which is not specifically limited in the present application.

Of course, in other alternative embodiments, the decoder may also decode the code stream to determine the target transformation mode, and this application does not make any specific limitation on this.

In some embodiments, the target transformation parameters are transformation parameters used by the target transformation method by default.

Of course, in other alternative embodiments, the decoder may also decode the code stream to determine the target transformation parameters, which is not specifically limited in the present application.

In some embodiments, based on the target transformation mode and the target transformation parameters, the contrast of the attribute information of the pixels in the current sub-block is adjusted to obtain the sub-block after the illumination transformation; wherein the contrast between the attribute information of the pixels in the sub-block after the illumination transformation is greater than the contrast between the attribute information of the pixels in the current sub-block.

In other words, the target transformation parameters used by the current sub-block can increase the contrast between the attribute information of the pixels in the current sub-block.

Exemplarily, the contrast of the attribute information of the pixels in the current sub-block may be the contrast between the texture information of the pixels in the current sub-block. In other words, the target transformation parameters used by the current sub-block can increase the contrast between the texture information of the pixels in the current sub-block.

Exemplarily, the contrast of the attribute information of the pixels in the current sub-block may be the contrast between the color information of the pixels in the current sub-block. In other words, the target transformation parameters used by the current sub-block can increase the contrast between the color information of the pixels in the current sub-block.

It is worth noting that when the target transformation mode and the target transformation mode are fixed, the difference between the contrast between the attribute information of the pixels in the sub-image block after the illumination transformation and the contrast between the attribute information of the pixels in the current sub-image block is also fixed. Therefore, the decoder can also determine the difference between the contrast between the attribute information of the pixels in the sub-image block after the illumination transformation and the contrast between the attribute information of the pixels in the current sub-image block by decoding the code stream, and then determine the target transformation mode and the target transformation parameters based on the difference between the contrast between the attribute information of the pixels in the sub-image block after the illumination transformation and the contrast between the attribute information of the pixels in the current sub-image block, as well as the contrast of the sub-image block after the illumination transformation. The present application does not make specific limitations on this.

In some embodiments, the method 200 may further include:

Based on the sub-image block after the illumination transformation, a decoded image of the current image is determined.

Exemplarily, the decoder determines a decoded image of the additional image to which the current sub-image block belongs based on the sub-image block after the illumination transformation.

The decoding method according to the embodiment of the present application is described in detail above from the perspective of a decoder. The encoding method according to the embodiment of the present application will be described below from the perspective of an encoder.

FIG8 is a schematic flow chart of a coding method 310 provided in an embodiment of the present application. It should be understood that the coding method 310 can be executed by an encoder. For example, it is applied to the coding framework 13 shown in FIG1. For ease of description, the following description is made by taking an encoder as an example.

As shown in FIG8 , the decoding method 310 may include part or all of the following:

S311, determining a current sub-block in the current image;

S312, determining whether to perform illumination transformation on the current sub-block;

S313, when it is determined to perform illumination transformation on the current sub-image block, perform illumination transformation on the current sub-image block to obtain a sub-image block after illumination transformation;

S314: Encode the sub-image block after the illumination transformation to obtain a bit stream.

Exemplarily, the current image may have an additional view.

Exemplarily, the current sub-image block may be a rectangular image block.

In this embodiment, by introducing illumination transformation for the current sub-block that needs illumination transformation on the encoder side, the encoder can enhance the quality of the current sub-block by illumination transformation of the current sub-block. For the decoder, the illumination transformation of the current sub-block can not only restore the illumination effect of the current sub-block, but also ensure that the sub-block decoded by the decoder is the sub-block whose quality has been enhanced by the encoder. Furthermore, the problem of quality degradation of its pixel information after video encoding and decoding can be alleviated, for example, the problem of texture information loss of the sub-block can be alleviated, thereby improving the decoding performance.

In some embodiments, the S314 may include:

Acquire a sub-image block identifier of the current sub-image block; the sub-image block identifier of the current sub-image block indicates whether to perform illumination transformation on the current sub-image block;

The sub-image block identifier of the current sub-image block and the sub-image block after the illumination transformation are encoded to obtain the code stream.

In this embodiment, by writing the sub-block identifier of the current sub-block into the bitstream, it is helpful for the decoder to decode the bitstream and determine whether to perform illumination transformation with the current sub-block. This can ensure that the decoder and the encoder have consistent understanding of whether illumination transformation is required for the current sub-block, thereby improving encoding and decoding efficiency.

Of course, in other alternative embodiments, other methods may be used to indicate to the decoder whether to perform illumination transformation on the current sub-block.

In a possible implementation, the encoder encodes the identifier of the sub-image block that needs to be transformed for illumination, so that the decoder determines whether to perform the illumination transformation based on the decoded sub-image block identifier. For example, the decoder may determine whether to perform the illumination transformation on the current sub-image block based on whether the bitstream carries the identifier of the current sub-image block. For example, if the bitstream carries the identifier of the current sub-image block, the decoder determines to perform the illumination transformation on the current sub-image block; if the bitstream does not carry the identifier of the current sub-image block, the decoder determines not to perform the illumination transformation on the current sub-image block.

In another possible implementation, the encoder may also encode a pixel-level illumination change identifier or a view-level illumination change identifier. For example, the decoder may determine whether to perform an illumination change on the current pixel based on an identifier corresponding to the current pixel in the current sub-image block, and the identifier corresponding to the current pixel indicates whether to perform an illumination change on the current pixel. For another example, the decoder may also determine whether to perform an illumination change on a sub-image block in an additional view based on an identifier corresponding to an additional view to which the current sub-image block belongs; wherein the identifier corresponding to the additional illustration indicates whether to perform illumination changes with the sub-image block in the additional illustration.

In some embodiments, the S312 may include:

Get the base view;

Performing illumination detection on a pixel pair formed by a pixel in the base view and a pixel in the current sub-image block to determine whether the pixel pair is a pixel pair with inconsistent illumination;

Based on the number of the pixel pairs with inconsistent illumination, it is determined whether to perform illumination transformation on the current sub-block.

Exemplarily, the encoder may first map any pixel of the base view to the additional view to which the current sub-image block belongs based on parameters of a camera used to capture the base view, parameters of a camera used to capture the additional view to which the current sub-image block belongs, depth information of the base view, and depth information of the additional view to which the current sub-image block belongs, so as to obtain one or more pixel pairs formed by pixels in the base view and pixels in the current sub-image block.

Exemplarily, the pixel in the pixel pair from the current sub-image block, when the pixel in the pixel pair from the base view is projected or mapped to the current sub-image block, is the pixel of the current sub-image block at the projection position or the mapping position.

FIG. 9 is an example of a mapping relationship between pixels in a base view and a pixel block where pixels in a current sub-image block are located, provided in an embodiment of the present application.

As shown in Figure 9, assuming that view v0 is the base view and view v1 is the additional view to which the current sub-image block belongs, any pixel of view v0 can be mapped to view v1 based on the parameters of the camera used to capture view v0, the parameters of the camera used to capture view v1, the depth information of view v0, and the depth information of view v1 to obtain view v0, so as to obtain one or more pixel pairs formed by pixels in view v0 and pixels in the current sub-image block; for example, the figure exemplarily shows a pixel pair formed by pixel 1 in view v0 and pixel 2 in the current sub-image block.

In some embodiments, a distribution difference value between color information of pixels in the base view and color information of pixels in the current sub-block is determined; and based on the distribution difference value, it is determined whether the pixel pair is a pixel pair with inconsistent illumination.

Exemplarily, the distribution difference value may be used to characterize the difference between the illumination effect of the color information of the pixel in the base view and the illumination effect of the color information of the pixel in the current sub-image block. For example, if the distribution difference value is smaller, it means that the illumination effect of the color information of the pixel in the base view is closer to the illumination effect of the color information of the pixel in the current sub-image block, and thus it can be shown that the pixel pair is not an illumination-inconsistent pixel pair. Conversely, if the distribution difference value is larger, it means that the illumination effect of the color information of the pixel in the base view is different from the illumination effect of the color information of the pixel in the current sub-image block, and thus it can be shown that the pixel pair is an illumination-inconsistent pixel pair.

In some embodiments, the maximum difference value among (1-R ₁ )/(1-R ₂ ), (1-G ₁ )/(1-G ₂ ) and (1-B ₁ )/(1-B ₂ ) is determined as the distribution difference value; wherein R ₁ , G ₁ , B ₁ respectively represent the values of the B component, G component, and B component of the pixel in the current sub-block; and R ₂ , G ₂ , B ₂ respectively represent the values of the B component, G component, and B component of the pixel in the base view.

For example, the encoder may first determine the difference between (1-R ₁ )/(1-R ₂ ) and (1-G ₁ )/(1-G ₂ ), the difference between (1-R ₁ )/(1-R ₂ ) and (1-B ₁ )/(1-B ₂ ), and the difference between (1-B ₁ )/(1-B ₂ ) and (1-G ₁ )/(1-G ₂ ), and convert the difference between (1-R ₁ )/(1-R ₂ ) and (1-G ₁ )/(1-G ₂ ), the difference between (1-R ₁ )/(1-R ₂ ) and (1-B ₁ )/(1-B ₂ ), and the difference between (1-B ₁ )/(1-B ₂ ) and (1-G ₁ )/(1-G _{2 ).} ) is determined as the distribution difference value.

Since the illumination effect of the pixels in the current sub-block is closer to the illumination effect of the highlight, the values of the B component, G component, and B component of the pixels in the current sub-block are closer to 1. Therefore, in the embodiment of the present application, the closer (1-R ₁ )/(1-R ₂ ), (1-G ₁ )/(1-G ₂ ), and (1-B ₁ )/(1-B ₂ ) are, the closer the illumination effect between the color information of the pixels in the current sub-block is to the highlight effect, and the detection of the highlighted sub-block can be achieved, which is equivalent to that the encoder can perform illumination transformation on the highlighted sub-block, and further, the problem of quality degradation of its pixel information after video encoding and decoding can be alleviated, for example, the problem of texture information loss of the sub-block can be alleviated, thereby improving the decoding performance.

Of course, in other alternative embodiments, the encoder may also determine the distribution difference value by taking the difference between two items of (1-R ₁ )/(1-R ₂ ), (1-G ₁ )/(1-G ₂ ) and (1-B ₁ )/(1-B ₂ ). This application does not impose any specific limitation on this.

For example, the encoder may determine the distribution difference value by the difference between (1-R ₁ )/(1-R ₂ ) and (1-G ₁ )/(1-G ₂ ). For another example, the encoder may determine the distribution difference value by the difference between (1-R ₁ )/(1-R ₂ ) and (1-B ₁ )/(1-B ₂ ). For another example, the encoder may determine the distribution difference value by the difference between (1-B ₁ )/(1-B ₂ ) and (1-G ₁ )/(1-G ₂ ).

In some embodiments, if the distribution difference value is less than or equal to a first threshold, the pixel pair is determined to be a pixel pair with inconsistent illumination; if the distribution difference value is greater than the first threshold, the pixel pair is determined not to be a pixel pair with inconsistent illumination.

Exemplarily, the first threshold may be a preset threshold, such as a threshold predefined by a standard.

Of course, in other alternative embodiments, the encoder may also determine whether the pixel pair is a pixel pair with inconsistent illumination based on the range to which the distribution difference value belongs. For example, if the range to which the distribution difference value belongs is a first range, the pixel pair is determined to be a pixel pair with inconsistent illumination, and if the range to which the distribution difference value belongs is a second range, the pixel pair is determined not to be a pixel pair with consistent illumination. Optionally, the maximum value of the first range is less than or equal to the minimum value of the second range.

In some embodiments, based on the number of the pixel pairs with inconsistent illumination, the proportion of the pixel pairs with inconsistent illumination is determined; and based on the proportion of the pixel pairs with inconsistent illumination, it is determined whether to perform illumination transformation on the current sub-image block.

Exemplarily, the encoder may determine the ratio of the number of the illumination-inconsistent pixel pairs to the number of pixels in the current sub-block as the proportion of the illumination-inconsistent pixel pairs.

In some embodiments, if the proportion of the pixel pairs with inconsistent lighting is greater than or equal to a second threshold, the current sub-block is determined to be a sub-block that requires lighting transformation; if the proportion of the pixel pairs with inconsistent lighting is less than the second threshold, the current sub-block is determined to be a sub-block that does not require lighting transformation.

Exemplarily, the second threshold may be a predefined threshold, such as a standard predefined threshold.

In some embodiments, it is determined whether the pixel pair is a similar pixel pair; when the pixel pair is not a similar pixel pair, illumination detection is performed on the pixel pair to determine whether the pixel pair is a pixel pair with inconsistent illumination.

Exemplarily, the encoder triggers illumination detection on the pixel pair only when it determines that the pixel pair is not a similar pixel pair, and further determines whether the pixel pair is a pixel pair with inconsistent illumination.

In some embodiments, the S312 may include:

determining a depth difference value between the pixel pairs;

Determine a brightness difference value between a pixel in the base view and a pixel block where a pixel in the current sub-block is located;

When the depth difference value is less than a third threshold and the brightness difference value is less than a fourth threshold, determining that the pixel pair is a similar pixel pair;

When the depth difference value is greater than or equal to the third threshold or the brightness difference value is greater than or equal to the fourth threshold, it is determined that the pixel pair is not a similar pixel pair.

Exemplarily, the encoder may determine the difference value between the brightness of the pixel in the base view and the brightness of each pixel in the pixel block, and determine the maximum value of the determined difference values as the maximum difference value. For example, assuming that the pixel block is a 3×3 pixel block centered on the pixel in the current sub-block, the encoder may determine the difference value (i.e., 9 difference values) between the brightness of the pixel in the base view and the brightness of each pixel in the 3×3 pixel block and determine the maximum value of the determined 9 difference values as the maximum difference value.

It is worth noting that the encoder may determine the depth difference value in a pixel-to-pixel manner and determine the brightness difference value in a pixel-to-block manner. However, this application does not specifically limit this. For example, in other alternative embodiments, the encoder may also determine the brightness difference value in a pixel-to-pixel manner.

Of course, in other alternative embodiments, the encoder may also perform illumination detection on the pixel pair when the depth difference value is greater than or equal to the third threshold value to determine whether the pixel pair is a pixel pair with inconsistent illumination. That is, when the encoder determines that the pixel pair is not a pixel pair with similar brightness, it may trigger illumination detection on the pixel pair to further determine whether the pixel pair is a pixel pair with inconsistent illumination.

FIG. 10 is an example of a similar pixel pair determination process 320 provided in an embodiment of the present application.

As shown in FIG. 10 , the determination process 320 may include:

S321 , the encoder determines whether the depth difference value is less than t1 .

Exemplarily, the encoder may calculate a depth difference value of a pixel pair formed by a pixel in the base view and a pixel in the additional image, and determine whether the depth difference value is less than t1. For example, if the depth difference value is less than t1, the encoder determines that the pixel pair is a pixel pair with similar depth, otherwise, the encoder determines that the pixel pair is not a pixel pair with similar depth. The pixel in the base view may be pixel 1 as shown in FIG. 9, and the pixel in the additional image may be pixel 2 as shown in FIG. 9.

S322, the encoder determines whether the brightness difference value is less than t2.

Exemplarily, the encoder may determine the difference value between the brightness of the pixel in the base view and the brightness of each pixel in the pixel block where the pixel in the additional image is located, and determine the maximum value of the determined difference values as the maximum difference value. For example, assuming that the pixel block is a 3×3 pixel block centered on the pixel in the additional image, the encoder may determine the difference value (i.e., 9 difference values) between the brightness of the pixel in the base view and the brightness of each pixel in the 3×3 pixel block and determine the maximum value of the determined 9 difference values as the brightness difference value, and determine whether the brightness difference value is less than t2. For example, if the brightness difference value is less than t2, the encoder determines that the pixel pair is a pixel pair with similar brightness, otherwise, the encoder determines that the pixel pair is not a pixel pair with similar brightness.

S323, execute the second stage.

Exemplarily, the above S321 and S322 can be understood as the first stage of similarity detection on the pixel pair. After the first stage is executed, the second stage can be executed on the suspected pixel pair output by the first stage. After the second stage is executed, the final similar pixel pair can be determined. Optionally, when the encoder performs the second stage detection on the suspected similar pixel pair, it can determine whether the suspected similar pixel pair is a similar pixel based on the color difference value between the suspected similar pixel pair. It is worth noting that the execution step of the second stage can be an optional step.

In this embodiment, after the encoder uses the pixel pruning module to detect the pixels in the additional view and the pixels in the base view, the pixels can be pruned in subsequent processing, that is, only the valid pixels with non-repetitive expression (also called valid pixel clusters) are retained, and then the valid pixel clusters are divided and packed to form image blocks of regular shapes (i.e. sub-blocks), and then the base view and one or more sub-blocks obtained based on the additional view are reorganized (for example, the base view and the sub-blocks obtained based on the additional view are spliced) to form a larger rectangular image (also called a spliced image or a spliced image), which can be used to generate video data so that the encoder can encode the video data to obtain a bit stream.

FIG. 11 is an example of combining illumination transformation with the determination process 330 of similar pixel pairs provided in an embodiment of the present application.

As shown in FIG. 11 , the determination process 330 may include:

S331 , the encoder determines whether the depth difference value is less than t1 .

S332, the encoder determines whether the brightness difference value is less than t2.

S333, execute the second stage.

Exemplarily, the above S321 and S322 can be understood as the first stage of similarity detection on the pixel pairs. After executing the first stage, the second stage can be executed on the suspected pixel pairs output by the first stage. After the second stage is executed, the final similar pixel pairs can be determined. Optionally, when the encoder performs the second stage detection on the suspected similar pixel pairs, it can determine whether the suspected similar pixel pairs are similar pixels based on the color difference value between the suspected similar pixel pairs. It is worth noting that the execution step of the second stage can be an optional step.

S334, the encoder determines whether the distribution difference value is less than t3.

Exemplarily, the encoder may also perform illumination detection on the pixel pair when the depth difference value is greater than or equal to t1 to determine whether the pixel pair is an illumination-inconsistent pixel pair. That is, when the encoder determines that the pixel pair is not a pixel pair with similar brightness, it may trigger illumination detection on the pixel pair to further determine whether the pixel pair is an illumination-inconsistent pixel pair. For example, if the distribution difference value of the pixel pair is less than t3, the pixel pair is determined to be an illumination-inconsistent pixel pair; otherwise, the pixel pair is determined not to be an illumination-inconsistent pixel pair.

S335 : The encoder determines whether to perform illumination transformation on the current sub-block based on the detection result of the pixel pairs with inconsistent illumination.

S336, the encoder performs a transformation on the current sub-block.

It should be understood that the above S335 and S336 may refer to the description in method 310, and will not be repeated here to avoid repetition.

In this embodiment, after the encoder uses the pixel pruning module to detect the pixels in the additional view and the pixels in the basic view, the pixels can be pruned in subsequent processing, that is, only the valid pixels with non-repeated expression (also called valid pixel clusters) are retained, and then the valid pixel clusters are divided and packed to form image blocks of regular shapes (i.e., sub-blocks), and then the sub-blocks that need to be illuminated in the one or more obtained sub-blocks are illuminated, and the illuminated sub-blocks are obtained; then the basic view and the sub-blocks that do not need to be illuminated obtained based on the additional view, as well as the illuminated sub-blocks that need to be illuminated are reorganized (for example, the basic view and the sub-blocks obtained based on the additional view are spliced) to form a larger rectangular image (also called a spliced image or a spliced image), which can be used to generate video data so that the encoder can encode the video data and obtain a bitstream.

In some embodiments, a target transformation method used by the current sub-image block is determined; target transformation parameters used by the current sub-image block are determined; and based on the target transformation method and the target transformation parameters, an illumination transformation is performed on the current sub-image block to obtain the sub-image block after the illumination transformation.

Exemplarily, when the encoder determines to perform illumination transformation on the current sub-image block, it first determines the target transformation method and the target transformation parameters, and then performs illumination transformation on the current sub-image block using the target transformation method based on the target transformation parameters to obtain the sub-image block after illumination transformation.

Of course, in other alternative embodiments, the encoder may also write the target transformation mode into the bitstream, and this application does not make any specific limitation on this.

Of course, in other alternative embodiments, the encoder may also write the target transformation parameters into the bitstream, which is not specifically limited in the present application.

In some embodiments, based on the target transformation method and the target transformation parameters, the contrast of the attribute information of the pixels in the current sub-block is adjusted to obtain the sub-block after the illumination transformation; wherein the contrast between the attribute information of the pixels in the sub-block after the illumination transformation is less than the contrast between the attribute information of the pixels in the current sub-block.

In other words, the target transformation parameters used by the current sub-image block can reduce the contrast between the attribute information of the pixels in the current sub-image block.

Exemplarily, the contrast of the attribute information of the pixels in the current sub-image block may be the contrast between the texture information of the pixels in the current sub-image block. In other words, the target transformation parameters used by the current sub-image block can reduce the contrast between the texture information of the pixels in the current sub-image block.

Exemplarily, the contrast of the attribute information of the pixels in the current sub-block may be the contrast between the color information of the pixels in the current sub-block. In other words, the target transformation parameters used by the current sub-block can reduce the contrast between the color information of the pixels in the current sub-block.

It is worth noting that, since the encoder strengthens the contrast between the attribute information of the pixels in the current sub-block, the decoder needs to weaken the contrast between the attribute information of the pixels in the current sub-block when restoring the attribute information of the pixels in the current sub-block. Of course, in other alternative embodiments, the encoder performs illumination transformation on the current sub-block, and for the decoder, the illumination transformation performed on the current sub-block can also be understood as the inverse illumination transformation performed by the encoder on the current sub-block. This application does not make specific limitations on this.

It is worth noting that when the target transformation mode and the target transformation mode are fixed, the difference between the contrast between the attribute information of the pixels in the sub-image block after the illumination transformation and the contrast between the attribute information of the pixels in the current sub-image block is also fixed. Therefore, the encoder can also determine the difference between the contrast between the attribute information of the pixels in the sub-image block after the illumination transformation and the contrast between the attribute information of the pixels in the current sub-image block by decoding the code stream, and then determine the target transformation mode and the target transformation parameters based on the difference between the contrast between the attribute information of the pixels in the sub-image block after the illumination transformation and the contrast between the attribute information of the pixels in the current sub-image block, as well as the contrast of the sub-image block after the illumination transformation. The present application does not make specific limitations on this.

In some embodiments, the sub-image block after the illumination transformation is spliced with a base view to obtain a spliced image; and pixels in the spliced image and pixels in the sub-image block after the illumination transformation are encoded to obtain the code stream.

Exemplarily, the encoder may remove redundant pixels between the base view and the additional view, that is, retain only valid pixels (also referred to as valid pixel clusters) that are not expressed repeatedly, and then divide and pack the valid pixel clusters to form image blocks of regular shapes (i.e., sub-blocks), and then perform illumination transformation on the sub-blocks that need to be illuminated in the one or more obtained sub-blocks, and obtain the sub-blocks after illumination transformation; and then reorganize the base view and the sub-blocks that do not need to be illuminated obtained based on the additional view, as well as the sub-blocks that need to be illuminated and obtained after illumination transformation (for example, the base view and the sub-blocks obtained based on the additional view are spliced) to form a larger rectangular image (also referred to as a spliced image or a spliced image), which can be used to generate video data so that the encoder can encode the video data and obtain a bitstream.

The preferred embodiments of the present application are described in detail above in conjunction with the accompanying drawings. However, the present application is not limited to the specific details in the embodiments mentioned above. Within the technical concept of the present application, the technical solution of the present application can be subjected to a variety of simple modifications, and these simple modifications all belong to the protection scope of the present application. For example, the various specific technical features described in the specific embodiments mentioned above can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present application will not further explain various possible combinations. For another example, the various different embodiments of the present application can also be combined arbitrarily, as long as they do not violate the idea of the present application, they should also be regarded as the contents disclosed in the present application. It should also be understood that in the various method embodiments of the present application, the size of the sequence number of each process mentioned above does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

The method embodiment of the present application is described in detail above. The device embodiment of the present application is described in detail below in conjunction with Figures 12 to 14.

FIG. 12 is a schematic block diagram of a decoder 400 according to an embodiment of the present application.

As shown in FIG. 12 , the decoder 400 may include:

A decoding unit 410, configured to decode the bitstream and determine a current sub-block in a current image;

A determination unit 420, configured to determine whether to perform illumination transformation on the current sub-block;

The transformation unit 430 is configured to perform a lighting transformation on the current sub-image block to obtain a sub-image block after the lighting transformation when it is determined to perform a lighting transformation on the current sub-image block.

In some embodiments, the determining unit 420 is specifically configured to:

In some embodiments, the transform unit 430 is specifically used to:

Determine a target transformation mode used by the current sub-block;

Determining target transformation parameters used by the current sub-block;

In some embodiments, the transform unit 430 is specifically used for:

Based on the target transformation mode and the target transformation parameters, the contrast of the attribute information of the pixels in the current sub-image block is adjusted to obtain the sub-image block after the illumination transformation; wherein the contrast between the attribute information of the pixels in the sub-image block after the illumination transformation is greater than the contrast between the attribute information of the pixels in the current sub-image block.

In some embodiments, the decoding unit 410 is further configured to:

FIG. 13 is a schematic block diagram of an encoder 500 according to an embodiment of the present application.

As shown in FIG. 13 , the encoder 500 may include:

A first determining unit 510, configured to determine a current sub-block in a current image;

A second determining unit 520, configured to determine whether to perform illumination transformation on the current sub-block;

A transformation unit 530, configured to perform a lighting transformation on the current sub-image block to obtain a sub-image block after the lighting transformation when it is determined to perform a lighting transformation on the current sub-image block;

The encoding unit 540 is used to encode the sub-block after the illumination transformation to obtain a code stream.

In some embodiments, the encoding unit 540 is specifically used to:

In some embodiments, the second determining unit 520 is specifically configured to:

Get the base view;

Based on the number of the pixel pairs with inconsistent illumination, it is determined whether to perform illumination transformation on the current sub-image block.

Determine a distribution difference value between color information of pixels in the base view and color information of pixels in the current sub-block;

Based on the distribution difference value, it is determined whether the pixel pair is a pixel pair with inconsistent illumination.

The maximum difference value among (1-R ₁ )/(1-R ₂ ), (1-G ₁ )/(1-G ₂ ) and (1-B ₁ )/(1-B ₂ ) is determined as the distribution difference value; wherein R ₁ , G ₁ , B ₁ respectively represent the values of the B component, G component, and B component of the pixel in the current sub-block; and R ₂ , G ₂ , B ₂ respectively represent the values of the B component, G component, and B component of the pixel in the base view.

If the distribution difference value is less than or equal to a first threshold, determining that the pixel pair is a pixel pair with inconsistent illumination;

If the distribution difference value is greater than the first threshold, it is determined that the pixel pair is not a pixel pair with inconsistent illumination.

Determining a proportion of the pixel pairs with inconsistent illumination based on the number of the pixel pairs with inconsistent illumination;

Based on the proportion of the pixel pairs with inconsistent illumination, it is determined whether to perform illumination transformation on the current sub-image block.

If the proportion of the pixel pairs with inconsistent illumination is greater than or equal to a second threshold, determining that the current sub-image block is a sub-image block that needs illumination transformation;

If the proportion of the pixel pairs with inconsistent lighting is less than the second threshold, the current sub-block is determined to be a sub-block that does not require lighting change.

Determining whether the pixel pair is a similar pixel pair;

When the pixel pair is not a similar pixel pair, light detection is performed on the pixel pair to determine whether the pixel pair is a pixel pair with inconsistent light.

determining a depth difference value between the pixel pairs;

In some embodiments, the transform unit 530 is specifically used to:

Determine a target transformation mode used by the current sub-block;

Determining target transformation parameters used by the current sub-block;

In some embodiments, the transform unit 530 is specifically used to:

Based on the target transformation mode and the target transformation parameters, the contrast of the attribute information of the pixels in the current sub-image block is adjusted to obtain the sub-image block after the illumination transformation; wherein the contrast between the attribute information of the pixels in the sub-image block after the illumination transformation is smaller than the contrast between the attribute information of the pixels in the current sub-image block.

In some embodiments, the encoding unit 540 is specifically used to:

Splicing the sub-image block after the illumination transformation with the base view to obtain a spliced image;

The pixels in the spliced image and the pixels in the sub-image block after the illumination transformation are encoded to obtain the code stream.

It should be understood that the device embodiment and the method embodiment may correspond to each other, and similar descriptions may refer to the method embodiment. To avoid repetition, it will not be repeated here. Specifically, the decoder 400 shown in Figure 12 may correspond to the corresponding subject in the method 200 for executing the embodiment of the present application, and the aforementioned and other operations and/or functions of each unit in the decoder 400 are respectively for implementing the corresponding process in the method 210. The encoder 500 shown in Figure 13 may correspond to the corresponding subject in the methods 310 to 330 for executing the embodiment of the present application, that is, the aforementioned and other operations and/or functions of each unit in the encoder 500 are respectively for implementing the corresponding processes in the methods 310 to 330.

It should also be understood that the various units in the decoder 400 or encoder 500 involved in the embodiment of the present application can be respectively or all merged into one or several other units to constitute, or some (some) units therein can also be split into multiple smaller units in function to constitute, which can achieve the same operation without affecting the realization of the technical effect of the embodiment of the present application. The units involved in the above are divided based on logical functions. In practical applications, the function of a unit can also be realized by multiple units, or the function of multiple units is realized by one unit. In other embodiments of the present application, the decoder 400 or encoder 500 may also include other units. In practical applications, these functions can also be implemented by other units to assist in implementation, and can be implemented by multiple units in collaboration. According to another embodiment of the present application, the decoder 400 or encoder 500 involved in the embodiment of the present application can be constructed by running a computer program (including program code) capable of executing each step involved in the corresponding method on a general computing device including a general-purpose computer such as a central processing unit (CPU), a random access storage medium (RAM), a read-only storage medium (ROM) and a processing element and a storage element. , And to implement the encoding method or decoding method of the embodiment of the present application. The computer program can be recorded on, for example, a computer-readable storage medium, and loaded into an electronic device through the computer-readable storage medium and run therein to implement the corresponding method of the embodiment of the present application.

In other words, the units mentioned above can be implemented in hardware form, can be implemented in software form, or can be implemented in the form of a combination of software and hardware. Specifically, the steps of the method embodiments in the embodiments of the present application can be completed by the hardware integrated logic circuit and/or software form instructions in the processor, and the steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software combination in the decoding processor can be executed. Optionally, the software can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory, and completes the steps in the method embodiments mentioned above in combination with its hardware.

FIG. 14 is a schematic structural diagram of an electronic device 600 provided in an embodiment of the present application.

As shown in FIG. 14 , the electronic device 600 includes at least a processor 610 and a computer-readable storage medium 620. The processor 610 and the computer-readable storage medium 620 may be connected via a bus or other means. The computer-readable storage medium 620 is used to store a computer program 621, which includes computer instructions, and the processor 610 is used to execute the computer instructions stored in the computer-readable storage medium 620. The processor 610 is the computing core and control core of the electronic device 600, which is suitable for implementing one or more computer instructions, and is specifically suitable for loading and executing one or more computer instructions to implement the corresponding method flow or corresponding function.

Exemplarily, the processor 610 may also be referred to as a central processing unit (CPU). The processor 610 may include, but is not limited to, a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, discrete hardware components, and the like.

Exemplarily, the computer-readable storage medium 620 may be a high-speed RAM memory, or a non-volatile memory (Non-Volatile Memory), such as at least one disk memory; optionally, it may also be at least one computer-readable storage medium located away from the aforementioned processor 610. Specifically, the computer-readable storage medium 620 includes, but is not limited to: a volatile memory and/or a non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM) and direct RAM bus random access memory (Direct Rambus RAM, DR RAM).

Exemplarily, the electronic device 600 may be an encoder or encoding framework involved in an embodiment of the present application; a first computer instruction is stored in the computer-readable storage medium 620; the first computer instruction stored in the computer-readable storage medium 620 is loaded and executed by the processor 610 to implement the corresponding steps in the encoding method provided in an embodiment of the present application; in other words, the first computer instruction in the computer-readable storage medium 620 is loaded by the processor 610 and the corresponding steps are executed. To avoid repetition, it will not be repeated here.

Exemplarily, the electronic device 600 may be a decoder or decoding framework involved in an embodiment of the present application; a second computer instruction is stored in the computer-readable storage medium 620; the second computer instruction stored in the computer-readable storage medium 620 is loaded and executed by the processor 610 to implement the corresponding steps in the decoding method provided in an embodiment of the present application; in other words, the second computer instruction in the computer-readable storage medium 620 is loaded by the processor 610 and the corresponding steps are executed, which will not be repeated here to avoid repetition.

According to another aspect of the present application, the present application also provides a coding and decoding system, including the encoder and decoder mentioned above.

According to another aspect of the present application, the present application also provides a computer-readable storage medium (Memory), which is a memory device in the electronic device 600 for storing programs and data. For example, a computer-readable storage medium 620. It can be understood that the computer-readable storage medium 620 here can include both the built-in storage medium in the electronic device 600 and the extended storage medium supported by the electronic device 600. The computer-readable storage medium provides a storage space, which stores the operating system of the electronic device 600. In addition, one or more computer instructions suitable for being loaded and executed by the processor 610 are also stored in the storage space, and these computer instructions can be one or more computer programs 621 (including program codes).

According to another aspect of the present application, the present application further provides a computer program product or a computer program, which includes a computer instruction, and the computer instruction is stored in a computer-readable storage medium. For example, a computer program 621. At this time, the data processing device 600 can be a computer, and the processor 610 reads the computer instruction from the computer-readable storage medium 620, and the processor 610 executes the computer instruction, so that the computer executes the encoding method or decoding method provided in the various optional methods mentioned above.

In other words, when implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process of the embodiment of the present application is run in whole or in part or the function of the embodiment of the present application is implemented. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center.

Those of ordinary skill in the art will appreciate that the units and process steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

Finally, it should be noted that the above content is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

A decoding method, characterized by comprising:

Decode the code stream and determine the current sub-block in the current image;

Determining whether to perform illumination transformation on the current sub-block;

When it is determined to perform illumination transformation on the current sub-image block, illumination transformation is performed on the current sub-image block to obtain a sub-image block after illumination transformation.
The method according to claim 1, characterized in that the determining whether to perform a lighting transformation on the current sub-block comprises:

Decoding the bitstream to determine a sub-image block identifier of the current sub-image block; the sub-image block identifier of the current sub-image block indicates whether to perform illumination transformation on the current sub-image block;

Based on the sub-image block identifier of the current sub-image block, it is determined whether to perform illumination transformation on the current sub-image block.
The method according to claim 1 or 2, characterized in that the step of performing illumination transformation on the current sub-image block to obtain the sub-image block after illumination transformation comprises:

Determine a target transformation mode used by the current sub-block;

Determining target transformation parameters used by the current sub-block;

Based on the target transformation mode and the target transformation parameters, the current sub-image block is subjected to illumination transformation to obtain the sub-image block after illumination transformation.
The method according to claim 3 is characterized in that the target transformation mode is a default transformation mode.
The method according to claim 3 is characterized in that the target transformation parameters are transformation parameters used by default in the target transformation mode.
The method according to claim 3, characterized in that the step of performing illumination transformation on the current sub-image block based on the target transformation mode and the target transformation parameters and obtaining the sub-image block after illumination transformation comprises:

Based on the target transformation mode and the target transformation parameters, the contrast of the attribute information of the pixels in the current sub-image block is adjusted to obtain the sub-image block after the illumination transformation; wherein the contrast between the attribute information of the pixels in the sub-image block after the illumination transformation is greater than the contrast between the attribute information of the pixels in the current sub-image block.
The method according to any one of claims 1 to 6, characterized in that the method further comprises:

Based on the sub-image block after the illumination transformation, a decoded image of the current image is determined.
A coding method, characterized by comprising:

Determine a current sub-block in a current image;

Determining whether to perform illumination transformation on the current sub-block;

When it is determined to perform illumination transformation on the current sub-image block, perform illumination transformation on the current sub-image block to obtain a sub-image block after illumination transformation;

The sub-image block after the illumination transformation is encoded to obtain a bit stream.
The method according to claim 8, characterized in that encoding the sub-image block after the illumination transformation to obtain a bit stream comprises:

Acquire a sub-image block identifier of the current sub-image block; the sub-image block identifier of the current sub-image block indicates whether to perform illumination transformation on the current sub-image block;

The sub-image block identifier of the current sub-image block and the sub-image block after the illumination transformation are encoded to obtain the code stream.
The method according to claim 8, characterized in that the determining whether to perform a lighting transformation on the current sub-block comprises:

Get the base view;

Performing illumination detection on a pixel pair formed by a pixel in the base view and a pixel in the current sub-image block to determine whether the pixel pair is a pixel pair with inconsistent illumination;

Based on the number of the pixel pairs with inconsistent illumination, it is determined whether to perform illumination transformation on the current sub-block.
The method according to claim 10, characterized in that the step of performing illumination detection on a pixel pair formed by a pixel in the base view and a pixel in the current sub-block to determine whether the pixel pair is a pixel pair with inconsistent illumination comprises:

Determine a distribution difference value between color information of pixels in the base view and color information of pixels in the current sub-block;

Based on the distribution difference value, it is determined whether the pixel pair is a pixel pair with inconsistent illumination.
The method according to claim 11, characterized in that the determining of the distribution difference value between the color information of the pixel in the base view and the color information of the pixel in the current sub-block comprises:

The maximum difference value among (1-R 1 )/(1-R 2 ), (1-G 1 )/(1-G 2 ) and (1-B 1 )/(1-B 2 ) is determined as the distribution difference value; wherein R 1 , G 1 , B 1 respectively represent the values of the B component, G component, and B component of the pixel in the current sub-block; and R 2 , G 2 , B 2 respectively represent the values of the B component, G component, and B component of the pixel in the base view.
The method according to claim 11, characterized in that the step of determining whether the pixel pair is a pixel pair with inconsistent illumination based on the distribution difference value comprises:

If the distribution difference value is less than or equal to a first threshold, determining that the pixel pair is a pixel pair with inconsistent illumination;

If the distribution difference value is greater than the first threshold, it is determined that the pixel pair is not a pixel pair with inconsistent illumination.
The method according to claim 10, characterized in that the determining whether to perform illumination transformation on the current sub-block based on the number of pixel pairs with inconsistent illumination comprises:

Determining a proportion of the pixel pairs with inconsistent illumination based on the number of the pixel pairs with inconsistent illumination;

Based on the proportion of the pixel pairs with inconsistent illumination, it is determined whether to perform illumination transformation on the current sub-image block.
The method according to claim 14, characterized in that the determining whether to perform illumination transformation on the current sub-block based on the proportion of the pixel pairs with inconsistent illumination comprises:

If the proportion of the pixel pairs with inconsistent illumination is greater than or equal to a second threshold, determining that the current sub-image block is a sub-image block that needs illumination transformation;

If the proportion of the pixel pairs with inconsistent illumination is less than the second threshold, it is determined that the current sub-image block is a sub-image block that does not require illumination transformation.
The method according to claim 10, characterized in that the step of performing illumination detection on a pixel pair formed by a pixel in the base view and a pixel in the current sub-block to determine whether the pixel pair is a pixel pair with inconsistent illumination comprises:

Determining whether the pixel pair is a similar pixel pair;

When the pixel pair is not a similar pixel pair, light detection is performed on the pixel pair to determine whether the pixel pair is a pixel pair with inconsistent light.
The method according to claim 16, characterized in that the determining whether the pixel pair is a similar pixel pair comprises:

determining a depth difference value between the pixel pairs;

Determine a brightness difference value between a pixel in the base view and a pixel block where a pixel in the current sub-block is located;

When the depth difference value is less than a third threshold and the brightness difference value is less than a fourth threshold, determining that the pixel pair is a similar pixel pair;

When the depth difference value is greater than or equal to the third threshold or the brightness difference value is greater than or equal to the fourth threshold, it is determined that the pixel pair is not a similar pixel pair.
The method according to any one of claims 8 to 17, characterized in that performing illumination transformation on the current sub-image block to obtain the sub-image block after illumination transformation comprises:

Determine a target transformation mode used by the current sub-block;

Determining target transformation parameters used by the current sub-block;

Based on the target transformation mode and the target transformation parameters, the current sub-image block is subjected to illumination transformation to obtain the sub-image block after illumination transformation.
The method according to claim 18 is characterized in that the target transformation mode is a default transformation mode.
The method according to claim 18 is characterized in that the target transformation parameters are transformation parameters used by default in the target transformation method.
The method according to claim 18, characterized in that the step of performing illumination transformation on the current sub-image block based on the target transformation mode and the target transformation parameters and obtaining the sub-image block after illumination transformation comprises:

Based on the target transformation mode and the target transformation parameters, the contrast of the attribute information of the pixels in the current sub-image block is adjusted to obtain the sub-image block after the illumination transformation; wherein the contrast between the attribute information of the pixels in the sub-image block after the illumination transformation is smaller than the contrast between the attribute information of the pixels in the current sub-image block.
The method according to any one of claims 8 to 21, characterized in that encoding the sub-image block after the illumination transformation to obtain a bit stream comprises:

Splicing the sub-image block after the illumination transformation with the base view to obtain a spliced image;

The pixels in the spliced image and the pixels in the sub-image block after the illumination transformation are encoded to obtain the code stream.
A decoder, comprising:

A decoding unit, used for decoding the code stream and determining a current sub-block in a current image;

A determination unit, used to determine whether to perform illumination transformation on the current sub-block;

A transformation unit is used to perform a lighting transformation on the current sub-image block when it is determined that the current sub-image block is to be subjected to a lighting transformation, so as to obtain a sub-image block after the lighting transformation.
An encoder, characterized in that it comprises:

A first determining unit, configured to determine a current sub-block in a current image;

A second determining unit, used to determine whether to perform illumination transformation on the current sub-block;

A transformation unit, configured to perform a lighting transformation on the current sub-image block to obtain a sub-image block after the lighting transformation when it is determined to perform a lighting transformation on the current sub-image block;

The encoding unit is used to encode the sub-image block after the illumination transformation to obtain a code stream.
An electronic device, comprising:

a processor adapted to execute a computer program;

A computer-readable storage medium having a computer program stored therein, wherein when the computer program is executed by the processor, the method according to any one of claims 1 to 7 or the method according to any one of claims 8 to 21 is implemented.
A computer-readable storage medium, characterized in that it is used to store a computer program, wherein the computer program enables a computer to execute the method according to any one of claims 1 to 7 or the method according to any one of claims 8 to 21.
A computer program product, comprising a computer program/instruction, characterized in that when the computer program/instruction is executed by a processor, the method according to any one of claims 1 to 7 or the method according to any one of claims 8 to 21 is implemented.
A code stream, characterized in that the code stream is a code stream in the method according to any one of claims 1 to 7 or a code stream generated by the method according to any one of claims 8 to 21.