WO2021237569A1

WO2021237569A1 - Encoding method, decoding method, apparatus and system

Info

Publication number: WO2021237569A1
Application number: PCT/CN2020/092920
Authority: WO
Inventors: 刘细华
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2020-05-28
Filing date: 2020-05-28
Publication date: 2021-12-02
Also published as: CN113994708A

Abstract

An encoding method, decoding method, apparatus and system. The encoding method comprises: acquiring a target image to be encoded, converting an RGB channel image corresponding to the target image into a YUV image, and fusing an alpha channel image corresponding to the target image with the YUV image so as to generate a new YUV image; and encoding the new YUV image. By fusing an alpha channel image with a YUV image, only one file thus requires being encoded and obtained; in addition, in a transmission process, transmission bandwidth can be reduced, only one file needs to be decoded at a decoding end, and decoding end performance requirements are relatively low, and at the same time, image and video lag can be avoided.

Description

Coding method, decoding method, device and system

Technical field

This application relates to the field of coding technology, and specifically to an encoding method, decoding method, device and system.

Background technique

More and more smart terminals provide various video taking and video editing functions. For example, users can use their mobile phones to take pictures or videos, and then process the pictures or videos and share them with friends. In order to make pictures or videos more lively and interesting, users can superimpose some animation materials on the pictures or videos. In order to better integrate the animation materials with the shooting materials, the animation materials often use transparent or semi-transparent materials, which are superimposed with the shooting materials. Fusion later. Since pictures or videos containing transparent or semi-transparent materials include not only red (Red, R for short), Green (G), and Blue (B for short) three-channel images, they also include Alpha channel images, so When encoding, transmitting and decoding such pictures or videos, the Alpha channel image needs to be separately encoded into one file, so two files need to be transmitted and decoded. The overall file takes up a lot of memory and the transmission process takes up more bandwidth resources. , The performance requirements for terminal equipment are also high, and it is not suitable for some terminal equipment with relatively poor performance, and it is easy to cause a stuck phenomenon.

Summary of the invention

In view of this, the present application provides an encoding method, decoding method, device, and system.

According to the first aspect of the present application, an encoding method is provided, the method including:

Obtain the target image to be encoded;

Converting the RGB channel image corresponding to the target image into a YUV image;

Fuse the Alpha channel image corresponding to the target image with the YUV image to generate a new YUV image;

Encoding the new YUV image.

According to a second aspect of the present application, there is provided a decoding method, the method including:

Acquiring a YUV image to be decoded, where the YUV image is obtained by fusing the original YUV image obtained by conversion of the RGB channel image corresponding to the target image and the Alpha channel image corresponding to the target image;

Obtaining the original YUV image and the Alpha channel image based on the YUV image restoration;

The original YUV image is converted to obtain the RGB channel image, and the target image is obtained based on the RGB channel image and the Alpha channel image.

According to a third aspect of the present application, an encoding device is provided. The encoding device includes a processor, a memory, and a computer program stored in the storage, and the processor implements the following steps in the claims when the processor executes the computer program:

Obtain the target image to be encoded;

Encoding the new YUV image.

According to a fourth aspect of the present application, a decoding device is provided, the decoding device includes a processor, a memory, and a computer program stored in the storage, and when the processor executes the computer program, the following steps are implemented:

According to a fifth aspect of the present application, there is provided a video transmission system, the video transmission system including an encoding device and a decoding device;

The encoding device is used to obtain a target image to be encoded, convert the RGB channel image corresponding to the target image into a YUV image, and fuse the Alpha channel image corresponding to the target image with the YUV image to generate a new YUV image , Encoding the new YUV image to send the encoded image;

The decoding device is configured to, after receiving the encoded image, process the encoded image to obtain the new YUV image, and restore the YUV image and the Alpha channel based on the new YUV image Image, converting the YUV image into an RGB image, and obtaining the target image based on the RGB image and the Alpha channel image.

Using the solution provided by this application, when encoding a target image with a transparent effect, the RGB image corresponding to the target image can be converted to a YUV image, and then the Alpha channel image corresponding to the target image can be fused with the YUV image to obtain a new The YUV image, and the new YUV image is encoded. By fusing the Alpha channel image with the YUV image, only one file needs to be encoded, the transmission bandwidth can be saved during transmission, and the decoding end only needs to decode one file, which requires lower performance on the decoding end, while avoiding images and videos The stuttering phenomenon.

Description of the drawings

In order to explain the technical solutions in the embodiments of the present application more clearly, the following will briefly introduce the drawings needed in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor.

Fig. 1 is a schematic diagram of encoding an image with a transparent effect in the related art.

Fig. 2 is a flowchart of an encoding method according to an embodiment of the present application.

Fig. 3(a) is a schematic diagram of the fusion of the Y channel image and the Alpha channel image in the YUV image according to an embodiment of the present application.

Fig. 3(b) is a schematic diagram of the fusion of the Y channel image and the Alpha channel image in the YUV image according to an embodiment of the present application.

Fig. 4 is a schematic diagram of the fusion of Y channel images and Alpha channel images in a YUV image according to an embodiment of the present application.

Fig. 5 is a schematic diagram of a decoding method according to an embodiment of the present application.

Fig. 6(a) is a schematic diagram of an encoding process of an embodiment of the present application.

Figure 6(b) is a schematic diagram of the decoding process of an embodiment of the present application.

Fig. 7 is a schematic diagram of the logical structure of an encoding device according to an embodiment of the present application.

Fig. 8 is a schematic diagram of a logical structure of a decoding device according to an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by a person of ordinary skill in the art without creative work shall fall within the protection scope of this application.

The shooting and editing of pictures and videos is becoming more and more common. After users take pictures or videos, they can superimpose some transparent or semi-transparent materials in the pictures or videos, and then merge them with the shooting materials, for example, in the image Add some translucent watermarks, logos, etc. Since pictures or videos containing transparent or semi-transparent materials include not only red, green, and blue (referred to as R, G, B) three-channel images, they also include Alpha channel images. Therefore, the encoding of such pictures or videos , During transmission and decoding, in order to comply with the relevant coding and decoding standards, it needs to be encoded into two files, as shown in Figure 1. At the encoding end, the image A to be transmitted needs to be encoded into two files, that is, the RGB channel image is encoded into One file (file 1), the Alpha channel image needs to be separately encoded into a file (file 2), and then the two files are transmitted to the decoding end. Similarly, on the decoding end, two files need to be decoded. File 2 is decoded to obtain an RGB channel image and an Alpha channel image, thereby obtaining image A. Due to the large size of the overall file, the transmission process requires more bandwidth resources. At the same time, for the video or picture decoding end, since two files need to be decoded at the same time, the performance requirements of the terminal device are relatively high. Therefore, for some performance comparisons Poor terminal equipment is not suitable, and it is easy to cause jams. However, the current shooting, editing, coding and decoding of pictures and videos are mostly performed on the mobile phone. Compared with the previous computer-side processing methods, the performance of the mobile phone is often inferior. Therefore, it is necessary to provide a way to contain transparent images. The encoding and decoding scheme of the picture or video in order to be equally applicable to terminal equipment with poor performance.

Based on this, this application provides an encoding method that can merge the YUV image obtained by converting the Alpha channel image corresponding to the image to be encoded and the RGB channel image corresponding to the image to be encoded into a new YUV image, and then encoding The processing not only conforms to the general coding and decoding standards, but also only needs to code and decode one file, which can save transmission resources and has lower requirements on terminal performance. Specifically, as shown in Figure 2, the method includes the following steps:

S202: Obtain a target image to be encoded;

S204: Convert the RGB channel image corresponding to the target image into a YUV image;

S206. Fusion the Alpha channel image corresponding to the target image with the YUV image to generate a new YUV image;

S208: Encode the new YUV image.

The encoding method of the present application can be used for an encoding device that performs encoding processing on images or videos. The encoding device can be integrated on terminal devices such as mobile phones, tablets, and computers, or it can be an independent device.

The target image to be encoded can be an image with a transparency effect, that is, an image containing four channels of R, G, B, and Alpha. The target image can be a single image, or it can be a frame or multiple frames of images contained in the video to be encoded. Generally speaking, when encoding an image using coding standards such as H264/H265, it is necessary to convert the image into a YUV format image, and then perform the encoding process. Therefore, after acquiring the target image, the RGB channel image corresponding to the target image can be converted into a YUV image, where the YUV image can be any of the formats of YUV444, YUV422 or YUV420, which can be specifically set according to the actual codec standard. Then the Alpha channel image corresponding to the target image can be fused with the converted YUV image to generate a new YUV image. Among them, when the YUV image and the Alpha channel image are fused to generate a new YUV image, the two images can be fused using the preset fusion rules. The fusion rules can be set by themselves, as long as the decoding can be restored according to the fusion rules. The original YUV image and Alpha channel image can be used.

After the new YUV image is obtained by fusion, general coding technology can be used to encode the new YUV image, for example, the H264/H265 standard is used to encode the generated new YUV image.

Through this encoding method, the image can not only meet the general encoding and decoding standards, and for one frame of target image, only one file needs to be encoded, which can reduce the data volume of the file and save the bandwidth resources of the transmission process. It is necessary to decode a file, which requires relatively low performance on the decoding side, and it is also applicable to the decoding side with poor performance.

Since the image in the YUV format includes a Y channel image, a U channel image, and a V channel image, in some embodiments, when the Alpha channel image corresponding to the target image is fused with the converted YUV image, the target image can be combined The corresponding Alpha channel image is fused with the image of the designated channel corresponding to the YUV image, where the image of the designated channel may be at least one of a Y channel image, a U channel image, or a V channel image. For example, the Alpha channel image and the Y channel image can be fused to obtain the Y channel image of a new YUV image, or the Alpha channel image can be fused with the U channel image or V channel image to obtain the U channel image or V channel image of the new YUV image. , You can also merge the Alpha channel image with the U channel image and the V channel image to obtain the U channel image and V channel image of the new YUV image.

In some embodiments, when the Alpha channel image corresponding to the target image is fused with the image of the specified channel corresponding to the YUV image, the Alpha channel image and the image of the specified channel corresponding to the YUV image may be compressed first, and then The compressed Alpha channel image and the compressed image of the specified channel are stitched together to obtain the image of the specified channel corresponding to the new YUV image.

Among them, when the Alpha channel image and the image of the specified channel are compressed, the width of the image can be compressed, and the height of the image can also be compressed. Therefore, in some embodiments, the width of the Alpha channel image and the width of the image of the specified channel corresponding to the YUV image can be respectively compressed, and then the compressed images are stitched horizontally to obtain the image of the specified channel of the new YUV image. ; Of course, it is also possible to separately compress the height of the Alpha channel image and the height of the image of the specified channel corresponding to the YUV image, and then stitch the compressed images vertically to obtain the image of the specified channel of the new YUV image. Taking the image of the designated channel as the Y channel image as an example, as shown in Figure 3(a), it is a schematic diagram of horizontally compressing the Y channel image and the Alpha channel image, and then stitching the Y channel image of a new YUV image. As shown in Figure 3(b), it is a schematic diagram of longitudinally compressing the Y channel image and the Alpha channel image, and then stitching to obtain the Y channel image of the new YUV image.

Among them, when the image of the specified channel and the image of the Alpha channel are compressed, the compression ratio may be the same or different. In some embodiments, the Alpha channel image can be compressed according to the first ratio, and the image of the specified channel can be compressed according to the second ratio. In order that the size of the finally stitched image is consistent with the size of the original image of the specified channel, the first ratio The sum to the second ratio is 1. Among them, the first ratio and the second ratio can be set according to actual needs. Generally, the larger the compression ratio, the more obvious the distortion of the image. Therefore, in the actual processing process, you can according to the requirements for the effect of the specified channel image or the alpha channel image Set the first ratio and the second ratio flexibly. For example, for some scenes, where the effect or definition of the Alpha channel image is higher, the compression ratio of the Alpha channel image can be appropriately smaller, and the compression of the image of the specified channel The ratio can be larger. For some scenes, where the effect or definition of the image of the specified channel is higher, the compression ratio of the image of the specified channel can be appropriately smaller.

In an embodiment, in order not to affect the compression effect of the Alpha channel image and the specified channel image as much as possible, the two images can be compressed by half. Therefore, the first ratio and the second ratio can both be set to 1/2.

In some embodiments, when the Alpha channel image corresponding to the target image is fused with the image of the specified channel corresponding to the converted YUV image, the pixel value of the specified pixel in the Alpha channel image can be used to replace the image of the specified channel. The pixel values of the pixels at the corresponding positions of these designated pixels are obtained to obtain the image of the designated channel corresponding to the new YUV image. Among them, the designated pixels may be some predefined pixels, for example, some rows of pixels, some columns of pixels, some designated number of pixels, and so on. In some embodiments, the designated pixel may be a pixel of a designated row in the Alpha channel image or a pixel of a designated column in the Alpha channel image. For example, you can replace the pixel values of the odd-numbered rows of pixels in the image of the specified channel with the pixel values of the odd-numbered rows of the Alpha channel image to obtain the image of the specified channel of the new YUV image. Of course, you can also use The pixel value of the odd-numbered column of the pixel in the alpha channel image replaces the pixel value of the odd-numbered column of the image of the specified channel to obtain the image of the specified channel of the new YUV image. Of course, it is also possible to replace the pixel value of the even-numbered pixel in the odd-numbered row of the image of the specified channel with the pixel value of the even-numbered pixel in the odd-numbered row of the Alpha channel image to obtain the image of the specified channel of the new YUV image , Or other alternative methods can also be used, and this application is not limited. Taking the specified channel image as the Y channel image as an example, as shown in Figure 4, the pixel values of the odd rows of the Y channel image are replaced with the pixel values of the odd rows of the Alpha channel image to obtain the Y of the new YUV image. Schematic diagram of channel image.

When the Alpha channel image and the designated channel image are fused, the size of the fused image needs to be consistent with the size of the designated channel image, and the fusion process will inevitably sacrifice the sharpness of the image. In some embodiments, for convenience The Alpha channel image and the specified channel image are fused to minimize the sharpness of the two images during the fusion process. The specified channel can select the channel with the same number of pixels as the Alpha channel, so that the image of the specified channel can be compared with the Alpha channel. The number of pixels in the channel image is the same, that is, the image size is the same. For example, for YUV images, there are multiple formats, such as YUV444, YUV422, and YUV420. For YUV444, a Y component corresponds to a U component and a V component, that is, a Y channel image, a U channel image, and a V channel. The number of pixels in the image is the same, which is the same as the number of pixels in the Alpha channel image. Therefore, when the image of the specified channel is selected for fusion with the Alpha channel image, either the Y channel image, the U channel or the V channel can be selected. Image. For the YUV420 image, every four Y components share a pair of U and V components. The number of pixels in the U and V channel images is much less than the number of pixels in the Y channel image. The number of pixels in the Y channel image is the same as Alpha. The number of pixels in the channel image is the same. If the Alpha channel image is fused with the U channel or V channel image, the size of the fused image and the U channel or V channel image must be the same, or the Alpha channel image and the U channel and V channel image should be fused , The fused image and the image size jointly formed by the U channel and the V channel image must be the same, and the size of the Alpha channel image (or the number of pixels) and the size of the Y channel image (or the number of pixels) are the same, that is The size of the Alpha channel image (or the number of pixels) is larger than any one or a combination of the U channel image and the V channel image. At this time, the resolution of the Alpha channel image will be greatly reduced, which will affect the display effect of the transparent image after encoding. Yes, because the color of the U channel and/or V channel image is relatively large compressed, when it is merged with the Alpha channel image, the resolution of the U channel and/or V channel image will be further reduced, and it will also affect the encoding U channel and/or V channel image display effect.

Therefore, in some embodiments, if the YUV image is an image in the YUV420 format, the image of the specified channel can be a Y-channel image. The Y-channel image of the new YUV image is obtained by fusing the Y-channel image with the Alpha channel image. The above examples are only exemplary, and the purpose is to illustrate the solution of the application, and does not limit the application.

After the new YUV image is obtained by fusion, a general coding standard can be used to encode the new YUV image to obtain an encoded image.

In the embodiment of the application, when encoding a target image with a transparent effect, the RGB image corresponding to the target image can be converted into a YUV image, and then the Alpha channel image corresponding to the target image is fused with the YUV image to obtain a new YUV image , And the new YUV image is encoded. By fusing the Alpha channel image with the YUV image, only one file needs to be encoded, the transmission bandwidth can be saved during transmission, and the decoding end only needs to decode one file, which requires lower performance on the decoding end, while avoiding images and videos The stuttering phenomenon.

Corresponding to the foregoing encoding method, this application also provides a decoding method. As shown in FIG. 5, the decoding method includes the following steps:

S502: Obtain a YUV image to be decoded, where the YUV image is obtained by fusing the original YUV image obtained by the conversion of the RGB channel image corresponding to the target image and the Alpha channel image corresponding to the target image;

S504: Obtain the original YUV image and the Alpha channel image based on the YUV image restoration;

S506: Convert the original YUV image to obtain the RGB channel image, and obtain the target image based on the RGB channel image and the Alpha channel image.

Similarly, the decoding method of the present application can be applied to a decoding device that decodes images or videos. The decoding device can be integrated on terminal devices such as mobile phones, tablets, and computers, or can be an independent device.

In the decoding process, the image to be decoded can be obtained first. The image to be decoded is a YUV image. The YUV image is obtained by fusing the original YUV image converted from the RGB channel image corresponding to the target image and the Alpha channel image corresponding to the target image , Where the original YUV image and the Alpha channel image are fused to obtain the YUV image, which can be fused by using predefined fusion methods and rules. For the specific fusion process, please refer to the implementations in the above coding methods. The description of the example will not be repeated here.

After obtaining the YUV image to be decoded, the original YUV image and Alpha channel image corresponding to the target image can be restored according to the YUV image, and then the original YUV image is converted to obtain the corresponding RGB channel image of the target image. The channel image and the Alpha channel image, that is, the decoded target image can be obtained.

Since when encoding the target image, the original YUV image obtained by converting the RGB channel image corresponding to the target image and the Alpha channel image corresponding to the target image are fused into a new YUV image, and then the encoding process is performed. Therefore, when decoding, it is also Only one file needs to be decoded, and during the decoding process, the fused YUV image can be restored to obtain the original YUV image Alpha channel image of the target image, and the original YUV image is converted into RGB channel image, according to the Alpha channel image and RGB channel image The channel image can decode the target image. For images or videos with transparent effects, since only one file needs to be decoded, the performance requirements of the decoding end device are relatively low.

In some embodiments, after receiving the image encoded by the encoding device, the image may be first subjected to a conventional decoding operation, for example, the image is decoded according to a general encoding and decoding standard. Therefore, the YUV image to be decoded in this application may be an image obtained after a specified decoding operation is processed. The specified decoding operation may be a decoding operation based on the H264/H265 standard, of course, it may also be based on other similar decoding standards. Operation.

In some embodiments, because during the encoding process, when the Alpha channel image corresponding to the target image is fused with the original YUV image, the Alpha channel image corresponding to the target image can be fused with the specified channel image corresponding to the YUV image. The image of the designated channel may be at least one of a Y channel image, a U channel image, or a V channel image. Therefore, when the original YUV image and the Alpha channel image are restored from the fused YUV image, the image of the specified channel corresponding to the original YUV image and the image of the specified channel can be restored according to the image of the specified channel corresponding to the fused YUV image. Alpha channel image, where the image of the designated channel may also be at least one of Y channel image, U channel image, or V channel image, which can be specifically determined according to the encoding mode. For example, if the specified channel image is a Y channel image during encoding, the Y channel image of the merged YUV is restored to obtain the Y channel image and the Alpha channel image of the original YUV image.

When the Alpha channel image and the designated channel image are fused, the size of the fused image needs to be consistent with the size of the designated channel image, and the fusion process will inevitably sacrifice the sharpness of the image. In some embodiments, for convenience The Alpha channel image and the designated channel image are fused to minimize the decrease in the sharpness of the two images during the fusion process. The designated channel image can be selected to have the same number of pixels as the Alpha channel image.

In some embodiments, if the YUV image is an image in the YUV420 format, the image of the designated channel may be a Y channel image, and the Y channel image of the new YUV image is obtained by fusing the Y channel image and the Alpha channel image.

In some embodiments, when the image of the specified channel corresponding to the original YUV image and the image of the Alpha channel are obtained by restoring the image of the specified channel corresponding to the fused YUV image, the YUV image may be corresponding to the original YUV image according to the preset split rule The pixels of the image of the specified channel are split to obtain some pixels of the image of the specified channel corresponding to the original YUV image and some pixels of the image of the Alpha channel, and then according to the resolution of the YUV image and the image of the specified channel Some pixels restore the image of the specified channel corresponding to the original YUV image, and restore the Alpha channel image according to the resolution of the YUV image and some pixels of the Alpha channel image. Among them, the split rule can be determined according to the fusion method of the original YUV image and the Alpha channel image. For example, when the original YUV image and the Alpha channel image are fused into a YUV image, the odd rows of pixels of the Alpha channel image are used to replace the odd rows of pixels of the Y channel image of the original YUV image. When splitting, you can Split the odd and even rows of pixels of the fused YUV image. The odd rows of pixels are part of the Alpha channel image, and the even rows of pixels are part of the Y channel image of the original YUV image. point. Of course, if the original YUV image and the Alpha channel image are fused into a YUV image, the Y channel image and the Alpha channel image of the original YUV image are compressed by half in height respectively, and then stitched vertically to obtain the YUV image, then when decoding , The pixels of the YUV image can be divided into 2 parts in the longitudinal direction, one part is part of the pixels of the Y channel image of the original YUV image, and the other part is part of the pixels of the Alpha channel image. Of course, if other fusion methods are used in the encoding process, the splitting rules can be adjusted according to the fusion method, and this application does not limit it.

In some embodiments, when restoring the image of the specified channel of the original YUV image and the Alpha channel image based on the partial pixels obtained by the YUV splitting, interpolation algorithm can be used to restore, or the pixels of adjacent pixels can be directly copied. The value is used as the pixel of the new pixel to restore the image. For example, the size of the specified channel image and the alpha channel image of the original YUV image can be determined according to the resolution of the fused YUV image, that is, the number of pixels, and then some pixels of the image of the specified channel of the original YUV image can be interpolated Processing, restore the image of the designated channel of the original YUV image, and perform interpolation processing on some pixels of the Alpha channel image to restore the Alpha channel image. Of course, you can also copy the pixel values of some pixels of the image of the specified channel as the pixel values of adjacent pixels to restore the image of the specified channel of the original YUV image, and copy the pixel values of some pixels of the Alpha channel image as the adjacent pixels. The pixel value of the pixel is restored to obtain the Alpha channel image. Of course, this application is not limited to the above-listed methods, and the methods of restoring an image according to a part of the pixels of the image in the prior art are all applicable to this application.

After restoring the specified channel image and Alpha channel image of the original YUV image, the original YUV image can be obtained, and then the original YUV image is converted into an RGB channel image. According to the RGB channel image and the Alpha channel image, the target image can be decoded.

In the embodiment of the present application, when decoding a target image with a transparent effect, the decoding end only needs to decode one file, which has lower performance requirements on the decoding end, and can avoid image and video jams.

In order to further explain the encoding and decoding method of the present application, the following will explain it in conjunction with a specific embodiment.

Generally, users can edit the video after shooting some small videos with mobile phones, tablets and other devices, such as adding some transparent animation materials to increase the interest of the video, and then send the processed video to their friends. For images and videos with transparent effects, when encoding, transmitting, and decoding, the Alpha channel image needs to be separately encoded into one file. Therefore, when transmitting and decoding, two files need to be transmitted and decoded. The overall file size is large and the transmission process It occupies a lot of bandwidth resources and has high requirements for terminal equipment performance. It is not suitable for some terminal equipment with relatively poor performance, and it is easy to cause a stuck phenomenon.

In order to avoid the above-mentioned problems, this embodiment provides a method for encoding images and videos with transparency effects. Since most hardware platforms support decoding of images or video files in H264YUV420 format, the following uses this format as an example to illustrate , The specific encoding and decoding process is as follows:

1. The coding process

For the specific encoding process, refer to Figure 6(a). Each frame image in the video is an RGBA image. The RGBA image contains both RGB channel images and Alpha channel images. Since H264 encoding needs to convert the image to the YUV format image, you can first convert the RGB channel image to the YUV420 format image. Hereinafter, it is called the original YUV420 image. The YUV420 format image means that every 4 Y components share a pair of U components. And V component, YUV420 image contains Y channel image, U channel image and V channel image, the size of Y channel image is the same as the original image size. The height or width of the Y channel image can be compressed at a ratio of 2:1 (in the figure, height compression is taken as an example). For example, an image with a width and height of 400x400 can be scaled 2:1 in height to become a 400x200 image. At the same time, the height or width of the Alpha channel image is compressed at a ratio of 2:1 (in the figure, the height compression is taken as an example). Of course, it can also be compressed at other ratios. For example, if the brightness of the encoded image is higher, then the Y channel The compression ratio of the image can be smaller, and the transparency of the image is required to be higher, then the compression ratio of the Alpha channel image can be set smaller. Then the compressed Y channel image and Alpha channel image are spliced together to form a spliced Y channel image. The spliced Y channel image and the original UV channel image form a new YUV420 image, which is called fusion hereinafter YUV420 image, and then the fused YUV420 image can be sent to the encoder for H264 encoding processing.

2. Decoding process

For the specific decoding process, please refer to Figure 6(b). After the decoder receives the image to be decoded encoded according to the above encoding process, it can first perform conventional decoding processing on the image to be decoded according to the H264 decoding standard to obtain the fused YUV420 image. . Then, the pixels of the Y-channel image of the fused YUV420 image are split according to the preset splitting rules, where the specific splitting rules can be determined according to the fusion method of the Y-channel image and the Alpha channel image in the encoding process. In the encoding process, the Y channel image and the Alpha channel image are highly compressed and vertically stitched. Therefore, the pixels of the Y channel image of the fused YUV420 image can be split into two parts, one part is the pixel of the Y channel image of the original YUV image Point, part of which is the pixel point of the Alpha channel image. Then the size of the original YUV image and the Alpha channel image can be determined according to the resolution of the fused YUV420 image, and the original YUV image and the Alpha channel image can be restored according to the split pixels, for example, it can be enlarged by a ratio of 1:2 In the specific restoration process, interpolation algorithm can be used to fill in the pixels of the image, or the neighboring pixels can be copied directly as new pixels to fill in the pixels of the image, and then the restored original YUV420 Y channel image and Alpha can be obtained. Channel image. After the original YUV420 image is obtained, the original YUV420 image can be converted into an RGB image, and then the RGB image and the restored Alpha channel image can be combined to obtain an RGBA image, thereby completing the image decoding process.

In this way, only one file needs to be transmitted during the transmission process, which can save bandwidth, and the decoder only needs to decode one file, which is also suitable for terminal devices with poor performance, which can reduce video jams.

Corresponding to the encoding method, the present application also provides an encoding device. As shown in FIG. 7, the encoding device includes a processor 71, a memory 72, a computer program stored in the memory 72, and the processor 71 The following steps of the claims are realized when the computer program is executed:

Obtain the target image to be encoded;

Encoding the new YUV image.

In some embodiments, when the processor is configured to fuse the Alpha channel image corresponding to the target image with the YUV image, it is specifically configured to:

The Alpha channel image corresponding to the target image is fused with the image of the designated channel corresponding to the YUV image, and the image of the designated channel includes at least one of a Y channel image, a U channel image, or a V channel image.

In some embodiments, when the processor is configured to fuse the Alpha channel image corresponding to the target image with the specified channel image corresponding to the YUV image, it is specifically configured to:

Respectively compressing the Alpha channel image corresponding to the target image and the designated channel image corresponding to the YUV image;

The compressed Alpha channel image is spliced with the compressed designated channel image to obtain the designated channel image corresponding to the new YUV image.

In some embodiments, when the processor is configured to respectively compress the Alpha channel image corresponding to the target image and the specified channel image corresponding to the YUV image, it is specifically configured to:

Respectively compress the width of the Alpha channel image corresponding to the target image and the specified channel image corresponding to the YUV image; or

The heights of the Alpha channel image corresponding to the target image and the specified channel image corresponding to the YUV image are compressed respectively.

The Alpha channel image is compressed at a first ratio, and the designated channel image is compressed at a second ratio, and the sum of the first ratio and the second ratio is 1.

In some embodiments, the first ratio and the second ratio are both 1/2.

Replace the pixel value of the pixel at the corresponding position of the specified pixel in the image of the specified channel with the pixel value of the specified pixel in the Alpha channel image to obtain the image of the specified channel corresponding to the new YUV image .

In some embodiments, the designated pixels include:

The pixels of the specified row in the Alpha channel image; or

The pixel points of the designated column in the Alpha channel image.

In some embodiments, the designated channel is at least one channel, and the images of all the designated channels have the same number of pixels as the Alpha channel image.

In some embodiments, the YUV image is an image in YUV420 format, and the image of the designated channel is a Y-channel image.

For the specific encoding details of the image and video by the encoding device, reference may be made to the description of each embodiment in the foregoing encoding method, which is not limited in this application.

The encoding device provided by the embodiments of the present application can convert the RGB image corresponding to the target image into a YUV image when encoding a target image with a transparent effect, and then merge the Alpha channel image corresponding to the target image with the YUV image, A new YUV image is obtained, and the new YUV image is encoded. By fusing the Alpha channel image with the YUV image, only one file needs to be encoded and the transmission bandwidth can be saved during the transmission process.

Corresponding to the decoding method, the present application also provides a decoding device. As shown in FIG. 8, the decoding device includes a processor 81, a memory 82, a computer program stored in the memory 82, and the processor 81 The following steps are implemented when the computer program is executed:

In some embodiments, when the processor is configured to obtain the original YUV image and the Alpha channel image based on the YUV image restoration, it is specifically configured to:

Based on the image restoration of the specified channel corresponding to the YUV image, the image of the specified channel corresponding to the original YUV image and the Alpha channel image are obtained, and the image of the specified channel includes the Y channel image, the U channel image, or the V channel image. At least one of.

In some embodiments, when the processor is configured to obtain the image of the specified channel corresponding to the original YUV image and the image of the Alpha channel based on the image restoration of the specified channel corresponding to the YUV image, it is specifically used for:

Split the pixels of the designated channel image corresponding to the YUV image based on a preset splitting rule to obtain some pixels of the designated channel image corresponding to the original YUV image and some pixels of the Alpha channel image , The split rule is determined based on the fusion mode of the original YUV image and the Alpha channel image;

Restore the image of the specified channel based on the resolution of the YUV image and some pixels of the image of the specified channel, and restore the Alpha based on the resolution of the YUV image and some pixels of the image of the Alpha channel Channel image.

In some embodiments, the processor is configured to restore the image of the specified channel based on the resolution of the YUV image and some pixels of the image of the specified channel, and based on the resolution of the YUV image and all pixels. When some pixels of the Alpha channel image are restored to the Alpha channel image, they are specifically used for:

Based on the resolution of the YUV image, perform interpolation processing on some pixels of the image of the specified channel to restore the image of the specified channel, perform interpolation processing on some of the pixels of the Alpha channel image, and restore the obtained The Alpha channel image; or

Based on the resolution of the YUV image, copy the pixel values of some pixels of the image of the specified channel as the pixel values of adjacent pixels, restore the image of the specified channel, and copy part of the image of the Alpha channel The pixel value of the pixel is used as the pixel value of the neighboring pixel, and the Alpha channel image is restored.

In some embodiments, the YUV image to be decoded is an image obtained after processing by a designated decoding operation, and the designated decoding operation includes a decoding operation based on the H264/H265 standard.

For the specific decoding details of the image and video by the decoding device, reference may be made to the description of each embodiment in the foregoing decoding method, which is not limited in this application.

The decoding device provided by the embodiment of the present application only needs to decode one file, which has low requirements on the performance of the decoding end, and can avoid the freezing phenomenon of images and videos at the same time.

Furthermore, this application also provides a video transmission system, characterized in that the video transmission system includes an encoding device and a decoding device;

The video transmission system provided by the embodiments of this application can convert the RGB image corresponding to the target image into a YUV image when encoding a target image with a transparent effect, and then fuse the Alpha channel image corresponding to the target image with the YUV image , A new YUV image is obtained, and the new YUV image is encoded. By fusing the Alpha channel image with the YUV image, only one file needs to be encoded, the transmission bandwidth can be saved during transmission, and the decoding end only needs to decode one file, which requires lower performance on the decoding end, while avoiding images and videos The stuttering phenomenon. Correspondingly, an embodiment of this specification also provides a computer storage medium in which a program is stored, and when the program is executed by a processor, the encoding method or the decoding method in any of the foregoing embodiments is implemented.

The embodiments of this specification may adopt the form of a computer program product implemented on one or more storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing program codes. Computer usable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disc (DVD) or other optical storage, Magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media can be used to store information that can be accessed by computing devices.

As for the device embodiment, since it basically corresponds to the method embodiment, the relevant part can refer to the part of the description of the method embodiment. The device embodiments described above are merely illustrative, where the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in One place, or it can be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the objectives of the solutions of the embodiments. Those of ordinary skill in the art can understand and implement without creative work.

It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply one of these entities or operations. There is any such actual relationship or order between. The terms "include", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements that are not explicitly listed. Elements, or also include elements inherent to such processes, methods, articles, or equipment. If there are no more restrictions, the element defined by the sentence "including a..." does not exclude the existence of other identical elements in the process, method, article, or equipment that includes the element.

The methods and devices provided by the embodiments of the present invention are described in detail above. Specific examples are used in this article to illustrate the principles and implementations of the present invention. The descriptions of the above embodiments are only used to help understand the methods and methods of the present invention. Core idea; At the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation and the scope of application. In summary, the content of this specification should not be construed as a limitation of the present invention .

Claims

An encoding method, characterized in that the method includes:

Obtain the target image to be encoded;

Converting the RGB channel image corresponding to the target image into a YUV image;

Fuse the Alpha channel image corresponding to the target image with the YUV image to generate a new YUV image;

Encoding the new YUV image.
The encoding method according to claim 1, wherein fusing the Alpha channel image corresponding to the target image with the YUV image comprises:

The Alpha channel image corresponding to the target image is fused with the image of the designated channel corresponding to the YUV image, and the image of the designated channel includes at least one of a Y channel image, a U channel image, or a V channel image.
The encoding method according to claim 2, wherein the fusion of the Alpha channel image corresponding to the target image with the image of the designated channel corresponding to the YUV image comprises:

Respectively compressing the Alpha channel image corresponding to the target image and the designated channel image corresponding to the YUV image;

The compressed Alpha channel image is spliced with the compressed designated channel image to obtain the designated channel image corresponding to the new YUV image.
The encoding method according to claim 3, wherein compressing the Alpha channel image corresponding to the target image and the image of the designated channel corresponding to the YUV image respectively comprises:

Respectively compress the width of the Alpha channel image corresponding to the target image and the specified channel image corresponding to the YUV image; or

The heights of the Alpha channel image corresponding to the target image and the specified channel image corresponding to the YUV image are compressed respectively.
The encoding method according to any one of claims 3 or 4, characterized in that, respectively compressing the Alpha channel image corresponding to the target image and the designated channel image corresponding to the YUV image, comprising:

The Alpha channel image is compressed at a first ratio, and the designated channel image is compressed at a second ratio, and the sum of the first ratio and the second ratio is 1.
The encoding method according to claim 5, wherein the first ratio and the second ratio are both 1/2.
The encoding method according to claim 3, wherein the fusion of the Alpha channel image corresponding to the target image with the image of the designated channel corresponding to the YUV image comprises:

Replace the pixel value of the pixel at the corresponding position of the specified pixel in the image of the specified channel with the pixel value of the specified pixel in the Alpha channel image to obtain the image of the specified channel corresponding to the new YUV image .
The encoding method according to claim 7, wherein the designated pixel point comprises:

The pixels of the specified row in the Alpha channel image; or

The pixel points of the designated column in the Alpha channel image.
The encoding method according to any one of claims 2-8, wherein the designated channel is at least one channel, and all the images of the designated channel have the same number of pixels as the Alpha channel image.
The encoding method according to any one of claims 2-9, wherein the YUV image is a YUV420 image, and the image of the designated channel is a Y channel image.
An encoding device, characterized in that the encoding device includes a processor, a memory, and a computer program stored on the memory, and the processor implements the following steps in the claims when the processor executes the computer program:

Obtain the target image to be encoded;

Converting the RGB channel image corresponding to the target image into a YUV image;

Fuse the Alpha channel image corresponding to the target image with the YUV image to generate a new YUV image;

Encoding the new YUV image.
11. The encoding device according to claim 11, wherein the processor is used for fusing an Alpha channel image corresponding to the target image with the YUV image, and is specifically used for:

The Alpha channel image corresponding to the target image is fused with the image of the designated channel corresponding to the YUV image, and the image of the designated channel includes at least one of a Y channel image, a U channel image, or a V channel image.
The encoding device according to claim 12, wherein the processor is configured to merge an Alpha channel image corresponding to the target image with an image of a designated channel corresponding to the YUV image, and is specifically configured to:

Respectively compressing the Alpha channel image corresponding to the target image and the designated channel image corresponding to the YUV image;

The compressed Alpha channel image is spliced with the compressed designated channel image to obtain the designated channel image corresponding to the new YUV image.
The encoding device according to claim 13, wherein the processor is configured to respectively compress the Alpha channel image corresponding to the target image and the designated channel image corresponding to the YUV image, and is specifically configured to:

Respectively compress the width of the Alpha channel image corresponding to the target image and the specified channel image corresponding to the YUV image; or

The heights of the Alpha channel image corresponding to the target image and the specified channel image corresponding to the YUV image are compressed respectively.
The encoding device according to any one of claims 13 or 14, wherein the processor is configured to respectively compress the Alpha channel image corresponding to the target image and the designated channel image corresponding to the YUV image, Specifically used for:

The Alpha channel image is compressed at a first ratio, and the designated channel image is compressed at a second ratio, and the sum of the first ratio and the second ratio is 1.
The encoding device according to claim 15, wherein the first ratio and the second ratio are both 1/2.
The encoding device according to claim 13, wherein the processor is used for fusing the Alpha channel image corresponding to the target image with the image of the designated channel corresponding to the YUV image, specifically for:

Replace the pixel value of the pixel at the corresponding position of the specified pixel in the image of the specified channel with the pixel value of the specified pixel in the Alpha channel image to obtain the image of the specified channel corresponding to the new YUV image .
The encoding device according to claim 17, wherein the designated pixel point comprises:

The pixels of the specified row in the Alpha channel image; or

The pixel points of the designated column in the Alpha channel image.
The encoding device according to any one of claims 12-18, wherein the designated channel is at least one channel, and all the images of the designated channel have the same number of pixels as the image of the Alpha channel.
The encoding device according to any one of claims 12-19, wherein the YUV image is an image in YUV420 format, and the image of the designated channel is a Y-channel image.
A decoding method, characterized in that the method includes:

Acquiring a YUV image to be decoded, where the YUV image is obtained by fusing the original YUV image obtained by conversion of the RGB channel image corresponding to the target image and the Alpha channel image corresponding to the target image;

Obtaining the original YUV image and the Alpha channel image based on the YUV image restoration;

The original YUV image is converted to obtain the RGB channel image, and the target image is obtained based on the RGB channel image and the Alpha channel image.
The decoding method according to claim 21, wherein obtaining the original YUV image and the Alpha channel image based on the YUV image restoration comprises:

Based on the image restoration of the specified channel corresponding to the YUV image, the image of the specified channel corresponding to the original YUV image and the Alpha channel image are obtained, and the image of the specified channel includes the Y channel image, the U channel image, or the V channel image. At least one of.
The decoding method according to claim 22, wherein the restoration of the image of the specified channel corresponding to the YUV image to obtain the image of the specified channel corresponding to the original YUV image and the image of the Alpha channel comprises:

Split the pixels of the designated channel image corresponding to the YUV image based on a preset splitting rule to obtain some pixels of the designated channel image corresponding to the original YUV image and some pixels of the Alpha channel image , The split rule is determined based on the fusion mode of the original YUV image and the Alpha channel image;

Restore the image of the specified channel based on the resolution of the YUV image and some pixels of the image of the specified channel, and restore the Alpha based on the resolution of the YUV image and some pixels of the image of the Alpha channel Channel image.
The decoding method according to claim 23, wherein the image of the specified channel is restored based on the resolution of the YUV image and some pixels of the image of the specified channel, and the image of the specified channel is restored based on the resolution of the YUV image And restoring the Alpha channel image with some pixels of the Alpha channel image includes:

Based on the resolution of the YUV image, perform interpolation processing on some pixels of the image of the specified channel to restore the image of the specified channel, perform interpolation processing on some of the pixels of the Alpha channel image, and restore the obtained The Alpha channel image; or

Based on the resolution of the YUV image, copy the pixel values of some pixels of the image of the specified channel as the pixel values of adjacent pixels, restore the image of the specified channel, and copy part of the image of the Alpha channel The pixel value of the pixel is used as the pixel value of the neighboring pixel, and the Alpha channel image is restored.
The decoding method according to any one of claims 22-24, wherein the designated channel is at least one channel, and the images of all the designated channels have the same number of pixels as the Alpha channel image.
The decoding method according to any one of claims 22-25, wherein the YUV image is an image in YUV420 format, and the image of the designated channel is a Y channel image.
The decoding method according to any one of claims 22-26, wherein the YUV image to be decoded is an image obtained after a specified decoding operation is processed, and the specified decoding operation includes decoding based on the H264/H265 standard operate.
A decoding device, characterized in that the decoding device includes a processor, a memory, and a computer program stored in the storage, and the processor implements the following steps when the processor executes the computer program:

Acquiring a YUV image to be decoded, where the YUV image is obtained by fusing the original YUV image obtained by conversion of the RGB channel image corresponding to the target image and the Alpha channel image corresponding to the target image;

Obtaining the original YUV image and the Alpha channel image based on the YUV image restoration;

The original YUV image is converted to obtain the RGB channel image, and the target image is obtained based on the RGB channel image and the Alpha channel image.
The decoding device according to claim 28, wherein the processor is configured to obtain the original YUV image and the Alpha channel image based on the YUV image restoration, and is specifically configured to:

Based on the image restoration of the specified channel corresponding to the YUV image, the image of the specified channel corresponding to the original YUV image and the Alpha channel image are obtained, and the image of the specified channel includes the Y channel image, the U channel image, or the V channel image. At least one of.
The decoding device according to claim 29, wherein the processor is configured to obtain the specified channel image corresponding to the original YUV image and the Alpha channel image based on the image restoration of the specified channel corresponding to the YUV image , Specifically used for:

Split the pixels of the designated channel image corresponding to the YUV image based on a preset splitting rule to obtain some pixels of the designated channel image corresponding to the original YUV image and some pixels of the Alpha channel image , The split rule is determined based on the fusion mode of the original YUV image and the Alpha channel image;

Restore the image of the specified channel based on the resolution of the YUV image and some pixels of the image of the specified channel, and restore the Alpha based on the resolution of the YUV image and some pixels of the image of the Alpha channel Channel image.
The decoding device according to claim 30, wherein the processor is configured to restore the image of the specified channel based on the resolution of the YUV image and some pixels of the image of the specified channel, and based on the When the resolution of the YUV image and some pixels of the Alpha channel image are used to restore the Alpha channel image, they are specifically used for:

Based on the resolution of the YUV image, perform interpolation processing on some pixels of the image of the specified channel to restore the image of the specified channel, perform interpolation processing on some of the pixels of the Alpha channel image, and restore the obtained The Alpha channel image; or

Based on the resolution of the YUV image, copy the pixel values of some pixels of the image of the specified channel as the pixel values of adjacent pixels, restore the image of the specified channel, and copy part of the image of the Alpha channel The pixel value of the pixel is used as the pixel value of the neighboring pixel, and the Alpha channel image is restored.
The decoding device according to any one of claims 29-31, wherein the designated channel is at least one channel, and the images of all the designated channels have the same number of pixels as the Alpha channel image.
The decoding device according to any one of claims 29-32, wherein the YUV image is an image in YUV420 format, and the image of the designated channel is a Y channel image.
The decoding device according to any one of claims 29-33, wherein the YUV image to be decoded is an image obtained after a specified decoding operation is processed, and the specified decoding operation includes decoding based on the H264/H265 standard operate.
A video transmission system, characterized in that, the video transmission system includes an encoding device and a decoding device;

The encoding device is used to obtain a target image to be encoded, convert the RGB channel image corresponding to the target image into a YUV image, and fuse the Alpha channel image corresponding to the target image with the YUV image to generate a new YUV image , Encoding the new YUV image to send the encoded image;

The decoding device is configured to, after receiving the encoded image, process the encoded image to obtain the new YUV image, and restore the YUV image and the Alpha channel based on the new YUV image Image, converting the YUV image into an RGB image, and obtaining the target image based on the RGB image and the Alpha channel image.