CN116801049A

CN116801049A - Video synthesis method, device, electronic equipment and storage medium

Info

Publication number: CN116801049A
Application number: CN202310822371.7A
Authority: CN
Inventors: 杨浩; 徐皖辉
Original assignee: Beijing Dajia Internet Information Technology Co Ltd
Current assignee: Beijing Dajia Internet Information Technology Co Ltd
Priority date: 2023-07-05
Filing date: 2023-07-05
Publication date: 2023-09-22

Abstract

The disclosure relates to a video synthesis method, a device, an electronic device and a storage medium, comprising: dividing each image in the sequence of images into three sub-images based on the resolution size of each image; acquiring transparency parameters of each pixel point in each sub-image; for each group of pixel points in the same position in the three sub-images, respectively storing transparency parameters of each pixel point in each group of pixel points into storage spaces of an R channel, a G channel and a B channel of the pixel point corresponding to the same position in a transparency parameter storage area to obtain a transparency image corresponding to each image; the storage spaces of the R channel, the G channel and the B channel of each pixel point in the transparency parameter storage area are preset to be empty; and synthesizing the transparency image corresponding to each image with each image to obtain a target video corresponding to the image sequence. The method can ensure the image quality of the video while reducing the size of the synthesized video.

Description

Video synthesis method, device, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of video processing technologies, and in particular, to a video synthesis method, apparatus, electronic device, storage medium, and program product.

Background

In the live and short video fields, it is often desirable to display video with transparent effects. The current method for realizing video with transparent effect is generally based on vector animation such as Lottie (an open source tool capable of adding animation effect to native application) or dynamic pictures such as GIF (Graphics Interchange Format, graphic interchange format), apng (Animated Portable Network Graphics, expansion of bitmap animation based on PNG), webp (a picture file format providing lossy compression and lossless compression at the same time), and the like, so as to generate video with transparent effect.

However, the method for storing the transparency parameters of the pixels in each video frame of the video with the transparency effect generated by the method is as follows: for each video frame, the transparency parameters of each pixel point in the video frame are stored through a transparency parameter storage area with the same size as the video frame, specifically, the transparency parameters of each pixel point in the video frame are stored in any one channel of RGB channels of the pixel point at the corresponding position in the transparency parameter storage area, so that the waste of the other two channels is caused, and the generated video file is overlarge.

Disclosure of Invention

The present disclosure provides a video synthesizing method, apparatus, electronic device, storage medium and program product, so as to at least solve the problem that the related art cannot simultaneously satisfy the video quality requirement and the size requirement of the video file. The technical scheme of the present disclosure is as follows:

According to a first aspect of an embodiment of the present disclosure, there is provided a video compositing method, including:

dividing each image in the image sequence into three sub-images based on the resolution size of each image;

acquiring transparency parameters of each pixel point in each sub-image;

for each group of pixel points in the same position in the three sub-images, respectively storing transparency parameters of each pixel point in each group of pixel points into storage spaces of an R channel, a G channel and a B channel of the pixel point corresponding to the same position in a transparency parameter storage area to obtain a transparency image corresponding to each image; each group of pixel points comprises a pixel point which is positioned at the same position in at least one sub-image, and the storage space of an R channel, a G channel and a B channel of each pixel point in the transparency parameter storage area is preset to be empty;

and synthesizing the transparency image corresponding to each image with each image to obtain the target video corresponding to the image sequence.

In an exemplary embodiment, the dividing each image into three sub-images based on the resolution size of each image includes:

Determining the storage type of pixel information of pixel points required by a target video to be synthesized; the storage type represents the position relation between the RGB parameter storage area and the transparency parameter storage area of the pixel point;

determining a segmentation mode of each image according to the storage type of the pixel information;

and dividing each image into three sub-images based on the dividing mode and the resolution size of each image.

In an exemplary embodiment, the determining the segmentation mode for each image according to the storage type of the pixel information includes:

when the storage type of the pixel information is a first storage type, determining that the segmentation mode of each image is a longitudinal segmentation mode along the height direction of the image resolution; the first storage type represents the position relationship between the RGB parameter storage area and the transparency parameter storage area of the pixel point, and is an adjacent relationship along the width direction of the image resolution;

the dividing each image into three sub-images based on the dividing mode and the resolution size of each image includes:

and dividing each image into three sub-images according to the longitudinal dividing mode and the resolution width of each image.

In an exemplary embodiment, the determining the segmentation mode for each image according to the storage type of the pixel information further includes:

when the storage type of the pixel information is the second storage type, determining that the segmentation mode of each image is a transverse segmentation mode along the width direction of the resolution of the image; the second storage type represents the position relationship between the RGB parameter storage area and the transparency parameter storage area of the pixel point, and is an adjacent relationship along the height direction of the image resolution;

the dividing each image into three sub-images based on the dividing mode and the resolution size of each image, further includes:

and dividing each image into three sub-images according to the transverse division mode and the resolution height of each image.

In an exemplary embodiment, the dividing each image into three sub-images based on the dividing method and the resolution size of each image further includes:

determining a resolution condition; the resolution condition is a condition of equally dividing an image into three sub-images;

when the resolution size meets the resolution condition, carrying out average segmentation on the image meeting the resolution condition based on the segmentation mode and the resolution size of each image to obtain three sub-images with the same image size.

In an exemplary embodiment, the method further comprises:

when the resolution size does not meet the resolution condition, taking the completeness of pixels in each sub-image obtained by segmentation as a segmentation condition, and segmenting the image which does not meet the resolution condition based on the segmentation mode and the resolution size of each image to obtain three sub-images with different sizes;

wherein the image size of at least one of the three sub-images of different sizes is different from the image size of the other sub-images.

In an exemplary embodiment, storing the transparency parameters of each pixel point in each group of pixel points in the storage spaces of the R channel, the G channel, and the B channel of the pixel point corresponding to the same position in the transparency parameter storage area, to obtain the transparency image corresponding to each image, includes:

when the sizes of the three sub-images are different, determining redundant pixel points in the three sub-images, and adding a target storage space of a complementary pixel point corresponding to the redundant pixel points in the transparency parameter storage area; the redundant pixel points are pixel points, of which the image with the largest image size is more than the image with the smallest image size, in the three sub-images; the target storage space comprises storage spaces of an R channel, a G channel and a B channel of the complementary pixel point;

And storing transparency parameters of redundant pixel points in the three sub-images in a storage space of one or two channels of an R channel, a G channel and a B channel of the complementary pixel points to obtain a transparency image corresponding to each image.

In an exemplary embodiment, after the synthesizing the transparency image corresponding to each image and each image, obtaining the target video corresponding to the image sequence further includes:

based on the transparency parameters of the pixels of each image in the image sequence, obtaining displacement information of the transparency parameters of the pixels of each image in a storage mode of the transparency images of each image;

and decoding the target video according to the displacement information.

According to a second aspect of embodiments of the present disclosure, there is provided a video compositing apparatus, comprising:

a segmentation unit configured to perform, for each image in a sequence of images, segmentation of said each image into three sub-images based on a resolution size of said each image;

an acquisition unit configured to perform acquisition of transparency parameters of respective pixel points in each sub-image;

A storage unit configured to execute transparency parameters of each pixel point in each group of pixel points in the same position in each of the three sub-images, and store the transparency parameters into storage spaces of an R channel, a G channel and a B channel of the pixel point corresponding to the same position in a transparency parameter storage area, respectively, so as to obtain a transparency image corresponding to each image; each group of pixel points comprises a pixel point which is positioned at the same position in at least one sub-image, and the storage space of an R channel, a G channel and a B channel of each pixel point in the transparency parameter storage area is preset to be empty;

and the synthesizing unit is configured to perform synthesizing processing on the transparency image corresponding to each image and each image to obtain the target video corresponding to the image sequence.

In an exemplary embodiment, the segmentation unit is further configured to perform determining a storage type of pixel information of pixel points required for the target video to be synthesized; the storage type represents the position relation between the RGB parameter storage area and the transparency parameter storage area of the pixel point; determining a segmentation mode of each image according to the storage type of the pixel information; and dividing each image into three sub-images based on the dividing mode and the resolution size of each image.

In an exemplary embodiment, the segmentation unit is further configured to determine, when the storage type of the pixel information is the first storage type, that the segmentation mode for each image is a longitudinal segmentation mode along the height direction of the image resolution; the first storage type represents the position relationship between the RGB parameter storage area and the transparency parameter storage area of the pixel point, and is an adjacent relationship along the width direction of the image resolution; and dividing each image into three sub-images according to the longitudinal dividing mode and the resolution width of each image.

In an exemplary embodiment, the dividing unit is further configured to determine, when the storage type of the pixel information is the second storage type, that the dividing manner for each image is a lateral dividing manner along the width direction of the image resolution; the second storage type represents the position relationship between the RGB parameter storage area and the transparency parameter storage area of the pixel point, and is an adjacent relationship along the height direction of the image resolution; and dividing each image into three sub-images according to the transverse division mode and the resolution height of each image.

In an exemplary embodiment, the segmentation unit is further configured to perform determining a resolution condition; the resolution condition is a condition of equally dividing an image into three sub-images; when the resolution size meets the resolution condition, carrying out average segmentation on the image meeting the resolution condition based on the segmentation mode and the resolution size of each image to obtain three sub-images with the same image size.

In an exemplary embodiment, the segmentation unit is further configured to perform, when the resolution size does not meet the resolution condition, segmentation with the integrity of pixels in each sub-image obtained by segmentation as a segmentation condition, and segment the image that does not meet the resolution condition based on the segmentation mode and the resolution size of each image, so as to obtain three sub-images with unequal sizes; wherein the image size of at least one of the three sub-images of different sizes is different from the image size of the other sub-images.

In an exemplary embodiment, the storage unit is further configured to determine redundant pixel points in the three sub-images when the sizes of the three sub-images are different, and add a target storage space of a supplementary pixel point corresponding to the redundant pixel points in the transparency parameter storage area; the redundant pixel points are pixel points, of which the image with the largest image size is more than the image with the smallest image size, in the three sub-images; the target storage space comprises storage spaces of an R channel, a G channel and a B channel of the complementary pixel point; and storing transparency parameters of redundant pixel points in the three sub-images in a storage space of one or two channels of an R channel, a G channel and a B channel of the complementary pixel points to obtain a transparency image corresponding to each image.

In an exemplary embodiment, the apparatus further includes a decoding unit configured to perform a storing manner in the transparency image of each image based on the transparency parameter of the pixel of each image in the image sequence, to obtain displacement information of the transparency parameter of the pixel of each image; and decoding the target video according to the displacement information.

According to a third aspect of embodiments of the present disclosure, there is provided an electronic device, comprising:

a processor;

a memory for storing the processor-executable instructions;

wherein the processor is configured to execute the instructions to implement the method of any of the above.

According to a fourth aspect of embodiments of the present disclosure, there is provided a computer readable storage medium, which when executed by a processor of an electronic device, causes the electronic device to perform the method as set forth in any one of the preceding claims.

According to a fifth aspect of embodiments of the present disclosure, there is provided a computer program product comprising instructions therein, which when executed by a processor of an electronic device, enable the electronic device to perform the method of any one of the above.

The technical scheme provided by the embodiment of the disclosure at least brings the following beneficial effects:

firstly, dividing each image into three sub-images based on the resolution size of each image in an image sequence, after acquiring the transparency parameters of each pixel point in each sub-image, storing the transparency parameters of each pixel point in each group of pixel points in the same position in the three sub-images into the storage spaces of R channels, G channels and B channels of the pixel points in the corresponding positions in a transparency parameter storage area respectively, thereby realizing the full use of the three storage channels of each pixel in the transparency parameter storage area, avoiding the waste of two channels in the three channels of each pixel, realizing the reduction of the transparency parameter storage area, further achieving the aim of reducing the size of the finally synthesized video.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure and do not constitute an undue limitation on the disclosure.

FIG. 1 is a schematic diagram of a prior art storage of RGB three primary color parameters and transparency parameters for a pixel in a video synthesized with an alpha channel.

Fig. 2 is a flow diagram illustrating a video compositing method according to an illustrative embodiment.

Fig. 3 is a schematic diagram illustrating a manner of storing transparency parameters of respective pixels in an image according to an exemplary embodiment.

Fig. 4 is a flow diagram illustrating the segmentation of each image into three sub-images, according to an exemplary embodiment.

Fig. 5 is a schematic diagram showing a manner of storing transparency parameters of respective pixels in an image according to another exemplary embodiment.

Fig. 6 is a schematic diagram showing the comparison of the effect of a transparency image after the optimization method of the present disclosure with that of a transparency image in the related art, according to an exemplary embodiment.

Fig. 7 is a flow chart illustrating a video compositing method according to another illustrative embodiment.

Fig. 8 is a block diagram illustrating a structure of a video composing apparatus according to an exemplary embodiment.

Fig. 9 is a block diagram of an electronic device, according to an example embodiment.

Detailed Description

In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

It should be noted that the embodiments described in the following exemplary examples do not represent all embodiments consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as detailed in the accompanying claims.

It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the foregoing figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the disclosure described herein may be capable of operation in sequences other than those illustrated or described herein. It should be further noted that, the user information (including, but not limited to, user equipment information, user personal information, etc.) and the data (including, but not limited to, data for presentation, analyzed data, etc.) related to the present disclosure are information and data authorized by the user or sufficiently authorized by each party.

Referring to fig. 1, a schematic diagram of a storage manner of RGB three primary color parameters and transparency parameters (i.e., alpha values) of pixels in a video synthesized with an alpha channel in the prior art is shown in fig. 1, where the pixels in the video with a transparent special effect have RGB three primary color parameters and transparency parameters, and the RGB three primary color parameters and the transparency parameters are stored in left and right areas respectively and are recorded as an RGB parameter storage area and a transparency parameter storage area, where the storage manner in the RGB parameter storage area and the transparency parameter storage area is an RGB three-channel storage, and the RGB three primary color parameters of a pixel occupy 3 channels in the RGB parameter storage area, and the transparency parameter occupies one channel in the transparency parameter storage area. In the storage mode shown in fig. 1, in each RGB three-channel in the transparency parameter storage area, only one channel stores an alpha value, and the other two channels are wasted but occupy storage resources, so that the obtained video file with transparent special effects is too large, and if the alpha channel is reduced, the image quality of the video will be affected, so that the conventional method has the problem that the video quality requirement and the video file size requirement cannot be met simultaneously.

Therefore, in order to solve the above-mentioned problem, the present disclosure proposes a video composition method for reducing the size of the entire video by using two channels free for each pixel in the transparency parameter storage area to store the transparency parameters of other pixels, thereby reducing the size of the transparency parameter storage area.

Referring to fig. 2, a flow chart of a video synthesizing method according to the present disclosure is provided, and this embodiment is illustrated by applying the method to a terminal, where it is understood that the method may also be applied to a server, and may also be applied to a system including the terminal and the server, and implemented through interaction between the terminal and the server. The terminal can be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, internet of things equipment and portable wearable equipment, and the internet of things equipment can be smart speakers, smart televisions, smart air conditioners, smart vehicle-mounted equipment and the like. The portable wearable device may be a smart watch, smart bracelet, headset, or the like. The server may be implemented as a stand-alone server or as a server cluster composed of a plurality of servers. In this embodiment, the method includes the steps of:

In step S210, for each image in the image sequence, each image is divided into three sub-images based on the resolution size of each image.

The image sequence may be a sequence of a plurality of images that are captured in succession and that can be combined into a video.

Wherein the resolution size includes a resolution width and a resolution height of the image.

In particular implementations, since the transparency parameters of pixels in the synthesized video with transparency effects are stored in R, G, B three channels, the present disclosure employs dividing each image in an image sequence into three sub-images so that the transparency parameters of two of the sub-image pixels are stored in the free two channels of pixels in the other sub-image.

More specifically, the division of each image may be performed longitudinally according to the resolution width, or may be performed transversely according to the resolution height, and the specific division manner needs to be determined according to the positional relationship between the RGB storage area and the transparency parameter storage area in the finally synthesized video.

In step S220, a transparency parameter of each pixel point in each sub-image is acquired.

In a specific implementation, the pixel points of each image in the image sequence store RGBA parameters, where RGB represents the red, green, and blue tristimulus values of the pixel, and a represents the transparency parameter (i.e., alpha value) of the pixel, so that the transparency parameter of each pixel point in each sub-image can be extracted.

In step S230, for each group of pixel points in the same position in the three sub-images, the transparency parameters of each pixel point in each group of pixel points are respectively stored into the storage spaces of the R channel, the G channel and the B channel of the pixel point corresponding to the same position in the transparency parameter storage area, so as to obtain a transparency image corresponding to each image; each group of pixel points comprises a pixel point which is positioned at the same position in at least one sub-image, and the storage space of an R channel, a G channel and a B channel of each pixel point in the transparency parameter storage area is preset to be empty.

In a specific implementation, each image in the image sequence has a corresponding transparency parameter storage area for storing transparency parameters of pixels in the image. Referring to fig. 3, a schematic diagram of a manner of storing transparency parameters of each pixel in an image according to the disclosure is shown in fig. 3, each image is set with an RGB parameter storage area (left half area in fig. 3) and a transparency parameter storage area (right half area in fig. 3), each image is divided into 3 sub-images, and then is marked as a sub-image 31, a sub-image 32 and a sub-image 33, in the transparency parameter storage area, pixel points a, B and c of the sub-image 31, the sub-image 32 and the sub-image 33 in the same position are used as a group of pixel points, and the transparency parameters of each pixel point in each group of pixel points are respectively stored in storage spaces of R channel, G channel and B channel of the pixel point corresponding to the same position in the transparency parameter storage area, so that R channel, G channel and B channel of the same pixel can be fully occupied, storage resources of 6 channels can be saved at a time, and the transparency parameter storage area after storing transparency parameters of each pixel in the three sub-images is regarded as one image, thereby obtaining an image corresponding to each image in an image sequence.

In step S240, a combination process is performed on the transparency image corresponding to each image and each image, so as to obtain a target video corresponding to the image sequence.

In the specific implementation, after the transparency image of each image in the image sequence is obtained, the transparency image corresponding to each image and the RGB color parameters of each image can be further synthesized to obtain the target video with the transparent special effect corresponding to the image sequence.

In another embodiment, the transparency parameter information may be stored as a separate data stream when generating the target video with the transparency effect and processed separately when decoding.

In the video synthesis method, firstly, based on the resolution size of each image in an image sequence, each image is divided into three sub-images, after the transparency parameters of all pixel points in each sub-image are acquired, the transparency parameters of all pixel points in each group of pixel points in the same position in the three sub-images are respectively stored into the storage spaces of R channels, G channels and B channels of the pixel points in the corresponding positions in a transparency parameter storage area, so that the full use of the three storage channels of each pixel in the transparency parameter storage area is realized, the waste of two channels in the three channels of each pixel is avoided, the reduction of the transparency parameter storage area is realized, and the aim of reducing the size of the finally synthesized video is fulfilled.

In an exemplary embodiment, as shown in fig. 4, dividing each image into three sub-images based on the resolution size of each image in step S210 may be implemented specifically by:

in step S410, a storage type of pixel information of a pixel point required for the target video to be synthesized is determined; the storage type represents the position relation between the RGB parameter storage area and the transparency parameter storage area of the pixel point;

in step S420, determining a division manner for each image according to the storage type of the pixel information;

in step S430, each image is divided into three sub-images based on the division manner and the resolution size of each image.

The dividing mode can comprise transverse dividing and longitudinal dividing.

In a specific implementation, the positional relationship between the RGB parameter storage area and the transparency parameter storage area may be an adjacent relationship along the width direction of the image resolution, as shown in fig. 3, the left side is the RGB parameter storage area, the right side is the transparency parameter storage area, and the positional relationship may also be an adjacent relationship along the height direction of the image resolution. Therefore, when each image in the image sequence is segmented, it is required to determine which type of storage type of pixel information in the target video to be synthesized needs to be set, and then determine a segmentation mode of each image according to the storage type, and further determine whether to segment the image according to the resolution height or the resolution width according to the segmentation mode.

Further, in an exemplary embodiment, when the storage type of the pixel information is the first storage type, it is determined that the division manner for each image is a longitudinal division manner in the image resolution height direction; the first storage type represents the position relationship between the RGB parameter storage area and the transparency parameter storage area of the pixel point, and is the adjacent relationship along the width direction of the image resolution; each image is divided into three sub-images according to a longitudinal division manner and a resolution width of each image.

In a specific implementation, when it is determined that the storage type of the pixel information in the target video needs to be set to a first storage type, that is, the positional relationship between the RGB parameter storage area and the transparency parameter storage area is an adjacent relationship along the width direction of the resolution of the image, the RGB three channels of each pixel are arranged transversely, so that the segmentation mode of each image should be a longitudinal segmentation mode, the resolution width of each image needs to be acquired correspondingly, and each image is segmented longitudinally into three sub-images according to the resolution width.

In another exemplary embodiment, when the storage type of the pixel information is the second storage type, it is determined that the division manner for each image is a lateral division manner in the image resolution width direction; the second storage type represents the position relationship between the RGB parameter storage area and the transparency parameter storage area of the pixel point, and is an adjacent relationship along the height direction of the image resolution; each image is divided into three sub-images according to a lateral division manner and a resolution height of each image.

In a specific implementation, when it is determined that the storage type of the pixel information in the target video needs to be set to the second storage type, that is, the positional relationship between the RGB parameter storage area and the transparency parameter storage area is an adjacent relationship along the height direction of the resolution of the image, the RGB three channels of each pixel are arranged longitudinally, so that the segmentation mode of each image should be a transverse segmentation mode, the resolution height of each image needs to be acquired correspondingly, and each image is transversely segmented into three sub-images according to the resolution height.

In the above embodiment, it is determined which type of storage type of the pixel information in the target video to be synthesized needs to be set, then the segmentation mode of each image is determined according to the storage type, and then the image segmentation is performed according to the resolution height or the resolution width according to the segmentation mode, so that the segmentation result is not matched with the requirement of the synthesized video, resulting in subsequent processing errors, and time and processing resources are wasted.

In an exemplary embodiment, before dividing each image into three sub-images based on the division manner and the resolution size of each image, further comprising:

Step S211, determining a resolution condition; the resolution condition is a condition in which an image is equally divided into three sub-images;

step S212, when the resolution size accords with the resolution condition, carrying out average segmentation on the image which accords with the resolution condition based on the segmentation mode and the resolution size of each image to obtain three sub-images with the same image size;

step S213, when the resolution size does not meet the resolution condition, the integrity of the pixels in each sub-image obtained by segmentation is taken as the segmentation condition, and the image which does not meet the resolution condition is segmented based on the segmentation mode and the resolution size of each image, so as to obtain three sub-images with different sizes; wherein the image size of at least one of the three sub-images of different sizes is different from the image size of the other sub-images.

The resolution condition may be that the number of pixels corresponding to the resolution size is a multiple of 3.

It should be noted that, in this embodiment, the unequal sizes of the three sub-images indicates that the sizes of the three sub-images are not completely equal, but the sizes of two sub-images may be equal.

In a specific implementation, since the number of channels corresponding to each pixel in the transparency parameter storage area storing the transparency parameter is three, each image in the image sequence needs to be divided into three sub-images, and the division of the images can be regarded as the division of the pixels, and the number of horizontal pixels (i.e. resolution width) and the number of vertical pixels (i.e. resolution height) in the image are not necessarily equally divided into three parts, therefore, in the process of dividing each image into three sub-images, each image may be divided into three sub-images with equal sizes, and each image may be divided into three sub-images with unequal sizes. Therefore, before the segmentation, it is necessary to determine whether the resolution size meets a preset resolution condition based on the segmentation mode. More specifically, it is necessary to determine whether to use resolution height division or resolution width division based on the division manner, and then determine whether the resolution height or resolution width is a multiple of 3.

When the resolution size is determined to meet the resolution condition based on the segmentation mode, namely, the resolution size is equal to the multiple of 3, the image can be equally segmented, and then the image meeting the preset resolution condition can be equally segmented based on the segmentation mode and the resolution size of each image, so that three equal sub-images are obtained. For example, if the division method is vertical division, it is determined whether the resolution width is a multiple of 3, and if so, the image may be vertically divided into three sub-images of equal size according to the resolution width. If the division mode is the transverse division, determining whether the resolution height is a multiple of 3, and if so, transversely dividing the image into three sub-images with the same size according to the resolution height.

On the contrary, when the resolution size is determined to be not in accordance with the resolution condition based on the division mode, that is, not a multiple of 3, it is indicated that the images cannot be equally divided, and in this case, in order to ensure the integrity of the pixels in the three sub-images obtained by division, each image may be divided into three sub-images of unequal sizes according to the division mode and the resolution size by using a video filter clipping model. For example, if the division mode is vertical division, it is determined whether the resolution width is a multiple of 3, if not, the image may be vertically divided into three sub-images of equal size according to the resolution width, for example, the resolution width is 8 pixels, and the image may be divided into three sub-images of 3 pixels, and 2 pixels in width, respectively. In addition, when the video filter clipping model is adopted for segmentation, the smaller and better the difference among the three sub-images is, so that the waste of transparency parameter storage area channels is reduced as much as possible. For example, a division method in which the resolution width is 8 pixels is divided into 3 pixels, 3 pixels and 2 pixels is preferable to a division method in which 2 pixels, 4 pixels and 4 pixels, because the number of channels is set by the sub-image with the largest resolution size among the three sub-images, the division method in which 3 pixels, 3 pixels and 2 pixels needs to set 3*3 =9 channels, and the division method in which 2 pixels, 4 pixels and 4 pixels needs to set 3×4=12 channels, so that the number of wasted channels is larger, and the smaller the difference between the three sub-images is, the better.

In the above embodiment, whether the resolution size meets the preset resolution condition is determined according to the segmentation mode, so that each image is segmented by adopting different segmentation modes according to different determination results, so that the difference between the three sub-images obtained by segmentation is as small as possible, and the waste of transparency parameter storage area channels is reduced as little as possible, so as to reduce the size of the synthesized video.

In an exemplary embodiment, in step S230, transparency parameters of each pixel in each group of pixels are stored in storage spaces of R channel, G channel and B channel of the pixel corresponding to the same position in the transparency parameter storage area, so as to obtain transparency images corresponding to each image, including: when the sizes of the three sub-images are equal, that is, the resolution size is a multiple of 3, dividing the width of each image into three parts equally, as shown in fig. 3, storing the transparency parameters of all pixels of the first part in the R channel of each pixel of the right transparency parameter storage area; the transparency parameters of all pixels of the second part are stored in the G channel of each pixel of the right transparency parameter storage area; the transparency parameters of all pixels of the third part are stored in the B channel of each pixel of the right transparency parameter storage area, so that the purposes of reducing the width of the right transparency parameter storage channel area by 2/3 and reducing the width of the whole video by 1/3 while completely storing the transparency parameters of all pixels are achieved, and the size of the synthesized target video is reduced on the basis of not affecting the video image quality.

In another exemplary embodiment, in step S230, transparency parameters of each pixel point in each group of pixel points are respectively stored in storage spaces of R channel, G channel and B channel of the pixel point corresponding to the same position in the transparency parameter storage area, so as to obtain a transparency image corresponding to each image, and the method further includes:

step S231, when the sizes of the three sub-images are different, determining redundant pixel points in the three sub-images, and adding a target storage space of the complementary pixel points corresponding to the redundant pixel points in the transparency parameter storage area; the redundant pixel points are pixel points, of which the image with the largest image size is more than the image with the smallest image size, in the three sub-images; the target storage space comprises storage spaces of an R channel, a G channel and a B channel of the complementary pixel points;

and step S232, storing transparency parameters of redundant pixel points in the three sub-images in a storage space of one or two channels of an R channel, a G channel and a B channel of the complementary pixel points to obtain a transparency image corresponding to each image.

In a specific implementation, when the sizes of the three sub-images are not equal, i.e. the resolution size is not a multiple of 3, only the case of dividing 3 by more than 1 or 3 by more than 2 will occur, i.e. 1 pixel or 2 pixels will be more. Therefore, a pixel point can be supplemented in the transparency parameter storage area, the transparency parameters of the redundant pixel points are stored in one or two channels of an R channel, a G channel and a B channel of the transparency parameter storage area supplemented pixel points, and a transparency image corresponding to each image is obtained.

More specifically, when 1 pixel is more than 1 pixel, the transparency parameter of the redundant pixel point may be stored in any one of the R channel, G channel, and B channel of the transparency parameter storage area supplementary pixel point; when more than 2 pixels are provided, the transparency parameters of the redundant 2 pixels can be stored in the transparency parameter storage area to supplement any two channels of the R channel, the G channel and the B channel of the pixels.

For example, referring to fig. 5, for a schematic diagram of a transparency parameter storage manner when the resolution size does not meet the resolution requirement, the upper schematic diagram in fig. 5 shows a storage manner in the case of more than 1 pixel, and the transparency parameter of the more than 1 pixel is stored in the B channel of the supplementary pixel. The lower diagram in fig. 5 shows the storage mode in case of more than 2 pixels, and the transparency parameters of the redundant pixels are stored in the R channel and the G channel of the supplementary pixels.

When the three sub-images provided in this embodiment are unequal in size, the transparency parameters of the redundant pixel points are stored in the R, G, B channel of the pixel point additionally added in the transparency parameter storage area, so that the transparency parameters of all pixels of each image can be completely stored, and the image quality of the special effect video is ensured not to be lost.

Referring to fig. 6, in order to compare the effect of the transparency image obtained by the optimization method of the present disclosure with that of the transparency image in the prior art, it can be seen from the graph that the transparency image obtained by the transparency parameter storage method of the present disclosure is far smaller than the transparency image obtained by the prior art, so that the method can reduce the occupation of the transparency parameter storage resource and reduce the size of the synthesized video.

In an exemplary embodiment, after the step S240 of synthesizing the transparency image corresponding to each image and each image to obtain the target video corresponding to the image sequence, the method further includes:

step S241, obtaining displacement information of transparency parameters of pixels of each image based on a storage mode of the transparency parameters of pixels of each image in the image sequence in the transparency image of each image;

step S242, decoding the target video according to the displacement information.

In a specific implementation, before decoding a target video with a transparent special effect, a storage type of pixel information needs to be set for the video to mark a decoding method for converting YUV into RGB by using a fragment shader in a decoding process, and after the target video with the transparent special effect is decoded into a data frame of YUV color space, the data frame is converted into RGB color space data by using OpenGL. In the decoding process, the YUV data is bound into textures and then uploaded to a fragment shader, and the fragment shader calculates displacement information of transparency parameters corresponding to each color pixel according to the storage mode of transparency parameters of transparency parameter storage area pixels in the decoding process, so as to restore the displacement information into transparency parameters of RGB color pixels for rendering, and decoding is completed.

In this embodiment, when decoding a target video with a transparent special effect, displacement information of transparency parameters corresponding to pixels of each image in the target video is calculated according to a transparency parameter storage mode of each pixel, so that decoding is performed according to the displacement information, and accuracy of a decoding result is ensured.

In another exemplary embodiment, as shown in fig. 7, there is a flowchart of a video compositing method according to an exemplary embodiment, in this embodiment, the method comprises the steps of:

step S710, obtaining an image sequence and determining the storage type of pixel information of pixel points in a target video to be synthesized; the storage type represents the position relation between the RGB parameter storage area and the transparency parameter storage area of the pixel point;

step S720, determining a segmentation mode of each image according to the storage type of the pixel information for each image in the image sequence;

step S730, determining whether the resolution size of each image meets a preset resolution condition based on the segmentation mode;

step S740, when the resolution size accords with the resolution condition, carrying out average segmentation on the image which accords with the resolution condition based on the segmentation mode and the resolution size of each image to obtain three sub-images with the same image size;

Step S750, when the resolution size does not meet the resolution condition, taking the integrity of pixels in each sub-image obtained by segmentation as a segmentation condition, and segmenting the image which does not meet the resolution condition based on a segmentation mode and the resolution size of each image to obtain three sub-images with different sizes;

step S760, obtaining transparency parameters of each pixel point in each sub-image;

step S770, for each group of pixel points in the same position in the three sub-images, storing the transparency parameters of each pixel point in each group of pixel points into the storage spaces of the R channel, the G channel and the B channel of the pixel point corresponding to the same position in the transparency parameter storage area, respectively, to obtain a transparency image corresponding to each image; each group of pixel points comprises a pixel point which is positioned at the same position in at least one sub-image, and the storage space of an R channel, a G channel and a B channel of each pixel point in the transparency parameter storage area is preset to be empty;

and step S780, synthesizing the transparency image corresponding to each image and each image to obtain the target video corresponding to the image sequence.

According to the video synthesis method provided by the embodiment, the transparency parameters of other pixels are stored by using two idle channels of each pixel in the transparency parameter storage area, so that the size of the transparency parameter storage area is reduced, the purpose of reducing the size of the whole video is achieved, and meanwhile, the special effect and the image quality of the video can be ensured not to be influenced.

It should be understood that, although the steps in the flowcharts related to the above embodiments are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.

It should be understood that the same/similar parts of the embodiments of the method described above in this specification may be referred to each other, and each embodiment focuses on differences from other embodiments, and references to descriptions of other method embodiments are only needed.

Based on the same inventive concept, the embodiment of the present disclosure also provides a video compositing apparatus for implementing the video compositing method referred to above.

Fig. 8 is a block diagram illustrating a structure of a video composing apparatus according to an exemplary embodiment. Referring to fig. 8, the apparatus includes:

a segmentation unit 810 configured to perform segmentation of each image in the sequence of images into three sub-images based on a resolution size of each image;

an acquiring unit 820 configured to perform acquisition of transparency parameters of respective pixel points in each sub-image;

a storage unit 830, configured to execute, for each group of pixel points in the same position in the three sub-images, to store transparency parameters of each pixel point in each group of pixel points into storage spaces of R channel, G channel and B channel of the pixel point corresponding to the same position in the transparency parameter storage area, respectively, so as to obtain a transparency image corresponding to each image; each group of pixel points comprises a pixel point which is positioned at the same position in at least one sub-image, and the storage space of an R channel, a G channel and a B channel of each pixel point in the transparency parameter storage area is preset to be empty;

and a synthesizing unit 840 configured to perform synthesizing processing on the transparency image corresponding to each image and each image, so as to obtain a target video corresponding to the image sequence.

In an exemplary embodiment, the segmentation unit 810 is further configured to perform a storage type of pixel information of pixel points required for determining the target video to be synthesized; the storage type represents the position relation between the RGB parameter storage area and the transparency parameter storage area of the pixel point; determining a segmentation mode for each image according to the storage type of the pixel information; each image is divided into three sub-images based on the division manner and the resolution size of each image.

In an exemplary embodiment, the segmentation unit 810 is further configured to perform determining that the segmentation method for each image is a longitudinal segmentation method along the height direction of the image resolution when the storage type of the pixel information is the first storage type; the first storage type represents the position relationship between the RGB parameter storage area and the transparency parameter storage area of the pixel point, and is the adjacent relationship along the width direction of the image resolution; each image is divided into three sub-images according to a longitudinal division manner and a resolution width of each image.

In an exemplary embodiment, the segmentation unit 810 is further configured to perform determining that the segmentation method for each image is a lateral segmentation method along the width direction of the image resolution when the storage type of the pixel information is the second storage type; the second storage type represents the position relationship between the RGB parameter storage area and the transparency parameter storage area of the pixel point, and is an adjacent relationship along the height direction of the image resolution; each image is divided into three sub-images according to a lateral division manner and a resolution height of each image.

In an exemplary embodiment, the segmentation unit 810 is further configured to perform determining a resolution condition; the resolution condition is a condition in which an image is equally divided into three sub-images; when the resolution size meets the resolution condition, the images meeting the resolution condition are divided averagely based on the dividing mode and the resolution size of each image, and three sub-images with the same image size are obtained.

In an exemplary embodiment, the segmentation unit 810 is further configured to perform, when the resolution size does not meet the resolution condition, segmentation with the integrity of the pixels in each of the segmented sub-images as the segmentation condition, and segment the image that does not meet the resolution condition based on the segmentation mode and the resolution size of each image, to obtain three sub-images with unequal sizes; wherein the image size of at least one of the three sub-images of different sizes is different from the image size of the other sub-images.

In an exemplary embodiment, the storage unit 830 is further configured to determine redundant pixel points in the three sub-images and add a target storage space of the supplementary pixel points corresponding to the redundant pixel points in the transparency parameter storage area when the sizes of the three sub-images are different; the redundant pixel points are pixel points, of which the image with the largest image size is more than the image with the smallest image size, in the three sub-images; the target storage space comprises storage spaces of an R channel, a G channel and a B channel of the complementary pixel points; and storing transparency parameters of redundant pixel points in the three sub-images in a storage space of one or two channels of an R channel, a G channel and a B channel of the complementary pixel points to obtain a transparency image corresponding to each image.

In an exemplary embodiment, the apparatus further includes a decoding unit configured to perform a transparency parameter based on the pixel point of each image in the image sequence, and obtain displacement information of the transparency parameter of the pixel point of each image in a storage manner in the transparency image of each image; and decoding the target video according to the displacement information.

The specific manner in which the various modules perform the operations in the apparatus of the above embodiments have been described in detail in connection with the embodiments of the method, and will not be described in detail herein.

Fig. 9 is a block diagram illustrating an electronic device 900 for implementing a video composition method, according to an example embodiment. For example, electronic device 900 may be a mobile phone, computer, digital broadcast terminal, messaging device, game console, tablet device, medical device, exercise device, personal digital assistant, and the like.

Referring to fig. 9, an electronic device 900 may include one or more of the following components: a processing component 902, a memory 904, a power component 906, a multimedia component 908, an audio component 910, an input/output (I/O) interface 912, a sensor component 914, and a communication component 916.

The processing component 902 generally controls overall operation of the electronic device 900, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 902 may include one or more processors 920 to execute instructions to perform all or part of the steps of the methods described above. Further, the processing component 902 can include one or more modules that facilitate interaction between the processing component 902 and other components. For example, the processing component 902 can include a multimedia module to facilitate interaction between the multimedia component 908 and the processing component 902.

The memory 904 is configured to store various types of data to support operations at the electronic device 900. Examples of such data include instructions for any application or method operating on the electronic device 900, contact data, phonebook data, messages, pictures, video, and so forth. The memory 904 may be implemented by any type of volatile or nonvolatile memory device or combination thereof, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, optical disk, or graphene memory.

The power supply component 906 provides power to the various components of the electronic device 900. Power supply components 906 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for electronic device 900.

The multimedia component 908 comprises a screen between the electronic device 900 and the user that provides an output interface. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may sense not only the boundary of a touch or slide action, but also the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 908 includes a front-facing camera and/or a rear-facing camera. When the electronic device 900 is in an operational mode, such as a shooting mode or a video mode, the front camera and/or the rear camera may receive external multimedia data. Each front and rear camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

The audio component 910 is configured to output and/or input audio signals. For example, the audio component 910 includes a Microphone (MIC) configured to receive external audio signals when the electronic device 900 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may be further stored in the memory 904 or transmitted via the communication component 916. In some embodiments, the audio component 910 further includes a speaker for outputting audio signals.

The I/O interface 912 provides an interface between the processing component 902 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: homepage button, volume button, start button, and lock button.

The sensor assembly 914 includes one or more sensors for providing status assessment of various aspects of the electronic device 900. For example, the sensor assembly 914 may detect an on/off state of the electronic device 900, a relative positioning of the components, such as a display and keypad of the electronic device 900, the sensor assembly 914 may also detect a change in position of the electronic device 900 or a component of the electronic device 900, the presence or absence of a user's contact with the electronic device 900, an orientation or acceleration/deceleration of the device 900, and a change in temperature of the electronic device 900. The sensor assembly 914 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 914 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 914 may also include an acceleration sensor, a gyroscopic sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 916 is configured to facilitate communication between the electronic device 900 and other devices, either wired or wireless. The electronic device 900 may access a wireless network based on a communication standard, such as WiFi, an operator network (e.g., 2G, 3G, 4G, or 5G), or a combination thereof. In one exemplary embodiment, the communication component 916 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component 916 further includes a Near Field Communication (NFC) module to facilitate short range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra Wideband (UWB) technology, bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the electronic device 900 may be implemented by one or more Application Specific Integrated Circuits (ASICs), digital Signal Processors (DSPs), digital Signal Processing Devices (DSPDs), programmable Logic Devices (PLDs), field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic elements for executing the methods described above.

In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory 904 including instructions executable by the processor 920 of the electronic device 900 to perform the above-described method. For example, the computer readable storage medium may be ROM, random Access Memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

In an exemplary embodiment, a computer program product is also provided, comprising instructions executable by the processor 920 of the electronic device 900 to perform the above-described method.

It should be noted that the descriptions of the foregoing apparatus, the electronic device, the computer readable storage medium, the computer program product, and the like according to the method embodiments may further include other implementations, and the specific implementation may refer to the descriptions of the related method embodiments and are not described herein in detail.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This disclosure is intended to cover any adaptations, uses, or adaptations of the disclosure following the general principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It is to be understood that the present disclosure is not limited to the precise arrangements and instrumentalities shown in the drawings, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method of video composition, comprising:

acquiring transparency parameters of each pixel point in each sub-image;

2. The method of claim 1, wherein the dividing each image into three sub-images based on the resolution size of each image comprises:

3. The method according to claim 2, wherein determining the division manner for each image according to the storage type of the pixel information includes:

4. The method according to claim 2, wherein determining the segmentation method for each image according to the storage type of the pixel information further comprises:

5. The method of any one of claims 2 to 4, wherein the dividing each image into three sub-images based on the division and the resolution size of each image comprises:

6. The method of claim 5, wherein the method further comprises:

7. The method according to claim 6, wherein storing the transparency parameters of each pixel in each group of pixels in the storage space of the R channel, the G channel, and the B channel of the pixel corresponding to the same position in the transparency parameter storage area, respectively, to obtain the transparency image corresponding to each image, includes:

8. The method according to claim 1, wherein after the synthesizing the transparency image corresponding to each image and each image, obtaining the target video corresponding to the image sequence further comprises:

obtaining displacement information of transparency parameters of pixel points of each image based on a storage mode of the transparency parameters of the pixel points of each image in the image sequence in the transparency image of each image;

And decoding the target video according to the displacement information.

9. A video compositing apparatus, comprising:

10. An electronic device, comprising:

a processor;

a memory for storing the processor-executable instructions;

wherein the processor is configured to execute the instructions to implement the video compositing method of any of claims 1-8.

11. A computer readable storage medium, characterized in that instructions in the computer readable storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the video compositing method of any of claims 1-8.