WO2020258907A1

WO2020258907A1 - Virtual article generation method, apparatus and device

Info

Publication number: WO2020258907A1
Application number: PCT/CN2020/077034
Authority: WO
Inventors: 刘伟; 庞炳新
Original assignee: 香港乐蜜有限公司
Priority date: 2019-06-28
Filing date: 2020-02-27
Publication date: 2020-12-30
Also published as: CN110213640A; CN110213640B

Abstract

Provided are a virtual article generation method, apparatus and device. A virtual article generation method is applied to a server, and comprises: when a generation instruction for a virtual article is detected, acquiring first image data, audio data and a playing timestamp of a first video corresponding to the generation instruction, wherein the first image data provides a picture of the first video, the audio data provides a sound of the first video, and the playing timestamp is an identifier used for ensuring content synchronization of the first image data and the audio data; adjusting the first image data according to preset appearance information of the virtual article, and transcoding the adjusted first image data to obtain picture data of the virtual article; and synchronously playing the picture data of the virtual article and the audio data according to the playing timestamp so as to obtain the virtual article corresponding to the generation instruction. The solution can achieve the effect of ensuring the synchronization of sounds and pictures in a diversified display effect of a generated virtual article.

Description

Method, device and equipment for generating virtual items

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on June 28, 2019 with the application number 201910578523.7 and the invention titled "Method, Apparatus and Equipment for Generating Virtual Objects", the entire contents of which are incorporated into this application by reference in.

Technical field

This application relates to the technical field of behavioral virtual items, in particular to a method, device and equipment for generating virtual items.

Background technique

Various types of Internet-related clients usually provide various configured virtual items so that users can use virtual items to realize virtual activities. Exemplarily, users can use virtual items to implement virtual transactions, dress up personal online communities, and so on. For example: buying virtual gifts in the live broadcast client to give to the anchor, and using virtual pendants in social software to decorate the avatar and personal homepage, etc. Among them, virtual items are usually static images, such as flower images, fireworks images, and so on. Since the static image has only one fixed picture, it is easy to cause the display effect of the generated virtual item to be relatively simple.

In this regard, sound effects can be added to virtual items to increase the variety of virtual items' display effects. However, in the related art, the way of adding sound effects is simply to play music when displaying virtual items, which may easily cause the picture and sound effects of the virtual items to be out of sync, leading to the problem of the sound and picture being out of sync with the display effects of the virtual items. Therefore, how to ensure the synchronization of audio and video of the diversified display effects of the generated virtual items is an urgent problem to be solved.

Summary of the invention

The purpose of the embodiments of the present application is to provide a method, device, and equipment for generating virtual items, so as to achieve the effect of improving the convenience of checking the configuration of virtual items. The specific technical solutions are as follows:

In the first aspect, an embodiment of the present application provides a method for generating a virtual item, which is applied to a server, and the method includes:

When the generation instruction of the virtual item is detected, acquiring the first image data, audio data, and playback timestamp of the first video corresponding to the generation instruction; wherein the first image data provides a picture of the first video; The audio data provides the sound of the first video; the playback timestamp is an identifier used to ensure synchronization of the content of the first image data and the audio data;

Adjusting the first image data according to preset shape information about the virtual item to obtain adjusted first image data, and transcoding the adjusted first image data to obtain the screen data of the virtual item;

According to the playback time stamp, the screen data and the audio data of the virtual item are played synchronously to obtain the virtual item corresponding to the generation instruction.

Optionally, before the step of transcoding the adjusted first image data to obtain screen data of the virtual item, the method further includes:

Acquiring second image data of a second video whose screen color is a transparent color; the second image data provides a screen of the second video;

Adjust the second image data according to preset shape information about the virtual item to obtain adjusted second image data;

The step of transcoding the adjusted first image data to obtain the screen data of the virtual item includes:

Acquiring a transparent position belonging to a transparent area in the adjusted first image data;

Masking the transparent position in the adjusted first image data to a transparent color by using the adjusted second image data to obtain masked image data;

Transcoding the masked image data to obtain the image data of the virtual item.

Optionally, the step of using the adjusted second image data to mask the transparent position in the adjusted first image data to a transparent color to obtain the masked image data includes:

In the transparent position in the adjusted first image data, the adjusted second image data is displayed to obtain the masked image data.

Optionally, the step of using the adjusted second image data to mask the transparent position in the adjusted first image data to a transparent color to obtain the masked image data includes :

Using the transparent position of the adjusted first image data as a transparent channel;

Filling the pixels in the transparent position in the adjusted second image data into the transparent channel to obtain transparent channel data;

Taking pixels other than the pixels in the transparent position in the adjusted first image data as non-transparent channel data;

Rendering the transparent channel data and the non-transparent channel data in the adjusted first image data to obtain masked image data.

Optionally, before the step of transcoding the masked image data to obtain the screen data of the virtual item, the method further includes:

Acquiring third image data and the positional relationship between the third image data and the masked image data; the third image data is used as a specific element of the virtual item;

Adding the third image data to the masked image data according to the position relationship to obtain special effect image data;

The step of transcoding the masked image data to obtain the image data of the virtual item includes:

Transcoding the special effect image data to obtain the screen data of the virtual item.

Optionally, the position relationship includes: taking each pixel of the masked image data as an element of a matrix, and setting each pixel of the masked image data in the masked image The position in the data is used as the position of the element in the matrix;

The positional relationship is the corresponding relationship between the elements of the second pixel matrix and the elements of the first pixel matrix; the second pixel matrix is the pixel matrix of pixels in the third image data; the first pixel matrix is The pixel matrix of the pixels in the masked image data;

The step of adding the third image data to the masked image data according to the position relationship to obtain special effect image data includes:

Respectively transforming the masked image data and the third image data into a first matrix and a second matrix;

In the first matrix, add elements in the second matrix according to the position relationship to obtain a third matrix;

The third matrix is converted into image data to obtain special effect image data.

In the second aspect, an embodiment of the present application provides a virtual item generation device applied to a server, and the device includes:

The data acquisition module is configured to acquire the first image data, audio data, and playback timestamp of the first video corresponding to the generation instruction when the generation instruction of the virtual item is detected; wherein, the first image data provides the The picture of the first video; the audio data provides the sound of the first video; the playback timestamp is an identifier used to ensure that the content of the first image data and the audio data are synchronized;

The screen data generation module is used to adjust the first image data according to the preset shape information about the virtual item to obtain the adjusted first image data, and to transcode the adjusted first image data to obtain the virtual Picture data of the item;

The virtual item display module is configured to synchronously play the screen data and the audio data of the virtual item according to the playback time stamp to obtain the virtual item corresponding to the generation instruction.

Optionally, the data acquisition module is specifically used for:

Before transcoding the adjusted first image data to obtain the screen data of the virtual item, obtain the second image data of the second video whose screen color is a transparent color; the second image data provides the second The picture of the video;

The picture data generation module includes: a mask sub-module and a transcoding sub-module;

The mask sub-module is configured to adjust the first image data and the second image data according to preset shape information about the virtual item to obtain adjusted second image data; and obtain the adjusted first image data; In one image data, the transparent position belonging to the transparent area; using the adjusted second image data, mask the transparent position in the adjusted first image data with a transparent color to obtain the masked Image data

The transcoding sub-module is used to transcode the masked image data to obtain the screen data of the virtual item.

Optionally, the mask submodule is specifically used for:

Use pixels other than the pixels in the transparent position in the adjusted first image data as non-transparent channel data;

Optionally, the data acquisition module is specifically used for:

Before the transcoding submodule transcodes the masked image data to obtain the screen data of the virtual item, obtains third image data, and the difference between the third image data and the masked image data The positional relationship between; the third image data is used as a specific element of the virtual item;

The transcoding submodule is specifically used for:

Transcoding the special effect image data to obtain virtual items.

The data acquisition module is specifically used for:

In a third aspect, an embodiment of the present application provides an electronic device, which includes:

Processor, communication interface, memory and communication bus. Among them, the processor, communication interface, and memory communicate with each other through the bus; the memory is used to store computer programs; the processor is used to execute the programs stored in the memory to realize The steps of the method for generating virtual items provided in the first aspect described above.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium in which a computer program is stored, and when the computer program is executed by a processor, the steps of the method for generating a virtual item provided in the first aspect are implemented.

In the solution provided by the embodiment of the present application, when the server detects the generation instruction of the virtual item, by acquiring the first image data, audio data, and playback timestamp of the first video corresponding to the generation instruction, it is possible to implement a preset virtual item For the shape information of the item, adjust the first image data, and transcode the adjusted first image data to obtain the picture data of the virtual item; and then according to the playback time stamp, the picture data and audio data of the virtual item are played synchronously, Obtain the virtual item corresponding to the generation instruction. In this solution, the screen data and audio data required to generate the virtual item are content extracted from the same video synchronized with audio and video, and when the virtual item is generated, the screen data and audio data of the virtual item are played according to the playback time. In this way, it can ensure the synchronization of the sound effect and the screen effect of the virtual item. It can be seen that through this solution, the effect of ensuring the synchronization of audio and video in the diversified display effects of the generated virtual items can be achieved.

Description of the drawings

In order to explain the embodiments of the present application and the technical solutions of the prior art more clearly, the following briefly introduces the drawings needed in the embodiments and the prior art. Obviously, the drawings in the following description are only the present For some of the embodiments of the application, for those of ordinary skill in the art, other drawings can be obtained from these drawings without creative work.

FIG. 1 is a schematic flowchart of a method for generating virtual items provided by an embodiment of the application;

2 is a schematic flowchart of a method for generating virtual items provided by another embodiment of the application;

FIG. 3 is a schematic structural diagram of a virtual item generating apparatus provided by an embodiment of the application;

4 is a schematic structural diagram of a virtual item generating apparatus provided by another embodiment of the application;

FIG. 5 is a schematic structural diagram of an electronic device provided by an embodiment of the application.

Detailed ways

In order to make the purpose, technical solutions, and advantages of the present application clearer, the following further describes the present application in detail with reference to the drawings and embodiments. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of this application.

The following first introduces a method for generating a virtual item according to an embodiment of the present application.

The method for generating virtual items provided by the embodiments of this application can be applied to a server corresponding to an Internet-related client. The server can include a desktop computer, a portable computer, an Internet TV, a smart mobile terminal, a wearable smart terminal, etc. This is not limited, and any server that can implement the embodiments of the present application falls within the protection scope of the embodiments of the present application.

In a specific application, there may be multiple clients related to the Internet. Exemplarily, the client related to the Internet may be a live broadcast client, or a social client, etc. Correspondingly, there can be multiple types of virtual items. For example, the virtual items may be gifts, pendants and other virtual items.

As shown in Fig. 1, the flow of the method for generating virtual items according to an embodiment of the present application may include the following steps:

S101: When a generation instruction of a virtual item is detected, first image data, audio data, and a playback time stamp of a first video corresponding to the generation instruction are acquired. Wherein, the first image data provides the picture of the first video; the audio data provides the sound of the first video; and the playback timestamp is an identifier for ensuring synchronization of the content of the first image data and the audio data.

In order to generate diversified virtual items, related personnel can issue generation instructions for different virtual items. Of course, the issuer of virtual item generation instructions is not limited to related personnel. For example, when a predetermined trigger is detected, the device automatically It is also reasonable to generate instructions for generating virtual items. In addition, since one generation instruction is used to generate a virtual item, and one first video is used to provide a picture and sound of a virtual item, one first video corresponds to one generation instruction. Correspondingly, when the generation instruction of the virtual item is detected, the first image data, audio data, and playback timestamp of the first video corresponding to the generation instruction can be acquired, so as to generate the corresponding generation instruction through the subsequent steps S102 to S103. Virtual item. Wherein, the different first videos may be independent videos that are not related to each other, or the different first videos may be different video segments obtained by segmentation from the same original video.

The first video may specifically include first image data and audio data, the first image data provides a picture of the first video, and the audio data provides a sound of the first video. Therefore, for example, the first image data may specifically be an image frame queue of the first video, and the audio data may specifically be an audio packet queue of the first video. Exemplarily, the method for acquiring the first image data and audio data may specifically include: using a preset model to read the first video to obtain an image frame queue and an audio packet queue, which are used as the first image data and audio data, respectively. Among them, the preset model is a model capable of video capture and decoding. Exemplarily, the preset model may be FFMPEG (Fast Forward MPEG, a tool with functions such as video capture, video format conversion, and video capture). At this time, FFMPEG can be used to read the first video to obtain First image data and audio data.

In addition, the playback time stamp is an identifier used to ensure that the content of the first image data and the audio data are synchronized. For example, the playback timestamp may include 1 second and 2 seconds, the playback timestamp of the video frame VF1 in the first image data is 1 second, and the playback timestamp of the video frame VF2 is 2 seconds; the playback of the audio package AP1 in the audio data The time stamp is 1 second, and the playback time stamp of the audio packet AP2 is 2 seconds. At this time, the first image data and audio data can be played synchronously according to the playback time stamp to ensure that the content of the first image data and audio data is synchronized: at 0-1 second, the playback time stamp is both 1 second. Frame VF1 and audio package AP1 are played synchronously. At 1-2 seconds, the video frame VF2 and audio package AP2 with a playback time stamp of 2 seconds are played synchronously. Based on this, the playback timestamp of the first video can be acquired, so that in the subsequent step S103, the screen data and audio data of the virtual item are synchronously played according to the acquired playback timestamp to obtain the virtual item corresponding to the generation instruction. In this way, the screen data and audio data of the virtual item can be played synchronously to ensure that the screen effect and sound effect of the generated virtual item are synchronized.

S102: Adjust the first image data according to the preset shape information about the virtual item to obtain the adjusted first image data, and transcode the adjusted first image data to obtain the screen data of the virtual item.

Wherein, the preset shape information about the virtual item is information about the shape of the virtual item, which may be of various types. Exemplarily, the preset shape information about the virtual item may include at least one of the preset size, color, brightness and shape of the virtual item and other information about the shape of the virtual item. Correspondingly, the adjustment of the first image data according to the preset shape information about the virtual item may specifically be: adjusting the appearance parameters of the video frame included in the first image data to be the same as the preset about the virtual item The shape information is the same. For example, if the preset shape information about the virtual item is size S1, color C1, brightness L1, and shape F1, then the size, color, brightness, shape and other parameters of the video frame included in the first image data can be set Values are adjusted to size S1, color C1, brightness L1, and shape F1.

In addition, in order to facilitate the use of the playback timestamp of the first video in the subsequent step S103 to synchronize the picture effect and sound effect of the virtual item, the adjusted first image data may be transcoded to obtain the picture data of the virtual item. At this time, the screen data of the virtual item is equivalent to a video without sound effects. In a specific application, transcoding the adjusted first image data is equivalent to encoding the adjusted first image data into a video without sound effects, and there may be multiple specific types. Exemplarily, an inter-frame compression encoding method may be used to transcode the adjusted first image data, or an intra-frame compression encoding method may be adopted to transcode the adjusted first image data. Any method that can transcode the adjusted first image data can be used in this application, and this embodiment does not limit this.

Among them, there is likely to be redundant data with great correlation between two consecutive image frames of the video. Therefore, inter-frame compression can be implemented by comparing data between different image frames on the time axis to increase the compression ratio and reduce The amount of data processing resources occupied. Intra-frame compression, when compressing an image frame, only considers the data of the image frame, and does not consider the redundant information between the image frame and adjacent image frames. It is similar to the compression of static images, and the compression ratio is relatively low. , The occupation of data processing resources is relatively large. Therefore, for application scenarios that need to reduce the occupation of data processing resources, such as the generation of virtual gifts in a live broadcast, inter-frame compression can be used to obtain virtual items, which is beneficial to reduce the occupation of data processing resources.

S103: Synchronously play the screen data and audio data of the virtual item according to the play timestamp to obtain the virtual item corresponding to the generation instruction.

Since the playback time stamp can ensure the synchronization of the content of the first image data and audio data, the screen data and audio data of the virtual item are played synchronously according to the playback time stamp to obtain the virtual item corresponding to the generation instruction, which can ensure that the generated virtual item The picture effects of the items are synchronized with the sound effects.

Exemplarily, according to the playback time stamp, the screen data and audio data of the virtual item are played synchronously to obtain the virtual item corresponding to the generation instruction, which may specifically include: sending the screen data of the virtual item to the image display device, so that the image is displayed The device selects data corresponding to the playback time stamp from the screen data of the virtual item for display; and synchronously sends the audio data to the audio output device, so that the audio output device plays the data corresponding to the playback time stamp in the audio data. For example, the playback timestamp from the 0th to the 10th second is T1. Specifically, the duration 10S or the number 1 or the number 0 can be identified as the timestamp T1 to mark the 0th to 10th time period. second. In the screen data of the virtual item, the data corresponding to the playback time stamp T1 is the video frame VF0 from the 0th second to the 10th second, and the data corresponding to the playback time stamp T1 in the audio data is the audio packet AP0 of the video frame VF0. While the image display device starts to display the video frame VF0, the audio playback device may start to play the audio package AP0.

In the solution provided by the embodiment of the present application, when the server detects the generation instruction of the virtual item, by acquiring the first image data, audio data, and playback timestamp of the first video corresponding to the generation instruction, it is possible to implement a preset virtual item The shape information of the item, the first image data is adjusted, and the adjusted first image data is transcoded to obtain the picture data of the virtual item; and then the picture data and audio data of the virtual item are played according to the playback timestamp to obtain Generate virtual items corresponding to the command. In this solution, the screen data and audio data required to generate the virtual item are content extracted from the same video synchronized with audio and video, and when the virtual item is generated, the screen data and audio data of the virtual item are played according to the playback time. In this way, it can ensure the synchronization of the sound effect and the screen effect of the virtual item. It can be seen that through this solution, the effect of ensuring the synchronization of audio and video in the diversified display effects of the generated virtual items can be achieved.

As shown in FIG. 2, the flow of the method for generating virtual items according to another embodiment of the present application, the method may include:

S201: When a generation instruction of a virtual item is detected, obtain the first image data, audio data, and playback timestamp of the first video corresponding to the generation instruction.

S201 is the same step as S101 in the embodiment of FIG. 1 of this application, and will not be repeated here. For details, refer to the description of the embodiment of FIG. 1 of this application.

S202: Adjust the first image data according to preset shape information about the virtual item to obtain adjusted first image data.

The step of adjusting the first image data in S202 is the same as the step of adjusting the first image data in S102 of the embodiment of FIG. 1 of this application, which will not be repeated here.

S203: Acquire second image data of the second video whose screen color is a transparent color; the second image data provides a screen of the second video.

In specific applications, a transparent part can be set in the screen of the virtual item to ensure that the display effect of the virtual item is more realistic and three-dimensional. For example, for a virtual item: a virtual gift car, the color of the screen other than the car itself can be set to a transparent color, so that the screen content of the virtual item displayed is the car itself, and there will be no large black or white areas. Content that has nothing to do with the car itself. For this reason, the data related to the transparent color can be used to mask the area of the virtual item that needs to have a transparent effect. Wherein, the data related to the transparent color may be second image data of the second video whose screen color is a transparent color, and the second image data provides a picture of the second video, or may be an image whose screen color is a transparent color.

The second image data and the first image data are both video providing pictures, and the difference is that the picture color of the second image data is a transparent color, and the second video picture is provided. Therefore, when the second image data of the second video is used as the data related to the transparent color, the processing logic similar to that of the first image data can be used. The difference is that the processing object is replaced with the second video and the second image data. Add separate processing logic. For example, a preset model for obtaining the first image data can be used to read the second video to obtain the second image data. Moreover, step S203 can be performed before or after step S202.

S204: Adjust the second image data according to preset shape information about the virtual item to obtain adjusted second image data.

Step S204 is similar to the process of adjusting the first image data in S102 of the embodiment of FIG. 1 of the present application, except that the object of adjustment in S204 is the second image data. Moreover, when step S203 is executed before step S202, step S204 may be executed before step S202, or simultaneously with step S202. Among them, adjusting two kinds of image data at the same time can improve efficiency. The same content will not be repeated here. For details, please refer to the description of the embodiment in FIG. 1 of the present application.

S205: Acquire a transparent position belonging to a transparent area in the adjusted first image data.

In a specific application, for a certain virtual item, the virtual item needs to be set to the position of the transparent part of the transparent color, that is, the transparent position belonging to the transparent area in the adjusted first image data. Therefore, the transparent position belonging to the transparent area in the adjusted first image data can be obtained, and used for masking in the subsequent step S206 to obtain the masked image data with the transparent color area. Also, for example, the transparent position may specifically be the two-dimensional coordinates of the transparent area in the two-dimensional coordinate system of the adjusted first image data.

Wherein, there may be multiple ways to obtain the transparent position belonging to the transparent area in the adjusted first image data.

Exemplarily, in an implementation manner, the adjusted shape and size of the first image data are preset shape information about the virtual item, which is fixed, and the position of the non-transparent area of the virtual item is preset , The transparent area is the area other than the non-transparent area in the adjusted first image data. Therefore, according to the position of the non-transparent area and the adjusted shape and size of the first image data, a position different from the position of the non-transparent area in the adjusted first image data can be stored as a transparent position in advance. Correspondingly, the pre-stored transparent position can be directly read.

Exemplarily, in another implementation manner, the position of the non-transparent area of the virtual item may be read, and a position different from the position of the non-transparent area may be determined from the adjusted first image data as the transparent position.

Any method that can obtain the transparent position belonging to the transparent area in the adjusted first image data can be used in this application, and this embodiment does not limit this.

S206: Using the adjusted second image data, mask the transparent position in the adjusted first image data into a transparent color to obtain masked image data.

The adjusted second image data corresponds to the adjusted first image bit by bit, and has the same size and shape, and the adjusted second image data is a transparent color. Therefore, the adjusted second image data can be used to mask the transparent position in the adjusted first image data into a transparent color to obtain masked image data. In a specific application, the adjusted second image data is used to mask the transparent position in the adjusted first image data into a transparent color, and there are multiple ways to obtain the masked image data. The following is a specific description in the form of an alternative embodiment.

In an alternative embodiment, the above step S206: using the adjusted second image data, mask the transparent position in the adjusted first image data to a transparent color to obtain the masked image data, specifically It can include the following steps:

At the transparent position in the adjusted first image data, the adjusted second image data is displayed to obtain the masked image data.

Wherein, displaying the adjusted second image data at the transparent position in the adjusted first image data specifically refers to: displaying the adjusted transparent position in each video frame included in the adjusted first image data The second image data. In addition, the transparent position in each video frame included in the adjusted first image data can display one video frame included in the second image data.

The masked image data includes adjusted second image data and adjusted first image data, where the display effect of the adjusted second image data is a transparent color; the display effect of the adjusted first image data is The screen content of the virtual item itself and the area that needs to be set to a transparent color. Therefore, the adjusted second image data can be displayed at the transparent position in the adjusted first image data, so that the area in the adjusted first image data that needs to be set to a transparent color for the adjusted second image data The transparent color is displayed, and the masked image data is obtained.

In a specific application, displaying the adjusted second image data at the transparent position in the adjusted first image data to obtain the masked image data may specifically include: combining the adjusted first image data with The adjusted second image data is copied to the buffer of the display module, so that while the display module displays the first image data, it simultaneously displays the adjusted second image data in a transparent position in the adjusted first image data to achieve Dual display. For example, in the Android (Android) operating system, the adjusted first image data and the adjusted second image data can be copied to the cache of the display module, so that the display module is in the NativeWindow (local window). Dual display.

In this optional embodiment, the acquisition of the masked image data can be achieved by dual display of the adjusted second image and the adjusted first image, without the need for complex image channel filling and rendering processes. Therefore, It has the advantage of a relatively simple implementation process, and can improve the efficiency of generating virtual items.

In another alternative embodiment, the above step S206: using the adjusted second image data, mask the transparent position in the adjusted first image data to a transparent color to obtain the masked image data, Specifically, it can include the following steps:

Use the adjusted transparent position of the first image data as a transparent channel;

Filling the pixels at the transparent position in the adjusted second image data into the transparent channel to obtain the transparent channel data;

Rendering the transparent channel data and the non-transparent channel data in the adjusted first image data to obtain the masked image data.

Wherein, each pixel in the video frame included in the adjusted first image data, the position of the video frame in the adjusted first image data belongs to the transparent position or the non-transparent position of the adjusted first image data; here Basically, when the pixel at the transparent position has a transparent color, the picture effect at that position is a transparent effect. Therefore, the adjusted transparent position of the first image data can be used as the transparent channel, so that the transparent channel can be filled with pixels with transparent colors in subsequent steps to achieve a transparent effect. In addition, the adjusted second image data corresponds to the pixels of the adjusted first image data bit by bit. Therefore, pixels in the transparent position in the adjusted second image data can be filled into the transparent channel to obtain the transparent channel Data, thereby obtaining adjusted first image data including a transparent channel and a non-transparent channel. On this basis, in order to obtain the masked image data that has a transparent color at the transparent position and can display the screen content of the virtual item, the transparent channel data and the transparent channel data can be adjusted according to the pixel distribution position in the adjusted first image data. Non-transparent channel data is rendered.

Exemplarily, rendering the transparent channel data and the non-transparent channel data in the adjusted first image data to obtain the masked image data may specifically include: changing the transparent channel in the adjusted first image data The texture data is generated from the data and the non-transparent channel data, and the Texture data is rendered using OpenGL (Open Graphics Library) to obtain the masked image data. In addition, the execution subject for rendering may specifically be a GPU (Graphics Processing Unit, image processor) to improve the efficiency of acquiring the masked image.

In this optional embodiment, the acquisition of the masked image data is to fill the adjusted pixels of the second image into the transparency channel of the adjusted first image, so as to make the adjustment transparent to the first image. The channel data and the non-transparent channel data are rendered to obtain the masked image data. Since transparent channel data and non-transparent channel data with different display effects can be rendered in a targeted manner, compared with the way of obtaining image data after masking without rendering, it is equivalent to performing secondary rendering, which can improve subsequent The display quality of the virtual item obtained based on the masked image data.

S207: Transcoding the masked image data to obtain screen data of the virtual item.

The foregoing step S207 is similar to the process of transcoding the adjusted first image data in S102 of the embodiment of FIG. 1 of the present application to obtain the image data of the virtual item, except that the object of the transcoding in S207 is the masked image data; And at this time, the screen data of the virtual item is screen data for masking the transparent area that needs to be set to a transparent color to a transparent color. The same content will not be repeated here. For details, please refer to the description of the embodiment in FIG. 1 of the present application.

S208: Synchronously play the screen data and audio data of the virtual item according to the play timestamp.

S208 is a similar step to S103 in the embodiment of FIG. 1 of the present application. The difference is that the screen data of the virtual item in S208 is the screen data that masks the transparent area that needs to be set to a transparent color as a transparent color, which can improve the generated virtual item The sense of three-dimensionality and realism. The same parts will not be repeated here. For details, please refer to the description of the embodiment in FIG. 1 of this application.

In the embodiment of FIG. 2 described above, by acquiring the second image data of the second video, without adding a separate processing logic for the second image, masking the adjusted transparent position of the first image data, Obtain the masked image, and then use the masked image to obtain the virtual item corresponding to the survival instruction, so as to realize the transparent effect of the transparent position of the virtual item, and improve the three-dimensional and real sense of the virtual item.

Optionally, in the foregoing step S206: transcoding the masked image data to obtain the virtual item screen data, the method for generating virtual items provided in the embodiment of the present application may further include the following steps:

Acquire the third image data and the positional relationship between the third image data and the masked image data; the third image data is used as a specific element of the virtual item;

According to the acquired position relationship, add third image data to the masked image data to obtain special effect image data;

Correspondingly, the above step S206: transcoding the masked image data to obtain the screen data of the virtual item may specifically include: transcoding the special effect image data to obtain the screen data of the virtual item.

Wherein, the third image data may specifically be a vector image. The specific elements of a virtual item can be of various types. Exemplarily, the specific element may be a user avatar, text input by the user, special effects, and so on. The acquisition of the positional relationship between the third image data and the masked image data may be multiple. Exemplarily, the position relationship may be fixed, and therefore, the pre-stored position relationship can be directly read. For example, the positional relationship may be the upper left corner or the upper right corner of the masked image data, etc., and the positional relationship is stored in advance so as to be read when the special effect image data is acquired. Or, for example, the positional relationship may be to search for the added position corresponding to the acquired type of the third image from the preset correspondence between the type of the third image data and the adding position, as the third image data and the shadow The positional relationship between the image data after the mask. For example, when the type of the third image data is user information, such as a user avatar and text input by the user, the corresponding adding position may be a transparent position, or a virtual gift boundary position, etc., which does not block the virtual gift position. When the type of the third image data is a special effect, such as a snow special effect, the corresponding adding position may be a transparent position or a designated position that does not block the virtual gift.

Any method for acquiring the positional relationship between the third image data and the masked image data can be used in this application, and this embodiment does not limit this.

On this basis, since the positional relationship can indicate the position of the third image data in the masked image data, according to the positional relationship, the third image data is added to the masked image data to obtain a special effect image data. Specifically, according to the adding position indicated by the positional relationship, the third image data may be added at the adding position of the masked image data to obtain the special effect image data. For example, the third image data is a user avatar, and the position relationship is that the third image data is at the upper left corner of the masked image data. At this time, a user avatar can be added to the upper left corner of the masked image data to obtain special effect image data with transparent effects and containing the user avatar.

Correspondingly, the special effect image data can be transcoded to obtain the picture data of the virtual item. This step is similar to the process of transcoding the adjusted first image data in S102 of the embodiment of FIG. 1 of the present application to obtain the image data of the virtual item, except that the object of the transcoding in this step is special effect image data. The same content will not be repeated here. For details, please refer to the description of the embodiment in FIG. 1 of the present application.

In this optional embodiment, by adding specific elements to the virtual item, the richness of the content expressed by the virtual item can be increased, and the diversification of the display effect can be improved. In addition, the specific element takes the form of the third image data. Therefore, it is relatively convenient to add the specific element in the form of the third image data to the masked image data that is also image data.

Optionally, the above-mentioned positional relationship is the corresponding relationship between the elements of the second pixel matrix and the elements of the first pixel matrix; the second pixel matrix is the matrix of pixels in the third image data; the first pixel matrix is the masked Pixel matrix of pixels in image data;

Correspondingly, the above step of adding third image data to the masked image data according to the position relationship to obtain the special effect image data may specifically include:

Respectively transform the masked image data and the third image data into a first matrix and a second matrix;

In the first matrix, according to the position relationship, add elements in the second matrix to obtain the third matrix;

Convert the third matrix into image data to obtain special effect image data.

Among them, the matrix of the pixels in the third image data: the second pixel matrix takes each pixel of the third image data as an element of the matrix, and uses the position of each pixel of the third image data in the third image data as The position of the elements of the matrix in the matrix, the resulting matrix. Similarly, the first pixel matrix takes each pixel of the masked image data as an element of the matrix, and uses the position of each pixel of the masked image data in the masked image data as an element in the The position in the matrix, the resulting matrix. Therefore, the correspondence between the elements of the second pixel matrix and the elements of the first pixel matrix can be taken as the positional relationship between the third image data and the masked image data.

On this basis, in order to add third image data to the masked image data according to the position relationship to obtain special effect image data, it is necessary to convert the masked image data into the first matrix and the third image data into The second matrix, so that in the first matrix, add elements in the second matrix according to the position relationship to obtain a third matrix; and convert the third matrix into image data to obtain special effect image data. Exemplarily, the positional relationship is that the elements S11 to S36 in the second matrix correspond to the elements F11 to F36 in the upper left corner of the first matrix bit by bit. Therefore, the first matrix can be added to the positions of the elements F11 to F36 in the first matrix. The elements S11 to S36 in the second matrix obtain the third matrix.

In this optional embodiment, the third image data as a specific element is added according to the pixel position. Compared with the traditional adding by drawing, the complexity of adding and the amount of data to be processed are relatively reduced. , Can reduce the consumption of data processing resources.

Corresponding to the foregoing method embodiment, an embodiment of the present application also provides a virtual item generation device.

As shown in FIG. 3, the virtual item generation device provided by an embodiment of the present application is applied to a server, and the device may include:

The data acquisition module 301 is configured to acquire the first image data, audio data, and playback time stamp of the first video corresponding to the generation instruction when the generation instruction of the virtual item is detected; wherein the first image data provides The picture of the first video; the audio data provides the sound of the first video; the playback timestamp is an identifier used to ensure that the content of the first image data and the audio data are synchronized;

The screen data generation module 302 is configured to adjust the first image data according to preset shape information about the virtual item to obtain adjusted first image data, and transcode the adjusted first image data to obtain Screen data of virtual items;

The virtual item display module 303 is configured to synchronously play the screen data and the audio data of the virtual item according to the playback timestamp to obtain the virtual item corresponding to the generation instruction.

As shown in Fig. 4, the device for generating virtual items provided by another embodiment of the present application is applied to a server, and the device may include:

The data acquisition module 401 is configured to acquire the first image data, audio data, and playback timestamp of the first video corresponding to the generation instruction when the generation instruction of the virtual item is detected; wherein the first image data provides The picture of the first video; the audio data provides the sound of the first video; the playback timestamp is an identifier used to ensure that the content of the first image data and the audio data are synchronized; the color of the acquired picture is Second image data of a second video in a transparent color; the second image data provides a picture of the second video;

The screen data generation module 402 includes: a mask sub-module 4021 and a transcoding sub-module 4022;

The mask sub-module 4021 is configured to adjust the second image data according to preset shape information about the virtual item to obtain adjusted second image data; obtain the adjusted first image data that is transparent The transparent position of the region; using the adjusted second image data to mask the transparent position in the adjusted first image data into a transparent color to obtain the masked image data;

The transcoding sub-module 4022 is used to transcode the masked image data to obtain screen data of virtual items;

The virtual item display module 403 is configured to synchronously play the screen data and the audio data of the virtual item according to the playback time stamp to obtain the virtual item corresponding to the generation instruction.

Optionally, the mask submodule 4021 is specifically used for:

Optionally, the data acquisition module 401 is specifically configured to:

Before the transcoding submodule 4022 transcodes the masked image data to obtain the screen data of the virtual item, acquire third image data, as well as the third image data and the masked image data The positional relationship between; the third image data is used as a specific element of the virtual item;

Correspondingly, the transcoding submodule 4022 is specifically used for:

Transcoding the special effect image data to obtain virtual items.

Optionally, the positional relationship is a correspondence relationship between elements of a second pixel matrix and elements of a first pixel matrix; the second pixel matrix is a pixel matrix of pixels in the third image data; A pixel matrix is the pixel matrix of pixels in the masked image data;

The data acquisition module 401 is specifically used for:

Corresponding to the foregoing embodiment, an embodiment of the present application also provides an electronic device. As shown in FIG. 5, the device may include:

A processor 501, a communication interface 502, a memory 503, and a communication bus 504, wherein the processor 501, the communication interface 502, and the memory communicate with each other through the communication bus 504 through 503;

The memory 503 is used to store computer programs;

The processor 501 is configured to implement the steps of any virtual item generation method in the foregoing embodiment when executing the computer program stored in the foregoing memory 503.

It is understandable that the electronic device in the embodiment of FIG. 5 of the present application may specifically be a server corresponding to a client related to the Internet.

The foregoing memory may include RAM (Random Access Memory, random access memory), and may also include NVM (Non-Volatile Memory, non-volatile memory), such as at least one disk storage. Optionally, the memory may also be at least one storage device located far away from the foregoing processor.

The above-mentioned processor may be a general-purpose processor, including CPU (Central Processing Unit), NP (Network Processor, network processor), etc.; it may also be DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array, field programmable gate array) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

The computer-readable storage medium provided by an embodiment of the present application is included in an electronic device. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, it realizes the generation of any virtual item in the above-mentioned embodiments. Method steps.

In another embodiment provided by the present application, a computer program product containing instructions is also provided, which when running on a computer, causes the computer to execute the method for generating virtual items described in any of the above embodiments.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in the form of a computer program product in whole or in part. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website, computer, server, or data center. Transmission to another website, computer, server, or data center via wired (such as coaxial cable, optical fiber, DSL (Digital Subscriber Line)) or wireless (such as infrared, radio, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center integrated with one or more available media. The available medium may be a magnetic medium, (for example, a floppy disk) , Hard disk, magnetic tape), optical media (for example: DVD (Digital Versatile Disc)), or semiconductor media (for example: SSD (Solid State Disk, solid state drive)), etc.

In this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such existence between these entities or operations. The actual relationship or order. Moreover, the terms "include", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device that includes a series of elements includes not only those elements, but also includes Other elements of, or also include elements inherent to this process, method, article or equipment. If there are no more restrictions, the element defined by the sentence "including a..." does not exclude the existence of other same elements in the process, method, article, or equipment including the element.

The various embodiments in this specification are described in a related manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and electronic device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and for related parts, please refer to the part of the description of the method embodiments.

The above are only preferred embodiments of the present application, and are not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of this application are all included in the protection scope of this application.

The above are only the preferred embodiments of this application and are not intended to limit this application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall be included in this application Within the scope of protection.

Claims

A method for generating virtual items is characterized by being applied to a server, and the method includes:

When the generation instruction of the virtual item is detected, acquiring the first image data, audio data, and playback timestamp of the first video corresponding to the generation instruction; wherein the first image data provides a picture of the first video; The audio data provides the sound of the first video; the playback timestamp is an identifier used to ensure synchronization of the content of the first image data and the audio data;

Adjusting the first image data according to preset shape information about the virtual item to obtain adjusted first image data, and transcoding the adjusted first image data to obtain the screen data of the virtual item;

According to the playback time stamp, the screen data and the audio data of the virtual item are played synchronously to obtain the virtual item corresponding to the generation instruction.
The method according to claim 1, characterized in that, before the step of transcoding the adjusted first image data to obtain the screen data of the virtual item, the method further comprises:

Acquiring second image data of a second video whose screen color is a transparent color; the second image data provides a screen of the second video;

Adjust the second image data according to preset shape information about the virtual item to obtain adjusted second image data;

The step of transcoding the adjusted first image data to obtain the screen data of the virtual item includes:

Acquiring a transparent position belonging to a transparent area in the adjusted first image data;

Masking the transparent position in the adjusted first image data to a transparent color by using the adjusted second image data to obtain masked image data;

Transcoding the masked image data to obtain the image data of the virtual item.
2. The method according to claim 2, wherein the adjusted second image data is used to mask the transparent position in the adjusted first image data with a transparent color to obtain a mask The image data steps after the mask include:

In the transparent position in the adjusted first image data, the adjusted second image data is displayed to obtain the masked image data.
2. The method according to claim 2, wherein the adjusted second image data is used to mask the transparent position in the adjusted first image data with a transparent color to obtain a mask The steps for the image data behind the mask include:

Using the transparent position of the adjusted first image data as a transparent channel;

Filling the pixels in the transparent position in the adjusted second image data into the transparent channel to obtain transparent channel data;

Taking pixels other than the pixels in the transparent position in the adjusted first image data as non-transparent channel data;

Rendering the transparent channel data and the non-transparent channel data in the adjusted first image data to obtain masked image data.
2. The method according to claim 2, wherein before the step of transcoding the masked image data to obtain the screen data of the virtual item, the method further comprises:

Acquiring third image data and the positional relationship between the third image data and the masked image data; the third image data is used as a specific element of the virtual item;

Adding the third image data to the masked image data according to the position relationship to obtain special effect image data;

The step of transcoding the masked image data to obtain the image data of the virtual item includes:

Transcoding the special effect image data to obtain the screen data of the virtual item.
The method according to claim 5, wherein the positional relationship is the correspondence between the elements of the second pixel matrix and the elements of the first pixel matrix; the second pixel matrix is the third image data A pixel matrix of middle pixels; the first pixel matrix is a pixel matrix of pixels in the masked image data;

The step of adding the third image data to the masked image data according to the position relationship to obtain special effect image data includes:

Respectively transforming the masked image data and the third image data into a first matrix and a second matrix;

In the first matrix, add elements in the second matrix according to the position relationship to obtain a third matrix;

The third matrix is converted into image data to obtain special effect image data.
A device for generating virtual items is characterized in that it is applied to a server, and the device includes:

The data acquisition module is configured to acquire the first image data, audio data, and playback timestamp of the first video corresponding to the generation instruction when the generation instruction of the virtual item is detected; wherein, the first image data provides the The picture of the first video; the audio data provides the sound of the first video; the playback timestamp is an identifier used to ensure that the content of the first image data and the audio data are synchronized;

The screen data generation module is used to adjust the first image data according to the preset shape information about the virtual item to obtain the adjusted first image data, and to transcode the adjusted first image data to obtain the virtual Picture data of the item;

The virtual item display module is configured to synchronously play the screen data and the audio data of the virtual item according to the playback time stamp to obtain the virtual item corresponding to the generation instruction.
The device according to claim 7, wherein the data acquisition module is specifically configured to:

Before transcoding the adjusted first image data to obtain the screen data of the virtual item, obtain the second image data of the second video whose screen color is a transparent color; the second image data provides the second The picture of the video;

The picture data generation module includes: a mask sub-module and a transcoding sub-module;

The mask sub-module is configured to adjust the second image data according to preset shape information about the virtual item to obtain adjusted second image data; obtain the adjusted first image data belonging to a transparent area The transparent position of; using the adjusted second image data, mask the transparent position in the adjusted first image data to a transparent color to obtain the masked image data;

The transcoding sub-module is used to transcode the masked image data to obtain the screen data of the virtual item.
The device according to claim 8, wherein the mask sub-module is specifically used for:

In the transparent position in the adjusted first image data, the adjusted second image data is displayed to obtain the masked image data.
The device according to claim 8, wherein the mask sub-module is specifically used for:

Using the transparent position of the adjusted first image data as a transparent channel;

Filling the pixels in the transparent position in the adjusted second image data into the transparent channel to obtain transparent channel data;

Taking pixels other than the pixels in the transparent position in the adjusted first image data as non-transparent channel data;

Rendering the transparent channel data and the non-transparent channel data in the adjusted first image data to obtain masked image data.
The device according to claim 8, wherein the data acquisition module is specifically configured to:

Before the transcoding submodule transcodes the masked image data to obtain the screen data of the virtual item, obtains third image data, and the difference between the third image data and the masked image data The positional relationship between; the third image data is used as a specific element of the virtual item;

Adding the third image data to the masked image data according to the position relationship to obtain special effect image data;

The transcoding submodule is specifically used for:

Transcoding the special effect image data to obtain virtual items.
The device according to claim 11, wherein the positional relationship is a correspondence between elements of the second pixel matrix and elements of the first pixel matrix; the second pixel matrix is the third image data A pixel matrix of middle pixels; the first pixel matrix is a pixel matrix of pixels in the masked image data;

The data acquisition module is specifically used for:

Respectively transforming the masked image data and the third image data into a first matrix and a second matrix;

In the first matrix, add elements in the second matrix according to the position relationship to obtain a third matrix;

The third matrix is converted into image data to obtain special effect image data.
An electronic device characterized by comprising a processor, a communication interface, a memory, and a communication bus. The processor, the communication interface, and the memory communicate with each other through the bus; the memory is used to store computer programs; the processor is used The program stored in the memory is executed to realize the method steps according to any one of claims 1-6.
A computer-readable storage medium, characterized in that a computer program is stored in the storage medium, and when the computer program is executed by a processor, the method steps according to any one of claims 1-6 are realized.