CN110493630A - The treating method and apparatus of virtual present special efficacy, live broadcast system - Google Patents

The treating method and apparatus of virtual present special efficacy, live broadcast system Download PDF

Info

Publication number
CN110493630A
CN110493630A CN201910859930.5A CN201910859930A CN110493630A CN 110493630 A CN110493630 A CN 110493630A CN 201910859930 A CN201910859930 A CN 201910859930A CN 110493630 A CN110493630 A CN 110493630A
Authority
CN
China
Prior art keywords
special efficacy
present
live video
client
position information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910859930.5A
Other languages
Chinese (zh)
Other versions
CN110493630B (en
Inventor
杨克敏
陈杰
欧燕雄
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou Cubesili Information Technology Co Ltd
Original Assignee
Guangzhou Huaduo Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Huaduo Network Technology Co Ltd filed Critical Guangzhou Huaduo Network Technology Co Ltd
Priority to CN201910859930.5A priority Critical patent/CN110493630B/en
Publication of CN110493630A publication Critical patent/CN110493630A/en
Priority to PCT/CN2019/125929 priority patent/WO2021047094A1/en
Application granted granted Critical
Publication of CN110493630B publication Critical patent/CN110493630B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/2187—Live feed
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/254—Management at additional data server, e.g. shopping server, rights management server
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
    • H04N21/4316—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/435—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
    • H04N21/4788—Supplemental services, e.g. displaying phone caller identification, shopping application communicating with other users, e.g. chatting

Abstract

Treating method and apparatus, live broadcast system, equipment and the storage medium of virtual present special efficacy provided by the embodiments of the present application, it is related to direct seeding technique field, the processing method gives instruction by receiving the virtual present that spectator client is sent, according to the characteristic area given instruction and obtain corresponding target special efficacy present and determining target special efficacy present；Synthesising position information of the target special efficacy present on live video is determined according to characteristic area；Live video and synthesising position information are sent to client, is synthesized and is shown so that target special efficacy present is added to live video according to synthesising position information by spectator client.Synthesising position information is carried out coding encapsulation outside live video using main broadcaster's client by the application, it is transmitted independently of live video, it decodes to obtain synthesising position information in spectator client, it is shown convenient for effect of the spectator client to virtual special efficacy present and carries out secondary editor, be conducive to improve the effect that special efficacy is shown.

Description

The treating method and apparatus of virtual present special efficacy, live broadcast system
Technical field
The invention relates to direct seeding technique fields, in particular to a kind of processing method of virtual present special efficacy With device, live broadcast system, a kind of computer equipment and storage medium are further related to.
Background technique
With network technical development, real-time video exchange such as network direct broadcasting, Video chat room become one kind and increasingly flow Capable entertainment way.In real-time video communication process, user can be increased by way of giving virtual present and showing special efficacy Between interactivity.
For example, main broadcaster user is broadcast live in direct broadcasting room in live scene, spectators user watches in spectator client and leading The live streaming process broadcast.In order to increase the interactivity between main broadcaster user and spectators user, spectators user can choose specific mesh Mark special efficacy present is given to main broadcaster, and target special efficacy present is added to the certain bits of main broadcaster's picture according to corresponding amusement template It sets, shows corresponding special efficacy.
The methods of exhibiting of existing present special efficacy is that present special efficacy is synthesized to video frame by main broadcaster's client, will be contained The video frame of present special efficacy is put into video and is streamed to other main broadcaster's clients or spectator client progress special efficacy displaying, above-mentioned exhibition The mode of showing makes present special efficacy be solidificated in video area, be unfavorable for it is subsequent secondary adjustment is carried out for present special efficacy, affect whole Body display effect.
Summary of the invention
The purpose of the application aims to solve at least one of above-mentioned technological deficiency, is especially detrimental to the progress of virtual present special efficacy Secondary adjustment, the bad problem of bandwagon effect.
In a first aspect, the embodiment of the present application provides a kind of processing method of virtual present special efficacy, comprising the following steps:
Receive virtual present gives instruction, gives the corresponding target special efficacy present of instruction acquisition and determining institute according to described State the characteristic area of target special efficacy present；
Synthesising position information of the target special efficacy present on the live video is determined according to the characteristic area；
The live video and the synthesising position information are sent to spectator client, so that the spectator client The target special efficacy present is added to the live video according to the synthesising position information to be synthesized and be shown.
In one embodiment, described to give the corresponding target special efficacy present of instruction acquisition according to and determine the target The step of characteristic area of special efficacy present includes:
Obtain target special efficacy present mark；
Target special efficacy present is obtained according to the target special efficacy present identifier lookup, determines that the target special efficacy present institute is right The characteristic area answered.
In one embodiment, described to determine the target special efficacy present on the live video according to the characteristic area Synthesising position information the step of include:
Obtain the current video frame image of the live video；
Extract the character contour key point in the current video frame image；
The characteristic area corresponding mesh on the current video frame image is determined according to the character contour key point Cursor position, to be synthesized in the target position to the target special efficacy present.
In one embodiment, described that the live video and the synthesising position information are sent to spectator client Step includes:
The live video and the synthesising position information are subjected to coding and are packaged into live video flow data, and passes through clothes Device be engaged in for the live video stream data forwarding to the spectator client.
In one embodiment, the target special efficacy present is the special efficacy present of Three-dimensional Display form.
In one embodiment, the synthesising position information includes: face information, human body contour outline information, gesture information and people At least one of body bone information.
Second aspect, the embodiment of the present application provide a kind of processing method of virtual present special efficacy, comprising the following steps:
Virtual present, which is sent, to main broadcaster's client gives instruction；Wherein, main broadcaster's client gives instruction according to It obtains corresponding target special efficacy present and determines the characteristic area of the target special efficacy present, institute is determined according to the characteristic area State synthesising position information of the target special efficacy present on the live video；
Receive the live video flow data that main broadcaster's client is sent；It wherein, include live streaming in the live video flow data The synthesising position information of video and target special efficacy present；
By the live video stream data forwarding to spectator client；Wherein, the spectator client is according to the synthesis The target special efficacy present is added to the live video and is synthesized and be shown by location information.
In one embodiment, described that the target special efficacy present is added to by the live streaming according to the synthesising position information Video is synthesized and includes: the step of being shown
Obtain the current video frame image of the live video；
The target special efficacy present is added into the current video frame image according to the synthesising position information, obtains special efficacy Frame image；
The special efficacy frame image rendering is shown to special efficacy display area.
The third aspect, the embodiment of the present application provide a kind of processing unit of virtual present special efficacy, comprising:
Feature obtains module, gives instruction for receive virtual present, gives instruction acquisition corresponding mesh according to described It marks special efficacy present and determines the characteristic area of the target special efficacy present；
Information determination module, for determining the target special efficacy present in the live video according to the characteristic area Synthesising position information；
Information sending module, for the live video and the synthesising position information to be sent to spectator client, So that the spectator client according to the synthesising position information by the target special efficacy present be added to the live video into Row is synthesized and is shown.
Fourth aspect, the embodiment of the present application provide a kind of processing unit of virtual present special efficacy, comprising:
Instruction sending module, for giving instruction to main broadcaster's client transmission virtual present；Wherein, the main broadcaster client End gives instruction according to and obtains corresponding target special efficacy present and determine the characteristic area of the target special efficacy present, according to The characteristic area determines synthesising position information of the target special efficacy present on the live video；
Data reception module, for receiving the live video flow data of main broadcaster's client transmission；Wherein, the live video It include the synthesising position information of live video and target special efficacy present in flow data；
Data forwarding module is used for the live video stream data forwarding to spectator client；Wherein, the spectators visitor The target special efficacy present is added to the live video according to the synthesising position information and is synthesized and open up by family end Show.
5th aspect, the embodiment of the present application provide a kind of live broadcast system, including main broadcaster's client, spectator client and with And server, main broadcaster's client are communicatively coupled with the spectator client by network through the server；
The server gives instruction for receive virtual present that the spectator client is sent, and objective to main broadcaster Instruction is given described in sending in family end；
Main broadcaster's client described give instruction and obtains target special efficacy present mark for receiving；According to the mesh Mark special efficacy present identifier lookup obtains target special efficacy present, determines characteristic area corresponding to the target special efficacy present；According to The characteristic area determines synthesising position information of the target special efficacy present on the live video；By the synthesising position Information and the live video are encoded into live video flow data and are sent to server；
The server is also used to the live video stream data forwarding to the spectator client；
The spectator client, for receiving live video flow data and target special efficacy present, from the live video stream The synthesising position information of live video and the target special efficacy present is obtained in data；It will be described according to the synthesising position information Target special efficacy present is added to the live video and is synthesized and be shown.
6th aspect, the embodiment of the present application provide a kind of computer equipment, including memory, processor and are stored in On reservoir and the computer program that can run on a processor, the processor realize such as any of the above-described reality when executing described program The step of applying the processing method of virtual present special efficacy described in example.
7th aspect, the embodiment of the present application provide a kind of storage medium comprising computer executable instructions, the meter Calculation machine executable instruction as computer processor when being executed for executing the virtual present special efficacy as described in above-mentioned any embodiment Processing method the step of.
Treating method and apparatus, live broadcast system, equipment and the storage medium of virtual present special efficacy provided by the above embodiment, The virtual present sent by reception spectator client gives instruction, obtains corresponding target special efficacy present according to instruction is given And determine the characteristic area of target special efficacy present；Synthesis position of the target special efficacy present on live video is determined according to characteristic area Confidence breath；Live video and synthesising position information are sent to client, so that spectator client is according to synthesising position information Target special efficacy present is added to live video to be synthesized and be shown.Relative to main broadcaster's client directly by target special efficacy Present is synthesized to live video, retransmits to each spectator client special to play virtual present in the video area of spectator client Effect, synthesising position information is carried out coding encapsulation outside live video using main broadcaster's client by the program, independently of live video Transmitted so that spectator client decodes to obtain synthesising position information, convenient for the effect to virtual special efficacy present show into The secondary editor of row, so that being conducive to spectator client realizes layering special effect processing and the special efficacy exhibition across live video play area Show, improves the effect that special efficacy is shown.
The additional aspect of the application and advantage will be set forth in part in the description, these will become from the following description It obtains obviously, or recognized by the practice of the application.
Detailed description of the invention
The application is above-mentioned and/or additional aspect and advantage will become from the following description of the accompanying drawings of embodiments Obviously and it is readily appreciated that, in which:
Fig. 1 is the system framework schematic diagram of the processing method for the virtual present special efficacy that an embodiment provides；
Fig. 2 is the structural schematic diagram for the live broadcast system that an embodiment provides；
Fig. 3 is a kind of flow chart of the processing method for virtual present special efficacy that an embodiment provides；
Fig. 4 is the flow chart for the synthesising position information acquisition method that an embodiment provides；
Fig. 5 is a kind of flow chart of the synthetic method for virtual present special efficacy that an embodiment provides；
Fig. 6 is a kind of synthetic effect figure of virtual present in direct seeding technique；
Fig. 7 is the effect picture for the virtual present synthesis that an embodiment provides；
Fig. 8 is a kind of flow chart of the rendering method for virtual present special efficacy that an embodiment provides；
Fig. 9 is a kind of rendering effect figure of virtual present in direct seeding technique；
Figure 10 is another flow chart of the processing method for the virtual present special efficacy that an embodiment provides；
Figure 11 is the timing diagram for the virtual present comp process that an embodiment provides；
Figure 12 is the structural schematic diagram of the processing unit for the virtual present special efficacy that an embodiment provides；
Figure 13 is another structural schematic diagram of the processing unit for the virtual present special efficacy that an embodiment provides.
Specific embodiment
Embodiments herein is described below in detail, examples of the embodiments are shown in the accompanying drawings, wherein from beginning to end Same or similar label indicates same or similar element or element with the same or similar functions.Below with reference to attached The embodiment of figure description is exemplary, and is only used for explaining the application, and cannot be construed to the limitation to the application.
Those skilled in the art of the present technique are appreciated that unless expressly stated, singular " one " used herein, " one It is a ", " described " and "the" may also comprise plural form.It is to be further understood that being arranged used in the description of the present application Diction " comprising " refer to that there are the feature, integer, step, operation, element and/or component, but it is not excluded that in the presence of or addition Other one or more features, integer, step, operation, element, component and/or their group.It should be understood that when we claim member Part is " connected " or when " coupled " to another element, it can be directly connected or coupled to other elements, or there may also be Intermediary element.In addition, " connection " used herein or " coupling " may include being wirelessly connected or wirelessly coupling.It is used herein to arrange Diction "and/or" includes one or more associated wholes for listing item or any cell and all combinations.
It will be appreciated by those skilled in the art that the application so-called " client ", " application ", " application program " and similar The concept of statement is same concept well known to those skilled in the art, refer to instructed by series of computation machine and related data money The computer software for being suitable for electronics operation of the organic construction in source.Unless specified, this name itself is not by programming language kind Class, rank, the operating system of operation of also not rely by it or platform are limited.In the nature of things, this genus is not also by any The terminal of form is limited.
Those skilled in the art of the present technique are appreciated that unless otherwise defined, all terms used herein (including technology art Language and scientific term), there is meaning identical with the general understanding of those of ordinary skill in the application fields.Should also Understand, those terms such as defined in the general dictionary, it should be understood that have in the context of the prior art The consistent meaning of meaning, and unless idealization or meaning too formal otherwise will not be used by specific definitions as here To explain.
In order to preferably illustrate the technical solution of the application, the processing method of the virtual present special efficacy of this programme is illustrated below The application environment that can be applicable in.As shown in Figure 1, the processing method that Fig. 1 is the virtual present special efficacy that an embodiment provides is System block schematic illustration, which may include server-side and client.It can wrap in live streaming platform in server-side Multiple virtual direct broadcasting rooms and server etc. are included, each virtual direct broadcasting room is corresponding to play different live contents.Client includes seeing Many clients and main broadcaster's client, typically, main broadcaster is broadcast live by main broadcaster's client, and spectators are selected by spectator client It selects and is broadcast live into a certain virtual direct broadcasting room viewing main broadcaster.Spectator client and main broadcaster's client can be by being mounted on terminal Live streaming application program (Application, APP) in equipment enters live streaming platform.
In the present embodiment, terminal device can be smart phone, tablet computer, electronic reader, desktop computer or pen The terminals such as this computer are remembered, to this and without limitation.Server is used to terminal device and provides the background server of background service, It can be realized with the server cluster that separate server or multiple servers form.
The processing method of virtual present special efficacy provided in this embodiment is suitable for giving virtual present during live streaming simultaneously The case where being shown to virtual present special efficacy can be spectators by spectator client and give virtual present to target main broadcaster, The special efficacy that virtual present is shown with main broadcaster's client locating for target main broadcaster and multiple spectator clients, is also possible to main broadcaster and passed through Main broadcaster's client gives virtual present to another main broadcaster, in the main broadcaster institute of the main broadcaster and reception virtual present that give virtual present Main broadcaster's client at place and multiple spectator clients show the special efficacy etc. of virtual present.
Below by taking spectator client gives virtual special efficacy present to target main broadcaster as an example, exemplary theory is carried out to this programme It is bright.
Fig. 2 is the structural schematic diagram for the live broadcast system that an embodiment provides, as shown in Fig. 2, the live broadcast system 200 includes: Main broadcaster's client 210, spectator client 230 and server 220.Main broadcaster's client 210 is through server 220 and spectator client 230 are communicatively coupled by network.
In the present embodiment, main broadcaster's client can be mounted to main broadcaster's client on computer computer, be also possible to It is installed on mobile terminal, such as main broadcaster's client on mobile phone or tablet computer；Similarly, spectator client can be mounted to calculate Spectator client on electromechanical brain is also possible to be installed on mobile terminal, such as the spectator client on mobile phone or tablet computer.
The server 220, for receiving the instruction of giving of virtual present that the spectator client 230 is sent, and to Main broadcaster's client 210 gives instruction described in sending；
Main broadcaster's client 210 described give instruction and obtains target special efficacy present mark for receiving；According to described Target special efficacy present identifier lookup obtains target special efficacy present, determines characteristic area corresponding to the target special efficacy present；Root Synthesising position information of the target special efficacy present on the live video is determined according to the characteristic area；By the synthesis position Confidence breath and the live video are encoded into live video flow data and are sent to server；
The server 220 is also used to the live video stream data forwarding to the spectator client 230；
The spectator client 230, for receiving live video flow data and target special efficacy present, from the live video The synthesising position information of live video and the target special efficacy present is obtained in flow data；According to the synthesising position information by institute It states target special efficacy present and is added to the live video and synthesized, obtain special efficacy frame image；It is arranged on live streaming window and shows Special efficacy display area, during playing the live video, in the synchronous rendering special efficacy frame of the special efficacy display area Image.
Fig. 3 is a kind of flow chart of the processing method for virtual present special efficacy that an embodiment provides, the virtual present special efficacy Processing method be implemented in client, such as main broadcaster's client.In some embodiments, it can also be performed in server-side, such as service Device.The present embodiment is illustrated by taking main broadcaster's client as an example.
Specifically, as shown in figure 3, the processing method of the virtual present special efficacy may comprise steps of:
S110, it receives virtual present and gives instruction, give the corresponding target special efficacy present of instruction acquisition simultaneously according to described Determine the characteristic area of the target special efficacy present.
For spectators during watching live video, to target, main broadcaster gives virtual present, by triggering in spectator client Correlation function key selection target special efficacy present gives instruction to server sending virtual present, and server receives this and gives Instruction is sent to be forwarded to main broadcaster's client, wherein the Comp Requests of the virtual present carry target special efficacy present mark and target The information such as main broadcaster's mark.
The virtual present that main broadcaster's client reception spectator client is sent gives instruction, parses this and gives entrained by instruction Information, get target special efficacy present mark and target main broadcaster mark.Server goes out according to target special efficacy present identifier lookup Corresponding target special efficacy present.Optionally, target special efficacy present can be the special efficacy present of two-dimentional display format, be also possible to The special efficacy present of Three-dimensional Display form, i.e., three-dimensional special efficacy present.In the present embodiment, which is preferably three-dimensional special It imitates present and three-dimensional special efficacy is built by three-dimensional special efficacy present such as AI (Artificial Intelligence) special efficacy present, Augmented reality impression, improves the treatment effect of virtual present special efficacy.
In embodiment, target special efficacy present carries out effect displaying, different target special efficacy presents according to character features It is corresponding with respective characteristic area, wherein characteristic area refers to the area of target special efficacy present corresponding setting on live video Domain.Illustratively, target special efficacy present can be angel's wing, mask and cap etc., and angel's wing can be set in main broadcaster's Back, then the corresponding characteristic area of angel's wing is back；Mask is worn on the ocular vicinity of main broadcaster, then the corresponding feature of mask Region is face；Cap is worn on the head of main broadcaster, then the corresponding characteristic area of cap is head etc..
S120, synthesising position letter of the target special efficacy present on the live video is determined according to the characteristic area Breath.
Wherein, synthesising position information may include: face information, human body contour outline information, gesture information and skeleton letter At least one of breath.In embodiment, synthesising position information may include being known based on main broadcaster's client to live video The target special efficacy present not obtained is synthesized in the target position of the live video.
Main broadcaster's client receives when the giving instruction of virtual present of spectator client transmission, obtains target main broadcaster place The live video of direct broadcasting room extracts current video frame image, according to target special efficacy present to current video from the live video Frame image is handled, to extract the relevant information for being synthesized to target special efficacy present, such as target special efficacy present Synthesising position information of the characteristic area in current video frame image.According to synthesising position information, target special efficacy present can be closed At the target position for arriving current video frame image, wherein the characteristic area of target special efficacy present and the mesh of current video frame image Cursor position corresponds.In embodiment, synthesising position information can be indicated by one or more character contour key points, In, each character contour key point has unique coordinate value in current video frame image, according to the one of character contour key point A or multiple available target special efficacy present additions of coordinate value are in the target position of current video frame image.
For the ease of more clearly illustrating that this programme, the present embodiment will do in conjunction with " mask " for target special efficacy present exemplary Explanation.
In embodiment, spectators give " mask " this target special efficacy present, spy corresponding to " mask " to target main broadcaster Sign region is face.The live video of direct broadcasting room, extracts current video frame image, to this where server obtains target main broadcaster Current video frame image is identified, a series of character contour key point is obtained, and from these character contour key points really The corresponding this feature region of fixed certain some objective contour point, such as objective contour point A (50,55), B (45,48) and C (45,60), general It is target special efficacy present in current video frame that objective contour point A, B and C, which carries out the obtained region of line according to specific algorithm, The target position of image.Optionally, being somebody's turn to do the synthesising position information of " mask " in current video frame image includes objective contour point A, the coordinate value of B and C.
S130, the live video and the synthesising position information are sent to the spectator client, so that described The target special efficacy present is added to the live video according to the synthesising position information and is synthesized simultaneously by spectator client It is shown.
After main broadcaster's client gets synthesising position information, synthesising position information and live video are subjected to coding and form number According to packet, it is sent to spectator client.After spectator client receives data packet, it is decoded and extracts live video and synthesis position Confidence breath.In the present embodiment, synthesising position information includes objective contour of the target special efficacy present in current video frame image The coordinate value of point.
After spectator client gets synthesising position information, the big of special efficacy display area is carried out in conjunction with current spectator client It is small that corresponding conversion is carried out to synthesis location information, determine that target special efficacy present is being worked as according to the synthesising position information after conversion Target special efficacy present is added on target position and synthesizes by the target position on preceding video frame images.
For example, the resolution sizes that main broadcaster's client recognizes current video frame image are 400*300, obtained synthesis The coordinate value of objective contour point A is (50,50) in location information, and the same current video frame figure that spectator client is shown The resolution sizes of picture are 800*600, correspondingly convert to synthesis location information, obtain current goal profile point A's ' Coordinate value is (100,100).Target special efficacy present is added to target position determined by the synthesising position information after converting On synthesized.It should be noted that same current video frame image refers to that the content of video frame images is identical, remaining feature is such as Resolution ratio, picture size etc. can be different.
In the present embodiment, the current video frame image zooming-out of live video can be come out, in spectator client by mesh Mark special efficacy present is added to synthesis special efficacy frame image on current video frame image, carries out effect in the special efficacy display area of pre-configuration It shows, which can be independently of video area, to improve the bandwagon effect of special efficacy.
The processing method of virtual present special efficacy provided in this embodiment is applied to main broadcaster's client, by receiving spectators visitor The virtual present that family end is sent gives instruction, obtains corresponding target special efficacy present and determining target special efficacy according to instruction is given The characteristic area of present；Synthesising position information of the target special efficacy present on live video is determined according to characteristic area；It will live streaming Video and synthesising position information are sent to client, so that spectator client is according to synthesising position information by target special efficacy present It is added to live video to be synthesized and be shown.Synthesising position information is followed live video to be sent to sight together by this programme Many clients.Target special efficacy present is directly synthesized to live video relative to main broadcaster's client, is retransmited to each spectators client End is to play virtual present special efficacy in the video area of spectator client, and the embodiment is using main broadcaster's client outside live video Synthesising position information is subjected to coding encapsulation, so that synthesising position information follows live video to be passed independently of live video It is defeated, it is carried out in order to which spectator client is shown according to effect of the synthesising position information combination spectator client to virtual special efficacy present Secondary editor, to be conducive to realize layering special effect processing in spectator client and across the special efficacy exhibition of live video play area Show, improves the effect that special efficacy is shown
In order to keep the technical program relatively sharp, more for ease of understanding, below to each step in the technical program Concrete implementation process and mode are described in detail.
In one embodiment, the instruction of giving according to of step S110 obtains corresponding target special efficacy present and determines institute The characteristic area for stating target special efficacy present, may comprise steps of:
S1101, target special efficacy present mark is obtained.
In embodiment, different virtual presents has unique mark, and different virtual presents has different displayings Form, for example, common virtual present and special virtual present can be divided into according to the type of virtual present.If what is given is general Logical virtual present, then carry out special efficacy displaying according to the present methods of exhibiting of the prior art, if what is given is special virtual present, Special efficacy displaying is carried out according to the processing method of virtual present special efficacy provided by this programme.
In embodiment, target special efficacy present mark is obtained, judges whether it is special according to target special efficacy present mark Virtual present, if so, thening follow the steps S1102.
S1102, target special efficacy present is obtained according to the target special efficacy present identifier lookup, determines the target special efficacy gift Characteristic area corresponding to object.
In embodiment, characteristic area corresponding to different target special efficacy presents is different, that is to say, different targets is special Effect present is added to the difference of the position in live video.For example, " angel's wing " is added to the back of main broadcaster, " cap " is added to The head etc. of main broadcaster.
Server gets target special efficacy present mark, finds out from the database of pre-configuration and the target special efficacy present Identify corresponding target special efficacy present and its characteristic area.Optionally, characteristic area can be marked with the character of certain settings Know, for example, there is alphabetical A to indicate head, letter b indicates back etc..
In one embodiment, target special efficacy present mark, target special efficacy present and characteristic area can establish association and close System, such as { 0001, angel's wing, A }, wherein 0001 indicates target special efficacy present mark, and " angel's wing " indicates target special efficacy The title of present, A indicate the characteristic area of target special efficacy present, such as back.Further, incidence relation can be passed through into data The mode of structure stores.
Fig. 4 is the flow chart for the synthesising position information acquisition method that an embodiment provides, as shown in figure 4, in an embodiment In, step S120's determines that synthesising position of the target special efficacy present on the live video is believed according to the characteristic area Breath, may comprise steps of:
S1201, the current video frame image for obtaining the live video.
Wherein, current video frame image can be a frame, be also possible to multiframe.
When the instruction of giving for the virtual present for receiving spectator client transmission, it is straight that main broadcaster's client obtains target live streaming place A frame or multiframe current video frame image for live video between broadcasting.When current video frame image is multiframe, which works as Preceding video frame images can be connection frame video image, be also possible to every frame video image.
Character contour key point in S1202, the extraction current video frame image.
In embodiment, main broadcaster's client pre-processes current video frame image, as image format conversion, filtering are gone Dry, binary conversion treatment etc. extracts the character contour by pretreated current video frame image, passes through algorithm further according to profile Operation obtains character contour key point.In general, needing current video frame image being converted to bitmap images.Bitmap is by picture Plain (Pixel) composition, pixel is the smallest information unit of bitmap, is stored in image grid, and each pixel has specific Position and color value, the position of pixel can be indicated according to the size of image with coordinate value (x, y).
It should be noted that the extracting mode of the character contour key point of current video frame image can pass through existing work Have with algorithm and realize, such as OpenCV, HOG and OTSU algorithm, current video frame can also be extracted by other means certainly The character contour key point of image.
The corresponding different human body information of the set of different character contour key points.For example, identifying current video frame image Face position, extract the profile key point at face position, in one embodiment, face information may include that 106 profiles close Key point, each profile key point correspond to a certain position of face, and each profile key point corresponds to unique coordinate value, indicates the wheel Position of the wide key point in current video frame image.Similarly, body contour includes 59 profile key points, and each profile is crucial The edge contour at the corresponding each position of human body of point, skeleton includes 22 profile key points, and each profile key point corresponds to people The coordinate value of body skeletal joint point, each profile key point indicates the position in current video frame image.
S1203, determine that the characteristic area is right on the current video frame image according to the character contour key point The target position answered, to be synthesized in the target position to the target special efficacy present.
Wherein, the corresponding characteristic area of target special efficacy present is corresponding in the target position of current video frame image.Example Such as, the characteristic area of target special efficacy present " angel's wing " is " back ", is identified from the character contour key point extracted Belong to the profile key point of " back " feature, and be determined as objective contour point, according to these objective contour points in current video frame Coordinate value on image, the target position synthesized on current video frame image where determining target special efficacy present, wherein mesh Cursor position can be the set of the coordinate value of objective contour point, is also possible to objective contour point line and is formed by region.
In one embodiment, the live video and the synthesising position information are sent to the visitor by step S120 Family end, may comprise steps of:
The live video and the synthesising position information are subjected to coding and are packaged into live video flow data, and passes through clothes Device be engaged in for the live video stream data forwarding to the spectator client.
In the present embodiment, main broadcaster's client is by live video and the synthesis position identified according to target special efficacy present Confidence breath carries out coding and is packaged into data packet, forms live video flow data.Main broadcaster's client sends live video flow data To server, so that server is by the live video stream data forwarding to spectator client.
Spectator client obtains synthesising position information after being decoded to live video flow data, and is believed according to synthesising position Breath handles target special efficacy present.
Exemplary one: Fig. 5 is a kind of flow chart of the synthetic method for virtual present special efficacy that an embodiment provides, Fig. 5 institute Show, spectator client carries out layering synthesis to target special efficacy present according to synthesising position information and handles, and main flow can be as follows:
S110a, live video flow data and target special efficacy present are received, live streaming is obtained from the live video flow data The synthesising position information of video and the target special efficacy present.
Wherein, the synthesising position information may include being identified to obtain to the live video based on main broadcaster's client Target special efficacy present synthesize target position on the live video.
In embodiment, user gives instruction to server transmission virtual present by spectator client.Main broadcaster client Instruction is given in termination receiving end server forwarding, obtains characteristic area corresponding to live video and target special efficacy present.It is optional , characteristic area can be main broadcaster's client and be identified according to instruction is given, and be also possible to server and receive give Instruction is sent to be identified to obtain and relay to main broadcaster's client.The present embodiment is with main broadcaster's client according to giving instruction identification target It is illustrated for characteristic area corresponding to special efficacy present.
Spectator client is decoded after receiving live video flow data, obtains synthesising position information and live video, And current video frame image is obtained from live video.It should be noted that main broadcaster's client synthesising position information for identification The current video frame image that corresponding current video frame image and spectator client are obtained from live video is same frame figure Picture, showing can be different in main broadcaster's client and the resolution ratio of spectator client, size and color etc..
Wherein, target special efficacy present can be the special efficacy present of two-dimentional display format, be also possible to Three-dimensional Display form Special efficacy present, i.e., three-dimensional special efficacy present.In the present embodiment, which is preferably three-dimensional special efficacy present, passes through three Dimension special efficacy present builds three-dimensional special efficacy, and augmented reality impression improves the bandwagon effect of virtual present special efficacy.
S120a, the live video is divided into foreground image layer and background image layer, and raw according to target special efficacy present At at least one virtual present special efficacy layer.
Specifically, obtaining current video frame image from live video；It is foreground area by current video frame image segmentation The background area and；Wherein, figure layer where the foreground area is foreground image layer；Figure layer where the background area is Background As layer.
In embodiment, spectator client obtains current video frame image from live video, wherein current video frame figure As multi-frame video frame image can be also possible to a frame video frame images.
Further, background segment processing is carried out to current video frame image.Existing algorithm be can use to current video Each pixel value of frame image compares, and is foreground area and background area by current video frame image segmentation, such as by pixel value Pixel value is less than the pixel of a certain threshold value as foreground area by region corresponding to the set greater than the pixel of a certain threshold value Region corresponding to the set of point is as background area.In embodiment, foreground area and background area are located at different Image layer, wherein the image layer where foreground area is foreground image layer, and the image layer where background area is background image Layer.
In one embodiment, foreground image layer may include the main broadcaster personage region in live video, and background image layer can To include the background area in live video in addition to main broadcaster personage region.In addition, in embodiment, it can be to target special efficacy present It is split, one or more virtual present special efficacy layers corresponding to target special efficacy present is generated, for example, " mask " present only has One virtual present special efficacy layer, " snowflake " present may include multiple virtual present special efficacy layers, and such as first snowflake is in virtual gift Object special efficacy layer A, second snowflake is in virtual present special efficacy layer B, and third piece and the 4th snowflake are in virtual present special efficacy layer C etc..
Spectator client gets the foreground image layer, background image layer and target special efficacy present of current video frame image Corresponding one or more virtual present special efficacy layers.Optionally, it can be performed corresponding processing and be cached.
S130a, according to the synthesising position information by each virtual present special efficacy layer and the foreground image layer and back Scape image layer is synthesized and is shown in sequence.
Illustratively, synthesising position information includes the position A (50,50) of character contour key point, B (55,60), C (70, 100), each figure layer includes foreground image layer a, background image layer b, virtual present special efficacy layer c, virtual present special efficacy layer d and virtual The synthesis sequence of present special efficacy layer e, each figure layer are b, c, a, d and e, and the position of c corresponding A, d correspond to the position of B, and e corresponds to C's Position.
Firstly, background image layer b is set to bottom, then according to location A by virtual present special efficacy layer c and foreground image Layer a is synthesized, and synthesizes virtual present special efficacy layer d then according to B location, finally according to location of C that virtual present is special Active layer e is synthesized, so that each section adds the target position corresponding to current video frame image in target special efficacy present After setting, the current video frame image of synthesis target special efficacy present is shown.
As shown in fig. 6, the synthetic effect figure that Fig. 6 is virtual present in a kind of direct seeding technique especially exists in the art During big special efficacy present is shown, virtual present is directly appended on live video, so that virtual present and live video picture Overlapping, blocks main broadcaster personage, influences user's viewing；And after using the technology of the application, it can be to avoid blocking main broadcaster personage, it can be with Obtain better special efficacy bandwagon effect.
Fig. 7 is the effect picture for the virtual present synthesis that an embodiment provides, as shown in fig. 7, according to the human body back of main broadcaster Foreground image layer setting where main broadcaster personage is blocked " angel's wing in the upper surface of special efficacy layer where " angel's wing " by profile information The setting regions of wing " achievees the effect that " angel's wing " being added to the back of main broadcaster personage, be believed according to the face mask of main broadcaster Breath sets special efficacy layer setting where " mask " in the face on foreground image layer, blocking main broadcaster personage where main broadcaster personage Region achievees the effect that for " mask " to be added on main broadcaster personage's eyes, enables target special efficacy present according to human body contour outline Feature is synthesized to the target position of the current video frame image of live video, obtains better special efficacy bandwagon effect.
The synthetic method of virtual present special efficacy provided in this embodiment, by receiving live video flow data and target special efficacy Present obtains the synthesising position information of live video and target special efficacy present from live video flow data；By live video point It is segmented into foreground image layer and background image layer, and at least one virtual present special efficacy layer is generated according to target special efficacy present；According to Each virtual present special efficacy layer and foreground image layer and background image layer are synthesized and are shown in sequence by synthesising position information. Spectator client is according to synthesising position information in the present embodiment, by the spy of at least one virtual present corresponding to target special efficacy present The foreground image layer and background image layer of active layer and current video frame image are synthesized and are shown in sequence, and root is realized Target special efficacy present is synthesized to the set mesh of current video frame image according to the obtained synthesising position information such as character contour In cursor position, avoid target special efficacy present be directly displayed at video area block main broadcaster personage and influence be broadcast live effect, simultaneously Also improve the bandwagon effect of virtual present special efficacy.
Exemplary two: Fig. 8 is a kind of flow chart of the rendering method for virtual present special efficacy that an embodiment provides, Fig. 8 institute Show, spectator client carries out across live video play area to target special efficacy present according to synthesising position information and renders, main Process can be as follows:
Specifically, as shown in figure 8, the rendering method of the virtual present special efficacy may comprise steps of:
S110b, live video flow data and target special efficacy present are received, live streaming is obtained from the live video flow data The synthesising position information of video and the target special efficacy present.
Wherein, the target special efficacy present in the present embodiment can be the special efficacy present of Three-dimensional Display form, i.e., three-dimensional special efficacy Present (the virtual special efficacy present of AI (Artificial Intelligence)).Three-dimensional special efficacy is built by three-dimensional special efficacy present, Augmented reality impression, improves the rendering effect of virtual present special efficacy.
This step is identical as step S110a, herein no longer narration in detail.
S120b, it the target special efficacy present is added to by the live video according to the synthesising position information closes At obtaining special efficacy frame image.
Spectator client gets synthesising position information, determines that target special efficacy present is regarded in live streaming according to synthesising position information Target special efficacy present is added on the target position and current video frame image by the target position of the current video frame image of frequency It is synthesized, obtains special efficacy frame image.Wherein, current video frame image can be a frame video frame images, be also possible to multiframe Video frame images.
In embodiment, current video frame image can be divided into foreground image layer and background image layer, target special efficacy gift Object may include one or more virtual present special efficacy layers, in embodiment, can split to target special efficacy present, generate One or more virtual present special efficacy layers corresponding to target special efficacy present, determine each virtual present according to synthesising position information Target position of the special efficacy layer on foreground image layer or background image layer, by each virtual present special efficacy layer, foreground image layer and back Scape image layer is synthesized, and special efficacy frame image is obtained.
In one embodiment, according to synthesising position information by each virtual present special efficacy layer and the foreground image layer and Background image layer is synthesized and is shown according to priority orders.
S130b, setting shows special efficacy display area, during playing the live video, In on live streaming window The special efficacy display area is synchronous to render the special efficacy frame image.
Live streaming window refer to live streaming application it is in the open state under corresponding window, be that can take up in maximized state Entire terminal device screen.In the present embodiment, special efficacy display area is set on live streaming window, special efficacy display area is set to On live video play area, special efficacy display area is greater than live video play area, so that corresponding to target special efficacy present Special efficacy can amplify render come, improve special efficacy show effect.Wherein, live video play area refers to for playing The region of live video.
After spectator client gets synthesising position information, the big of special efficacy display area is carried out in conjunction with current spectator client It is small that corresponding conversion is carried out to synthesis location information, determine that target special efficacy present is being worked as according to the synthesising position information after conversion Target special efficacy present is added on target position and synthesizes by the target position on preceding video frame images.
For example, the resolution sizes that main broadcaster's client recognizes current video frame image are 400*300, obtained synthesis The coordinate value of objective contour point A is (50,50) in location information, and the same current video frame figure that spectator client is shown The resolution sizes of picture are 800*600, correspondingly convert to synthesis location information, obtain current goal profile point A's ' Coordinate value is (100,100).Target special efficacy present is added to target position determined by the synthesising position information after converting On synthesized.It should be noted that same current video frame image refers to that the content of video frame images is identical, remaining feature is such as Resolution ratio, picture size etc. can be different.
In the present embodiment, the special efficacy of picture frame image shows the thread different with live video broadcasting occupancy, so that Wherein for a thread during playing live video, special efficacy frame image synchronization can be rendered into special efficacy show area by another thread Domain improves the bandwagon effect of special efficacy to reach video playing progress synchronous with special efficacy displaying.
It should be noted that the areas transparent that will be blocked in special efficacy layer corresponding to special efficacy present by main broadcaster personage, with So that the special efficacy across live video play area is shown without influencing the normal video playing of live video play area.
As shown in figure 9, the rendering effect figure that Fig. 9 is virtual present in a kind of direct seeding technique especially exists in the art During the virtual special efficacy present of AR (Augmented Reality, augmented reality) is shown, AR virtual present real can only be broadcast live Video playback area, bandwagon effect are bad；And after using the technology of the application, can across live video play area show, can To obtain better special efficacy bandwagon effect.
With continued reference to Fig. 7, as shown in fig. 7, special efficacy display area is arranged in live video play area, in special efficacy show area The area in domain is greater than live video play area, and virtual present special efficacy is rendered into special efficacy display area, so that virtual present is special Efficiency reaches across live video region and is shown, as shown in Figure 7 in virtual special efficacy present in " angel's wing ", obtain more preferable Special efficacy bandwagon effect.
The rendering method of virtual present special efficacy provided in this embodiment, by receiving live video flow data and target special efficacy Present obtains the synthesising position information of live video and target special efficacy present from live video flow data；Wherein, synthesising position Information includes being synthesized on live video based on the target special efficacy present that main broadcaster's client identifies live video Target position；Target special efficacy present is added to live video according to synthesising position information to synthesize, obtains special efficacy frame image； Setting shows special efficacy display area on live streaming window, during playing live video, in the synchronous wash with watercolours of special efficacy display area Special efficacy frame image is contaminated, so that virtual present special efficacy is not intended to be limited to the live video play area of client, it can be across view Frequency play area is rendered and is shown.
Figure 10 is another flow chart of the processing method for the virtual present special efficacy that an embodiment provides, the virtual present special efficacy Processing method be applied to server-side, can be executed by server.
Specifically, as shown in Figure 10, the processing method of virtual present special efficacy the following steps are included:
S410, instruction is given to main broadcaster's client transmission virtual present.
Wherein, main broadcaster's client gives instruction according to and obtains corresponding target special efficacy present and determine the mesh The characteristic area for marking special efficacy present, determines conjunction of the target special efficacy present on the live video according to the characteristic area At location information.
For spectators during watching live video, to target, main broadcaster gives virtual present, by triggering in spectator client Correlation function key selection target special efficacy present, so that spectator client gives instruction to server sending virtual present, Server, which receives this, to be given instruction and is forwarded to main broadcaster's client, wherein the Comp Requests of the virtual present carry target spy Imitate the information such as present mark and target main broadcaster mark.
Server is given the virtual present that spectator client is sent after instruction is transmitted to main broadcaster's client, according to acquisition The target special efficacy present arrived, which identifies, determines corresponding target special efficacy present and its corresponding characteristic area, while obtaining live streaming view Current video frame image in frequency identifies that the characteristic area for obtaining the target special efficacy present is being worked as to current video frame image The point of objective contour corresponding to preceding video frame images, obtains synthesising position information.
S420, the live video flow data that main broadcaster's client is sent is received.
It wherein, include the synthesising position information of live video and target special efficacy present in the live video flow data.
Main broadcaster's client encodes synthesising position information and live video, forms live video flow data, so that Synthesising position information independently follows live video to be transmitted.In embodiment, server receive by main broadcaster's client according to The live video flow data that the communication protocol of setting is transmitted.
S430, by the live video stream data forwarding to spectator client；Wherein, the spectator client is according to The target special efficacy present is added to the live video and is synthesized and be shown by synthesising position information.
Server is forwarded to each spectators in same direct broadcasting room after receiving the live video stream that main broadcaster's client is sent Client so that spectator client decoding live video flow data obtains synthesising position information, and is believed according to the synthesising position Target special efficacy present is added on live video on target position and synthesizes by breath, and to the target special efficacy present after synthesis into Row is shown.
The processing method of virtual present special efficacy provided in this embodiment is applied to server-side, by sending out to main broadcaster's client Virtual present is sent to give instruction；Wherein, main broadcaster's client obtains corresponding target special efficacy present and determining mesh according to instruction is given The characteristic area for marking special efficacy present, determines synthesising position information of the target special efficacy present on live video according to characteristic area； Receive the live video flow data that main broadcaster's client is sent；It wherein, include that live video and target are special in live video flow data Imitate the synthesising position information of present；By live video stream data forwarding to spectator client；Wherein, spectator client is according to synthesis Target special efficacy present is added to live video and is synthesized and be shown by location information.This programme will be synthesized by server Location information follows live video to be sent to spectator client together, so that synthesising position information is followed independently of live video Live video is transmitted, in order to spectator client according to synthesising position information combination spectator client to virtual special efficacy present Effect show and carry out secondary editor, to be conducive to realize layering special effect processing and across live video broadcasting in spectator client The special efficacy in region is shown, improves the effect that special efficacy is shown.
In order to more clearly illustrate the technical solution of the application, the example under several scenes will be combined, further progress is said It is bright.
Scene one: referring to Figure 11, and Figure 11 is the timing diagram for the virtual present comp process that an embodiment provides；The example In, spectators give three-dimensional special efficacy present " angel's wing " to main broadcaster, and correspondence is identified as ID1648, then its main flow can be as Under:
S11, spectator client sending are given gifts and are requested to server.
Spectators user W is given gifts by spectator client sending and is requested to server, wherein virtual present ID1648.
S12, server carry out business processing.
Server receives give gifts request after, do corresponding business processing (such as deducting fees).
S13, server broadcast are given gifts information.
Spectators user W is broadcast to all users in channel to the information of giving gifts that main broadcaster gives ID1648 present, including Main broadcaster's client and spectator client.
S14, main broadcaster's client receive after giving gifts information, inquire the virtual present, identify synthesising position information.
After main broadcaster's client receives the broadcast for information of giving gifts, according to virtual present ID1648, inquires present and configure this Virtual present is three-dimensional special efficacy present (such as AI (Artificial Intelligence) present), the synthesising position for needing to identify Information includes face and back, then main broadcaster's client terminal start-up carries out recognition of face and background segment identification.
S15, main broadcaster's client transmit synthesising position information package to live video stream.
Recognition of face and background segment are identified obtained synthesising position information (can be AI information) by main broadcaster's client It is bundled in live video stream, follows live video to be transferred to server together by live video stream.
S16, server forward the live video stream.
Server will include that the live video of synthesising position information is streamed to spectator client.
S17, spectator client obtain synthesising position information, carry out virtual present synthesis and show.
Spectator client decodes from live video stream and obtains synthesising position information, by synthesising position information and virtual present It combines, play out angel's wing special efficacy: main broadcaster grows book angel wing behind.
Scene two: spectators give three-dimensional special efficacy present " pet bird " to main broadcaster, and correspondence is identified as ID1649, then it is main Process can be such that
S21, spectator client sending are given gifts and are requested to server.
Spectators user Q is given gifts by spectator client sending and is requested to server, wherein virtual present ID1649.
S22, server carry out business processing；
Server receives give gifts request after, do corresponding business processing (such as deducting fees).
S23, server broadcast are given gifts information.
Spectators user Q is broadcast to all users in channel to the information of giving gifts that main broadcaster gives ID1649 present, including Main broadcaster's client and spectator client.
S24, main broadcaster's client receive after giving gifts information, inquire the virtual present, identify synthesising position information.
After main broadcaster's client receives the broadcast for information of giving gifts, according to virtual present ID1649, inquires present and configure this Virtual present is three-dimensional special efficacy present (such as AI (Artificial Intelligence) present), the synthesising position for needing to identify Information includes face and human body contour outline, then main broadcaster's client terminal start-up carries out recognition of face and human body contour outline identification.
S25, main broadcaster's client transmit synthesising position information package to live video stream.
Recognition of face and human body contour outline are identified obtained synthesising position information (can be AI information) by main broadcaster's client It is bundled in live video stream, follows live video to be transferred to server together by live video stream.
S26, server forward the live video stream.
Server will include that the live video of synthesising position information is streamed to spectator client.
S27, spectator client obtain synthesising position information, carry out virtual present synthesis and show.
Spectator client decodes from live video stream and obtains synthesising position information, by synthesising position information and virtual present It combines, play out " pet bird " special efficacy: bird flies on main broadcaster's shoulder from video exterior domain.
Above example is only used for auxiliary and illustrates technical scheme, and the diagramatic content and detailed process being related to are not constituted Restriction to the usage scenario of technical scheme.
The related embodiment of the processing unit of virtual present special efficacy is described in detail below.
Figure 12 is the structural schematic diagram of the processing unit for the virtual present special efficacy that an embodiment provides, as shown in figure 12, should The processing unit 100 of virtual present special efficacy may include: that feature obtains module 110, information determination module 120 and information transmission mould Block 130.
Wherein, feature obtains module 110, gives instruction for receive virtual present, gives instruction acquisition according to described Corresponding target special efficacy present and the characteristic area for determining the target special efficacy present；
Information determination module 120, for determining that the target special efficacy present is regarded in the live streaming according to the characteristic area Synthesising position information on frequency；
Information sending module 130, for the live video and the synthesising position information to be sent to spectators client End, so that the target special efficacy present is added to the live video according to the synthesising position information by the spectator client It is synthesized and is shown.
The processing unit of virtual present special efficacy provided in this embodiment is applied to main broadcaster's client, obtains mould by feature The virtual present that the reception spectator client of block 110 is sent gives instruction, obtains corresponding target special efficacy gift according to instruction is given Object and the characteristic area for determining target special efficacy present；Information determination module 120 determines that target special efficacy present exists according to characteristic area Synthesising position information on live video；Live video and synthesising position information are sent to client by information sending module 130 End is synthesized and is opened up so that target special efficacy present is added to live video according to synthesising position information by spectator client Show.Synthesising position information is followed live video to be sent to spectator client together by this programme.It is direct relative to main broadcaster's client Target special efficacy present is synthesized to live video, is retransmited to each spectator client to be played in the video area of spectator client Synthesising position information is carried out coding encapsulation outside live video using main broadcaster's client, made by virtual present special efficacy, the embodiment Synthesising position information is obtained independently of live video, live video is followed to be transmitted, in order to which spectator client is according to synthesis position Confidence breath combines spectator client to show the effect of virtual special efficacy present and carries out secondary editor, to be conducive in spectators client End realizes that layering special effect processing and the special efficacy across live video play area are shown, improves the effect that special efficacy is shown.
In one embodiment, it includes: mark acquiring unit and characteristics determining unit that feature, which obtains module 110,；
Wherein, mark acquiring unit, for obtaining target special efficacy present mark；Characteristics determining unit, for according to Target special efficacy present identifier lookup obtains target special efficacy present, determines characteristic area corresponding to the target special efficacy present.
In one embodiment, information determination module 120 includes: frame image acquisition unit, key point extraction unit and mesh Cursor position determination unit；
Wherein, frame image acquisition unit, for obtaining the current video frame image of the live video；Key point is extracted single Member, for extracting the character contour key point in the current video frame image；Target position determination unit, for according to Character contour key point determines the characteristic area corresponding target position on the current video frame image, in the mesh Cursor position synthesizes the target special efficacy present.
In one embodiment, information sending module 120 is used to carry out the live video and the synthesising position information Coding is packaged into live video flow data, and passes through server for the live video stream data forwarding to the spectators client End.
In one embodiment, the target special efficacy present is the special efficacy present of Three-dimensional Display form.
In one embodiment, the synthesising position information includes: face information, human body contour outline information, gesture information and people At least one of body bone information.
Figure 13 is another structural schematic diagram of the processing unit for the virtual present special efficacy that an embodiment provides, in the present embodiment The processing unit of virtual present special efficacy be applied to server-side, such as server, as shown in figure 13, the processing of virtual present special efficacy fill Setting 400 includes: instruction sending module 410, data reception module 420 and data forwarding module 430.
Wherein, instruction sending module 410, for giving instruction to main broadcaster's client transmission virtual present；Wherein, described Main broadcaster's client gives instruction according to and obtains corresponding target special efficacy present and determine the feature of the target special efficacy present Region determines synthesising position information of the target special efficacy present on the live video according to the characteristic area；
Data reception module 420, for receiving the live video flow data of main broadcaster's client transmission；Wherein, the live streaming It include the synthesising position information of live video and target special efficacy present in video stream data；
Data forwarding module 430 is used for the live video stream data forwarding to spectator client；Wherein, the sight Many clients according to the synthesising position information by the target special efficacy present be added to the live video carry out synthesis go forward side by side Row is shown.
The processing unit of the virtual present special efficacy of above-mentioned offer can be used for executing the virtual gift that above-mentioned any embodiment provides The processing method of object special efficacy has corresponding function and beneficial effect.
The embodiment of the present application also provides a kind of computer equipment, including memory, processor and storage are on a memory simultaneously The computer program that can be run on a processor when the processor executes described program, is realized as in above-mentioned any embodiment Virtual present special efficacy processing method.
Optionally, which can be mobile terminal, tablet computer, computer computer or server etc..It is above-mentioned When the computer equipment of offer executes the processing method for the virtual present special efficacy that any of the above-described embodiment provides, there is corresponding function Energy and beneficial effect.
The embodiment of the present application also provides a kind of storage medium comprising computer executable instructions, and the computer is executable Instruction by computer processor when being executed for executing a kind of processing method of virtual present special efficacy, comprising:
Receive virtual present gives instruction, gives the corresponding target special efficacy present of instruction acquisition and determining institute according to described State the characteristic area of target special efficacy present；
Synthesising position information of the target special efficacy present on the live video is determined according to the characteristic area；
The live video and the synthesising position information are sent to spectator client, so that the spectator client The target special efficacy present is added to the live video according to the synthesising position information to be synthesized and be shown.
Alternatively, the computer executable instructions are special for executing a kind of virtual present when being executed by computer processor The processing method of effect, comprising:
Virtual present, which is sent, to main broadcaster's client gives instruction；Wherein, main broadcaster's client gives instruction according to It obtains corresponding target special efficacy present and determines the characteristic area of the target special efficacy present, institute is determined according to the characteristic area State synthesising position information of the target special efficacy present on the live video；
Receive the live video flow data that main broadcaster's client is sent；It wherein, include live streaming in the live video flow data The synthesising position information of video and target special efficacy present；
By the live video stream data forwarding to spectator client；Wherein, the spectator client is according to the synthesis The target special efficacy present is added to the live video and is synthesized and be shown by location information.
Certainly, a kind of storage medium comprising computer executable instructions, computer provided by the embodiment of the present application The processing method for the virtual present special efficacy that executable instruction is not limited to the described above operates, and the application can also be performed and arbitrarily implement Relevant operation in the processing method of virtual present special efficacy provided by example, and have corresponding function and beneficial effect.
By the description above with respect to embodiment, it is apparent to those skilled in the art that, the application It can be realized by software and required common hardware, naturally it is also possible to which by hardware realization, but in many cases, the former is more Good embodiment.Based on this understanding, the technical solution of the application substantially in other words contributes to the prior art Part can be embodied in the form of software products, which can store in computer readable storage medium In, floppy disk, read-only memory (Read-Only Memory, ROM), random access memory (Random such as computer Access Memory, RAM), flash memory (FLASH), hard disk or CD etc., including some instructions are with so that a computer is set Standby (can be personal computer, server or the network equipment etc.) executes virtual present described in the application any embodiment The processing method of special efficacy.
It should be understood that although each step in the flow chart of attached drawing is successively shown according to the instruction of arrow, These steps are not that the inevitable sequence according to arrow instruction successively executes.Unless expressly stating otherwise herein, these steps Execution there is no stringent sequences to limit, can execute in the other order.Moreover, at least one in the flow chart of attached drawing Part steps may include that perhaps these sub-steps of multiple stages or stage are not necessarily in synchronization to multiple sub-steps Completion is executed, but can be executed at different times, execution sequence, which is also not necessarily, successively to be carried out, but can be with other At least part of the sub-step or stage of step or other steps executes in turn or alternately.
The above is only some embodiments of the application, it is noted that for the ordinary skill people of the art For member, under the premise of not departing from the application principle, several improvements and modifications can also be made, these improvements and modifications are also answered It is considered as the protection scope of the application.

Claims (13)

1. a kind of processing method of virtual present special efficacy, which comprises the following steps:
Receive virtual present gives instruction, gives the corresponding target special efficacy present of instruction acquisition and the determining mesh according to described Mark the characteristic area of special efficacy present；
Synthesising position information of the target special efficacy present on the live video is determined according to the characteristic area；
The live video and the synthesising position information are sent to spectator client so that the spectator client according to The target special efficacy present is added to the live video and is synthesized and be shown by the synthesising position information.
2. the processing method of virtual present special efficacy according to claim 1, which is characterized in that described to give finger according to It enables the corresponding target special efficacy present of acquisition and includes: the step of determining the characteristic area of the target special efficacy present
Obtain target special efficacy present mark；
Target special efficacy present is obtained according to the target special efficacy present identifier lookup, is determined corresponding to the target special efficacy present Characteristic area.
3. the processing method of virtual present special efficacy according to claim 1, which is characterized in that described according to the characteristic area Domain determines that the target special efficacy present includes: in the step of synthesising position information on the live video
Obtain the current video frame image of the live video；
Extract the character contour key point in the current video frame image；
The characteristic area corresponding target position on the current video frame image is determined according to the character contour key point It sets, to be synthesized in the target position to the target special efficacy present.
4. the processing method of virtual present special efficacy according to claim 1, which is characterized in that described by the live video And synthesising position information the step of being sent to spectator client, includes:
The live video and the synthesising position information are subjected to coding and are packaged into live video flow data, and passes through server By the live video stream data forwarding to the spectator client.
5. the processing method of virtual present special efficacy according to any one of claims 1 to 4, which is characterized in that the target Special efficacy present is the special efficacy present of Three-dimensional Display form.
6. the processing method of virtual present special efficacy according to any one of claims 1 to 4, which is characterized in that the synthesis Location information includes: at least one of face information, human body contour outline information, gesture information and skeleton information.
7. a kind of processing method of virtual present special efficacy, which comprises the following steps:
Virtual present, which is sent, to main broadcaster's client gives instruction；Wherein, main broadcaster's client is given instruction according to and is obtained Corresponding target special efficacy present and the characteristic area for determining the target special efficacy present, determine the mesh according to the characteristic area Mark synthesising position information of the special efficacy present on the live video；
Receive the live video flow data that main broadcaster's client is sent；It wherein, include live video in the live video flow data With the synthesising position information of target special efficacy present；
By the live video stream data forwarding to spectator client；Wherein, the spectator client is according to the synthesising position The target special efficacy present is added to the live video and is synthesized and be shown by information.
8. the processing method of virtual present special efficacy according to claim 7, which is characterized in that described according to the synthesis position The target special efficacy present is added to that the live video is synthesized and the step of being shown includes: by confidence breath
Obtain the current video frame image of the live video；
The target special efficacy present is added into the current video frame image according to the synthesising position information, obtains special efficacy frame figure Picture；
The special efficacy frame image rendering is shown to special efficacy display area.
9. a kind of processing unit of virtual present special efficacy characterized by comprising
Feature obtains module, gives instruction for receive virtual present, gives instruction according to described and obtains corresponding target spy It imitates present and determines the characteristic area of the target special efficacy present；
Information determination module, for determining conjunction of the target special efficacy present on the live video according to the characteristic area At location information；
Information sending module, for the live video and the synthesising position information to be sent to spectator client, so that The target special efficacy present is added to the live video according to the synthesising position information and closed by the spectator client At and be shown.
10. a kind of processing unit of virtual present special efficacy characterized by comprising
Instruction sending module, for giving instruction to main broadcaster's client transmission virtual present；Wherein, main broadcaster's client root According to the characteristic area for giving instruction and obtaining corresponding target special efficacy present and determining the target special efficacy present, according to described Characteristic area determines synthesising position information of the target special efficacy present on the live video；
Data reception module, for receiving the live video flow data of main broadcaster's client transmission；Wherein, the live video fluxion Including the synthesising position information of live video and target special efficacy present in；
Data forwarding module is used for the live video stream data forwarding to spectator client；Wherein, the spectator client The target special efficacy present is added to the live video according to the synthesising position information to be synthesized and be shown.
11. a kind of live broadcast system, which is characterized in that including main broadcaster's client, spectator client and and server, the main broadcaster Client is communicatively coupled with the spectator client by network through the server；
The server, for receiving the instruction of giving of virtual present that the spectator client is sent, and to main broadcaster's client Instruction is given described in transmission；
Main broadcaster's client described give instruction and obtains target special efficacy present mark for receiving；It is special according to the target Effect present identifier lookup obtains target special efficacy present, determines characteristic area corresponding to the target special efficacy present；According to described Characteristic area determines synthesising position information of the target special efficacy present on the live video；By the synthesising position information Live video flow data, which is encoded into, with the live video is sent to server；
The server is also used to the live video stream data forwarding to the spectator client；
The spectator client, for receiving live video flow data and target special efficacy present, from the live video flow data The middle synthesising position information for obtaining live video and the target special efficacy present；According to the synthesising position information by the target Special efficacy present is added to the live video and is synthesized and be shown.
12. a kind of computer equipment including memory, processor and stores the meter that can be run on a memory and on a processor Calculation machine program, which is characterized in that the processor is realized when executing described program as claim 1-8 is described in any item virtual The step of processing method of present special efficacy.
13. a kind of storage medium comprising computer executable instructions, which is characterized in that the computer executable instructions by For executing the step of the processing method of virtual present special efficacy as described in claim any one of 1-8 when computer processor executes Suddenly.
CN201910859930.5A 2019-09-11 2019-09-11 Processing method and device for special effect of virtual gift and live broadcast system Active CN110493630B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201910859930.5A CN110493630B (en) 2019-09-11 2019-09-11 Processing method and device for special effect of virtual gift and live broadcast system
PCT/CN2019/125929 WO2021047094A1 (en) 2019-09-11 2019-12-17 Virtual gift special effect processing method and apparatus, and live streaming system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910859930.5A CN110493630B (en) 2019-09-11 2019-09-11 Processing method and device for special effect of virtual gift and live broadcast system

Publications (2)

Publication Number Publication Date
CN110493630A true CN110493630A (en) 2019-11-22
CN110493630B CN110493630B (en) 2020-12-01

Family

ID=68557666

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910859930.5A Active CN110493630B (en) 2019-09-11 2019-09-11 Processing method and device for special effect of virtual gift and live broadcast system

Country Status (2)

Country Link
CN (1) CN110493630B (en)
WO (1) WO2021047094A1 (en)

Cited By (39)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110958463A (en) * 2019-12-06 2020-04-03 广州华多网络科技有限公司 Method, device and equipment for detecting and synthesizing virtual gift display position
CN111277854A (en) * 2020-03-04 2020-06-12 网易（杭州）网络有限公司 Display method and device of virtual live broadcast room, electronic equipment and storage medium
CN111405343A (en) * 2020-03-18 2020-07-10 广州华多网络科技有限公司 Live broadcast interaction method and device, electronic equipment and storage medium
CN111432235A (en) * 2020-04-01 2020-07-17 网易（杭州）网络有限公司 Live video generation method and device, computer readable medium and electronic equipment
CN111464828A (en) * 2020-05-14 2020-07-28 广州酷狗计算机科技有限公司 Virtual special effect display method, device, terminal and storage medium
CN111464830A (en) * 2020-05-19 2020-07-28 广州酷狗计算机科技有限公司 Method, device, system, equipment and storage medium for image display
CN111698523A (en) * 2019-12-06 2020-09-22 广州华多网络科技有限公司 Method, device, equipment and storage medium for presenting text virtual gift
CN111784418A (en) * 2020-07-27 2020-10-16 网易（杭州）网络有限公司 Display control method and device for live broadcast room, computer medium and electronic equipment
CN111970575A (en) * 2020-07-24 2020-11-20 网易（杭州）网络有限公司 Virtual gift processing method, storage medium, processor and electronic equipment
CN112383788A (en) * 2020-11-11 2021-02-19 成都威爱新经济技术研究院有限公司 Live broadcast real-time image extraction system and method based on intelligent AI technology
WO2021047094A1 (en) * 2019-09-11 2021-03-18 广州华多网络科技有限公司 Virtual gift special effect processing method and apparatus, and live streaming system
WO2021047430A1 (en) * 2019-09-11 2021-03-18 广州华多网络科技有限公司 Virtual gift special effect synthesis method and apparatus, and live streaming system
WO2021047420A1 (en) * 2019-09-11 2021-03-18 广州华多网络科技有限公司 Virtual gift special effect rendering method and apparatus, and live streaming system
WO2021047419A1 (en) * 2019-09-12 2021-03-18 广州华多网络科技有限公司 Live broadcast interaction method, live broadcast system, electronic device and storage medium
CN112565806A (en) * 2020-12-02 2021-03-26 广州繁星互娱信息科技有限公司 Virtual gift presenting method, device, computer equipment and medium
CN112929680A (en) * 2021-01-19 2021-06-08 广州虎牙科技有限公司 Live broadcast room image rendering method and device, computer equipment and storage medium
CN113038228A (en) * 2021-02-25 2021-06-25 广州方硅信息技术有限公司 Virtual gift transmission and request method, device, equipment and medium thereof
CN113139913A (en) * 2021-03-09 2021-07-20 杭州电子科技大学 New view correction generation method for person portrait
CN113163220A (en) * 2021-02-26 2021-07-23 广州方硅信息技术有限公司 Virtual gift positioning display method and device, equipment and medium thereof
CN113196785A (en) * 2021-03-15 2021-07-30 百果园技术(新加坡)有限公司 Live video interaction method, device, equipment and storage medium
CN113315924A (en) * 2020-02-27 2021-08-27 北京字节跳动网络技术有限公司 Image special effect processing method and device
CN113395533A (en) * 2021-05-24 2021-09-14 广州博冠信息科技有限公司 Virtual gift special effect display method and device, computer equipment and storage medium
CN113473211A (en) * 2021-08-20 2021-10-01 广州博冠信息科技有限公司 Virtual gift processing method and device, storage medium and electronic equipment
CN113596498A (en) * 2021-07-29 2021-11-02 广州方硅信息技术有限公司 Virtual gift display method, device, system and storage medium in live broadcast
CN113766258A (en) * 2021-07-27 2021-12-07 广州方硅信息技术有限公司 Live broadcast room virtual gift presentation processing method, equipment and storage medium
CN113873267A (en) * 2021-08-20 2021-12-31 广州方硅信息技术有限公司 Live broadcast room virtual gift crown name method, server, live broadcast system and storage medium
CN114025188A (en) * 2021-10-28 2022-02-08 广州虎牙科技有限公司 Live broadcast advertisement display method, system, device, terminal and readable storage medium
CN114125488A (en) * 2021-12-09 2022-03-01 小象(广州)商务有限公司 Virtual gift display method and system in live broadcast
WO2022042089A1 (en) * 2020-08-28 2022-03-03 北京达佳互联信息技术有限公司 Interaction method and apparatus for live broadcast room
CN114222147A (en) * 2021-11-03 2022-03-22 广州方硅信息技术有限公司 Live broadcast layout adjustment method and device, storage medium and computer equipment
CN114237792A (en) * 2021-12-13 2022-03-25 广州繁星互娱信息科技有限公司 Virtual object display method and device, storage medium and electronic equipment
CN114422821A (en) * 2022-02-14 2022-04-29 广州方硅信息技术有限公司 Live broadcast home page interaction method, device, medium and equipment based on virtual gift
CN114501041A (en) * 2021-04-06 2022-05-13 北京字节跳动网络技术有限公司 Special effect display method, device, equipment, storage medium and product
CN114567793A (en) * 2022-02-23 2022-05-31 广州博冠信息科技有限公司 Method and device for realizing live broadcast interactive special effect, storage medium and electronic equipment
CN115134627A (en) * 2022-07-01 2022-09-30 广州博冠信息科技有限公司 Live broadcast room information processing method and device, readable storage medium and electronic equipment
CN115484472A (en) * 2022-09-23 2022-12-16 广州方硅信息技术有限公司 Special effect playing and processing method and device for live broadcast room, electronic equipment and storage medium
CN115706821A (en) * 2021-08-17 2023-02-17 上海哔哩哔哩科技有限公司 Virtual gift display method and device
WO2023029768A1 (en) * 2021-08-30 2023-03-09 北京字跳网络技术有限公司 Interaction method and apparatus, electronic device, readable storage medium, and program product
CN115706821B (en) * 2021-08-17 2024-04-30 上海哔哩哔哩科技有限公司 Virtual gift display method and device

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101452582A (en) * 2008-12-18 2009-06-10 北京中星微电子有限公司 Method and device for implementing three-dimensional video specific action
CN107071580A (en) * 2017-03-20 2017-08-18 北京潘达互娱科技有限公司 Data processing method and device
CN107483892A (en) * 2017-09-08 2017-12-15 北京奇虎科技有限公司 Video data real-time processing method and device, computing device
CN107613360A (en) * 2017-09-20 2018-01-19 北京奇虎科技有限公司 Video data real-time processing method and device, computing device
CN107682729A (en) * 2017-09-08 2018-02-09 广州华多网络科技有限公司 It is a kind of based on live interactive approach and live broadcast system, electronic equipment
CN107888965A (en) * 2017-11-29 2018-04-06 广州酷狗计算机科技有限公司 Image present methods of exhibiting and device, terminal, system, storage medium
CN108022279A (en) * 2017-11-30 2018-05-11 广州市百果园信息技术有限公司 Special video effect adding method, device and intelligent mobile terminal
CN108156507A (en) * 2017-12-27 2018-06-12 广州酷狗计算机科技有限公司 Virtual objects presentation method, device and storage medium
CN108391153A (en) * 2018-01-29 2018-08-10 北京潘达互娱科技有限公司 Virtual present display methods, device and electronic equipment
CN108900858A (en) * 2018-08-09 2018-11-27 广州酷狗计算机科技有限公司 A kind of method and apparatus for giving virtual present
CN109729288A (en) * 2018-12-17 2019-05-07 广州城市职业学院 A kind of short video-generating device and method

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007243573A (en) * 2006-03-08 2007-09-20 Toshiba Corp Server apparatus
KR101533065B1 (en) * 2008-12-01 2015-07-01 삼성전자주식회사 Method and apparatus for providing animation effect on video telephony call
CN104780458A (en) * 2015-04-16 2015-07-15 美国掌赢信息科技有限公司 Method and electronic equipment for loading effects in instant video
CN105334963B (en) * 2015-10-29 2018-11-20 广州华多网络科技有限公司 A kind of virtual objects methods of exhibiting and system
CN106231434B (en) * 2016-07-25 2019-09-10 武汉斗鱼网络科技有限公司 A kind of living broadcast interactive special efficacy realization method and system based on Face datection
CN106658035A (en) * 2016-12-09 2017-05-10 武汉斗鱼网络科技有限公司 Dynamic display method and device for special effect gift
CN110493630B (en) * 2019-09-11 2020-12-01 广州华多网络科技有限公司 Processing method and device for special effect of virtual gift and live broadcast system

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101452582A (en) * 2008-12-18 2009-06-10 北京中星微电子有限公司 Method and device for implementing three-dimensional video specific action
CN107071580A (en) * 2017-03-20 2017-08-18 北京潘达互娱科技有限公司 Data processing method and device
CN107483892A (en) * 2017-09-08 2017-12-15 北京奇虎科技有限公司 Video data real-time processing method and device, computing device
CN107682729A (en) * 2017-09-08 2018-02-09 广州华多网络科技有限公司 It is a kind of based on live interactive approach and live broadcast system, electronic equipment
CN107613360A (en) * 2017-09-20 2018-01-19 北京奇虎科技有限公司 Video data real-time processing method and device, computing device
CN107888965A (en) * 2017-11-29 2018-04-06 广州酷狗计算机科技有限公司 Image present methods of exhibiting and device, terminal, system, storage medium
CN108022279A (en) * 2017-11-30 2018-05-11 广州市百果园信息技术有限公司 Special video effect adding method, device and intelligent mobile terminal
CN108156507A (en) * 2017-12-27 2018-06-12 广州酷狗计算机科技有限公司 Virtual objects presentation method, device and storage medium
CN108391153A (en) * 2018-01-29 2018-08-10 北京潘达互娱科技有限公司 Virtual present display methods, device and electronic equipment
CN108900858A (en) * 2018-08-09 2018-11-27 广州酷狗计算机科技有限公司 A kind of method and apparatus for giving virtual present
CN109729288A (en) * 2018-12-17 2019-05-07 广州城市职业学院 A kind of short video-generating device and method

Cited By (53)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021047420A1 (en) * 2019-09-11 2021-03-18 广州华多网络科技有限公司 Virtual gift special effect rendering method and apparatus, and live streaming system
WO2021047094A1 (en) * 2019-09-11 2021-03-18 广州华多网络科技有限公司 Virtual gift special effect processing method and apparatus, and live streaming system
WO2021047430A1 (en) * 2019-09-11 2021-03-18 广州华多网络科技有限公司 Virtual gift special effect synthesis method and apparatus, and live streaming system
WO2021047419A1 (en) * 2019-09-12 2021-03-18 广州华多网络科技有限公司 Live broadcast interaction method, live broadcast system, electronic device and storage medium
CN111698523B (en) * 2019-12-06 2021-11-12 广州方硅信息技术有限公司 Method, device, equipment and storage medium for presenting text virtual gift
CN111698523A (en) * 2019-12-06 2020-09-22 广州华多网络科技有限公司 Method, device, equipment and storage medium for presenting text virtual gift
CN110958463A (en) * 2019-12-06 2020-04-03 广州华多网络科技有限公司 Method, device and equipment for detecting and synthesizing virtual gift display position
CN113315924A (en) * 2020-02-27 2021-08-27 北京字节跳动网络技术有限公司 Image special effect processing method and device
CN111277854A (en) * 2020-03-04 2020-06-12 网易（杭州）网络有限公司 Display method and device of virtual live broadcast room, electronic equipment and storage medium
CN111405343A (en) * 2020-03-18 2020-07-10 广州华多网络科技有限公司 Live broadcast interaction method and device, electronic equipment and storage medium
CN111432235A (en) * 2020-04-01 2020-07-17 网易（杭州）网络有限公司 Live video generation method and device, computer readable medium and electronic equipment
CN111464828A (en) * 2020-05-14 2020-07-28 广州酷狗计算机科技有限公司 Virtual special effect display method, device, terminal and storage medium
CN111464830A (en) * 2020-05-19 2020-07-28 广州酷狗计算机科技有限公司 Method, device, system, equipment and storage medium for image display
CN111464830B (en) * 2020-05-19 2022-07-15 广州酷狗计算机科技有限公司 Method, device, system, equipment and storage medium for image display
CN111970575A (en) * 2020-07-24 2020-11-20 网易（杭州）网络有限公司 Virtual gift processing method, storage medium, processor and electronic equipment
CN111784418B (en) * 2020-07-27 2023-08-08 网易（杭州）网络有限公司 Display control method and device for live broadcasting room, computer medium and electronic equipment
CN111784418A (en) * 2020-07-27 2020-10-16 网易（杭州）网络有限公司 Display control method and device for live broadcast room, computer medium and electronic equipment
WO2022042089A1 (en) * 2020-08-28 2022-03-03 北京达佳互联信息技术有限公司 Interaction method and apparatus for live broadcast room
CN112383788A (en) * 2020-11-11 2021-02-19 成都威爱新经济技术研究院有限公司 Live broadcast real-time image extraction system and method based on intelligent AI technology
CN112565806A (en) * 2020-12-02 2021-03-26 广州繁星互娱信息科技有限公司 Virtual gift presenting method, device, computer equipment and medium
CN112565806B (en) * 2020-12-02 2023-08-29 广州繁星互娱信息科技有限公司 Virtual gift giving method, device, computer equipment and medium
CN112929680B (en) * 2021-01-19 2023-09-05 广州虎牙科技有限公司 Live broadcasting room image rendering method and device, computer equipment and storage medium
CN112929680A (en) * 2021-01-19 2021-06-08 广州虎牙科技有限公司 Live broadcast room image rendering method and device, computer equipment and storage medium
CN113038228B (en) * 2021-02-25 2023-05-30 广州方硅信息技术有限公司 Virtual gift transmission and request method, device, equipment and medium thereof
CN113038228A (en) * 2021-02-25 2021-06-25 广州方硅信息技术有限公司 Virtual gift transmission and request method, device, equipment and medium thereof
CN113163220B (en) * 2021-02-26 2023-02-28 广州方硅信息技术有限公司 Virtual gift positioning display method and device, equipment and medium thereof
CN113163220A (en) * 2021-02-26 2021-07-23 广州方硅信息技术有限公司 Virtual gift positioning display method and device, equipment and medium thereof
CN113139913A (en) * 2021-03-09 2021-07-20 杭州电子科技大学 New view correction generation method for person portrait
CN113139913B (en) * 2021-03-09 2024-04-05 杭州电子科技大学 New view correction generation method for portrait
CN113196785B (en) * 2021-03-15 2024-03-26 百果园技术(新加坡)有限公司 Live video interaction method, device, equipment and storage medium
CN113196785A (en) * 2021-03-15 2021-07-30 百果园技术(新加坡)有限公司 Live video interaction method, device, equipment and storage medium
CN114501041B (en) * 2021-04-06 2023-07-14 抖音视界有限公司 Special effect display method, device, equipment and storage medium
CN114501041A (en) * 2021-04-06 2022-05-13 北京字节跳动网络技术有限公司 Special effect display method, device, equipment, storage medium and product
WO2022213979A1 (en) * 2021-04-06 2022-10-13 北京字节跳动网络技术有限公司 Special effect display method, apparatus and device, storage medium, and product
CN113395533A (en) * 2021-05-24 2021-09-14 广州博冠信息科技有限公司 Virtual gift special effect display method and device, computer equipment and storage medium
CN113766258A (en) * 2021-07-27 2021-12-07 广州方硅信息技术有限公司 Live broadcast room virtual gift presentation processing method, equipment and storage medium
CN113766258B (en) * 2021-07-27 2023-04-11 广州方硅信息技术有限公司 Live broadcast room virtual gift presentation processing method and equipment and storage medium
CN113596498A (en) * 2021-07-29 2021-11-02 广州方硅信息技术有限公司 Virtual gift display method, device, system and storage medium in live broadcast
CN115706821A (en) * 2021-08-17 2023-02-17 上海哔哩哔哩科技有限公司 Virtual gift display method and device
CN115706821B (en) * 2021-08-17 2024-04-30 上海哔哩哔哩科技有限公司 Virtual gift display method and device
CN113473211A (en) * 2021-08-20 2021-10-01 广州博冠信息科技有限公司 Virtual gift processing method and device, storage medium and electronic equipment
CN113873267A (en) * 2021-08-20 2021-12-31 广州方硅信息技术有限公司 Live broadcast room virtual gift crown name method, server, live broadcast system and storage medium
CN113873267B (en) * 2021-08-20 2024-01-19 广州方硅信息技术有限公司 Live broadcast room virtual gift naming method, server, live broadcast system and storage medium
WO2023029768A1 (en) * 2021-08-30 2023-03-09 北京字跳网络技术有限公司 Interaction method and apparatus, electronic device, readable storage medium, and program product
CN114025188A (en) * 2021-10-28 2022-02-08 广州虎牙科技有限公司 Live broadcast advertisement display method, system, device, terminal and readable storage medium
CN114222147B (en) * 2021-11-03 2023-10-03 广州方硅信息技术有限公司 Live broadcast layout adjustment method and device, storage medium and computer equipment
CN114222147A (en) * 2021-11-03 2022-03-22 广州方硅信息技术有限公司 Live broadcast layout adjustment method and device, storage medium and computer equipment
CN114125488A (en) * 2021-12-09 2022-03-01 小象(广州)商务有限公司 Virtual gift display method and system in live broadcast
CN114237792A (en) * 2021-12-13 2022-03-25 广州繁星互娱信息科技有限公司 Virtual object display method and device, storage medium and electronic equipment
CN114422821A (en) * 2022-02-14 2022-04-29 广州方硅信息技术有限公司 Live broadcast home page interaction method, device, medium and equipment based on virtual gift
CN114567793A (en) * 2022-02-23 2022-05-31 广州博冠信息科技有限公司 Method and device for realizing live broadcast interactive special effect, storage medium and electronic equipment
CN115134627A (en) * 2022-07-01 2022-09-30 广州博冠信息科技有限公司 Live broadcast room information processing method and device, readable storage medium and electronic equipment
CN115484472A (en) * 2022-09-23 2022-12-16 广州方硅信息技术有限公司 Special effect playing and processing method and device for live broadcast room, electronic equipment and storage medium

Also Published As

Publication number Publication date
WO2021047094A1 (en) 2021-03-18
CN110493630B (en) 2020-12-01

Similar Documents

Publication Publication Date Title
CN110493630A (en) The treating method and apparatus of virtual present special efficacy, live broadcast system
CN110475150A (en) The rendering method and device of virtual present special efficacy, live broadcast system
CN110536151A (en) The synthetic method and device of virtual present special efficacy, live broadcast system
CN108010037B (en) Image processing method, device and storage medium
TWI826321B (en) A method for enhancing quality of media
CN1849823B (en) Video communications method and system
CN102271241A (en) Image communication method and system based on facial expression/action recognition
CN110213601A (en) A kind of live broadcast system and live broadcasting method based on cloud game, living broadcast interactive method
CN106789991A (en) A kind of multi-person interactive method and system based on virtual scene
US9210372B2 (en) Communication method and device for video simulation image
CN106713988A (en) Beautifying method and system for virtual scene live
CN108236784B (en) Model training method and device, storage medium and electronic device
CN110809173B (en) Virtual live broadcast method and system based on AR augmented reality of smart phone
CN109302628A (en) A kind of face processing method based on live streaming, device, equipment and storage medium
CN110446056A (en) A kind of method for processing video frequency, equipment, system and computer readable storage medium
CN110969572A (en) Face changing model training method, face exchanging device and electronic equipment
CN106993215A (en) A kind of information processing method and processing routine device
JP2000023036A (en) Image data processing method and its device and storage medium
EP3808073A1 (en) Method and system for automatic real-time frame segmentation of high resolution video streams into constituent features and modifications of features in each frame to simultaneously create multiple different linear views from same video source
CN116301322A (en) Output method and electronic equipment
CN116420351A (en) Providing 3D representations of sending participants in a virtual conference
CN106791890A (en) A kind of method and apparatus for building multi-angle frame TV program
CN113706673A (en) Cloud rendering framework platform applied to virtual augmented reality technology
CN113378923A (en) Image generation device acquisition method and image generation device
CN112702625A (en) Video processing method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20210115

Address after: 511442 3108, 79 Wanbo 2nd Road, Nancun Town, Panyu District, Guangzhou City, Guangdong Province

Patentee after: GUANGZHOU CUBESILI INFORMATION TECHNOLOGY Co.,Ltd.

Address before: 511442 29 floor, block B-1, Wanda Plaza, Huambo business district, Panyu District, Guangzhou, Guangdong.

Patentee before: GUANGZHOU HUADUO NETWORK TECHNOLOGY Co.,Ltd.