CN116962742A - Live video image data transmission method, device and live video system - Google Patents
Live video image data transmission method, device and live video system Download PDFInfo
- Publication number
- CN116962742A CN116962742A CN202210593874.7A CN202210593874A CN116962742A CN 116962742 A CN116962742 A CN 116962742A CN 202210593874 A CN202210593874 A CN 202210593874A CN 116962742 A CN116962742 A CN 116962742A
- Authority
- CN
- China
- Prior art keywords
- image
- video
- information
- live broadcast
- anchor
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 71
- 230000005540 biological transmission Effects 0.000 title claims abstract description 48
- 238000003709 image segmentation Methods 0.000 claims abstract description 45
- 238000012545 processing Methods 0.000 claims abstract description 26
- 230000011218 segmentation Effects 0.000 claims abstract description 12
- 230000000694 effects Effects 0.000 claims description 90
- 241000209140 Triticum Species 0.000 claims description 24
- 235000021307 Triticum Nutrition 0.000 claims description 24
- 230000003993 interaction Effects 0.000 claims description 17
- 238000009877 rendering Methods 0.000 claims description 10
- 230000006399 behavior Effects 0.000 claims description 5
- 238000007781 pre-processing Methods 0.000 claims description 5
- 238000013144 data compression Methods 0.000 claims description 4
- 239000000284 extract Substances 0.000 claims description 3
- 230000008569 process Effects 0.000 description 23
- 238000010586 diagram Methods 0.000 description 17
- 230000006870 function Effects 0.000 description 16
- 238000007906 compression Methods 0.000 description 8
- 230000006835 compression Effects 0.000 description 8
- 230000002452 interceptive effect Effects 0.000 description 8
- 238000004364 calculation method Methods 0.000 description 6
- 230000015572 biosynthetic process Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 5
- 230000002829 reductive effect Effects 0.000 description 5
- 238000003786 synthesis reaction Methods 0.000 description 5
- 230000009471 action Effects 0.000 description 4
- 230000003796 beauty Effects 0.000 description 3
- 230000006837 decompression Effects 0.000 description 3
- 238000010899 nucleation Methods 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000000227 grinding Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000013500 data storage Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000000670 limiting effect Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000002156 mixing Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000036961 partial effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 230000005236 sound signal Effects 0.000 description 1
- 238000009331 sowing Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000000153 supplemental effect Effects 0.000 description 1
- 230000002194 synthesizing effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/23424—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2187—Live feed
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
- H04N21/234345—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44016—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving splicing one content stream with another content stream, e.g. for substituting a video clip
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440245—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
- H04N21/4788—Supplemental services, e.g. displaying phone caller identification, shopping application communicating with other users, e.g. chatting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Marketing (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Reverberation, Karaoke And Other Acoustics (AREA)
- Processing Or Creating Images (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The application relates to a video image data transmission method and device for live webcasting and a live webcasting system, wherein the method comprises the following steps: collecting video images of a host, and extracting each image frame of the video images; identifying target areas of each image frame, and acquiring image segmentation information of the image frames according to an identification result; respectively associating the image frames of the image frames with image segmentation information, and compressing the image frames into video image data in a set format; transmitting the video image data to a live broadcast server, so that the live broadcast server performs segmentation processing on corresponding image frames according to the image segmentation information; according to the technical scheme, the data processing efficiency of the live broadcast server can be improved on the basis of ensuring live broadcast interactivity.
Description
The application claims priority of China patent application filed by China patent office at 14 month 04 in 2022, with application number of 202210387780. X and the name of 'online live wheat connecting interaction method, device and live broadcasting system', and the application file content is incorporated by reference.
Technical Field
The application relates to the technical field of network live broadcasting, in particular to a video image data transmission method and device for network live broadcasting and a live broadcasting system.
Background
With the iterative upgrade of new applications of new technologies of the mobile internet, the network live broadcast industry enters a rapid development period, and the media attribute, the social attribute, the business attribute and the entertainment attribute of the network live broadcast industry are increasingly prominent, so that the network ecology is deeply influenced. In network live broadcast, video data collected by a client must be transmitted to a server in time for processing, and high requirements are required for real-time performance and video quality of video data transmission, on one hand, the required transmission video data volume needs to be reduced as much as possible, so that the time delay is ensured to meet the requirements, and meanwhile, the original video data and related information of other live video are required to be transmitted to the live broadcast server as much as possible, so that more interactive application can be performed in the network live broadcast process, and the calculation pressure of the live broadcast server is reduced, so that the influence on the live broadcast interaction effect caused by carrying out a large amount of data calculation on the live broadcast server is avoided.
Currently, in the network live broadcast process, a method is that video data of a host broadcast is collected at a client side, cut and the like are processed, and then the processed video data is uploaded to a live broadcast server, but the original image information collected at the host broadcast side and the image related information thereof are easily caused to be incapable of being applied to the live broadcast server in the mode; and the original video data and the image related information of the anchor are transmitted to the server for processing, but the mode can lead to the increase of the calculation pressure of the live broadcast server, and the video processing efficiency and the interaction effect are affected.
Therefore, the existing live video data transmission scheme is difficult to ensure the data processing efficiency of the live server on the basis of improving live interaction, and the real-time effect of network live broadcast is easily affected in the real-time live broadcast process.
Disclosure of Invention
Based on the foregoing, it is necessary to provide a method, an apparatus and a live broadcast system for transmitting video image data of live broadcast according to at least one technical defect, which can ensure the data processing efficiency of the live broadcast server and improve the real-time effect of live broadcast on the basis of improving the live broadcast interactivity.
A video image data transmission method of live webcast includes:
collecting video images of a host, and extracting each image frame of the video images;
identifying target areas of each image frame, and acquiring image segmentation information of the image frames according to an identification result;
respectively associating the image frames of the image frames with image segmentation information, and compressing the image frames into video image data in a set format;
and transmitting the video image data to a live broadcast server, so that the live broadcast server performs segmentation processing on corresponding image frames according to the image segmentation information.
In one embodiment, identifying a target area of each image frame and acquiring image segmentation information of the image frame according to the identification result includes:
and recognizing the human images of each image frame to obtain a human image region, and acquiring Alpha images of the image frames according to the human image region.
In one embodiment, the associating the image frames of the image frames with the image segmentation information, and compressing the image frames into the video image data in the set format includes:
transversely splicing each image frame with the corresponding Alpha image to obtain spliced image frames;
coding the spliced image frames based on YUV format to obtain video coding data;
and compressing the video coding data into video image data in a set format.
In one embodiment, the anchor comprises at least two anchors connected with the wheat, and the image segmentation information is image matting information of video images of the anchor;
and the live broadcast server also extracts the portrait images from the video images of each wheat-linking anchor according to the portrait matting information, and synthesizes the portrait images of each wheat-linking anchor with the background image to obtain a virtual same-platform scene.
In one embodiment, the compressing the video encoded data into video image data in a set format includes:
acquiring specific information and image related information of a host;
adding the special effect information and the image related information to SEI information;
the video stream encoding and the SEI information are compressed into video image data in a set format.
In one embodiment, the obtaining the anchor effect information includes:
carrying out AI identification on the portrait images of each image frame of the video image to obtain an AI key point of the anchor;
and acquiring special effect information of the portrait image of the anchor according to the AI key points.
In one embodiment, the live broadcast server further performs live broadcast and wheat connection interaction based on the virtual same-platform scene, draws special effect information on the virtual same-platform scene, and pushes a wheat connection video stream generated by the live broadcast and wheat connection interaction to a viewer side.
In one embodiment, the special effects information includes a virtual special effects ID and a display location;
the virtual special effect ID is used for inquiring virtual special effect content on the live broadcast server, and the display position is used for converting the virtual special effect content position on the live broadcast server according to the anchor AI key point to obtain a rendering position.
In one embodiment, the method for transmitting video image data of live webcast further includes:
receiving video image data uploaded by other anchor terminals forwarded by a live broadcast server;
decompressing the video image data to obtain video coding data and SEI information;
decoding the video coding data to obtain image frames of other anchor terminals and Alpha images of the other anchor terminals; acquiring special effect information and image related information of other anchor from the SEI information; the Alpha image is used for matting out the portrait image from the image frame.
In one embodiment, the image related information includes one or more of AI keypoints, image URLs, anchor behavior information, anchor on-hook effect information, or virtual gift information.
A live video image data transmission apparatus, comprising:
the image acquisition module is used for acquiring video images of the anchor and extracting each image frame of the video images;
the image preprocessing module is used for identifying target areas of each image frame and acquiring image segmentation information of the image frames according to an identification result;
the data compression module is used for respectively associating the image frames of the image frames with the image segmentation information and compressing the image frames into video image data in a set format;
And the data transmission module is used for transmitting the video image data to a live broadcast server, so that the live broadcast server performs segmentation processing on the corresponding image frames according to the image segmentation information.
A live broadcast system, comprising at least one live broadcast client and a live broadcast server; wherein the client is configured to perform the live video image data transmission method of any one of the above.
A computer device, comprising:
one or more processors;
a memory;
one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more applications configured to perform the live video image data transmission method described above.
A computer readable storage medium storing at least one instruction, at least one program, code set, or instruction set, the at least one instruction, the at least one program, the code set, or instruction set being loaded by the processor and performing the live video image data transmission method described above.
According to the technical scheme of the embodiments, during network live broadcast, a live broadcast terminal collects video images of a live broadcast and extracts image frames, target areas of the image frames are identified to obtain image segmentation information, the image frames of the image frames are respectively associated with the image segmentation information and then compressed into video image data in a set format, and the video image data are transmitted to a live broadcast server for segmentation processing; according to the technical scheme, the target area is firstly identified and processed at the anchor end, so that image segmentation information is transmitted to the live broadcast server in an associated mode to be segmented and processed, original video image data is transmitted to the live broadcast server, follow-up application interactivity can be confirmed, meanwhile, an identification process is completed at the anchor end, the live broadcast server can achieve segmentation and processing without identification, and data processing efficiency of the live broadcast server is improved on the basis of guaranteeing live broadcast interactivity.
Furthermore, by carrying out image recognition on the image frames to obtain Alpha images and transversely splicing each image frame with the corresponding Alpha images, video image data can be compressed in a compression mode of H264, HEVC and the like, so that compatible use on the existing anchor terminal is ensured, the use cost is reduced, the complexity of encoding cannot be increased, and the encoding efficiency is high.
Furthermore, in the live broadcasting process of the link wheat, video image data uploaded by other anchor terminals can be forwarded to the anchor terminal locally through the live broadcasting server, so that the anchor terminal can locally synthesize live video images of the link wheat, and the real-time performance of the link wheat anchor is ensured.
Furthermore, when video image data of network live broadcast are transmitted, AI key points and special effect information of a main broadcasting end can be added into the video image data, and the rendering position of virtual special effect content can be obtained on a live broadcast server through AI key point conversion, so that quick virtual special effect content rendering is realized, and the virtual special effect content display effect is improved.
Drawings
FIG. 1 is an exemplary network topology of a live network;
FIG. 2 is a flow chart of a method of video image data transmission for live webcasting in one embodiment;
FIG. 3 is an exemplary image segmentation information diagram;
FIG. 4 is a schematic diagram illustrating the lateral stitching of an image frame with an Alpha image;
FIG. 5 is a schematic illustration of an exemplary YUVA coding;
FIG. 6 is a schematic diagram of an exemplary compression format;
FIG. 7 is a diagram of an example multi-anchor live-link framework;
fig. 8 is a schematic structural diagram of a live video image data transmission apparatus according to an embodiment;
FIG. 9 is a schematic diagram of an exemplary live system architecture;
FIG. 10 is a timing diagram of an exemplary anchor interacting with a live server;
fig. 11 is a block diagram of an exemplary apparatus for performing a live video image data transmission method.
Detailed Description
The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.
In the embodiments of the present application, reference to "first," "second," etc. is used to distinguish between identical items or similar items that have substantially the same function and function, "at least one" means one or more, "a plurality" means two or more, for example, a plurality of objects means two or more. The words "comprise" or "comprising" and the like mean that information preceding the word "comprising" or "comprises" is meant to encompass the information listed thereafter and equivalents thereof as well as additional information not being excluded. Reference to "and/or" in embodiments of the application indicates that there may be three relationships, and the character "/" generally indicates that the associated object is an "or" relationship.
Referring to fig. 1, fig. 1 is an exemplary network topology of a live network; at least one main broadcasting end is connected to a live broadcasting server, network live broadcasting is carried out on audience terminals through the live broadcasting server, only one main broadcasting end is shown in the figure, in practical situations, a plurality of main broadcasting ends can carry out live broadcasting, in the network live broadcasting network, the live broadcasting server can process video image data uploaded by the main broadcasting end, a live broadcasting video stream is generated and is sent to each audience terminal, and video image data are transmitted between the main broadcasting end and the live broadcasting server.
Based on the network live network topological graph, the embodiment provides a network live video image data transmission method which is applied to a main broadcasting end (also called a main broadcasting client), and mainly realizes the functions of main broadcasting video acquisition, conversion, coding transmission and the like for the main broadcasting end; the video playing system can comprise a playing tool and a client side, wherein the playing tool is video preprocessing software, can be realized through a mobile phone end, integrates a virtual camera, has various functions such as beautifying, drawing and the like, the client side is a software client side based on voice and network live broadcasting, can be realized based on a PC end, and a host in live broadcasting can provide a live broadcasting template of various types (entertainment/friend making/war/game/education and the like) for the host to select.
As shown in fig. 2, fig. 2 is a flowchart of a video image data transmission method of live webcasting according to an embodiment, including the following steps:
s10, acquiring video images of a host, and extracting each image frame of the video images.
When the live broadcasting is started, the main broadcasting end is connected with the live broadcasting server, and when the live broadcasting and the wheat linking are needed to be carried out with other main broadcasting ends, the interconnection among the main broadcasting participating in the wheat linking is also established through the live broadcasting server. Before network live broadcast, the anchor can set the relevant mode of the anchor, set the output video image parameters, etc.
In the live broadcast process, video image data of a host broadcast are collected in real time, and then each image frame data is extracted from the video image.
S20, identifying target areas of the image frames, and acquiring image segmentation information of the image frames according to identification results.
In this step, the image division information of the target area is determined by identifying the target area of each image frame, and for the target area, the designated target area in the image frame, such as a portrait image, a virtual special effect image, etc., the image division information of the target area of each image frame can be formed by preprocessing on the anchor side, for example, referring to fig. 3, fig. 3 is an exemplary image division information schematic diagram, and the contour edge division line of the portrait image is identified as the image division information when the portrait image and the background image are divided.
In the process, the image segmentation information can be generated by using an AI algorithm after identification, the AI algorithm occupies a large calculation amount, and compared with the conventional AI algorithm, the image segmentation is performed after identification, in the embodiment, the segmentation preprocessing process is performed at the data source end, and then the segmentation processing link is placed on the live broadcast server, so that the calculation amount of the live broadcast server can be greatly reduced.
In one embodiment, the image segmentation information acquisition process may include the following on the basis of the image segmentation of the portrait based on the anchor:
and recognizing the human images of each image frame to obtain a human image region, and acquiring Alpha images of the image frames according to the human image region.
In the above technical solution, the AI algorithm may be used to perform algorithm recognition on the portrait image in the image frame, so that the portrait image area of the anchor may be recognized to perform matting, where the matting is to obtain a useful partial image, and then an Alpha image is generated according to the portrait image area, and for the Alpha image in this embodiment, the Alpha image is a black and white picture generated based on the Alpha channel principle.
The Alpha Channel is also called an Alpha Channel (english is Alpha Channel), and refers to transparency and translucence of a picture, and in general, the Alpha Channel is a Channel in video image data encoded by multiple channels, and the Channel can be used as a transparent effect in the process of the above picture, and the general Alpha value is between 0 and 1.
In this embodiment, a new Alpha image is generated based on the Alpha channel, and the image is used to carry image segmentation information (or matting information).
And S30, respectively associating the image frames of the image frames with the image segmentation information, and compressing the image frames into video image data in a set format.
In this step, the image division information can be associated with each image frame, so that the image division information obtained when the image frame is preprocessed can be made to take the image frame and compressed into video image data of a set format.
Taking the image segmentation information as an example, each image frame carries the image segmentation information and is compressed into video image data with a set format, when the video image data is required to be used, the image segmentation information can be obtained after decompression of the set format, and the image segmentation information is utilized to rapidly segment the image from the image frame.
In one embodiment, as in the previous embodiment, alpha images are employed to carry image segmentation information, whereby the process of compressing video image data may include the following:
and S301, transversely splicing each image frame with the corresponding Alpha image to obtain spliced image frames.
Specifically, instead of adding an Alpha transparent channel (such as conventional YUVA or RGBA) as a video image format, the scheme of this embodiment uses an image splicing manner to splice the image frames and corresponding Alpha images into a new image frame, so that when the original image frame is used, the image segmentation information can be obtained from the Alpha images decoded at the splicing positions of the original image frame, and then the image frames can be segmented by using the image segmentation information.
The embodiment adopts the spliced Alpha image scheme, and only expands the image frame by one area, so that the method can be directly compatible and used under the condition of not changing the original coding mode of the anchor end, other clients such as web and old version software can not be modified, alpha images can be seen, and the method can be realized by using widely used coding algorithms such as H264 and HEVC, and software and hardware can be realized.
In the case of the splicing method, a transverse splicing method or an up-down longitudinal splicing method may be adopted, and the description is not expanded one by one, referring to fig. 4, fig. 4 is a schematic diagram showing the transverse splicing of an example image frame and an Alpha image, after the image frame and the Alpha image are spliced, two image frames including a left image frame and a right image frame are formed, and when encoding, some marking information, for example, a L, R marking mark is used for distinguishing the left image frame and the right image frame, so that the required original image frame and the required image segmentation information can be conveniently decoded.
S302, encoding the spliced image frames based on YUV format to obtain video encoding data.
In the process, a coding format can be selected to code the spliced image, and the spliced Alpha image is expanded to be equivalent to a YUV area on the right side of the image frame for storing Alpha information, and the Alpha image only needs 8 bits and only needs Y components, so that the coding does not bring about the increase of complexity cost.
For example, the anchor end performs video encoding in YUV format, so that the spliced image can also be encoded in YUV format, and the spliced image can be distinguished by L, R marks as described above, so that L, R marks can be added before each YUV data of the left and right images to distinguish, for example, LY1, LU2, …, LV1, RY1, …, RU1, RV1, …, and the like; for the spliced image encoding process, reference may be made to fig. 5, where fig. 5 is a schematic diagram of an exemplary YUVA encoding, and the YUVA encoding shown in the figure is different from the conventional YUVA channel encoding, and the encoding is performed by adding an extended region as a part of an image, instead of adding a channel based on YUV three-channel data, so that the encoding process is simpler.
And S303, compressing the video coding data into video image data in a set format.
In this step, after encoding the video stream, in order to transmit the video image to the live broadcast server, compression transmission is required to be performed on the video encoded data at the anchor side; as described above, in compression transmission, the video image data may be compressed by compression methods such as H264 and HEVC.
In order to improve the application function of video image data in a live broadcast server and enhance interactivity, when the video image data is compressed, some control information, special effect information of images and image related information can be added into the video image data, so that the video image data can be further expanded and used in function on the live broadcast server, special effect information such as some mapping special effect content on a main broadcasting end can be redrawn on the live broadcast server, and meanwhile interaction can be carried out between the live broadcast server and other continuous wheat main broadcasting, so that the live broadcast interactivity effect is improved.
Accordingly, as an embodiment, the video image data compression process of step S303 may include the following:
(1) And acquiring the special effect information and the image related information of the anchor.
The special effect information may be a mapping special effect, a virtual special effect and the like on the anchor side, and the image related information may be information content related to the current anchor image, such as picture information, URL connection, virtual gift information and the like, through which some application information related to the anchor image may be added, so that other applications may be performed on the live broadcast server.
As an embodiment, based on the special effect information described in the foregoing embodiment, taking the mapped special effect as an example, the corresponding method for obtaining the special effect information may include the following steps:
a. carrying out AI identification on the portrait images of each image frame of the video image to obtain an AI key point of the anchor; specifically, the image of the anchor is identified by using an AI algorithm on the anchor, so that AI key points, such as face key points, hand key points, trunk key points, etc., can be obtained.
b. Acquiring special effect information of the portrait image of the anchor according to the AI key points; specifically, the anchor special effect position and related information can be determined through the AI key points, so that the anchor special effect position and related information can be added into video image data for transmission.
As an embodiment, the special effect information includes a virtual special effect ID and a display position, the virtual special effect content can be queried on the live broadcast server through the virtual special effect ID, in general, the virtual special effect has a corresponding special effect server, and the corresponding special effect content can be downloaded to the special effect server through the virtual special effect ID; the display position is mainly used for determining the position area of the virtual special effect relative to the portrait image, so that the position of the virtual special effect content can be converted on the live broadcast server according to the AI key point of the anchor, the rendering position is obtained, the downloaded virtual special effect content is used for redrawing the virtual special effect of the anchor end on the corresponding position, the separation transmission of the virtual special effect and the anchor portrait image is realized, and the virtual special effect can be prevented from being filtered in the processing process of the image frame by the anchor end.
As in the above embodiment, the image related information may include AI key points, image URLs, anchor action information, anchor on-hook special effect information, virtual gift information, and so on, and in practical application, corresponding image related information may be added according to functional requirements, so that the transmitted video image data may carry more useful information, and thus may be used on a live broadcast server.
(2) The special effect information and the image related information are added to the SEI information.
In H264, SEI (Supplemental Enhancement Information ), which is custom information in H264.
(3) The video stream encoding and the SEI information are compressed into video image data in a set format.
Specifically, the encoding algorithms such as H264 and HEVC may be used to compress, through SEI information, required special effect information and image related information may be compressed into video image data for transmission, and the video image data may be decompressed on a live broadcast server to obtain information content carried therein, for a compression format of transmission, referring to fig. 6, fig. 6 is a schematic diagram of an example compression format, where a spliced image of an image frame and an Alpha image is encoded according to a YUVA video frame format, and AI key points, special effect information, and a background image URL are added to the SEI information for transmission.
According to the scheme of the embodiment, the special effect content is separated from the portrait image, so that more special effect content can be accessed at the anchor end, the special effect content of each link can be uniformly displayed, and the special effect content display is not affected by the processing of dividing, matting and the like of the original image frame.
And S40, transmitting the video image data to a live broadcast server, so that the live broadcast server performs segmentation processing on the corresponding image frames according to the image segmentation information.
In the step, video image data (in the form of video stream information) of a main broadcasting end are transmitted to a live broadcasting server, when the video image data are used on the live broadcasting server, the image segmentation information can be obtained after decompression of a set format, and the image segmentation information is utilized to rapidly segment the image from the image frame, so that the algorithm calculation force on the live broadcasting server is reduced, the recording information of the image of the original image frame is ensured, and various special effect contents are added before transmission, so that the identification influence on the image of the original image is not influenced.
Based on the technical scheme of the embodiment, the main broadcasting end can acquire the original video image through the camera, the broadcasting tool can perform face beautifying, skin grinding, face thinning and other face beautifying functions on the original video image, the client side performs image matting processing on the original video image, in the process, the original video image is further subjected to image matting and behavior data (such as arm motion, gesture, whole outline of a body and the like) extraction to obtain Alpha video image data, the client side receives video frame data with the Alpha image output by the broadcasting tool, the original video image of the video frame data and the Alpha image are transversely spliced, compression and transmission are performed based on conventional YUV video frame formats, and meanwhile, related image information such as AI key point information, face, gesture, head and other key point information, broadcasting special effect information, playing gift information and the like are transmitted by using SEI information.
In one embodiment, as in the technical solution of the foregoing embodiment, when the method is applied to live webcasting, a plurality of anchor terminals participate in the live broadcast process, so when a hybrid video image needs to be obtained in a hybrid mode, the webcast needs to quickly obtain a preview video image, and in a conventional mode, the live broadcast server returns to the anchor terminal after performing hybrid synthesis, the process is relatively complex, and the influence of network bandwidth and the like is easy to generate time delay, so that the anchor cannot preview the hybrid video image in time.
In one embodiment, the technical scheme of the application can be used for continuously seeding a plurality of anchor terminals in a continuously seeding scene, therefore, in the embodiment, the anchor includes at least two anchor terminals, the image segmentation information is the image matting information of the video image of the anchor, and on a live seeding server, the images of the video images of each anchor are segmented through the image matting information and are overlapped on a background image in a mixing manner, thereby realizing the interaction purpose of virtual same-platform scenes.
Because relevant details of the original video image are reserved in the uploaded video image data, the video image data can be processed on a live broadcast server at a later stage, so that the whole technical scheme has better expansibility; and the time delay of live interaction among the anchor is hardly increased compared with that of the common interaction mode, and the real-time performance is good.
As shown in fig. 1, a live broadcast system architecture is shown, a live broadcast end a and a live broadcast end B participate in live broadcast are connected with a live broadcast server to perform live broadcast, and meanwhile, image matting information (described as Alpha images) is utilized to scratch out image images from video images of each live broadcast according to the image matting information on the live broadcast server, and the image images of each live broadcast are synthesized with background images to obtain a virtual same-platform scene.
The virtual same-platform scene refers to a synthesized virtual scene formed by carrying out mixed drawing on the basis of the portrait image and the background image, and can be expressed as a concept fused in one same-platform scene; the anchor side A and the anchor side B upload video image data of the wheat-connected anchor to a live broadcast server, and the live broadcast server can decode the video image data to obtain an original video image of the anchor, and then uses Alpha images to perform matting and synthesis to obtain a virtual same-platform scene; each wheat linking anchor can carry out live wheat linking interaction in a virtual same-platform scene, virtual gift giving, live broadcasting interaction and the like, including same-platform KTV, same-platform dancing, same-platform knowledge competition and the like, and the live broadcasting server pushes the generated wheat linking video stream to the audience terminal.
Preferably, when the live broadcast server pushes the link video stream to the audience, the link video stream can be pushed to a CDN (Content Delivery Network ) network to be delivered to the audience by acquiring a link video image and an interactive special effect generated by the link anchor performing interactive live broadcast on the virtual same scene, and encoding the link video image and the interactive special effect into the link video stream.
According to the technical scheme, the virtual same-platform wheat connecting effect is achieved, live broadcast content is enriched, live broadcast interactivity is enhanced, immersive experience of live broadcast wheat connecting is improved, different region anchor wheat connecting is achieved under a unified virtual scene, multiple interactive playing methods are derived based on the virtual same-platform scene, and the wheat connecting interaction effect is greatly improved.
In order to improve the speed of obtaining the mixed picture by the anchor side, the embodiment also provides a technical scheme that the live broadcast server forwards video image data of other anchors to the anchor side for mixed picture synthesis, and specifically, the method for transmitting the video image data of the network live broadcast of the embodiment can further comprise:
s501, receiving video image data uploaded by other anchor broadcast forwarded by a live broadcast server; specifically, taking fig. 7 as an example, fig. 7 is an exemplary frame diagram of a plurality of live broadcast frames of a host, in which, taking a host a and a host B as examples, a live broadcast server forwards video image data uploaded by the host a and the host B to each other. Meanwhile, at the local of the anchor, the anchor can add beauty Yan Texiao, such as fat face, red face, sunglasses and the like, and the green background of the green curtain video image is segmented based on the green curtain matting to obtain Alpha images.
S502, decompressing the video image data to obtain video coding data and SEI information; since special effect information, AI key points, image related information and the like are added in the SEI data of the video image data, the special effect information, the AI key points, the image related information and the like in the video coding data and the SEI information of the original image frame can be obtained through decompression.
S503, decoding the video coding data to obtain image frames of other anchor terminals and Alpha images of the other anchor terminals; and acquiring special effect information and image related information of other anchor from the SEI information.
Specifically, firstly, decoding is carried out through video coding data to obtain other anchor image frames and corresponding Alpha images, the Alpha images are utilized to key the image frames, and the portrait images in the image frames can be key, namely, the picture segmentation process is completed to obtain portrait images; and then acquiring special effect information, AI key points, image related information and the like of other anchor terminals from the SEI information, calculating a rendering position by using the AI key points, re-rendering and drawing special effect content locally by using the special effect information, downloading background images by using the image related information, acquiring behavior actions of the anchor, playing special effect information, playing gift information and the like.
For example, the anchor terminal a may receive the video image data forwarded by the live broadcast server and uploaded by the anchor terminal B, and then synthesize the portrait image of itself with the portrait image transmitted by the anchor terminal B to obtain a mixed-picture video image of the live broadcast link, so that the anchor terminal may obtain a preview video image quickly locally, and draw the map special effect content belonging to the anchor terminal B on the mixed-picture video image.
The technical scheme of the embodiment can be applied to various network live broadcast scenes, particularly to the currently popular live broadcast scenes with the wheat, and by the technical scheme, the interactive effect of live broadcast with the wheat can be obviously improved, meanwhile, the fluency of a live broadcast server can be ensured, and the compatibility of a main broadcasting end is also ensured; since the details of the original video image frames are reserved in the uploaded video image data, the original video image frames can be used for further processing in the later period, and the expansibility is better; in the wheat connecting process, the time delay is hardly increased compared with the common interaction mode, and the real-time performance is good.
An embodiment of a live video image data transmission device is set forth below.
Referring to fig. 8, fig. 8 is a schematic structural diagram of a live video image data transmission apparatus according to an embodiment, including:
the image acquisition module 10 is used for acquiring video images of the anchor and extracting each image frame of the video images;
the image preprocessing module 20 is used for identifying target areas of each image frame and acquiring image segmentation information of the image frames according to an identification result;
a data compression module 30, configured to associate the image frames of the image frames with the image segmentation information, and compress the image frames into video image data in a set format;
the data transmission module 40 is configured to transmit the video image data to a live broadcast server, so that the live broadcast server performs segmentation processing on the corresponding image frames according to the image segmentation information.
The live video image data transmission device of the present embodiment may perform a live video image data transmission method provided by the embodiment of the present application, and its implementation principle is similar, and actions performed by each module in the live video image data transmission device of each embodiment of the present application correspond to steps in the live video image data transmission method of each embodiment of the present application, and detailed functional descriptions of each module in the live video image data transmission device of the present application may be specifically referred to the description in the corresponding live video image data transmission method shown in the foregoing, which is not repeated herein.
An embodiment of a live system is set forth below.
The live broadcast system of the application comprises at least one main broadcasting end and a live broadcast server connected with the main broadcasting end, and referring to fig. 9, fig. 9 is a schematic diagram of an exemplary live broadcast system structure, wherein the live broadcast server is connected with a viewer end; as an embodiment, the anchor side and the live broadcast server are configured to execute the video image data transmission method of the network live broadcast of any embodiment, where the anchor side outputs video image data and uploads the video image data to the live broadcast server for live broadcast interactive application, and the live broadcast server issues the video image data uploaded by other anchor sides to each anchor side for local use.
In order to facilitate the more detailed implementation of the technical solution of the present application, the following describes an example of a scenario of applying the video image data transmission solution provided by the present application. As in the previous embodiments, referring to fig. 10, fig. 10 is an exemplary timing diagram of interaction between a hosting side and a live server; the anchor terminal comprises an opening tool and a client terminal, and the anchor terminal A and the anchor terminal B are taken as examples for illustration.
For the anchor side, the following functions are mainly realized:
(1) The functions of capturing, converting, encoding, compressing and transmitting the video of the anchor are realized;
the sowing tool is responsible for collecting cameras, extracting portrait data by matting the green screen background of the anchor, and outputting image frame data with Alpha channels; for the processing flow of the starting tool, after the camera acquires the original video image, the original video image is subjected to the beauty function processing of beauty, skin grinding, face thinning and the like; then, the original video image is subjected to image matting by using an AI-based algorithm, including image matting and behavior data (such as data of arm actions, gestures, whole contours of a body and the like) of the original video image are extracted to obtain an Alpha image, and the Alpha image is stored by using a YUV format.
The client receives the image frames with Alpha images output by the playing tool, transversely splices the image frames with the Alpha images to obtain spliced images, encodes the spliced images into video coding data, acquires AI key points, map special effects and the like of the anchor end, adds the AI key points, map special effects and the like to SEI information, and compresses the information content by using encoding algorithms such as H264, HEVC and the like.
(2) Synthesizing a mixed picture video image function of the live continuous wheat broadcast locally;
on the basis of the connection of the anchor A and the anchor B, the live broadcast server transmits video image data uploaded by the anchor B to the anchor A, and the anchor A performs mixed drawing synthesis by using the personal image of the anchor A, the personal image of the anchor B and the background image to generate a preview mixed drawing video image.
For a live broadcast server, the following functions are mainly realized:
(1) And (3) dividing video image data uploaded by the anchor terminal, and performing interactive application functions, for example, in the live broadcast of the link, when mixed drawing synthesis is required on a live broadcast server, the video image data uploaded by the anchor terminal A and the anchor terminal B are subjected to image matting by utilizing Alpha images to obtain a portrait image of the anchor A or the anchor B, and then the portrait image and a background image are overlapped and rendered to obtain mixed drawing video images, and the mixed drawing video images are pushed to each audience terminal for viewing.
(2) Realizing a virtual special effect content rendering function; the anchor A and anchor B respectively need the virtual gift special effect that shows in the process of linking the wheat, have transmitted to the live broadcast server through AI key point and special effect information, the live broadcast server can convert and obtain the rendering position of virtual special effect content on the mixed video image according to AI key point and position information, then superpose and carry out the rendering composition on the mixed video image.
Embodiments of a computer device and a computer-readable storage medium are set forth below.
The application provides a technical scheme of computer equipment, which is used for realizing related functions of a video image data transmission method of network live broadcast.
In one embodiment, the present application provides a computer device comprising:
one or more processors;
a memory;
one or more applications, wherein the one or more applications are stored in memory and configured to be executed by the one or more processors, the one or more applications configured for the live video image data transmission method of any of the embodiments.
As shown in FIG. 11, FIG. 11 is a block diagram of an example computer device. The computer device may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, an exercise device, a personal digital assistant, or the like. Referring to fig. 11, the apparatus 1000 may include one or more of the following components: a processing component 1002, a memory 1004, a power component 1006, a multimedia component 1008, an audio component 1010, an input/output (I/O) interface 1012, a sensor component 1014, and a communication component 1016.
The processing component 1002 generally controls overall operation of the apparatus 1000, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations.
The memory 1004 is configured to store various types of data to support operations at the device 1000. Such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.
The power supply component 1006 provides power to the various components of the device 1000.
The multimedia component 10010 includes a screen between the device 1000 and the user that provides an output interface. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). In some embodiments, the multimedia assembly 1008 includes a front-facing camera and/or a rear-facing camera.
The audio component 1010 is configured to output and/or input audio signals.
The I/O interface 1012 provides an interface between the processing assembly 1002 and peripheral interface modules, which may be a keyboard, click wheel, buttons, and the like. These buttons may include, but are not limited to: homepage button, volume button, start button, and lock button.
The sensor assembly 1014 includes one or more sensors for providing status assessment of various aspects of the device 1000. The sensor assembly 1014 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact.
The communication component 1016 is configured to facilitate communication between the apparatus 1000 and other devices, either wired or wireless. The device 1000 may access a wireless network based on a communication standard, such as WiFi, an operator network (e.g., 2G, 3G, 4G, or 5G), or a combination thereof.
The application provides a technical scheme of a computer readable storage medium, which is used for realizing related functions of a video image data transmission method of network live broadcast. The computer readable storage medium stores at least one instruction, at least one program, code set, or instruction set, the at least one instruction, at least one program, code set, or instruction set being loaded by a processor and performing the live video image data transmission method of any embodiment.
In an exemplary embodiment, the computer-readable storage medium may be a non-transitory computer-readable storage medium including instructions, such as a memory including instructions, for example, the non-transitory computer-readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
The foregoing examples illustrate only a few embodiments of the application, which are described in detail and are not to be construed as limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of protection of the present application is to be determined by the appended claims.
Claims (14)
1. A method for transmitting video image data of live webcast, comprising:
collecting video images of a host, and extracting each image frame of the video images;
identifying target areas of each image frame, and acquiring image segmentation information of the image frames according to an identification result;
respectively associating the image frames of the image frames with image segmentation information, and compressing the image frames into video image data in a set format;
and transmitting the video image data to a live broadcast server, so that the live broadcast server performs segmentation processing on corresponding image frames according to the image segmentation information.
2. The method for transmitting video image data of live broadcast according to claim 1, wherein identifying a target area of each image frame and acquiring image division information of the image frame according to the identification result comprises:
and recognizing the human images of each image frame to obtain a human image region, and acquiring Alpha images of the image frames according to the human image region.
3. The method for transmitting video image data of live broadcast according to claim 1, wherein the step of associating the image frames of the respective image frames with the image division information and compressing the image frames into video image data in a set format includes:
Transversely splicing each image frame with the corresponding Alpha image to obtain spliced image frames;
coding the spliced image frames based on YUV format to obtain video coding data;
and compressing the video coding data into video image data in a set format.
4. The method for transmitting video image data of live webcast of claim 3, wherein the anchor includes at least two anchors connected with a wheat, and the image segmentation information is image matting information of video images of the anchor;
and the live broadcast server also extracts the portrait images from the video images of each wheat-linking anchor according to the portrait matting information, and synthesizes the portrait images of each wheat-linking anchor with the background image to obtain a virtual same-platform scene.
5. The method for transmitting live video image data as claimed in claim 4, wherein said compressing the video encoded data into video image data in a set format comprises:
acquiring specific information and image related information of a host;
adding the special effect information and the image related information to SEI information;
the video stream encoding and the SEI information are compressed into video image data in a set format.
6. The method for transmitting video image data of live webcast of claim 5, wherein the obtaining the effect information of the anchor includes:
carrying out AI identification on the portrait images of each image frame of the video image to obtain an AI key point of the anchor;
and acquiring special effect information of the portrait image of the anchor according to the AI key points.
7. The method according to claim 6, wherein the live broadcast server further performs live broadcast and link-microphone interaction based on the virtual peer scene, draws special effect information on the virtual peer scene, and pushes a link-microphone video stream generated by the live broadcast and link-microphone interaction to a viewer.
8. The method for transmitting video image data of live broadcast according to claim 7, wherein the special effect information includes a virtual special effect ID and a display position;
the virtual special effect ID is used for inquiring virtual special effect content on the live broadcast server, and the display position is used for converting the virtual special effect content position on the live broadcast server according to the anchor AI key point to obtain a rendering position.
9. The method for transmitting live video image data as in claim 5, further comprising:
Receiving video image data uploaded by other anchor terminals forwarded by a live broadcast server;
decompressing the video image data to obtain video coding data and SEI information;
decoding the video coding data to obtain image frames of other anchor terminals and Alpha images of the other anchor terminals; acquiring special effect information and image related information of other anchor from the SEI information; the Alpha image is used for matting out the portrait image from the image frame.
10. The method of claim 9, wherein the image related information includes one or more of AI keypoints, image URLs, anchor behavior information, anchor on-the-fly special effect information, or virtual gift information.
11. A live video image data transmission apparatus, comprising:
the image acquisition module is used for acquiring video images of the anchor and extracting each image frame of the video images;
the image preprocessing module is used for identifying target areas of each image frame and acquiring image segmentation information of the image frames according to an identification result;
the data compression module is used for respectively associating the image frames of the image frames with the image segmentation information and compressing the image frames into video image data in a set format;
And the data transmission module is used for transmitting the video image data to a live broadcast server, so that the live broadcast server performs segmentation processing on the corresponding image frames according to the image segmentation information.
12. A live broadcast system, which is characterized by comprising at least one live broadcast client and a live broadcast server; wherein the client is configured to perform the live video image data transmission method of any of claims 1-10.
13. A computer device, comprising:
one or more processors;
a memory;
one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more applications configured to perform the live video image data transmission method of any of claims 1-10.
14. A computer readable storage medium, characterized in that the storage medium stores at least one instruction, at least one program, a set of codes or a set of instructions, the at least one instruction, the at least one program, the set of codes or the set of instructions being loaded by the processor and executing the live video image data transmission method of any one of claims 1-10.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210387980 | 2022-04-14 | ||
| CN202210387980X | 2022-04-14 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116962742A true CN116962742A (en) | 2023-10-27 |
Family
ID=88441504
Family Applications (7)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210593874.7A Pending CN116962742A (en) | 2022-04-14 | 2022-05-27 | Live video image data transmission method, device and live video system |
| CN202210594789.2A Pending CN116962745A (en) | 2022-04-14 | 2022-05-27 | Mixed drawing method, device and live broadcast system of video image |
| CN202210594781.6A Pending CN116962744A (en) | 2022-04-14 | 2022-05-27 | Live webcast link interaction method, device and live broadcast system |
| CN202210593879.XA Pending CN116962743A (en) | 2022-04-14 | 2022-05-27 | Video image coding and matting method and device and live broadcast system |
| CN202210837532.5A Pending CN116962747A (en) | 2022-04-14 | 2022-07-15 | Real-time chorus synchronization method and device based on network live broadcast and network live broadcast system |
| CN202210837530.6A Pending CN116962746A (en) | 2022-04-14 | 2022-07-15 | Online chorus method and device based on continuous wheat live broadcast and online chorus system |
| CN202210864547.0A Pending CN116962748A (en) | 2022-04-14 | 2022-07-21 | Live video image rendering method and device and live video system |
Family Applications After (6)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210594789.2A Pending CN116962745A (en) | 2022-04-14 | 2022-05-27 | Mixed drawing method, device and live broadcast system of video image |
| CN202210594781.6A Pending CN116962744A (en) | 2022-04-14 | 2022-05-27 | Live webcast link interaction method, device and live broadcast system |
| CN202210593879.XA Pending CN116962743A (en) | 2022-04-14 | 2022-05-27 | Video image coding and matting method and device and live broadcast system |
| CN202210837532.5A Pending CN116962747A (en) | 2022-04-14 | 2022-07-15 | Real-time chorus synchronization method and device based on network live broadcast and network live broadcast system |
| CN202210837530.6A Pending CN116962746A (en) | 2022-04-14 | 2022-07-15 | Online chorus method and device based on continuous wheat live broadcast and online chorus system |
| CN202210864547.0A Pending CN116962748A (en) | 2022-04-14 | 2022-07-21 | Live video image rendering method and device and live video system |
Country Status (1)
| Country | Link |
|---|---|
| CN (7) | CN116962742A (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117196999B (en) * | 2023-11-06 | 2024-03-12 | 浙江芯劢微电子股份有限公司 | Self-adaptive video stream image edge enhancement method and system |
-
2022
- 2022-05-27 CN CN202210593874.7A patent/CN116962742A/en active Pending
- 2022-05-27 CN CN202210594789.2A patent/CN116962745A/en active Pending
- 2022-05-27 CN CN202210594781.6A patent/CN116962744A/en active Pending
- 2022-05-27 CN CN202210593879.XA patent/CN116962743A/en active Pending
- 2022-07-15 CN CN202210837532.5A patent/CN116962747A/en active Pending
- 2022-07-15 CN CN202210837530.6A patent/CN116962746A/en active Pending
- 2022-07-21 CN CN202210864547.0A patent/CN116962748A/en active Pending
Also Published As
| Publication number | Publication date |
|---|---|
| CN116962743A (en) | 2023-10-27 |
| CN116962748A (en) | 2023-10-27 |
| CN116962746A (en) | 2023-10-27 |
| CN116962747A (en) | 2023-10-27 |
| CN116962745A (en) | 2023-10-27 |
| CN116962744A (en) | 2023-10-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106303289B (en) | Method, device and system for fusion display of real object and virtual scene | |
| CN106789991B (en) | Multi-person interactive network live broadcast method and system based on virtual scene | |
| CN110798697B (en) | Video display method, device and system and electronic equipment | |
| CN110784730B (en) | Live video data transmission method, device, equipment and storage medium | |
| WO2018045927A1 (en) | Three-dimensional virtual technology based internet real-time interactive live broadcasting method and device | |
| US11863801B2 (en) | Method and device for generating live streaming video data and method and device for playing live streaming video | |
| CN106331880B (en) | Information processing method and system | |
| KR100889367B1 (en) | System and Method for Realizing Vertual Studio via Network | |
| US11451858B2 (en) | Method and system of processing information flow and method of displaying comment information | |
| CN110809173B (en) | Virtual live broadcast method and system based on AR augmented reality of smart phone | |
| CN109547724B (en) | Video stream data processing method, electronic equipment and storage device | |
| CN112135155B (en) | Audio and video connecting and converging method and device, electronic equipment and storage medium | |
| CN111464828A (en) | Virtual special effect display method, device, terminal and storage medium | |
| CN109168032B (en) | Video data processing method, terminal, server and storage medium | |
| WO2023241459A1 (en) | Data communication method and system, and electronic device and storage medium | |
| CN116962742A (en) | Live video image data transmission method, device and live video system | |
| KR101915792B1 (en) | System and Method for Inserting an Advertisement Using Face Recognition | |
| CN113411537A (en) | Video call method, device, terminal and storage medium | |
| CN116801006A (en) | Live-broadcast continuous-wheat confluence method, equipment and storage medium | |
| JP2020115299A (en) | Virtual space information processing device, method and program | |
| CN112565799B (en) | Video data processing method and device | |
| CN115174954A (en) | Video live broadcast method and device, electronic equipment and storage medium | |
| CN112423108B (en) | Method and device for processing code stream, first terminal, second terminal and storage medium | |
| CN110798715A (en) | Video playing method and system based on image string | |
| CN112087660A (en) | Video processing method, device and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |