CN106792092B - Live video stream split-mirror display control method and corresponding device thereof - Google Patents
Live video stream split-mirror display control method and corresponding device thereof Download PDFInfo
- Publication number
- CN106792092B CN106792092B CN201611177834.5A CN201611177834A CN106792092B CN 106792092 B CN106792092 B CN 106792092B CN 201611177834 A CN201611177834 A CN 201611177834A CN 106792092 B CN106792092 B CN 106792092B
- Authority
- CN
- China
- Prior art keywords
- video stream
- original
- frame
- image part
- pushed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 43
- 230000002194 synthesizing effect Effects 0.000 claims abstract description 7
- 238000004891 communication Methods 0.000 claims description 10
- 238000012545 processing Methods 0.000 claims description 9
- 238000007906 compression Methods 0.000 claims description 6
- 230000006835 compression Effects 0.000 claims description 6
- 230000003321 amplification Effects 0.000 claims description 4
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 4
- 238000006073 displacement reaction Methods 0.000 claims description 3
- 230000015572 biosynthetic process Effects 0.000 claims description 2
- 238000003786 synthesis reaction Methods 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 9
- 230000015654 memory Effects 0.000 description 9
- 230000008569 process Effects 0.000 description 9
- 230000000694 effects Effects 0.000 description 6
- 239000000243 solution Substances 0.000 description 5
- 238000004590 computer program Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 230000001815 facial effect Effects 0.000 description 3
- 230000001413 cellular effect Effects 0.000 description 2
- 238000005034 decoration Methods 0.000 description 2
- 238000009432 framing Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000014509 gene expression Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 239000003637 basic solution Substances 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000012905 input function Methods 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 238000012856 packing Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
- H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440263—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the spatial resolution, e.g. for displaying on a connected PDA
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
- H04N21/47205—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/15—Conference systems
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Human Computer Interaction (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The invention discloses a live video stream split-mirror display control method and a corresponding device thereof, wherein the method comprises the following steps: receiving the setting of a cast user for a mirror area in an original video stream preview window of the cast user; acquiring an image part corresponding to the original picture size occupied by the lens splitting area in the original video stream; enlarging the image portion from the original frame to a predetermined frame; synthesizing the image part amplified to the preset frame and the original video stream into a to-be-pushed video stream with a preset default frame; and uploading the video stream to be pushed to a server to be pushed to member users participating in live broadcast. By using the live video stream split-mirror display control method and the corresponding device thereof, a live room user can clearly see the details of a local picture which a main broadcasting user wants to emphasize while watching a panoramic original picture.
Description
Technical Field
The invention relates to the technical field of network live broadcast, in particular to a live broadcast video stream split-mirror display control method and a corresponding device thereof.
Background
The network live broadcast absorbs and continues the advantages of the internet, the online live broadcast is carried out by utilizing a video mode, the contents such as product display, related conferences, background introduction, scheme evaluation, online investigation, conversation interview, online training and the like can be released to the internet on site, and the popularization effect of the activity site is enhanced by utilizing the characteristics of intuition, quickness, good expression form, rich contents, strong interactivity, unlimited region, divisible audience and the like of the internet.
In the prior art, when a participant in a live broadcast room watches the live broadcast of a main broadcast, only one complete picture can be seen generally, when the main broadcast needs to show a local detail in the complete picture or needs to emphasize and explain a local content to the participant, the main broadcast needs to enlarge the whole picture and lock a preview window in the local picture to be shown, so that the participant can see or notice the local picture, but the original complete picture cannot be displayed at the moment, so that the main broadcast cannot explain or explain the local picture in comparison with the original complete picture at the same time, and the participant in the live broadcast room can also omit other contents in the original complete picture because only the local picture is seen.
Meanwhile, for one end of a member participating in a live broadcast room, if the complete picture is too large, picture details in some areas can be displayed unclear due to the limitation of a user interface of the live broadcast room, if a user needs to know local detail content in the live broadcast room, a certain display effect can be achieved by amplifying the picture, data received directly in the live broadcast room often pass through compressed original video data, the picture is directly amplified to cause pixel reduction, the picture is blurred, and the watching requirement of the user cannot be met.
Therefore, there is a need to provide a basic solution for solving the related technical problem, which can be extended and developed in the industry.
Disclosure of Invention
In view of the above problems, the present invention provides a live video stream split-mirror display control method and device, which solve the problem in the prior art that a main broadcast cannot show a complete picture and local details to a user in a live broadcast room at the same time.
The invention provides a live video stream split-mirror display control method in a first aspect, which comprises the following steps: receiving the setting of a cast user for a mirror area in an original video stream preview window of the cast user; acquiring an image part corresponding to the original picture size occupied by the lens splitting area in the original video stream; enlarging the image portion from the original frame to a predetermined frame; synthesizing the image part amplified to the preset frame and the original video stream into a to-be-pushed video stream with a preset default frame; and uploading the video stream to be pushed to a server to be pushed to member users participating in live broadcast.
Preferably, the setting of the mirroring area in the original video stream preview window by the receiving anchor user comprises: displaying an original video stream in the preview window; receiving a frame selection instruction acted in the preview window by an anchor user, and determining the geometric dimension of a frame body framed by the frame selection instruction and the relative position relation of the frame body relative to the preview window; and calculating the original frame size of the frame body in the original video stream according to the geometric size, and positioning the image part of the original video stream corresponding to the original frame size according to the relative position relation, thereby completing the setting of the split mirror area.
Furthermore, the same video stream to be pushed comprises a plurality of the lens splitting areas.
Preferably, the process of enlarging the image portion from the original frame to the predetermined frame is as follows: and processing the image part to have the same resolution as the preset default frame according to the resolution of the preset default frame of the video stream to be pushed and to be in accordance with the preset size, thereby forming the preset frame.
Further, when the resolution of the image part of the original video stream is higher than the resolution of a preset default frame of the video stream to be pushed, if the resolution of the image part is higher than the former when the image part is filled to a preset size, image compression is carried out on the image part to match the former; if the image portion is filled to a predetermined size with a lower resolution than the former, it is digitally enlarged to match the former.
Further, when the image part enlarged to the predetermined frame and the original video stream are synthesized into a to-be-pushed video stream with a preset default frame, the size of the predetermined frame is smaller than that of the preset default frame and larger than that of the original frame of the image part, and the image part is synthesized into a foreground image or an adjacent image of the original video stream in the to-be-pushed video stream.
Preferably, the method comprises the following steps: and outputting the video stream to be pushed to another preview window for a main broadcasting user to preview the video.
Preferably, the method comprises the following steps: receiving adjustment setting of a main broadcasting user on the split-mirror area, and synchronizing the synthesized image part in the video stream to be pushed into the image part corresponding to the split-mirror area after the split-mirror area is adjusted; or, the displacement of the image part in the original frame of the original video stream is dynamically tracked by using the image characteristics of the image part to synchronously adjust the corresponding split mirror area, so that the synthesized image part in the video stream to be pushed keeps basically consistent.
Further, the method comprises the following steps: and receiving a deleting instruction of a main broadcasting user for deleting the lens splitting area, and recovering the image content of the video stream to be pushed as the image content of the original video stream.
The second aspect of the present invention provides a live video stream split-mirror display control apparatus, including: the video capture module is used for receiving the setting of a split mirror area in an original video stream preview window of a anchor user and acquiring an image part corresponding to the original frame size occupied by the split mirror area in the original video stream; the video amplification module is used for amplifying the image part from an original frame to a preset frame; the video synthesis module is used for synthesizing the image part amplified to the preset frame and the original video stream into a to-be-pushed video stream with a preset default frame; and the video transmission module is used for uploading the video stream to be pushed to a server so as to push the video stream to member users participating in live broadcast.
Compared with the prior art, the scheme provided by the invention mainly has the following advantages:
(1) by enlarging the image portion of the original video stream corresponding to the size of the original frame occupied by the split-mirror area to a predetermined frame size, the local details that the anchor wants to emphasize can be shown to the live broadcast room user.
(2) Because the image is compressed in the transmission process, the pixels of the original image are higher than the pixels of the image received by the participating members, the image is amplified and then transmitted, the influence of the amplification on the pixels of the image received by the participating members in the original image is small, and the amplified image received by the participating members is clear.
(3) The image part amplified to the preset frame and the original video stream are synthesized into a to-be-pushed video stream with a preset default frame, so that the single-path video stream simultaneously transmits the amplified frames of the original frame and the local image part, and the effect of enhancing the live broadcast display content without increasing the transmission bandwidth is achieved.
(4) And the framing area is created and deleted by the anchor, and the user interfaces of the participating members are controlled to display and close the enlarged pictures of the image part, so that the integral control is facilitated.
These and other aspects of the invention are apparent from and will be elucidated with reference to the embodiments described hereinafter.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a flowchart of a live video stream split-mirror display control method according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating an embodiment of a step S101 in the live video stream split-mirror display control method of FIG. 1 according to the present invention;
fig. 3 is a flowchart illustrating another embodiment of the step S101 in the live video stream split-mirror display control method in fig. 1 according to the present invention;
FIG. 4 is a schematic diagram of an original frame to which the live video stream split-mirror display control method is applied according to the present invention;
FIG. 5 is a diagram illustrating a live video stream split-mirror display control of the original frame shown in FIG. 4 according to an embodiment of the present invention;
FIG. 6 is a diagram illustrating a live video stream split-mirror display control of the original frame shown in FIG. 4 according to another embodiment of the present invention;
FIG. 7 is a diagram illustrating a live video stream split-mirror display control of the original frame shown in FIG. 4 according to another embodiment of the present invention;
FIG. 8 is a diagram illustrating a live video stream split-mirror display control of the original frame shown in FIG. 4 according to another embodiment of the present invention;
fig. 9 is a schematic diagram of a live video stream split-mirror display control device according to an embodiment of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention.
In some of the flows described in the present specification and claims and in the above figures, a number of operations are included that occur in a particular order, but it should be clearly understood that these operations may be performed out of order or in parallel as they occur herein, with the order of the operations being indicated as 101, 102, etc. merely to distinguish between the various operations, and the order of the operations by themselves does not represent any order of performance. Additionally, the flows may include more or fewer operations, and the operations may be performed sequentially or in parallel.
It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
As will be appreciated by those skilled in the art, a "terminal" as used herein includes both devices having a wireless signal receiver, which are devices having only a wireless signal receiver without transmit capability, and devices having receive and transmit hardware, which have devices having receive and transmit hardware capable of two-way communication over a two-way communication link. Such a device may include: a cellular or other communication device having a single line display or a multi-line display or a cellular or other communication device without a multi-line display; PCS (Personal Communications Service), which may combine voice, data processing, facsimile and/or data communication capabilities; a PDA (Personal Digital Assistant), which may include a radio frequency receiver, a pager, internet/intranet access, a web browser, a notepad, a calendar and/or a GPS (Global Positioning System) receiver; a conventional laptop and/or palmtop computer or other device having and/or including a radio frequency receiver. As used herein, a "terminal" or "terminal device" may be portable, transportable, installed in a vehicle (aeronautical, maritime, and/or land-based), or situated and/or configured to operate locally and/or in a distributed fashion at any other location(s) on earth and/or in space. As used herein, a "terminal Device" may also be a communication terminal, a web terminal, a music/video playing terminal, such as a PDA, an MID (Mobile Internet Device) and/or a Mobile phone with music/video playing function, or a smart tv, a set-top box, etc.
Before describing the embodiments of the present invention in detail, it is necessary to provide the following instructive description of the application scenarios and the principles of the present invention.
The internet generally includes a user terminal (user terminal device), a network, and a server (e.g., a Web server of a website). The user end may be an Internet terminal Device of the user, such as a desktop computer (PC), a Laptop computer (Laptop), an intelligent Device with a web browsing function, such as a Personal Digital Assistant (PDA), a Mobile Internet Device (MID), a smart Phone (Phone), and the like. These terminal devices may request a service from another process, such as a server-provided process, in an internet environment, typically the internet environment. The server is typically a remote computer system accessible via a communications medium such as the internet, typically the internet. Furthermore, a server may typically provide services to multiple clients from the internet. The service providing process includes receiving the request from the user end, collecting the information and feedback information of the user end, etc. In essence, the server plays the role of an information provider for the computer network. The server is typically located at the party providing the service or is configured by a service provider to serve the content, such service provider may be, for example, a website of an internet service company or the like.
In the live webcast process, a user enters a live webcast room through a live webcast application program on terminal equipment or a corresponding live webcast room site. There are two members within the webcast room, the anchor member and the participating members respectively. The anchor member can play live performances such as singing and dancing, video or audio files are recorded through the terminal equipment and uploaded to the live webcast server, and the live webcast server issues the video or audio files to the live webcast room for the participating members to watch.
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The first aspect of the present invention provides a live video stream split-mirror display control method, as shown in fig. 1, including the following steps:
step S101: receiving the setting of a cast user for a mirror area in an original video stream preview window of the cast user;
step S102: acquiring an image part corresponding to the original picture size occupied by the lens splitting area in the original video stream;
step S103: enlarging the image portion from the original frame to a predetermined frame;
step S104: synthesizing the image part amplified to the preset frame and the original video stream into a to-be-pushed video stream with a preset default frame;
step S105: and uploading the video stream to be pushed to a server to be pushed to member users participating in live broadcast.
Wherein the content of the first and second substances,
in step S101, the original video stream may be video data of a live frame of a anchor live broadcast room captured by a camera, or video data acquired from a video memory of an anchor terminal device, and becomes a video stream to be pushed after being processed. Such as a picture of a live performance of the anchor captured by a camera or a game picture in progress of the anchor acquired from a video memory of the anchor terminal.
Referring to fig. 2, as an embodiment, the receiving the setting S101 of the mirroring area by the anchor user in the original video stream preview window may include the following steps:
step S201, displaying the original video stream in the preview window. And displaying a preview picture of the original video stream on the interface of the anchor terminal through the play system of the anchor terminal.
Step S202, receiving a frame selection instruction acted in the preview window by the anchor user, and determining the geometric dimension of the frame body framed by the frame selection instruction and the relative position relation of the frame body relative to the preview window. The frame selection instruction refers to an instruction sent by a main broadcasting user on a main broadcasting terminal interface through input equipment such as a mouse or a touch screen, and the frame selection frame body is a graphic area selected from a preview window of the main broadcasting interface; preferably, the graphic area is a regular geometric figure, such as a circular area or a rectangular area. Determining the geometric dimension of the frame body, namely determining the dimension of the graphic area; for example, when the selected figure is a circle, the radius or the diameter of the selected figure is determined, and when the selected figure is a rectangle, the side length of the selected figure is determined. Determining the relative position relationship of the graphic area relative to the preview window, namely determining the position data such as coordinates of the graphic area in the preview window; for example, when the framed graphic is a circle, the center of the circle or the coordinates of a quadrant point in the preview window can be selected and determined according to the system setting, and when the framed graphic is a rectangle, the coordinates of any corner point in the preview window can be selected and determined according to the system setting.
Step S203, calculating an original frame size of the frame in the original video stream according to the geometric size, and positioning an image portion of the original video stream corresponding to the original frame size according to the relative position relationship. The step of calculating the original frame size of the frame in the original video stream according to the geometric size specifically includes: the original frame size corresponding to the geometric size is ensured through the compression ratio of the preview frame in the preview window and the original frame of the video stream; for example, the original frame size is 1000 × 800, the preview frame size is 500 × 400, and the preview frame size is 1/2 of the original frame size, and at this time, the selected frame is a circle with a radius of 20, so that the original frame size of the frame in the original video stream is a circle radius of 40. The specific step of positioning the image part of the original video stream corresponding to the original frame size according to the relative position relationship is as follows: converting the relative position relation of the frame body relative to the preview window into the relative position relation of the original frame corresponding to the frame body relative to the original frame of the video stream through the compression ratio of the preview frame in the preview window and the original frame of the video stream, and determining the image part of the original video stream corresponding to the frame body by combining the obtained size of the original frame; for example, when the circle with the radius of 40 is determined to correspond to the image part in the video stream according to the coordinates of the center of the circle with the radius of 20 in the preview window, the image part is the split mirror area, and the setting of the split mirror area is completed.
Referring to fig. 3, as another embodiment, the step of receiving the setting S101 of the mirroring area in the original video stream preview window by the anchor user may further include the following steps:
step S301, displaying an original video stream in the preview window. This step is the same as the step S201, and is not described again.
Step S302, receiving the click command acted by the anchor user in the preview window, and identifying the item characteristics of the item at the click position in the preview picture. The article features comprise contour features of the article and facial features of the person; for example, a live broadcast user live broadcasts a street dance show with a plurality of persons performing on the same station, an anchor needs to show a single picture of a lead dancer, each performer is identified through face identification, the anchor clicks the lead dancer as a target figure, the facial features of the lead dancer are identified, and then the outline features of the whole body of the lead dancer are extracted.
Step S303, determining an image portion of the original video stream corresponding to the item feature according to the item feature and a preset range. The preset range may be a circular range centered on the outline of the article; for example, the circular area with the distance between the two farthest points in the outline of the dancer as the diameter is determined, that is, the image portion corresponding to the smallest circular area which takes the dancer as the center and contains the dancer in the original video stream is determined, and at this time, the image portion is the split mirror area, so that the setting of the split mirror area is completed.
In step S102, the frame refers to a picture formed according to a certain resolution and size, and the original frame refers to a local frame in an original frame of the original video stream corresponding to the split mirror region. For example, if a picture displayed on a computer is obtained from a video memory, the picture is a picture in accordance with the resolution and size of the computer screen and is the same as the original picture; if the image is read into the memory from the camera and then displayed on the computer, the image and the original image can be different frames, and the image displayed on the computer is the frame processed by the original image to adapt to the display requirement. As already explained in step S203 and step S303, the method for determining the position of the image portion in the original frame corresponding to the split mirror area is to cut the video content of the image portion in the original frame by one step, and may also cut the data of the corresponding split mirror area in the original video stream data by performing the screenshot process on the image portion in the original frame.
In step S103, the predetermined frame is usually set by an anchor user default or by a software default, and includes setting a size of the predetermined frame, and may further include setting the predetermined frame to be placed at a specific position of a preview window of the video stream to be pushed (which may also be considered to be performed in a next step). Preferably, the process of enlarging the image portion from the original frame to the predetermined frame is as follows: and processing the image part to have the same resolution as the preset default frame according to the resolution of the preset default frame of the video stream to be pushed and to be in accordance with the preset size, thereby forming the preset frame. When the resolution of an image part of the original video stream is higher than the resolution of a preset default frame of the video stream to be pushed, if the resolution of the image part is higher than the resolution of the image part when the image part is filled to a preset size, image compression is carried out on the image part to match the image part; if the image portion is filled to a predetermined size with a lower resolution than the former, it is digitally enlarged to match the former.
In step S104, the preset default frame corresponds to a frame that is finally output to the server, that is, the size and resolution of the finally pushed video stream. And when the image part enlarged to the preset frame and the original video stream are synthesized into a to-be-pushed video stream with a preset default frame, the size of the preset frame is smaller than that of the preset default frame and larger than that of the original frame of the image part, and the image part is synthesized into a foreground image or an adjacent image of the original video stream in the to-be-pushed video stream.
Referring to fig. 4, in the original picture a of the original video stream, the portion selected by the dashed box is determined to be the image portion a corresponding to the split mirror region.
Referring to fig. 5, the image portion a is synthesized into a foreground image a0 of the original video stream in the video stream to be pushed, at this time, the original video stream is still the original frame a without being processed, and meanwhile, the foreground image a0 has a partial coverage effect on the original frame a. Wherein the position and size of the foreground image a0 on the original picture can be set and adjusted by the anchor.
Referring to fig. 6 to 8, the image part a is synthesized into an adjacent image a1 of the original video stream in the video stream to be pushed, and the adjacent image a1 and the processed original picture a form a splicing effect in the video stream to be pushed. Take 1/2 of the size of the adjacent image a1 occupying the original frame a, which is spliced up and down as an example; as shown in fig. 6, the original picture a is subjected to aspect ratio locking scaling processing, the length and width of the original picture a are converted into half of the original size to form a first image a1, and the adjacent image a1 and the first image a1 are spliced up and down in the video stream to be pushed; as shown in fig. 7, the original picture a is moved to display a part to be shown to form a second image a2, and the adjacent image a1 and the second image a2 are spliced up and down in the video stream to be pushed; as shown in fig. 8, the original picture a is subjected to a vertical narrowing process, the length of the original picture a is kept unchanged, and half of the original size converted from the original picture a into the original size is converted into a third image A3, and the adjacent image a1 and the third image A3 are spliced up and down in the video stream to be pushed. In addition, splicing modes such as left-right splicing or diagonal splicing can be adopted, the specific implementation principle is the same as that of the up-down splicing, and the detailed description is omitted.
In step S105, the video stream to be pushed, which includes the image portion and the original video stream and is synthesized in step S104, is uploaded to a server through a network, and is pushed to a member user participating in live broadcast through packing and compression of the server, so that a picture of the image portion and a picture of the original video stream in a split view is displayed on an interface of the member user participating in live broadcast.
In addition to this, the present invention is,
as an embodiment, the same video stream to be pushed includes a plurality of the split mirror regions. For example, in a video conference, facial images of each participant in a video stream are synthesized in a form of a circular split-mirror area, so that expressions and speaking actions of the participants can be displayed, and each participant can really participate in the video conference.
Preferably, the method comprises the following steps: and outputting the video stream to be pushed to another preview window for a main broadcasting user to preview the video. The content in the another preview window may be the same as any one of the contents in fig. 5 to fig. 8, that is, the video stream to be pushed, which is composed in step S104 and includes the image portion and the original video stream, is displayed in another preview window for previewing. The operation of this step may be performed before the video stream to be pushed is uploaded to the server, or may be performed by receiving a display of the video stream to be pushed by the server. At this time, the anchor user can preview or synchronize the video frames received by the member users participating in live broadcasting in the other preview window, and control the frame contents of the video stream pushed to the participating members by adjusting the positions of the split-view areas shown in fig. 4 or the enlarged frames shown in fig. 5, so that the anchor user has better user experience and can adjust and watch the video frames at the same time.
Preferably, the method comprises the following steps: and receiving adjustment setting of a main broadcasting user on the split-mirror area, and synchronizing the synthesized image part in the video stream to be pushed into the image part corresponding to the split-mirror area after the split-mirror area is adjusted. As an implementation manner, this step of operation may be understood as related to step 202, that is, in association with step 202, the anchor user re-frames the split area, re-obtains the original frame corresponding to the split area in the original video stream, and further enlarges the original frame into a predetermined frame, and finally synthesizes the image portion of the re-framed split area in the original video stream into the video stream to be pushed, so as to implement the adjustment and the synchronous update.
Preferably, the method comprises the following steps: and dynamically tracking the displacement of the image part in the original frame of the original video stream by using the image characteristics of the image part to synchronously adjust the corresponding split mirror area, so that the synthesized image part in the video stream to be pushed keeps basically consistent. As an embodiment, it may be default that when determining the split mirror area for the first time, the image features of the corresponding image portion are extracted, and when tracking, the corresponding image features are searched in the whole image (in units of frames) of the whole original video stream, and then the image features are quickly located, that is, this image portion can be tracked, so as to achieve the synchronous adjustment of the corresponding split mirror area. As another embodiment, it can be understood in combination with steps S302 and S303 that the selected item can be considered to be re-clicked by the anchor, so as to update the item feature, or to realize the dynamic tracking for the position change of the updated same item feature in different frames. The substantial coincidence is that the shape, quality and the like of the image may be different due to factors such as frame processing lag in shifting, or influence of light and angle on the camera after the object is moved, and in this case, it cannot be guaranteed that the partial mirror region and the synthesized image portion are completely coincident, and there is a corresponding deviation within an acceptable range, but the image characteristics can still be considered as being coincident as long as they exist.
Further, the method comprises the following steps: and receiving a deleting instruction of a main broadcasting user for deleting the lens splitting area, and recovering the image content of the video stream to be pushed as the image content of the original video stream. When the details needing to be emphasized or the contents needing to be displayed are finished, the framing area is deleted, the video stream to be pushed is recovered to be the original video stream, and a single original video stream picture is displayed on a user interface of a participant. It can be understood that part of the plurality of the split-mirror regions can be deleted, and the image content of the video stream to be pushed is re-synthesized into the image part corresponding to the reserved split-mirror region and the image content of the original video stream; for example, some participants leave in the video conference in advance, and some other participants continue the conference, and the split mirror area corresponding to the participant who leaves needs to be deleted.
The second aspect of the present invention provides a live video stream split-mirror display control apparatus, including: the video capture module 41 is configured to receive a setting of a split-mirror area in an original video stream preview window of a anchor user, and obtain an image portion corresponding to an original frame size occupied by the split-mirror area in the original video stream; a video enlargement module 42 for enlarging the image portion from the original frame to a predetermined frame; the video synthesizing module 43 is configured to synthesize the image portion enlarged to the predetermined frame and the original video stream into a to-be-pushed video stream with a preset default frame; and the video communication module 44 is configured to upload the video stream to be pushed to a server to be pushed to member users participating in live broadcasting.
For a specific description of the functional modules included in the live video stream split-mirror display control apparatus, reference may be made to the embodiments of the live video stream split-mirror display control method, which are not described herein again.
The live video stream split-mirror display control device can be any terminal equipment such as a mobile phone, a tablet personal computer and a personal computer. Taking a mobile phone as an example, the mobile phone comprises a processor, an input unit and an output unit. The input unit and the output unit respectively comprise a screen with a touch input function and a corresponding circuit module. As a specific embodiment, the following steps may be included: displaying a preview picture of an original video stream on a screen; responding to two times of clicking operations of a main broadcasting user on a touch screen, and selecting a split mirror area in a preview picture by taking two points as rectangular diagonals; the processor acquires an image part corresponding to the original frame size occupied by the lens dividing area in the original video stream; generating an image portion preview on a screen with the image portion; receiving an amplification command of sliding and expanding two fingers of a anchor user outwards after simultaneously contacting the touch screen, and previewing and amplifying the image part to a preset size; the processor enlarges the image part from the original frame to a preset frame according to the enlargement operation in the previous step; simultaneously displaying a preview of a preset picture and a preview picture of an original video stream on a screen; receiving dragging commands of a main broadcast user on a touch screen for previewing the preset picture and the previewing picture; the processor synthesizes the image part amplified to the preset frame and the original video stream into a to-be-pushed video stream with a preset default frame; and finally, the processor uploads the video stream to be pushed to a server to be pushed to member users participating in live broadcast.
While the above detailed description has been directed to a mobile phone for controlling the live video stream mirroring display provided by the present invention, for a person skilled in the art, according to the concepts of the embodiments of the present invention, the specific implementation and the application scope may be changed, and in summary, the content of the present description should not be construed as limiting the present invention.
Those skilled in the art will appreciate that the present invention includes apparatus relating to performing one or more of the operations described in the present invention. These devices may be specially designed and manufactured for the required purposes, or they may comprise known devices in general-purpose computers. These devices have stored therein computer programs that are selectively activated or reconfigured. Such a computer program may be stored in a device (e.g., computer) readable medium, including, but not limited to, any type of disk including floppy disks, hard disks, optical disks, CD-ROMs, and magnetic-optical disks, ROMs (Read-Only memories), RAMs (Random Access memories), EPROMs (Erasable programmable Read-Only memories), EEPROMs (Electrically Erasable programmable Read-Only memories), flash memories, magnetic cards, or optical cards, or any type of media suitable for storing electronic instructions, and each coupled to a bus. That is, a readable medium includes any medium that stores or transmits information in a form readable by a device (e.g., a computer).
It will be understood by those within the art that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by computer program instructions. Those skilled in the art will appreciate that the computer program instructions may be implemented by a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, implement the features specified in the block or blocks of the block diagrams and/or flowchart illustrations of the present disclosure.
Those of skill in the art will appreciate that various operations, methods, steps in the processes, acts, or solutions discussed in the present application may be alternated, modified, combined, or deleted. Further, various operations, methods, steps in the flows, which have been discussed in the present application, may be interchanged, modified, rearranged, decomposed, combined, or eliminated. Further, steps, measures, schemes in the various operations, methods, procedures disclosed in the prior art and the present invention can also be alternated, changed, rearranged, decomposed, combined, or deleted.
The foregoing is only a partial embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.
Claims (9)
1. A live video stream split-mirror display control method is characterized by comprising the following steps:
receiving the setting of a cast user for a mirror area in an original video stream preview window of the cast user;
acquiring an image part corresponding to the original picture size occupied by the lens splitting area in the original video stream;
enlarging the image portion from the original frame to a predetermined frame; processing the image part to be in the same resolution as a preset default picture of a video stream to be pushed, wherein the image part conforms to the preset size;
synthesizing the image part amplified to the preset frame and the original video stream into a to-be-pushed video stream with a preset default frame;
and uploading the video stream to be pushed to a server to be pushed to member users participating in live broadcast.
2. The method of claim 1, wherein receiving a setting of a mirroring area by a anchor user in an original video stream preview window thereof comprises:
displaying an original video stream in the preview window;
receiving a frame selection instruction acted in the preview window by an anchor user, and determining the geometric dimension of a frame body framed by the frame selection instruction and the relative position relation of the frame body relative to the preview window;
and calculating the original frame size of the frame body in the original video stream according to the geometric size, and positioning the image part of the original video stream corresponding to the original frame size according to the relative position relation, thereby completing the setting of the split mirror area.
3. The method of claim 1, wherein: the same video stream to be pushed comprises a plurality of the split mirror areas.
4. The method of claim 1, wherein:
when the resolution of an image part of the original video stream is higher than the resolution of a preset default frame of the video stream to be pushed, if the resolution of the image part is higher than the resolution of the image part when the image part is filled to a preset size, image compression is carried out on the image part to match the image part; if the image portion is filled to a predetermined size with a lower resolution than the former, it is digitally enlarged to match the former.
5. The method of claim 1, wherein: and when the image part enlarged to the preset frame and the original video stream are synthesized into a to-be-pushed video stream with a preset default frame, the size of the preset frame is smaller than that of the preset default frame and larger than that of the original frame of the image part, and the image part is synthesized into a foreground image or an adjacent image of the original video stream in the to-be-pushed video stream.
6. The method of claim 1, comprising the steps of:
and outputting the video stream to be pushed to another preview window for a main broadcasting user to preview the video.
7. The method of claim 1, comprising the steps of:
receiving adjustment setting of a main broadcasting user on the split-mirror area, and synchronizing the synthesized image part in the video stream to be pushed into the image part corresponding to the split-mirror area after the split-mirror area is adjusted;
or, the displacement of the image part in the original frame of the original video stream is dynamically tracked by using the image characteristics of the image part to synchronously adjust the corresponding split mirror area, so that the synthesized image part in the video stream to be pushed keeps basically consistent.
8. The method of claim 1, comprising the steps of:
and receiving a deleting instruction of a main broadcasting user for deleting the lens splitting area, and recovering the image content of the video stream to be pushed as the image content of the original video stream.
9. A live video stream split-mirror display control device is characterized by comprising:
the video capture module is used for receiving the setting of a split mirror area in an original video stream preview window of a anchor user and acquiring an image part corresponding to the original frame size occupied by the split mirror area in the original video stream;
the video amplification module is used for amplifying the image part from an original frame to a preset frame; processing the image part to be in the same resolution as a preset default picture of a video stream to be pushed, wherein the image part conforms to the preset size;
the video synthesis module is used for synthesizing the image part amplified to the preset frame and the original video stream into a to-be-pushed video stream with a preset default frame;
and the video communication module is used for uploading the video stream to be pushed to a server so as to push the video stream to member users participating in live broadcast.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611177834.5A CN106792092B (en) | 2016-12-19 | 2016-12-19 | Live video stream split-mirror display control method and corresponding device thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611177834.5A CN106792092B (en) | 2016-12-19 | 2016-12-19 | Live video stream split-mirror display control method and corresponding device thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106792092A CN106792092A (en) | 2017-05-31 |
| CN106792092B true CN106792092B (en) | 2020-01-03 |
Family
ID=58890175
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611177834.5A Active CN106792092B (en) | 2016-12-19 | 2016-12-19 | Live video stream split-mirror display control method and corresponding device thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106792092B (en) |
Families Citing this family (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107948509A (en) * | 2017-11-27 | 2018-04-20 | 广州华多网络科技有限公司 | Adjusting method, storage device and the terminal of live picture focal length |
| CN108055568A (en) * | 2017-12-07 | 2018-05-18 | 北京奇虎科技有限公司 | Video broadcasting method and device in a kind of electronic equipment |
| CN108762757B (en) * | 2018-05-21 | 2022-02-08 | 武汉斗鱼网络科技有限公司 | Quick development framework and method for platform service of live broadcast room, server and medium |
| CN108986117B (en) * | 2018-07-18 | 2021-06-04 | 阿里巴巴（中国）有限公司 | Video image segmentation method and device |
| CN109121000A (en) * | 2018-08-27 | 2019-01-01 | 北京优酷科技有限公司 | A kind of method for processing video frequency and client |
| CN109413352B (en) * | 2018-11-08 | 2020-06-23 | 北京微播视界科技有限公司 | Video data processing method, device, equipment and storage medium |
| CN115119004B (en) * | 2019-05-13 | 2024-03-29 | 阿里巴巴集团控股有限公司 | Data processing method, information display device, server and terminal equipment |
| WO2020248697A1 (en) * | 2019-06-10 | 2020-12-17 | 海信视像科技股份有限公司 | Display device and video communication data processing method |
| CN111601066B (en) * | 2020-05-26 | 2022-03-25 | 维沃移动通信有限公司 | Information acquisition method and device, electronic equipment and storage medium |
| CN111757137A (en) * | 2020-07-02 | 2020-10-09 | 广州博冠光电科技股份有限公司 | Multi-channel close-up playing method and device based on single-shot live video |
| CN111757138A (en) * | 2020-07-02 | 2020-10-09 | 广州博冠光电科技股份有限公司 | Close-up display method and device based on single-shot live video |
| CN111757143B (en) * | 2020-07-22 | 2023-01-17 | 四川新视创伟超高清科技有限公司 | Video playing method based on cloud streaming media picture cutting |
| CN112261424B (en) * | 2020-10-19 | 2022-11-18 | 北京字节跳动网络技术有限公司 | Image processing method, image processing device, electronic equipment and computer readable storage medium |
| CN112261428A (en) * | 2020-10-20 | 2021-01-22 | 北京字节跳动网络技术有限公司 | Picture display method and device, electronic equipment and computer readable medium |
| CN112637624B (en) * | 2020-12-14 | 2023-07-18 | 广州繁星互娱信息科技有限公司 | Live stream processing method, device, equipment and storage medium |
| CN112969099A (en) * | 2021-02-07 | 2021-06-15 | 海信集团控股股份有限公司 | Camera device, first display equipment, second display equipment and video interaction method |
| CN113852757B (en) * | 2021-09-03 | 2023-05-26 | 维沃移动通信（杭州）有限公司 | Video processing method, device, equipment and storage medium |
| CN113891105A (en) * | 2021-09-28 | 2022-01-04 | 广州繁星互娱信息科技有限公司 | Picture display method and device, storage medium and electronic equipment |
| EP4171022B1 (en) * | 2021-10-22 | 2023-11-29 | Axis AB | Method and system for transmitting a video stream |
| CN114666477B (en) * | 2022-03-24 | 2023-10-13 | 重庆紫光华山智安科技有限公司 | Video data processing method, device, equipment and storage medium |
| CN116055789B (en) * | 2023-03-24 | 2023-08-11 | 杭州星犀科技有限公司 | Live broadcast picture amplifying method, system, equipment and medium based on android system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102208171A (en) * | 2010-03-31 | 2011-10-05 | 安凯(广州)微电子技术有限公司 | Local detail playing method on portable high-definition video player |
| WO2013013449A1 (en) * | 2011-07-26 | 2013-01-31 | 中兴通讯股份有限公司 | Video telephone image magnification method, terminal and video telephone system |
| CN104410916A (en) * | 2014-12-03 | 2015-03-11 | 广州华多网络科技有限公司 | On-line living broadcast method and equipment |
| CN105307051A (en) * | 2015-05-04 | 2016-02-03 | 维沃移动通信有限公司 | Video processing method and device |
| CN105357542A (en) * | 2015-11-20 | 2016-02-24 | 广州华多网络科技有限公司 | Live broadcast method, device and system |
-
2016
- 2016-12-19 CN CN201611177834.5A patent/CN106792092B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102208171A (en) * | 2010-03-31 | 2011-10-05 | 安凯(广州)微电子技术有限公司 | Local detail playing method on portable high-definition video player |
| WO2013013449A1 (en) * | 2011-07-26 | 2013-01-31 | 中兴通讯股份有限公司 | Video telephone image magnification method, terminal and video telephone system |
| CN104410916A (en) * | 2014-12-03 | 2015-03-11 | 广州华多网络科技有限公司 | On-line living broadcast method and equipment |
| CN105307051A (en) * | 2015-05-04 | 2016-02-03 | 维沃移动通信有限公司 | Video processing method and device |
| CN105357542A (en) * | 2015-11-20 | 2016-02-24 | 广州华多网络科技有限公司 | Live broadcast method, device and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106792092A (en) | 2017-05-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106792092B (en) | Live video stream split-mirror display control method and corresponding device thereof | |
| CN106658205B (en) | Live broadcast room video stream synthesis control method and device and terminal equipment | |
| CN106921866B (en) | Multi-video-director method and equipment for assisting live broadcast | |
| JP6966421B2 (en) | Composite and scaling angle-separated subscenes | |
| CN110798697B (en) | Video display method, device and system and electronic equipment | |
| EP3562163A1 (en) | Audio-video synthesis method and system | |
| US10574933B2 (en) | System and method for converting live action alpha-numeric text to re-rendered and embedded pixel information for video overlay | |
| JP2018513583A (en) | Audio video file live streaming method, system and server | |
| US20060242676A1 (en) | Live streaming broadcast method, live streaming broadcast device, live streaming broadcast system, program, recording medium, broadcast method, and broadcast device | |
| JP2012080431A (en) | Panoramic image display device and panoramic image display method | |
| US8830264B2 (en) | Method and apparatus for providing image to device | |
| US20150078723A1 (en) | Method and apparatus for smart video rendering | |
| KR20180038256A (en) | Method, and system for compensating delay of virtural reality stream | |
| CN113282257A (en) | Method, terminal device, device and readable storage medium for synchronous display | |
| CN114339405B (en) | Remote manufacturing method and device for AR video data stream, equipment and storage medium | |
| CN113596571B (en) | Screen sharing method, device, system, storage medium and computer equipment | |
| JP2010157906A (en) | Video display device | |
| CN116761009A (en) | Video playing method and device in meta-universe panoramic live broadcast scene and live broadcast system | |
| KR102029604B1 (en) | Editing system and editing method for real-time broadcasting | |
| WO2019118890A1 (en) | Method and system for cloud video stitching | |
| CN112601110B (en) | Method and apparatus for content recording and sharing | |
| CN114827647A (en) | Live broadcast data generation method, device, equipment, medium and program product | |
| CN114095772A (en) | Virtual object display method and system under live microphone connection and computer equipment | |
| KR102149004B1 (en) | Method and apparatus for generating multi channel images using mobile terminal | |
| WO2016117480A1 (en) | Telecommunication system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |