CN114466212A - Live broadcast method, device, electronic equipment and medium - Google Patents

Live broadcast method, device, electronic equipment and medium Download PDF

Info

Publication number
CN114466212A
CN114466212A CN202210116786.8A CN202210116786A CN114466212A CN 114466212 A CN114466212 A CN 114466212A CN 202210116786 A CN202210116786 A CN 202210116786A CN 114466212 A CN114466212 A CN 114466212A
Authority
CN
China
Prior art keywords
target
client
data
video stream
auxiliary
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210116786.8A
Other languages
Chinese (zh)
Inventor
谷典
左振领
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Baidu Online Network Technology Beijing Co Ltd
Original Assignee
Baidu Online Network Technology Beijing Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Baidu Online Network Technology Beijing Co Ltd filed Critical Baidu Online Network Technology Beijing Co Ltd
Priority to CN202210116786.8A priority Critical patent/CN114466212A/en
Publication of CN114466212A publication Critical patent/CN114466212A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21Server components or server architectures
    • H04N21/218Source of audio or video content, e.g. local disk arrays
    • H04N21/2187Live feed
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41Structure of client; Structure of client peripherals
    • H04N21/414Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance
    • H04N21/41407Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/439Processing of audio elementary streams
    • H04N21/4398Processing of audio elementary streams involving reformatting operations of audio signals
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H04N21/4402Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440263Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the spatial resolution, e.g. for displaying on a connected PDA
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H04N21/4402Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440281Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the temporal resolution, e.g. by frame skipping
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/462Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
    • H04N21/4627Rights management associated to the content

Abstract

The present disclosure provides a live broadcast method, apparatus, electronic device and medium, and relates to the technical field of computers, in particular to the technical field of cloud phones, cloud computing and cloud services. The specific implementation scheme is as follows: acquiring sound and shape data of a main broadcast and a target control instruction from a target client, and controlling target cloud equipment according to the target control instruction to generate target video stream data; and sending the target video streaming data and the sound shape data to the target client and at least one other client except the target client, so that the target client and the other clients display according to the target video streaming data and the sound shape data. The method and the device have the advantages that the effect of reducing the requirement of live broadcast on the hardware of the client is achieved, the low-performance client can still conduct live broadcast normally, and the popularization range of live broadcast is widened.

Description

Live broadcast method, device, electronic equipment and medium
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a live broadcast method, apparatus, electronic device, and medium, and to the field of cloud phones, cloud computing, and cloud service technologies.
Background
Along with the improvement of living standard, the popularization rate of the smart phone is higher and higher, and the types of applications in the smart phone are more diversified, so that great convenience is brought to life of people. Live broadcast applications are one type of applications which are popular at present, and the anchor shows colorful live broadcast contents to audiences through a smart phone.
The existing live broadcast technology needs to render a display picture and encode video through a smart phone of a main broadcast, and then push the video stream obtained by rendering and encoding video to audiences through a server.
Disclosure of Invention
The present disclosure provides a method, apparatus, electronic device, and medium for reducing a hardware requirement of a live broadcast on a client.
According to an aspect of the present disclosure, there is provided a live broadcasting method including:
acquiring sound and shape data of a main broadcast and a target control instruction from a target client, and controlling target cloud equipment according to the target control instruction to generate target video stream data;
and sending the target video streaming data and the sound shape data to the target client and at least one other client except the target client, so that the target client and the other clients display according to the target video streaming data and the sound shape data.
According to another aspect of the present disclosure, there is provided a live broadcasting apparatus including:
the video stream data generation module is used for acquiring sound and shape data of a main broadcast and a target control instruction from a target client, controlling target cloud equipment according to the target control instruction and generating target video stream data;
and the data sending module is used for sending the target video streaming data and the sound shape data to the target client and at least one other client except the target client so that the target client and the other client can display the target video streaming data and the sound shape data.
According to another aspect of the present disclosure, there is provided an electronic device including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any of the present disclosure.
According to another aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of the present disclosure.
According to another aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, performs the method of any one of the present disclosure.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.
Drawings
The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:
fig. 1 is a flow chart of some live methods disclosed in accordance with embodiments of the present disclosure;
FIG. 2A is a flow chart of further live methods disclosed in accordance with embodiments of the present disclosure;
fig. 2B is a schematic structural diagram of some live broadcast systems disclosed in accordance with an embodiment of the present disclosure;
fig. 3 is a schematic structural diagram of some live devices disclosed in accordance with an embodiment of the present disclosure;
fig. 4 is a block diagram of an electronic device for implementing the live broadcast method disclosed in the embodiments of the present disclosure.
Detailed Description
Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
In the research and development process, the applicant finds that the flow of the existing live broadcast technology can be summarized as follows: the anchor program operates live broadcast application running in the smart phone, the smart phone generates a screen display picture in a rendering mode in the operation process, the smart phone performs video coding on the screen display picture to obtain video streaming data, the smart phone sends the video streaming data to the server, the server sends the video streaming data to other smart phones which are connected to the server, and the other smart phones perform video decoding on the video streaming data. The audience can watch the screen display picture of the anchor smart phone through other smart phones, so that live broadcast of live broadcast application is realized.
However, the existing live broadcasting technology requires the anchor smartphone to perform picture rendering to generate the screen display picture, and perform video coding and decoding on the screen display picture to generate the video stream data. This undoubtedly has a high requirement on the hardware of the anchor smart phone, which causes the problem of stuttering or distortion of pictures in the live broadcast process of the low-performance smart phone, and the live broadcast cannot be performed normally.
Fig. 1 is a flowchart of some live broadcast methods disclosed in the embodiment of the present disclosure, and this embodiment may be applied to a case where a director performs live broadcast through a cloud device. The method of the present embodiment may be executed by a live broadcast apparatus disclosed in the embodiments of the present disclosure, and the apparatus may be implemented by software and/or hardware, and may be integrated on any electronic device with computing capability.
As shown in fig. 1, the live broadcasting method disclosed in this embodiment may include:
s101, acquiring sound and shape data of the anchor and a target control instruction from a target client, controlling target cloud equipment according to the target control instruction, and generating target video stream data.
The target client represents a smart terminal device, and may be, for example, a smart phone, a smart watch, a tablet computer, a notebook computer, or any electronic device with a smart operating system. The anchor represents a holder of the target client, and the anchor performs live broadcasting through the target client. The voice and shape data comprise voice data and shape data of the anchor, wherein the voice data can be collected real voice of the anchor, or synthetic voice generated by a TTS (Text To Speech, TTS) technology, and the like; the appearance data may be a real appearance image of the collected anchor, or may be a virtual anchor image generated by a character model technology, and the like.
The cloud equipment refers to equipment which applies a cloud computing technology to a server and realizes cloud service through the server, and can realize numerous functions through a network by means of a self-contained system and the server erected by manufacturers. The types of the cloud device include, but are not limited to, a cloud mobile phone, a cloud computer, a cloud tablet, a cloud television, and the like, and the specific type of the cloud device is not limited in this embodiment. In this embodiment, the server is equipped with at least one cloud device, each client corresponds to one cloud device having a control authority, and the target cloud device represents a cloud device having a control authority of the target client.
In one implementation mode, the anchor clicks a pre-installed cloud equipment application icon in a target client, the target client correspondingly generates a connection request and sends the connection request to the server, the server performs identity verification on the target client, and after the identity verification is passed, communication connection with the target client is established. Further, the server controls the target cloud equipment to render the cloud equipment interface, performs video coding on the rendered interface, generates video stream data, and sends the video stream data to the target client. And after receiving the video stream data, the target client decodes the video so as to display the cloud equipment interface through a screen. When the target client logs in the cloud device for the first time, the cloud device interface may be a desktop interface of the cloud device.
The anchor can apply control operation on the cloud equipment interface through the screen, the target client determines screen coordinates related to the control operation according to the control operation of the user, and generates a target control instruction containing the screen coordinates. The embodiment does not limit the specific form of the control operation, and all the ways of controlling the cloud device interface can be regarded as the control operation of the cloud device interface.
For example, when the cloud device interface is a desktop interface, the anchor can click any application icon in the desktop interface through the screen, and the target client generates a target control instruction containing the application screen coordinates so as to start an application program corresponding to the application icon installed in the cloud device.
For another example, when the cloud device interface is a game interface of a game application, the anchor may perform a press and drag operation through a screen, and the target client generates a target manipulation instruction including screen coordinates according to the screen coordinates involved in the press and drag operation, so as to control movement or attack of a game character.
For another example, when the cloud device interface is a teaching interface of an educational application, the anchor may perform a left-hand stroke or a right-hand stroke operation through the screen to control page turning of a teaching material or a presentation, or perform a press and drag operation through the screen to write a text or a formula, or the like. And the target client generates a target control instruction containing the screen coordinates according to the screen coordinates related to the left-hand stroke or right-hand stroke operation and the screen coordinates related to the pressing and dragging operation.
The target client side collects sound and shape data of the anchor through collection equipment installed on the target client side, wherein the sound and shape data can be collected in real time or can be collected when the anchor applies control operation on a cloud equipment interface. Optionally, when the anchor applies a control operation on the cloud device interface, the target client acquires real voice data of the user through an installed sound pickup device, such as a microphone, and also acquires real appearance data of the user through an installed image acquisition device, such as a video camera or a still camera. And the target client performs associated packaging on the real voice data and the real appearance image data to generate sound and shape data.
And the target client sends the target control instruction and the audio data to the server together. And the server receives the audio data and the target control instruction, and controls the cloud equipment to perform corresponding control operation in the cloud equipment interface according to the screen coordinate carried in the target control instruction. The cloud equipment adopts a carried graphics processor to render the interface of the cloud equipment to obtain interface display content, and adopts a carried video encoder to carry out video encoding on the interface display content to generate target video stream data containing the interface display content.
The target video stream data is generated by acquiring the sound and shape data of the anchor and the target control instruction from the target client and controlling the target cloud equipment according to the target control instruction, so that the effect of generating the target video stream data according to the control operation of the anchor on the target cloud equipment at the target client is realized, and a foundation is laid for subsequent stream pushing.
And S102, sending the target video streaming data and the graphic data to the target client and at least one other client except the target client, so that the target client and the other clients can display according to the target video streaming data and the graphic data.
Besides the communication connection with the target client, the server is also in communication connection with at least one other client. The other clients also represent a smart terminal device, and may be, for example, a smart phone, a smart watch, a tablet computer, a notebook computer, or any electronic device with a smart operating system.
In one embodiment, the server sends the target video stream data to the target client, and the target client performs video decoding on the target video stream data and is used for displaying interface display content of the cloud equipment in a screen of the target client. And the server side sends the target video stream data and the audio-visual data of the anchor program to other client sides, and the other client sides carry out video decoding on the target video stream data and analyze the audio-visual data, so that the interface display content of the cloud equipment and the appearance of the anchor program can be displayed on the screens of the other client sides at the same time, and the voice of the anchor program can be played through the loudspeakers of the other client sides.
According to the method, the sound and shape data and the target control instruction of the anchor broadcast are obtained from the target client, the target cloud equipment is controlled according to the target control instruction to generate target video stream data, and then the target video stream data and the sound and shape data are sent to the target client and at least one other client except the target client, so that the target client and the other clients can display according to the target video stream data and the sound and shape data; and moreover, the target video stream data and the audio-visual data of the anchor are displayed simultaneously, so that the live broadcast vitality is improved, and the live broadcast effect is optimized.
On the basis of the above embodiment, after "generating target video stream data" in S101, the method further includes:
and performing associated storage on the target video stream data and the audio data to generate live playback data.
In one embodiment, after the server generates the target video stream data, the server stores the target video stream data and the audio data in association with each other to generate live playback data. When the target client and/or other clients initiate a live playback request to the server, the server sends all target video stream data and audio and video data stored in a related mode to the target client and/or other clients together, and the target client and/or other clients can watch live playback.
The target video stream data and the audio data are stored in an associated mode to generate live playback data, so that a live playback function is provided for a user, the requirement of the user for watching live playback is met, and the user experience is improved.
Fig. 2A is a flowchart of another live broadcast method disclosed according to an embodiment of the present disclosure, which is further optimized and expanded based on the foregoing technical solution, and may be combined with each of the foregoing optional embodiments.
As shown in fig. 2A, the live broadcasting method disclosed in this embodiment may include:
s201, acquiring an operation request for the target cloud equipment from the target client, and performing identity verification on the target client according to identity information contained in the operation request.
The identity information represents identification information including, but not limited to, a password, a client device code, an account password, and the like.
In one implementation, the target client generates a control request carrying identity information and sends the control request to the server. The server side obtains an operation request for the target cloud equipment from the target client side, and analyzes the operation request to obtain identity information. The server compares the identity information of the target cloud equipment with the pre-stored standard identity information so as to verify the identity of the target client.
S202, under the condition that the identity authentication is passed, the control authority of the target cloud equipment is issued to the target client.
In one embodiment, if it is determined that the identity information of the target cloud device is the same as the standard identity information, the control authority of the target cloud device is issued to the target client. And if the identity information of the target cloud equipment is different from the standard identity information, refusing to issue the control authority to the target client, and prompting the target client to perform identity authentication again.
S203, acquiring sound-shape data of the anchor and a target control instruction from the target client, and acquiring first resolution and first coordinate information of the target client from the target control instruction.
The first coordinate information represents screen coordinates involved in the control operation of the anchor on the cloud equipment interface. For example, when the anchor clicks a certain point a in the cloud device interface, the screen coordinates of the point a are used as the first coordinate information. The first resolution represents a display resolution of the target client screen.
In one embodiment, the target client generates a target control instruction carrying the first resolution and the first coordinate information, and sends the target control instruction to the server. The server analyzes the target control instruction to acquire a first resolution and first coordinate information.
And S204, determining second coordinate information corresponding to the first coordinate information in the target cloud equipment according to the first resolution of the target client and the second resolution of the target cloud equipment, and controlling the target cloud equipment according to the second coordinate information to generate target video stream data.
Wherein the second resolution of the target cloud device represents a display resolution of the cloud device. Since the first coordinate information represents absolute coordinates, and the first resolution of the target client and the second resolution of the target cloud device are not the same in general, it is necessary to determine the second coordinate information corresponding to the first coordinate information in the target cloud device.
In one embodiment, a first ratio of the number of pixels of the first resolution in the X-axis direction to the X-coordinate of the first coordinate information and a second ratio of the number of pixels of the first resolution in the Y-axis direction to the Y-coordinate of the first coordinate information are determined to determine the relative coordinate of the first coordinate information at the first resolution. And determining absolute coordinate information according to the relative coordinates and the second resolution of the target cloud equipment, wherein the absolute coordinate information is used as second coordinate information corresponding to the first coordinate information in the target cloud equipment. And the server controls the cloud equipment to perform corresponding control operation in the cloud equipment interface according to the determined second coordinate information, and generates target video stream data.
And S205, sending the target video streaming data and the graphics data to the target client and at least one other client except the target client, so that the target client and the other clients can display according to the target video streaming data and the graphics data.
According to the method and the device, the control request for the target cloud equipment is obtained from the target client, the target client is subjected to identity verification according to the identity information contained in the control request, and the control authority for the target cloud equipment is issued to the target client under the condition that the identity verification is passed, so that the effect of performing identity verification on the target client is achieved, and the control authority for the target client can be obtained only when the identity verification is passed; the first resolution and the first coordinate information of the target client are obtained from the target control instruction, the second coordinate information corresponding to the first coordinate information in the target cloud equipment is determined according to the first resolution of the target client and the second resolution of the target cloud equipment, the target cloud equipment is controlled according to the second coordinate information, target video stream data are generated, the effect of controlling the target cloud equipment according to the second coordinate information after the relative coordinate conversion is achieved, the accuracy of controlling the target cloud equipment is guaranteed, and misoperation of the target cloud equipment is avoided.
On the basis of the above embodiment, the method further includes:
the method comprises the steps of obtaining a permission issuing request for target cloud equipment from a target client, determining at least one auxiliary client from other clients according to a client identifier contained in the permission issuing request, and issuing control permission for the target cloud equipment to the auxiliary client.
The client identifier includes, but is not limited to, identification information having uniqueness, such as a client device number or a name.
In one implementation, the target client generates a permission issuing request carrying a client identifier, and sends the permission issuing request to the server. The server analyzes the permission issuing request to obtain at least one client identifier, and at least one other client corresponding to the at least one client identifier is used as an auxiliary client, so that the control permission of the target cloud equipment is issued to the auxiliary client, and a user of the auxiliary client can operate and control the target cloud equipment.
By issuing the control authority of the target cloud equipment to the auxiliary client, a user of the auxiliary client can control the target cloud equipment, and the live broadcast interactivity is improved. For example, in a teaching scene, a student can control the target cloud equipment through the auxiliary client to interact with a teacher, such as asking questions or answering questions; for another example, in a game scenario, the spectator may play a game or the like with the game anchor by manipulating the target cloud device through the auxiliary client.
On the basis of the above embodiment, after issuing the control right to the target cloud device to the auxiliary client, the method further includes:
and acquiring an auxiliary control instruction from the auxiliary client, controlling the target cloud equipment according to the auxiliary control instruction, generating auxiliary video streaming data, and further sending the auxiliary video streaming data to the auxiliary client, the target client and other clients except the auxiliary client.
In one embodiment, a user of the auxiliary client can apply a control operation on the target cloud equipment interface through a screen, the auxiliary client determines screen coordinates related to the control operation according to the control operation of the user, and generates an auxiliary control instruction containing the screen coordinates. And the server side acquires an auxiliary control instruction from the auxiliary client side, and controls the cloud equipment to perform corresponding control operation in the cloud equipment interface according to the analyzed screen coordinate to generate auxiliary video stream data. And the server side sends the auxiliary video streaming data to the auxiliary client side, the target client side and other client sides except the auxiliary client side, and the auxiliary video streaming data are subjected to video decoding to display a target cloud equipment interface.
The auxiliary control instruction is obtained from the auxiliary client, the target cloud equipment is controlled according to the auxiliary control instruction, auxiliary video streaming data are generated, and then the auxiliary video streaming data are sent to the auxiliary client, the target client and other clients except the auxiliary client, so that the target cloud equipment can be controlled by a user of the auxiliary client, and the interaction of live broadcast is improved.
On the basis of the above embodiment, the method further includes:
and acquiring other control instructions from at least one other client, controlling at least one other cloud device according to the other control instructions, generating at least one piece of other video stream data, generating video stream summary data according to the at least one piece of other video stream data, sending the at least one piece of other video stream data to the at least one other client, and sending the video stream summary data to the target client.
And the other cloud equipment represents cloud equipment of which other clients have control authority. One other cloud device corresponds to one other client.
In one embodiment, users of other clients can apply control operation on other cloud equipment interfaces through the screen, and the other clients determine screen coordinates related to the control operation according to the control operation of the users and generate other control instructions containing the screen coordinates. And the server side acquires other control instructions from other clients, controls other cloud equipment to perform corresponding control operation in the cloud equipment interface according to the analyzed screen coordinates, and generates other video stream data.
And the server side summarizes at least one piece of other video stream data into a piece of video stream summarized data. And respectively sending at least one piece of other video stream data to the corresponding other clients, and sending the video stream summary data to the target client. And displaying other clients according to the corresponding other video stream data respectively, and displaying the target client according to the video stream summarized data.
For example, in a teaching scene, it is assumed that a student uses another client to control another corresponding cloud device to answer questions, and the other client sends other video stream data generated according to the answer interfaces of the other cloud devices to the other client corresponding to the other cloud device, so that the student watches the answer interfaces of the other cloud device through screens of the other client. For example, the student a controls other cloud devices a to answer questions through other clients a, then the other cloud devices a generate other video stream data and send the other video stream data to the other clients a, and the student a watches answer interfaces of the other cloud devices a through screens of the other clients a.
And, the teacher is the user of the target client. The server side collects at least one piece of other video stream data generated by at least one piece of other cloud equipment to generate video stream summary data, and sends the video stream summary data to the target client side, so that a teacher can watch answer interfaces of other cloud equipment corresponding to students through a screen of the target client side.
The other control instructions are obtained from at least one other client, at least one other cloud device is controlled according to the other control instructions, at least one piece of other video stream data is generated, video stream summary data are generated according to the at least one piece of other video stream data, the at least one piece of other video stream data are sent to the at least one other client, the video stream summary data are sent to the target client, the effect that the target client synchronously displays interfaces of the other cloud devices is achieved, and the method and the device are suitable for scenes such as invigilation or monitoring.
Fig. 2B is a schematic structural diagram of some live broadcast systems disclosed according to the embodiment of the present disclosure, and as shown in fig. 2B, the live broadcast system includes a server 200, a target cloud device 201 and at least one other cloud device 202 are built in the server 200, the live broadcast system further includes a target client 203 and at least one other client 204, and the target client 203 and the at least one other client 204 are in communication connection with the server 200.
In this embodiment, the server 200 is an ARM architecture server. In order to satisfy the requirement that a plurality of users run a plurality of services simultaneously, a platform layer needs to perform virtualized container isolation design on an instance, and since the server 200 adopts an ARM structure, each ARM mainboard can only be butted with the natural characteristic used by one user, the safety isolation of a single ARM mainboard is not needed, and the structural design scheme is simplified.
The server 200 supports various operating system platform ecosystems, including but not limited to android, Ubuntu and the like based on a Linux kernel system, and simultaneously supports a windows11 system.
Server 200 also has rich software resource package adaptation support including, but not limited to, json, ext3, vfat, ntfs, iso9660, jffs, and c, among many file types.
Fig. 3 is a schematic structural diagram of some live broadcast apparatuses disclosed according to an embodiment of the present disclosure, which may be applied to a case where a host performs live broadcast through cloud equipment. The device of the embodiment can be implemented by software and/or hardware, and can be integrated on any electronic equipment with computing capability.
As shown in fig. 3, the live device 30 disclosed in the present embodiment may include a video stream data generating module 31 and a data transmitting module 32, where:
the video stream data generating module 31 is configured to obtain audio-visual data of a main broadcast and a target control instruction from a target client, and control a target cloud device according to the target control instruction to generate target video stream data;
a data sending module 32, configured to send the target video stream data and the sound shape data to the target client and at least one other client except the target client, so that the target client and the other client display the target video stream data and the sound shape data.
Optionally, the video stream data generating module 31 is specifically configured to:
acquiring first resolution and first coordinate information of the target client from the target control instruction;
determining second coordinate information corresponding to the first coordinate information in the target cloud equipment according to the first resolution of the target client and the second resolution of the target cloud equipment;
and controlling the target cloud equipment according to the second coordinate information to generate target video stream data.
Optionally, the system further includes a video stream distribution module, specifically configured to:
acquiring other control instructions from the at least one other client, and controlling at least one other cloud device according to the other control instructions to generate at least one piece of other video stream data;
generating video stream summary data according to the at least one piece of other video stream data;
sending the at least one other piece of video stream data to the at least one other client, and sending the video stream summary data to the target client.
Optionally, the system further includes a data storage module, specifically configured to:
and performing associated storage on the target video stream data and the sound-shape data to generate live playback data.
Optionally, the system further includes an identity verification module, specifically configured to:
acquiring an operation request for the target cloud equipment from a target client, and performing identity verification on the target client according to identity information contained in the operation request;
and under the condition that the identity authentication is passed, sending the control authority of the target cloud equipment to the target client.
Optionally, the system further includes a permission issuing module, specifically configured to:
acquiring a permission issuing request for the target cloud equipment from the target client, and determining at least one auxiliary client from other clients according to a client identifier contained in the permission issuing request;
and sending the control authority of the target cloud equipment to the auxiliary client.
Optionally, the system further comprises an auxiliary control module, specifically configured to:
acquiring an auxiliary control instruction from the auxiliary client, and controlling target cloud equipment according to the auxiliary control instruction to generate auxiliary video stream data;
and sending the auxiliary video streaming data to the auxiliary client, the target client and other clients except the auxiliary client.
The live broadcasting device 30 disclosed in the embodiment of the present disclosure can execute the live broadcasting method disclosed in the embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method. Reference may be made to the description in the method embodiments of the present disclosure for details that are not explicitly described in this embodiment.
In the technical scheme of the disclosure, the acquisition, storage, application and the like of the personal information of the related user all accord with the regulations of related laws and regulations, and do not violate the customs of public sequences.
The present disclosure also provides an electronic device, a readable storage medium, and a computer program product according to embodiments of the present disclosure.
FIG. 4 shows a schematic block diagram of an example electronic device 400 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 4, the apparatus 400 includes a computing unit 401 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM)402 or a computer program loaded from a storage unit 408 into a Random Access Memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the device 400 can also be stored. The calculation unit 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input/output (I/O) interface 405 is also connected to bus 404.
A number of components in device 400 are connected to I/O interface 405, including: an input unit 406 such as a keyboard, a mouse, or the like; an output unit 407 such as various types of displays, speakers, and the like; a storage unit 408 such as a magnetic disk, optical disk, or the like; and a communication unit 409 such as a network card, modem, wireless communication transceiver, etc. The communication unit 409 allows the device 400 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.
Computing unit 401 may be a variety of general and/or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The computing unit 401 executes the respective methods and processes described above, such as the live method. For example, in some embodiments, the live method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 408. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 400 via the ROM 402 and/or the communication unit 409. When the computer program is loaded into RAM 403 and executed by computing unit 401, one or more steps of the live method described above may be performed. Alternatively, in other embodiments, the computing unit 401 may be configured to perform the live method by any other suitable means (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), blockchain networks, and the internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service are overcome.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, and the present disclosure is not limited herein.
The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.

Claims (17)

1. A live method, comprising:
acquiring sound and shape data of a main broadcast and a target control instruction from a target client, and controlling target cloud equipment according to the target control instruction to generate target video stream data;
and sending the target video streaming data and the sound shape data to the target client and at least one other client except the target client, so that the target client and the other clients display according to the target video streaming data and the sound shape data.
2. The method of claim 1, wherein the controlling the target cloud device according to the target manipulation instruction to generate target video stream data comprises:
acquiring a first resolution and first coordinate information of the target client from the target control instruction;
determining second coordinate information corresponding to the first coordinate information in the target cloud equipment according to the first resolution of the target client and the second resolution of the target cloud equipment;
and controlling the target cloud equipment according to the second coordinate information to generate target video stream data.
3. The method of claim 1, further comprising:
acquiring other control instructions from the at least one other client, and controlling at least one other cloud device according to the other control instructions to generate at least one piece of other video stream data;
generating video stream summary data according to the at least one piece of other video stream data;
sending the at least one other piece of video stream data to the at least one other client, and sending the video stream summary data to the target client.
4. The method of claim 1, further comprising, after generating the target video stream data:
and performing associated storage on the target video stream data and the sound-shape data to generate live playback data.
5. The method of claim 1, before obtaining the anchor's voice and shape data and the target manipulation instructions from the target client, further comprising:
acquiring an operation request for the target cloud equipment from a target client, and performing identity verification on the target client according to identity information contained in the operation request;
and under the condition that the identity authentication is passed, sending the control authority of the target cloud equipment to the target client.
6. The method of claim 5, further comprising:
acquiring a permission issuing request for the target cloud equipment from the target client, and determining at least one auxiliary client from other clients according to a client identifier contained in the permission issuing request;
and sending the control authority of the target cloud equipment to the auxiliary client.
7. The method of claim 6, after issuing the manipulation right to the target cloud device to the assistant client, further comprising:
acquiring an auxiliary control instruction from the auxiliary client, and controlling target cloud equipment according to the auxiliary control instruction to generate auxiliary video stream data;
and sending the auxiliary video streaming data to the auxiliary client, the target client and other clients except the auxiliary client.
8. A live device, comprising:
the video stream data generation module is used for acquiring sound and shape data of a main broadcast and a target control instruction from a target client, controlling target cloud equipment according to the target control instruction and generating target video stream data;
and the data sending module is used for sending the target video streaming data and the sound shape data to the target client and at least one other client except the target client so that the target client and the other clients display the target video streaming data and the sound shape data.
9. The apparatus according to claim 8, wherein the video stream data generating module is specifically configured to:
acquiring a first resolution and first coordinate information of the target client from the target control instruction;
determining second coordinate information corresponding to the first coordinate information in the target cloud equipment according to the first resolution of the target client and the second resolution of the target cloud equipment;
and controlling the target cloud equipment according to the second coordinate information to generate target video stream data.
10. The apparatus according to claim 8, further comprising a video stream distribution module, specifically configured to:
acquiring other control instructions from the at least one other client, and controlling at least one other cloud device according to the other control instructions to generate at least one piece of other video stream data;
generating video stream summary data from the at least one other video stream data;
sending the at least one other piece of video stream data to the at least one other client, and sending the video stream summary data to the target client.
11. The apparatus of claim 8, further comprising a data storage module, specifically configured to:
and performing associated storage on the target video stream data and the sound-shape data to generate live playback data.
12. The apparatus according to claim 8, further comprising an authentication module, specifically configured to:
acquiring an operation request for the target cloud equipment from a target client, and performing identity verification on the target client according to identity information contained in the operation request;
and under the condition that the identity authentication is passed, sending the control authority of the target cloud equipment to the target client.
13. The apparatus according to claim 12, further comprising a rights issuing module, specifically configured to:
acquiring a permission issuing request for the target cloud equipment from the target client, and determining at least one auxiliary client from other clients according to a client identifier contained in the permission issuing request;
and sending the control authority of the target cloud equipment to the auxiliary client.
14. The device according to claim 13, further comprising an auxiliary manipulation module, in particular for:
acquiring an auxiliary control instruction from the auxiliary client, and controlling target cloud equipment according to the auxiliary control instruction to generate auxiliary video stream data;
and sending the auxiliary video streaming data to the auxiliary client, the target client and other clients except the auxiliary client.
15. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.
16. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-7.
17. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-7.
CN202210116786.8A 2022-02-07 2022-02-07 Live broadcast method, device, electronic equipment and medium Pending CN114466212A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210116786.8A CN114466212A (en) 2022-02-07 2022-02-07 Live broadcast method, device, electronic equipment and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210116786.8A CN114466212A (en) 2022-02-07 2022-02-07 Live broadcast method, device, electronic equipment and medium

Publications (1)

Publication Number Publication Date
CN114466212A true CN114466212A (en) 2022-05-10

Family

ID=81410837

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210116786.8A Pending CN114466212A (en) 2022-02-07 2022-02-07 Live broadcast method, device, electronic equipment and medium

Country Status (1)

Country Link
CN (1) CN114466212A (en)

Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105681387A (en) * 2015-11-26 2016-06-15 乐视云计算有限公司 Method, device and system for uploading live video
CN111131907A (en) * 2019-12-31 2020-05-08 广州市百果园信息技术有限公司 Live content introduction method and device, storage medium and equipment
CN111818004A (en) * 2019-12-11 2020-10-23 厦门雅基软件有限公司 Cloud game live broadcasting method, cloud game server and computer readable storage medium
US20210092481A1 (en) * 2019-09-19 2021-03-25 Baidu Online Network Technology (Beijing) Co., Ltd. Video playing method and device, electronic device, and readable storage medium
CN112631447A (en) * 2020-12-24 2021-04-09 广州虎牙科技有限公司 Mouse synchronous processing method and device, electronic equipment and storage medium
CN112771582A (en) * 2018-10-02 2021-05-07 索尼公司 Image processing apparatus and method
CN112969099A (en) * 2021-02-07 2021-06-15 海信集团控股股份有限公司 Camera device, first display equipment, second display equipment and video interaction method
CN113015021A (en) * 2021-03-12 2021-06-22 腾讯科技(深圳)有限公司 Cloud game implementation method, device, medium and electronic equipment
CN113163215A (en) * 2020-01-22 2021-07-23 华为技术有限公司 Live broadcast method and live broadcast system based on cloud mobile phone
CN113448475A (en) * 2021-06-30 2021-09-28 广州博冠信息科技有限公司 Interaction control method and device for virtual live broadcast room, storage medium and electronic equipment
CN113655975A (en) * 2021-08-10 2021-11-16 北京百度网讯科技有限公司 Image display method, image display device, electronic apparatus, and medium
CN113703973A (en) * 2021-08-26 2021-11-26 北京百度网讯科技有限公司 Cloud task processing method, system, device, equipment and storage medium
CN113778599A (en) * 2021-09-07 2021-12-10 杭州雾联科技有限公司 Operation response method, operation method, system and related device of cloud game
CN113941156A (en) * 2020-07-16 2022-01-18 深圳瞬玩科技有限公司 Multi-player online game method and system based on cloud mobile phone

Patent Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105681387A (en) * 2015-11-26 2016-06-15 乐视云计算有限公司 Method, device and system for uploading live video
CN112771582A (en) * 2018-10-02 2021-05-07 索尼公司 Image processing apparatus and method
US20210092481A1 (en) * 2019-09-19 2021-03-25 Baidu Online Network Technology (Beijing) Co., Ltd. Video playing method and device, electronic device, and readable storage medium
CN111818004A (en) * 2019-12-11 2020-10-23 厦门雅基软件有限公司 Cloud game live broadcasting method, cloud game server and computer readable storage medium
CN111131907A (en) * 2019-12-31 2020-05-08 广州市百果园信息技术有限公司 Live content introduction method and device, storage medium and equipment
CN113163215A (en) * 2020-01-22 2021-07-23 华为技术有限公司 Live broadcast method and live broadcast system based on cloud mobile phone
CN113941156A (en) * 2020-07-16 2022-01-18 深圳瞬玩科技有限公司 Multi-player online game method and system based on cloud mobile phone
CN112631447A (en) * 2020-12-24 2021-04-09 广州虎牙科技有限公司 Mouse synchronous processing method and device, electronic equipment and storage medium
CN112969099A (en) * 2021-02-07 2021-06-15 海信集团控股股份有限公司 Camera device, first display equipment, second display equipment and video interaction method
CN113015021A (en) * 2021-03-12 2021-06-22 腾讯科技(深圳)有限公司 Cloud game implementation method, device, medium and electronic equipment
CN113448475A (en) * 2021-06-30 2021-09-28 广州博冠信息科技有限公司 Interaction control method and device for virtual live broadcast room, storage medium and electronic equipment
CN113655975A (en) * 2021-08-10 2021-11-16 北京百度网讯科技有限公司 Image display method, image display device, electronic apparatus, and medium
CN113703973A (en) * 2021-08-26 2021-11-26 北京百度网讯科技有限公司 Cloud task processing method, system, device, equipment and storage medium
CN113778599A (en) * 2021-09-07 2021-12-10 杭州雾联科技有限公司 Operation response method, operation method, system and related device of cloud game

Similar Documents

Publication Publication Date Title
US11158102B2 (en) Method and apparatus for processing information
US10348795B2 (en) Interactive control management for a live interactive video game stream
CN111773667B (en) Live game interaction method and device, computer readable medium and electronic equipment
CN111625214B (en) Audio control method, device, equipment and storage medium
CN111277845B (en) Game live broadcast control method and device, computer storage medium and electronic equipment
WO2020010819A1 (en) Live broadcast room-based data interaction method and device, terminal, and storage medium
WO2021143182A1 (en) Game processing method and apparatus, electronic device, and computer-readable storage medium
US10271105B2 (en) Method for playing video, client, and computer storage medium
CN111654715B (en) Live video processing method and device, electronic equipment and storage medium
WO2023071917A1 (en) Virtual object interaction method and device, and storage medium and computer program product
CN113965809A (en) Method and device for simultaneous interactive live broadcast based on single terminal and multiple platforms
CN110795004A (en) Social method and device
CN114222195A (en) Live broadcast stream pushing method and device, electronic equipment and storage medium
CN112354176A (en) Cloud game implementation method, cloud game implementation device, storage medium and electronic equipment
CN113655975B (en) Image display method, image display device, electronic apparatus, and medium
CN114339069A (en) Video processing method and device, electronic equipment and computer storage medium
CN111770388B (en) Content processing method, device, equipment and storage medium
WO2020253452A1 (en) Status message pushing method, and method, device and apparatus for switching interaction content in live broadcast room
CN114466212A (en) Live broadcast method, device, electronic equipment and medium
CN113411661B (en) Method, apparatus, device, storage medium and program product for recording information
CN112929685B (en) Interaction method and device for VR live broadcast room, electronic device and storage medium
CN112714331B (en) Information prompting method and device, storage medium and electronic equipment
KR20220041791A (en) Method for processing live broadcast information, apparatus, device, storage medium and computer program
CN114125498A (en) Video data processing method, device, equipment and storage medium
CN114449355A (en) Live broadcast interaction method, device, equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination