WO2017211112A1

WO2017211112A1 - Video live streaming method, user device, and unmanned aerial vehicle controller

Info

Publication number: WO2017211112A1
Application number: PCT/CN2017/079373
Authority: WO
Inventors: 高扬
Original assignee: 中兴通讯股份有限公司
Priority date: 2016-06-06
Filing date: 2017-04-01
Publication date: 2017-12-14
Also published as: CN107465935A; CN107465935B

Abstract

A video live streaming method, comprising: a first user device sends to a network server providing an unmanned aerial vehicle video live streaming function a request message for acquiring an operation interface of unmanned aerial vehicle video live streaming; the first user device receives the operation interface of unmanned aerial vehicle video live streaming returned by the network server, and then triggers connection and pairing with an unmanned aerial vehicle controller to interact and acquire a description parameter of a session description protocol; and the first user device receives and live streams a video stream transmitted via a media channel established by the unmanned aerial vehicle controller by means of a signaling system.

Description

Video live broadcast method, user equipment and drone remote controller

Technical field

The present application relates to, but is not limited to, the field of communication technologies, and more particularly to a video live broadcast method, a user equipment, and a drone remote controller.

Background technique

A drone is a non-manned aerial vehicle that is operated by a radio remote control device or an onboard computer program control system. The drone has a simple structure and low cost of use, which not only can complete the tasks performed by manned aircraft, but also is suitable for tasks that are not suitable for manned aircraft. In emergencies, emergency and early warning have a great effect. Aerial photography of drones is based on drones as an aerial platform, with on-board remote sensing equipment such as high-resolution CCD (Charge-coupled Device) digital cameras, light optical cameras, infrared scanners, laser scanners. The magnetic measuring instrument acquires information, processes the image information with a computer, and creates an image according to a certain precision requirement. UAV technology has outstanding features in design and optimization combination. It is a new application technology that integrates high-altitude shooting, remote control, telemetry technology, video image microwave transmission and computer image information processing.

At present, with the popularity of some entry-level commercial drones (such as the Dajiang UAV), drone aerial photography has entered the field of ordinary people from the professional level. Uploading aerial video and sharing video links among friends has become a fashion. The new perspective brought by aerial photography can really bring shocking beauty.

The drone aerial photography system consists of two parts:

The drone and camera, as shown in Figure 1, include an aircraft (unmanned aerial vehicle) and a camera for aerial photography;

The drone remote control, as shown in Figure 2, is a general commercial drone, and there is also a mobile phone holder (for the user to view the aerial video in real time).

The video image captured by the camera is transmitted to the remote control side of the drone through the wireless image transmission technology between the drone and the drone remote control. The drone remote control can store the video image on the local memory of the remote control (such as SD (Secure Digital) card); It is connected to the mobile phone via WiFi (Wireless Fidelity) and stored on the mobile phone (displayed to the user in real time).

Summary of invention

The following is an overview of the topics detailed in this document. This Summary is not intended to limit the scope of the claims.

Although drone aerial photography is a great technological advancement, from the perspective of communication, this aerial photography mode is still a recording and broadcasting mode, which cannot be broadcasted in the first time, cannot be interactive online, and is still in the real-time and interactive original target of communication. With distance, the gap is like the gap between digital cameras and smartphones that can communicate with video calls in real time.

The embodiment of the invention provides a video live broadcast method, a user equipment and a drone remote controller, so as to realize live video broadcast on an image taken by the drone.

The embodiment of the invention provides a video live broadcast method, including:

The first user equipment sends a request message for obtaining an operation interface of the drone video live broadcast to the network server that provides the drone video live broadcast function;

After receiving the operation interface of the drone video live broadcast returned by the network server, the first user equipment triggers a connection pairing with the remote control remote controller, and acquires a description parameter of the session description protocol.

The first user equipment receives and broadcasts a video stream transmitted by the UAV remote controller through a media channel established by the signaling system.

An embodiment of the present invention provides a user equipment, including:

The first sending module is configured to send a request message for acquiring an operation interface of the drone video live broadcast to the network server that provides the drone video live broadcast function;

The first pairing module is configured to: after receiving the operation interface of the drone video live broadcast returned by the network server, trigger a connection pairing with the drone remote controller to acquire a description parameter of the session description protocol;

The first playing module is configured to receive and broadcast a video stream transmitted by the UAV remote controller through a media channel established by the signaling system.

The network server that provides the live broadcast function of the drone receives the request message of the operation interface of the first user equipment for obtaining the live broadcast of the drone video;

The network server sends an operation interface of the drone video live broadcast to the first user equipment.

An embodiment of the present invention provides a network server, including:

The first receiving module is configured to receive a request message of the first user equipment for acquiring an operation interface of the drone video live broadcast;

The sixth sending module is configured to send an operation interface of the drone video live broadcast to the first user equipment.

After receiving the invitation message of the video call of the signaling system, the user equipment returns an acknowledgement message to the signaling system;

The user equipment receives and broadcasts a video stream transmitted by the UAV remote controller through a media channel established by the signaling system.

An embodiment of the present invention provides a user equipment, including:

a fifth transmission module, configured to receive an invitation message of the video call of the signaling system, and return an acknowledgement message to the signaling system;

The second playing module is configured to receive and broadcast the video stream transmitted by the drone remote controller through the media channel established by the signaling system.

The UAV remote controller performs connection and pairing with the first user equipment, and acquires description parameters of the session description protocol.

The UAV remote control receives the video stream sent by the UAV, and sends the video stream to the conference function entity of the first user equipment or the signaling system by using a media channel established by the signaling system.

The embodiment of the invention further provides a drone remote controller, comprising:

The second pairing module is configured to perform connection and pairing with the first user equipment, and acquire a description parameter of the session description protocol.

The sixth transmission module is configured to receive the video stream sent by the drone, and send the video stream to the conference function entity of the first user equipment or the signaling system by using a media channel established by the signaling system.

The embodiment of the invention further provides a computer readable storage medium storing computer executable instructions, which are implemented by the processor to implement the video live broadcast method.

In summary, the embodiment of the present invention provides a live video method, a user equipment, and a remote controller for a drone, which can realize live video broadcast on an image taken by the drone, and can also broadcast the aerial video of the drone to a friend in real time. And allow interaction between friends to improve the user experience.

Other aspects will be apparent upon reading and understanding the drawings and detailed description.

BRIEF abstract

Figure 1 is a schematic view of a drone and a camera;

2 is a schematic diagram of a drone remote controller (including a mobile phone bracket) according to an embodiment of the present invention;

3 is a system architecture diagram of a live video method according to an embodiment of the present invention;

4 is a flowchart of a video live broadcast method on a user equipment side according to an embodiment of the present invention;

FIG. 5 is a flowchart of a video live broadcast method on a network server side according to an embodiment of the present invention;

FIG. 6 is a flowchart of a video live broadcast method on a remote user equipment side according to an embodiment of the present invention;

FIG. 7 is a flowchart of a video live broadcast method on a remote user equipment side according to an embodiment of the present invention;

8 is a flowchart of pairing a UAV with a UE and establishing a live video broadcast according to an embodiment of the present invention;

FIG. 9 is a flowchart of creating a conference for live video broadcast according to an embodiment of the present invention;

FIG. 10 is a flowchart of inviting a friend to enter a live video broadcast according to an embodiment of the present invention;

FIG. 11 is a flowchart of adding a URI to a live video broadcast according to an embodiment of the present invention;

FIG. 12 is a flowchart of the control of the camera being handed over to the friend control according to an embodiment of the present invention; FIG.

FIG. 13 is a flowchart of controlling the flight of a drone to a friend control according to an embodiment of the present invention; FIG.

FIG. 14 is a schematic diagram of a user equipment according to an embodiment of the present invention; FIG.

FIG. 15 is a schematic diagram of a network server according to an embodiment of the present invention; FIG.

FIG. 16 is a schematic diagram of a user equipment on a remote side according to an embodiment of the present invention;

Figure 17 is a schematic diagram of a remote control of a drone according to an embodiment of the present invention.

Detailed

Embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be noted that, in the case of no conflict, the features in the embodiments and the embodiments in the present application may be arbitrarily combined with each other.

From the above background, if a technology can be proposed, the aerial video of the drone can be broadcast live to friends in real time, and the interaction between the friends is allowed, which is revolutionary for the aerial photography and communication of the drone.

The embodiment of the invention provides a video live broadcast method: the drone side is used as a fusion device, and the mobile phone is connected to the signaling system (for example, IMS (IP Multimedia Subsystem)) to perform live video broadcast and friend interaction. From the perspective of easy deployment and promotion, the interface HTML (Hyper Text Mark-up Language) is controlled, and the control program is scripted.

The functional entity involved in the embodiment of the present invention, as shown in FIG. 3, includes:

Signaling system, signaling system with registration, routing, call and other functions, typical signaling system, such as IMS;

Web Server, which provides pages and corresponding scripts for users to call related services; typical pages, such as HTML5 pages; typical scripts such as JavaScript scripts (hereinafter referred to as JS scripts);

User equipment (UE), a typical user equipment such as a smart phone. Hereinafter, UE_A is the UE of the calling user; UE_B is the UE of the called user;

Remote device: The remote device that the user (UE_A) sets up for the call. The typical remote device is the UE_B or the conference server (including the AS (Application Server) and the media. Preparation) and so on.

External device: A device that is paired with a user device and has media capabilities.

UAV remote control: It belongs to a specific external device, which is paired with the user equipment and has video transmission capability. Alternatively, the control channel can be established through the user equipment. At the same time, the UAV remote control can also communicate with the drone, control the drone, and communicate with the drone to transmit video signals.

Unmanned aerial vehicle: A device that can accept the drone remote control to fly, and can load the imaging device and transmit the video signal to the drone remote control.

Network Interface:

Control interface 1 (CI1), that is, the interface between the UE and the signaling system; the interface content is control signaling, such as a user's registration message, call message, subscription message, and the like. Typical control interface protocols, such as SIP (Session Initiation Protocol), XMPP (Extensible Messaging and Presence Protocol), HTTP+JSON (JavaScript Object Notation), and so on.

Control interface 2 (CI2), that is, the interface between the UE and the Web Server; the interface content is an HTTP request, a response, such as a user requesting to obtain a call related page, a script, and the like. Typical control interface protocols such as HTTP+HTML5.

Control interface 3 (CI3), the interface between the Web Server and the signaling system; the content of the interface is that the Web Server requests the signaling system to obtain a type of information, which is used to construct the content of the page; or the Web Server sends a message to the signaling system. Class request to implement signaling operations.

Control interface 4 (CI4), a connection interface between the UE and an external device, and a control interface for exchanging SDP. The SDP includes a video media description and an optional media description of the control channel.

Among them: the UAV remote control and the mobile phone are generally connected by WiFi (Wireless Fidelity), so the CI4 can be based on the WiFi connection, such as using UPnP (Universal Plug and Play) technology.

CI5: Control interface 5, the control interface between the UAV remote control and the UAV, including the flight control part and the control part of the imaging device.

Media interface 1 (MI1), the media interface between the UE_A drone and the drone remote (external device). The interface content is the video stream captured by the drone; the protocol used by this interface is unmanned Machine manufacturers choose their own.

Media Interface 2 (MI2), the media interface between the UAS remote control (external device) of UE_A and the remote device (such as UE_B). The content of the interface is a video stream taken by the drone; a typical protocol, such as RTP (Real-time Transport Protocol);

Among them, the mobile phone can be used as a 4G wireless router, and the mobile phone assigns a network address to the remote control (external device) of the drone. Because from the media channel, the mobile phone is only used as a data communication device (4G wireless router). From the application layer, the media channel is directly connected to the remote control (external device) and the remote device (such as UE_B).

Media Interface 3 (MI3), the media interface between the UAS remote control (external device) of UE_A and the external device of the remote device (such as UE_B). The interface content is the video stream captured by the drone; a typical protocol, such as RTP; exists when the remote device (such as UE_B) uses an external device.

According to the method provided by the embodiment of the present invention, a UAV side can be implemented as a converged device, and the mobile phone is connected to the signaling system (in this embodiment, the signaling system is IMS), and the live video and the friend interaction are performed. The function greatly improves the user VoLTE (IMS-based voice service) and IMS experience.

FIG. 4 is a flowchart of a video live broadcast method on the user equipment side according to an embodiment of the present invention. As shown in FIG. 4, the method in this embodiment includes:

Step 11: The first user equipment sends a request message for obtaining an operation interface of the drone video live broadcast to the network server that provides the drone video live broadcast function;

Step 12: After receiving the operation interface of the drone video live broadcast returned by the network server, the first user equipment triggers connection pairing with the remote control remote controller, and acquires description parameters of the session description protocol.

Step 13: The first user equipment receives and broadcasts a video stream transmitted by the UAV remote controller through a media channel established by the signaling system.

The method of this embodiment can realize an image captured by a video live drone.

In an embodiment, the first user equipment triggers a connection pairing with the drone remote controller, or the first user equipment receives and broadcasts the drone remote controller through a signaling system. After the video stream transmitted by the vertical media channel, it may also include:

The first user equipment sends an invitation message of the video call to the second user equipment by using the signaling system, so that the second user equipment receives the live video stream.

The signaling system is an IMS.

The description parameters of the session description protocol include a video media description.

The invitation message of the video call carries a video media description of the UAV remote controller and a user interface uniform resource identifier indication.

In an embodiment, after the first user equipment triggers connection and pairing with the UAV remote controller, or the first user equipment receives and broadcasts the media channel transmission established by the UAV remote controller through the signaling system. After the video stream, you can also include:

Sending, by the first user equipment, a request message for acquiring a group relationship and a buddy list to the network server, and receiving a group relationship and a buddy list returned by the network server;

The first user equipment selects one or more conference members from the group relationship and the buddy list, and sends a call request message for creating a video conference to the signaling system, and carries the video media description and the remote control of the drone remote controller. The selected conference member list is such that the selected one or more conference members receive the live video stream through the signaling system.

In an embodiment, the first user equipment sends an invitation message to the third user equipment to enter the video conference by using the signaling system, and carries the user interface uniform resource identifier indication, so that the third user equipment receives the live broadcast. Video stream.

In an embodiment, the first user equipment sends the unified resource identifier of the video conference to the friend by using the group message by using the grouping message, so that the specified friend obtains the live video through the unified resource identifier. The operation interface is configured to receive the live video stream.

In this way, the interaction between the user and the friend can be realized, and the user experience of the image taken by the drone is improved.

In an embodiment, the first user equipment acquires media description information controlled by the camera to the drone remote controller;

The first user equipment sends an invitation message for controlling the camera to the fourth user equipment by using the signaling system, and carries the media description information controlled by the camera, so that the fourth user equipment passes the The drone remote controls the camera of the drone.

In an embodiment, the user equipment acquires media description information of the drone flight control from the UAV remote controller;

Sending, by the signaling system, an invitation message for controlling the flight of the drone to the fifth user equipment, and carrying the media description information of the flight control of the drone, so that the fifth user equipment passes the The human-machine remote control performs flight control on the drone.

FIG. 5 is a flowchart of a video live broadcast method on a network server side according to an embodiment of the present invention. As shown in FIG. 5, the method in this embodiment includes:

Step 21: The network server that provides the live broadcast function of the drone receives the request message of the operation interface of the first user equipment for obtaining the live broadcast of the drone video;

Step 22: The network server sends an operation interface of the drone video live broadcast to the first user equipment.

In an embodiment, after the network server sends the operation interface of the live broadcast of the drone to the first user equipment, the network server further includes:

After receiving the request message of acquiring the group relationship and the buddy list of the first user equipment, the network server sends the request message to the signaling system;

After receiving the group relationship and the buddy list returned by the signaling system, the network server sends the group relationship and the buddy list to the first user equipment.

FIG. 6 is a flowchart of a video live broadcast method on a remote user equipment side according to an embodiment of the present invention. As shown in FIG. 6 , the method in this embodiment includes:

Step 31: After receiving the invitation message of the video call of the signaling system, the user equipment returns an acknowledgement message to the signaling system.

Step 32: The user equipment receives and broadcasts a video stream transmitted by the UAV remote controller through a media channel established by the signaling system.

In an embodiment, after the user equipment receives the invitation message of the video call of the signaling system, the method further includes:

User interface uniform resource carried by the user equipment according to the invitation message of the video call The identifier of the identifier or the uniform resource identifier of the video conference is sent to the network server that provides the live broadcast function of the drone to send a request message for obtaining an operation interface of the live broadcast of the drone;

The user equipment receives an operation interface of the drone video broadcast returned by the network server.

In an embodiment, after receiving the video call request message of the signaling system, the user equipment further includes:

The user equipment receives an invitation message for controlling a camera, and the invitation message for controlling the camera carries media description information controlled by the camera;

The user equipment acquires a control page of the camera from the drone remote controller according to the media description information controlled by the camera.

In an embodiment, after the user equipment obtains the control page of the camera from the drone remote controller according to the media description information controlled by the camera, the method further includes:

The user equipment sends the control instruction of the camera to the drone remote controller according to a control page of the camera.

The user equipment receives an invitation message for controlling the flight of the drone, and the invitation message for controlling the flight of the drone carries the media description information of the flight control of the drone;

And the user equipment sends the control instruction of the drone flight to the drone remote controller according to the media description information of the drone flight control.

FIG. 7 is a flowchart of a video live broadcast method on a remote user equipment side according to an embodiment of the present invention. As shown in FIG. 7, the method in this embodiment includes:

Step 41: The UAV remote controller performs connection and pairing with the first user equipment, and acquires description parameters of the session description protocol.

Step 42: The UAV remote controller receives the video stream sent by the UAV, and sends the video stream to the conference function entity of the first user equipment or the signaling system by using a media channel established by the signaling system.

In an embodiment, the UAV remote controller sends the video stream to the conference function entity of the first user equipment or the signaling system by using a media channel established by the signaling system, and further includes:

After receiving the request of the first user equipment for obtaining the media description controlled by the camera, the UAV remote controller sends the media description information controlled by the camera to the first user equipment;

After receiving the request of the second user equipment to obtain the control page of the camera, the UAV remote controller sends the control page of the camera to the second user equipment;

After receiving the control instruction of the operation camera of the second user equipment, the UAV remote controller sends the control instruction of the operation camera to the drone.

In an embodiment, the UAV remote controller sends the video stream to the conference function entity of the user equipment or the signaling system by using a media channel established by the signaling system, and further includes:

After receiving the request of the first user equipment for obtaining the media description of the drone flight control, the drone remote controller transmits the media description information of the drone flight control to the first user equipment;

After receiving the control instruction of the third user equipment to operate the drone, the UAV remote controller sends a control signal to the drone according to the control instruction.

The application of the embodiment of the present invention will be described in detail below with reference to the accompanying drawings.

FIG. 8 is a flowchart of pairing a UAV with a UE and establishing a live video broadcast according to an embodiment of the present invention. As shown in FIG. 8, the method includes the following steps:

Step 101: The UE_A sends a Get (Get) request message to the Web Server that provides the live broadcast function of the drone, and requests an operation interface (HTML page) of the live broadcast of the drone video;

Step 102, the Web Server returns the operation interface (HTML page) of the drone video live broadcast to the UE_A through the 200 OK message;

Step 103, the script in the page triggers the connection pairing of the UE_A with the UAV remote controller (external device);

Step 104: The script control, the UE_A and the UAV remote controller (external device) exchange the description parameters of the SDP (Session Description Protocol), including at least the video media description; in this step, the UE_A obtains the UAV remote controller. SDP description parameters;

In step 105, the UE_A initiates a video call to the UE_B, and sends an INVITE (Invite Message) to the signaling system, which carries a UI (User Interface) URI (Uniform Resource Identifier) Header indication, and the SDP carries the UAV remote control. Video media description of the device (external device);

Step 106, the signaling system forwards the INVITE to UE_B;

Step 107: The UE_B sends a Get request message to the Web Server according to the UI URI Header indication, requesting an operation interface (HTML page) of the drone video live broadcast;

Step 108, the Web Server returns the operation interface (HTML page) of the drone video live broadcast to the UE_B through the 200 OK message;

Step 109, UE_B returns 200 OK of the INVITE to the Web Server;

UE_B returns 200 OK of the INVITE to the signaling system.

Step 110, the signaling system forwards the 200 OK of the INVITE to the UE_A;

Step 111, UE_A interacts with the UAV remote controller (external device) SDP; in this step, the UAV remote controller obtains the SDP description parameter of UE_B;

Step 112, UE_A sends an ACK;

Step 113, the signaling system forwards the ACK to the UE_B;

Step 114: The drone sends a video stream to the drone remote controller;

Step 115: The UAV remote controller broadcasts the video stream through a media channel established by an signaling system (IMS). In this step, the UAV remote controller sends the video directly to the UE_B through the UE_A, so that the UE_B can play the video stream in a live broadcast manner.

If the UE_B does not need to use the operation interface (HTML page) of the special live broadcast of the UAV, the INVITE message sent by the UE_A may be without the UI URI Header indication, and the UE_B does not need to perform Step 107-Step 108, that is, UE_B is used. The traditional VoLTE terminal can accept live video.

In this embodiment, the UE_A may also invite the UE_B live video after establishing a live video broadcast with the drone remote controller (ie, step 105-step 113 may be performed after step 115).

FIG. 9 is a flowchart of creating a conference for live video broadcast according to an embodiment of the present invention, as shown in FIG. 9 Show, including the following steps:

Step 201: The UE_A sends a Get request to the Web Server that provides the live broadcast function of the drone, and requests an operation interface (HTML page) of the live broadcast of the drone.

Step 202, the Web Server returns the operation interface (HTML page) of the drone video live broadcast to the UE_A through the 200 OK message;

Step 203, the script in the page triggers the connection pairing of the UE_A with the UAV remote controller (external device);

Step 204: UE_A sends a POST request to the Web server to request a group relationship and a buddy list.

Step 205: The Web Server sends a request to the signaling system through the CI3 interface, requesting a group relationship and a buddy list;

Step 206: The signaling system returns the requested group relationship and the buddy list through the CI3 interface.

Step 207, the Web Server constructs a new page by group relationship and buddy list, and brings it to UE_A in 200 OK;

Step 208: The script controls, and the UE_A interacts with the UAV remote controller (external device) to obtain the SDP, and at least includes a video media description.

Step 209: UE_A selects a required live broadcast viewer on the page, initiates a call for creating a video conference to the signaling system, sends an INVITE to the signaling system, carries a UI URI Header indication, and carries the drone remote controller (external device) in the SDP. Video media description and carrying a list of meeting members (live audience);

Step 210: The conference function of the signaling system sends an INVITE to the UE_B (the user equipment (UE) of the selected conference member (ie, the live video viewer)), and carries the UI URI Header indication;

Step 211, UE_B sends a Get request to the Web Server according to the UI URI Header indication, requesting an operation interface (HTML page) of the drone video live broadcast;

Step 212, the Web Server returns the operation interface (HTML page) of the drone video live broadcast to the UE_B through 200 OK;

Step 213, UE_B returns 200 OK of the INVITE;

Step 214, the signaling system forwards the 200 OK of the INVITE to the UE_A;

Step 215, UE_A interacts with the UAV remote controller (external device) SDP;

Step 216, UE_A sends an ACK;

Step 217, the signaling system forwards the ACK to the UE_B;

Step 218, the drone sends a video stream to the drone remote controller;

Step 219: The UAV remote controller transmits the video stream to the conference function entity of the signaling system through the media channel established by the IMS;

Step 220: The conference function entity sends the video copy (such as by the Video Relay function or the traditional MRFP) to the user equipment of the conference member (for example, UE_B);

If UE_B does not need to use the special drone video live operation interface (HTML page), the INVITE sent by the conference function may be without the UI URI Header indication, then UE_B has no steps 211-212. That is, UE_B can accept live video broadcasts using a conventional VoLTE terminal. The conference function may determine whether the INVITE sent to the UE URI header carries the UI URI header according to whether the UE_B is registered with the UI URI Header capability indication.

FIG. 10 is a flowchart of inviting a friend to enter a live video broadcast according to an embodiment of the present invention. As shown in FIG. 10, the method includes the following steps:

Step 301, UE_A has created a video conference (unmanned aerial video broadcast);

Step 302: UE_A sends a Refer message to UE_B, carrying a UI URI Header indication, and ReferTo is set to be a video conference (unmanned video live broadcast);

Step 303, the signaling system forwards the Refer to the UE_B;

Step 304, UE_B returns 200 OK;

Step 305, the signaling system forwards 200 OK to UE_A;

Step 306, UE_B sends a Get request message to the Web Server according to the UI URI Header indication, requesting an operation interface (HTML page) of the drone video live broadcast;

Step 307, the Web Server returns the operation interface (HTML page) of the drone video live broadcast to the UE_B through 200 OK;

Step 308, UE_B sends an INVITE request to join the conference;

Step 309, the conference function of the signaling system returns 200OK to UE_B;

Step 310, UE_B sends an ACK;

Step 311, the drone sends a video stream to the drone remote controller;

Step 312: The UAV remote controller broadcasts the video stream to the conference function entity of the signaling system through the media channel established by the IMS.

In step 313, the conference function entity sends the video to the user equipment UE_B of the conference member, for example, by a Video Relay function or a conventional MRFP (Multimedia Resource Function Processor).

FIG. 11 is a flowchart of adding a URI to a live video broadcast according to an embodiment of the present invention. As shown in FIG. 11, the method in this embodiment includes:

Step 401, UE_A has created a video conference (unmanned aerial video broadcast);

In step 402, the UE_A sends the URI of the video conference (the broadcast of the drone video) to the friend through the group message. The URI of the live broadcast of the typical drone video may be a SIP (Session Initiation Protocol) URI. It can also be an HTTP URI wrapped by Web Server;

Step 403, the signaling system forwards the group message to the group member (such as UE_B);

Step 404: UE_B clicks the HTTP URI in the group message, sends a Get request to the Web server, and requests an operation interface (HTML page) of the drone video live broadcast;

Step 405, the Web Server returns the operation interface (HTML page) of the drone video live broadcast to the UE_B through 200 OK;

Step 406, UE_B sends an INVITE request to join the conference;

Step 407, the conference function of the signaling system returns 200OK to UE_B;

Step 408, UE_B sends an ACK;

Step 409, the drone sends a video stream to the drone remote controller;

Step 410: The UAV remote controller broadcasts the video stream to the conference function of the signaling system through the media channel established by the IMS.

In step 411, the conference function sends a video copy (such as through a Video Relay function, or a conventional MRFP) to UE_B.

FIG. 12 is a flowchart of the control of the camera to the buddy control according to the embodiment of the present invention. As shown in FIG. 12, the method in this embodiment includes the following steps:

Step 501, UE_A has created a video conference (unmanned aerial video broadcast);

Step 502: UE_A interacts with the UAV remote controller to obtain a media description (SDP) controlled by the camera. A common practice is that the camera device itself provides a web address for control. In this manner, for example, the SDP is provided with the web address. description of;

Step 503, UE_A uses the SDP to send an INVITE to UE_B.

Step 504, the signaling system forwards the INVITE to the UE_B;

Step 505, UE_B returns 200 OK;

Step 506, the signaling system forwards 200 OK;

Step 507, UE_A sends an ACK;

Step 508, the signaling system forwards the ACK.

Step 509: UE_B sends a Get message to the remote control of the UAV to the web address carried in the SDP.

Step 510, the drone remote control with the camera control page (HTML) in 200 OK;

Step 511, UE_B operates the control of the camera according to the interface, and the control instruction is sent by the page script to the remote control of the drone in the POST;

Step 512, the UAV remote controller interacts with the UAV to adjust the camera angle;

Step 513, the drone remote controller returns 200OK to UE_B;

Step 514, currently switching to a new perspective video stream, the drone sends the video stream to the drone remote controller;

Step 515: The UAV remote controller transmits the video stream to the conference function entity of the signaling system through the media channel established by the IMS;

In step 516, the conference function entity sends a video copy (eg, via the Video Relay function, or a conventional MRFP) to UE_B.

FIG. 13 is a flowchart of the control of the drone flight to the buddy control according to the embodiment of the present invention. As shown in FIG. 13, the method in this embodiment includes the following steps:

Step 601, UE_A has created a video conference (unmanned aerial video broadcast);

Step 602: UE_A interacts with the UAV remote controller to obtain a media description (SDP) of the UAV flight control; for this special application, the general practice is that the UAV establishes a proprietary control channel through the SDP (SDP description Is m=application);

Step 603, UE_A uses the SDP to send an INVITE to UE_B.

Step 604, the signaling system forwards the INVITE to the UE_B;

Step 605, UE_B returns 200 OK;

Step 606, the signaling system forwards 200 OK;

Step 607, UE_A sends an ACK;

Step 608, the signaling system forwards the ACK.

Step 609, UE_B uses a proprietary control channel to control flight and send control commands to the remote control of the drone;

Step 610: The drone remote controller sends a control signal to the drone according to the control instruction;

Step 611, currently switching to a new view video stream, the drone sends the video stream to the drone remote controller;

Step 612: The UAV remote controller broadcasts the video stream to the conference function of the signaling system through the media channel established by the IMS.

In step 613, the conference function sends a video copy (such as through the Video Relay function, or a conventional MRFP) to the UE_B.

Among them: UE_B control channel operation interface, you can use UE_B drone video live interface.

FIG. 14 is a schematic diagram of a user equipment according to an embodiment of the present invention. As shown in FIG. 14, the user equipment in this embodiment includes:

The first sending module 51 is configured to send a request message for acquiring an operation interface of the drone video live broadcast to the network server that provides the drone video live broadcast function;

The first pairing module 52 is configured to, after receiving the operation interface of the drone video broadcast returned by the network server, trigger a connection pairing with the UAV remote controller, and acquire a description parameter of the session description protocol.

The first playing module 53 is configured to receive and broadcast a video stream transmitted by the UAV remote controller through a media channel established by the signaling system.

In an embodiment, the user equipment in this embodiment may further include:

The second sending module is configured to send an invitation message of the video call to the second user equipment by using the signaling system, where the video call request message carries a video media description of the UAV remote controller and a user interface uniform resource identifier indication.

In an embodiment, the user equipment in this embodiment may further include:

The first transmission module is configured to send a request message for acquiring the group relationship and the buddy list to the network server, and receive the group relationship and the buddy list returned by the network server;

a second transmission module, configured to select one or more conference members from the group relationship and the buddy list, send a call request message for creating a video conference to the signaling system, and carry the video media description of the remote control of the drone and A list of selected conference members, and receiving an acknowledgement message for creating a video conference returned by the signaling system.

In an embodiment, the user equipment in this embodiment may further include:

The third sending module is further configured to send an invitation message to the third user equipment to enter the video conference by using the signaling system, and carry the user resource uniform resource identifier indication; or use the grouping message to send the video conference by using the grouping message. The uniform resource identifier is sent to the specified friend.

In an embodiment, the user equipment in this embodiment may further include:

a first obtaining module, configured to obtain, by the remote control device, a media description information controlled by the camera;

The fourth sending module is further configured to send an invitation message for controlling the camera to the fourth user equipment by using the signaling system, and carrying the media description information controlled by the camera.

In an embodiment, the user equipment in this embodiment may further include:

a second obtaining module, configured to acquire media description information of the drone flight control from the drone remote controller;

The fifth sending module is further configured to send, by using the signaling system, an invitation message for controlling the flight of the drone to the fifth user equipment, and carrying the media description information of the flight control of the drone.

FIG. 15 is a schematic diagram of a network server according to an embodiment of the present invention. As shown in FIG. 15, the network server of this embodiment includes:

In an embodiment, the network server of this embodiment may further include:

The third transmission module is configured to: after receiving the request message of acquiring the group relationship and the buddy list of the user equipment, sending the request message to the signaling system;

The fourth transmission module is configured to send the group relationship and the buddy list to the user equipment after receiving the group relationship and the buddy list returned by the signaling system.

FIG. 16 is a schematic diagram of a user equipment on the remote side according to an embodiment of the present invention. As shown in FIG. 16, the user equipment in this embodiment includes:

The fifth transmission module 71 is configured to: after receiving the invitation message of the video call of the signaling system, return an acknowledgement message to the signaling system;

The second playing module 72 is configured to receive and broadcast a video stream transmitted by the UAV remote controller through the media channel established by the signaling system.

In an embodiment, the user equipment in this embodiment may further include:

a third obtaining module, after the fifth transmission module receives the invitation message of the video call of the signaling system, according to the user resource uniform resource identifier indication carried in the video call request message or the uniform resource identifier of the video conference, The web server of the drone video live broadcast function sends a request message for obtaining an operation interface of the drone video live broadcast;

The third receiving module is configured to receive an operation interface of the drone video broadcast returned by the network server.

In an embodiment, the user equipment in this embodiment may further include:

a second receiving module, configured to receive an invitation message for controlling the camera, where the invitation message for controlling the camera carries media description information controlled by the camera;

The second obtaining module is configured to acquire a control page of the camera from the drone remote controller according to the media description information controlled by the camera.

In an embodiment, the user equipment in this embodiment may further include:

The first control module is configured to send the control command of the camera to the drone remote controller according to the control page of the camera.

In an embodiment, the user equipment in this embodiment may further include:

The fourth receiving module is configured to receive an invitation message for controlling the flight of the drone, and the invitation message for controlling the flight of the drone carries the media description information of the flight control of the drone;

The second control module is configured to receive the media description information according to the drone flight control, and send the control instruction of the drone flight to the drone remote controller.

FIG. 17 is a schematic diagram of a remote control of a drone according to an embodiment of the present invention. As shown in FIG. 17, the remote control of the drone of the embodiment includes:

The second pairing module 81 is configured to perform connection pairing with the first user equipment, and acquire a description parameter of the session description protocol.

The sixth transmission module 82 is configured to receive the video stream sent by the drone, and send the video stream to the conference function entity of the first user equipment or the signaling system by using a media channel established by the signaling system.

In an embodiment, the UAV remote controller of this embodiment may further include:

The seventh transmission module is configured to: after receiving the request message of the media description of the camera control by the first user equipment, send the media description information controlled by the camera to the first user equipment;

The eighth transmission module is configured to receive the control page of the second user equipment acquiring the camera Sending a control page of the camera to the second user equipment after the request;

The ninth transmission module is configured to send the control instruction of the operation camera to the drone after receiving the control instruction of the operation camera of the second user equipment.

The tenth transmission module is configured to: after receiving the request for acquiring the media description of the drone flight control of the first user equipment, send the media description information of the drone flight control to the first user equipment;

The eleventh transmission module is configured to, after receiving the control instruction of the third user equipment to operate the drone, send a control signal to the drone according to the control instruction.

One of ordinary skill in the art will appreciate that all or a portion of the steps described above can be accomplished by a program that instructs the associated hardware, such as a read-only memory, a magnetic or optical disk, and the like. All or part of the steps of the above embodiments may also be implemented using one or more integrated circuits. Correspondingly, each module/unit in the foregoing embodiment may be implemented in the form of hardware or in the form of a software function module. Embodiments of the invention are not limited to any specific form of combination of hardware and software.

The above is only the embodiment of the present invention. Of course, the present application can also have various other embodiments. Those skilled in the art can make various corresponding according to the present application without departing from the spirit and spirit of the present application. Changes and modifications, but such corresponding changes and modifications are intended to fall within the scope of the appended claims.

Industrial applicability

The embodiment of the invention provides a video live broadcast method, a user equipment and a drone remote controller, which can realize live broadcast of images taken by the drone, and can also broadcast the aerial video of the drone to friends in real time, and allow friends. The interaction between the two increases the user experience.

Claims

A video live broadcast method includes:

The first user equipment sends a request message for obtaining an operation interface of the drone video live broadcast to the network server that provides the drone video live broadcast function;

After receiving the operation interface of the drone video live broadcast returned by the network server, the first user equipment triggers a connection pairing with the remote control remote controller, and acquires a description parameter of the session description protocol.

The first user equipment receives and broadcasts a video stream transmitted by the UAV remote controller through a media channel established by the signaling system.
The video live broadcast method according to claim 1, wherein: the first user equipment triggers a connection pairing with the drone remote controller, or the first user equipment receives and broadcasts the drone remote controller After the video stream transmitted by the media channel established by the signaling system, the method further includes:

The first user equipment sends an invitation message of the video call to the second user equipment by using the signaling system, so that the second user equipment receives the live video stream.
A video live broadcast method according to claim 2, wherein:

The signaling system is an IP Multimedia Subsystem IMS;

The description parameters of the session description protocol include a video media description.

The invitation message of the video call carries a video media description of the UAV remote controller and a user interface uniform resource identifier indication.
The video live broadcast method according to claim 1, wherein: the first user equipment triggers a connection pairing with the drone remote controller, or the first user equipment receives and broadcasts the drone remote controller After the video stream transmitted by the media channel established by the signaling system, the method further includes:

Sending, by the first user equipment, a request message for acquiring a group relationship and a buddy list to the network server, and receiving a group relationship and a buddy list returned by the network server;

The first user equipment selects one or more conference members from the group relationship and the buddy list, and sends a call request message for creating a video conference to the signaling system, and carries the remote control of the drone. The video media description of the device and the selected list of conference members are such that the selected one or more conference members receive the live video stream through the signaling system.
The video live broadcast method according to claim 4, wherein: the first user equipment receives and broadcasts a video stream transmitted by the UAV remote controller through a media channel established with the UAV remote controller, Also includes:

The first user equipment sends an invitation message to the third user equipment to the third user equipment, and carries the user interface uniform resource identifier indication, so that the third user equipment receives the live video stream.
The video live broadcast method according to claim 4, wherein: the first user equipment receives and broadcasts a video stream transmitted by the UAV remote controller through a media channel established with the UAV remote controller, Also includes:

Sending, by the signaling system, the unified resource identifier of the video conference to the designated friend by using the grouping message, so that the specified friend obtains the operation interface of the live video broadcast through the unified resource identifier, thereby receiving The live video stream.
The video live broadcast method according to claim 4, wherein: the first user equipment receives and broadcasts a video stream transmitted by the UAV remote controller through a media channel established with the UAV remote controller, Also includes:

The first user equipment acquires media description information controlled by the camera from the remote control of the drone;

The first user equipment sends an invitation message for controlling the camera to the fourth user equipment by using the signaling system, and carries the media description information controlled by the camera, so that the fourth user equipment operates through the remote control of the drone Man-machine camera.
The video live broadcast method according to claim 4, wherein: the first user equipment receives and broadcasts a video stream transmitted by the UAV remote controller through a media channel established with the UAV remote controller, Also includes:

The first user equipment acquires media description information of the drone flight control from the drone remote controller;

The first user equipment sends a control drone flight to the fifth user equipment through a signaling system The invitation message carries the media description information of the drone flight control, so that the fifth user equipment performs flight control on the drone through the drone remote controller.
A user equipment comprising:

The first sending module is configured to send a request message for acquiring an operation interface of the drone video live broadcast to the network server that provides the drone video live broadcast function;

The first pairing module is configured to: after receiving the operation interface of the drone video live broadcast returned by the network server, trigger a connection pairing with the drone remote controller to acquire a description parameter of the session description protocol;

The first playing module is configured to receive and broadcast a video stream transmitted by the UAV remote controller through a media channel established by the signaling system.
The user equipment of claim 9, further comprising:

The second sending module is configured to send an invitation message of the video call to the second user equipment by using the signaling system, where the video call request message carries a video media description of the UAV remote controller and a user interface uniform resource identifier indication.
The user equipment of claim 9, further comprising:

The first transmission module is configured to send a request message for acquiring the group relationship and the buddy list to the network server, and receive the group relationship and the buddy list returned by the network server;

a second transmission module, configured to select one or more conference members from the group relationship and the buddy list, send a call request message for creating a video conference to the signaling system, and carry the video media description of the remote control of the drone and A list of selected conference members, and receiving an acknowledgement message for creating a video conference returned by the signaling system.
The user equipment of claim 11 further comprising

The third sending module is further configured to send an invitation message to the third user equipment to enter the video conference by using the signaling system, and carry the user resource uniform resource identifier indication; or use the grouping message to send the video conference by using the grouping message. The uniform resource identifier is sent to the specified friend.
The user equipment of claim 11, further comprising:

a first obtaining module, configured to obtain a media description of the camera control from the drone remote controller information;

The fourth sending module is configured to send an invitation message for controlling the camera to the fourth user equipment by using the signaling system, and carrying the media description information controlled by the camera.
The user equipment of claim 11, further comprising:

a second obtaining module, configured to acquire media description information of the drone flight control from the drone remote controller;

The fifth sending module is configured to send, by using a signaling system, an invitation message for controlling the flight of the drone to the fifth user equipment, and carrying the media description information of the flight control of the drone.
A video live broadcast method includes:

The network server that provides the live broadcast function of the drone receives the request message of the operation interface of the first user equipment for obtaining the live broadcast of the drone video;

The network server sends an operation interface of the drone video live broadcast to the first user equipment.
The video live broadcast method of claim 15, wherein: after the network server sends the operation interface of the drone video live broadcast to the first user equipment, the method further includes:

After receiving the request message of acquiring the group relationship and the buddy list of the first user equipment, the network server sends the request message to the signaling system;

After receiving the group relationship and the buddy list returned by the signaling system, the network server sends the group relationship and the buddy list to the first user equipment.
A web server that includes:

The first receiving module is configured to receive a request message of the first user equipment for acquiring an operation interface of the drone video live broadcast;

The sixth sending module is configured to send an operation interface of the drone video live broadcast to the first user equipment.
The network server of claim 17 further comprising:

The third transmission module is configured to: after receiving the request message of acquiring the group relationship and the buddy list of the user equipment, sending the request message to the signaling system;

The fourth transmission module is configured to send the group relationship and the buddy list to the user equipment after receiving the group relationship and the buddy list returned by the signaling system.
A video live broadcast method includes:

After receiving the invitation message of the video call of the signaling system, the user equipment returns an acknowledgement message to the signaling system;

The user equipment receives and broadcasts a video stream transmitted by the UAV remote controller through a media channel established by the signaling system.
The video broadcast method of claim 19, wherein: after the user equipment receives the invitation message of the video call of the signaling system, the method further includes:

The user equipment sends an operation interface for obtaining a live broadcast of the drone video to the network server that provides the live broadcast function of the drone according to the user resource uniform resource identifier indication or the uniform resource identifier of the video conference carried in the invitation message of the video call. Request message

The user equipment receives an operation interface of the drone video broadcast returned by the network server.
The video broadcast method of claim 20, wherein the user equipment receives and broadcasts a video stream transmitted by the media channel of the UAV remote controller through the signaling system, and further includes:

The user equipment receives an invitation message for controlling a camera, and the invitation message for controlling the camera carries media description information controlled by the camera;

The user equipment acquires a control page of the camera from the drone remote controller according to the media description information controlled by the camera.
The method of claim 21, wherein: after the user equipment obtains the control page of the camera from the remote control of the drone according to the media description information controlled by the camera, the method further includes:

The user equipment sends a control instruction of the camera to the drone remote controller according to a control page of the camera.
A video live broadcast method according to claim 20, wherein: said user equipment receives and broadcasts a video stream transmitted by the drone remote controller through a media channel established by the signaling system, and further include:

The user equipment receives an invitation message for controlling the flight of the drone, and the invitation message for controlling the flight of the drone carries the media description information of the flight control of the drone;

And the user equipment sends the control instruction of the drone flight to the drone remote controller according to the media description information of the drone flight control.
A user equipment comprising:

a fifth transmission module, configured to receive an invitation message of the video call of the signaling system, and return an acknowledgement message to the signaling system;

The second playing module is configured to receive and broadcast the video stream transmitted by the drone remote controller through the media channel established by the signaling system.
The user equipment of claim 24, further comprising

a third acquiring module, after the fifth transmission module receives the video call request message of the signaling system, according to the user resource uniform resource identifier indication carried in the invitation message of the video call or the uniform resource identifier of the video conference, providing The web server of the drone video live broadcast function sends a request message for obtaining an operation interface of the drone video live broadcast;

The second receiving module is configured to receive an operation interface of the drone video broadcast returned by the network server.
The user equipment of claim 25, further comprising:

a third receiving module, configured to receive an invitation message for controlling the camera, where the invitation message for controlling the camera carries media description information controlled by the camera;

The fourth obtaining module is configured to obtain a control page of the camera from the drone remote controller according to the media description information controlled by the camera.
The user equipment of claim 26, further comprising:

The first control module is configured to send the control command of the camera to the drone remote controller according to the control page of the camera.
The user equipment of claim 25, further comprising:

a fourth receiving module, configured to receive an invitation message for controlling drone flight, the control is absent The invitation message of the man-machine flight carries the media description information of the flight control of the drone;

The second control module is configured to receive the media description information according to the drone flight control, and send the control instruction of the drone flight to the drone remote controller.
A video live broadcast method includes:

The UAV remote controller performs connection and pairing with the first user equipment, and acquires description parameters of the session description protocol.

The UAV remote control receives the video stream sent by the UAV, and sends the video stream to the conference function entity of the first user equipment or the signaling system by using a media channel established by the signaling system.
A video live broadcast method according to claim 29, wherein: said UAV remote controller transmits said video stream to a conference function of said first user equipment or signaling system through a media channel established by a signaling system After the entity, it also includes:

After receiving the request of the first user equipment for obtaining the media description controlled by the camera, the UAV remote controller sends the media description information controlled by the camera to the first user equipment;

After receiving the request of the second user equipment to obtain the control page of the camera, the UAV remote controller sends the control page of the camera to the second user equipment;

After receiving the control instruction of the operation camera of the second user equipment, the UAV remote controller sends the control instruction of the operation camera to the drone.
The video live broadcast method according to claim 29, wherein the UAV remote controller sends the video stream to the conference function entity of the user equipment or the signaling system through a media channel established by the signaling system. ,Also includes:

After receiving the request of the first user equipment for obtaining the media description of the drone flight control, the drone remote controller transmits the media description information of the drone flight control to the first user equipment;

After receiving the control instruction of the third user equipment to operate the drone, the UAV remote controller sends a control signal to the drone according to the control instruction.
A drone remote control comprising:

The second pairing module is configured to perform connection and pairing with the first user equipment, and acquire a description parameter of the session description protocol.

The sixth transmission module is configured to receive the video stream sent by the drone, and send the video stream to the conference function entity of the first user equipment or the signaling system by using a media channel established by the signaling system.
The drone remote controller of claim 32, further comprising:

The seventh transmission module is configured to: after receiving the request message of the media description of the camera control by the first user equipment, send the media description information controlled by the camera to the first user equipment;

The eighth transmission module is configured to: after receiving the request of the second user equipment to obtain the control page of the camera, send the control page of the camera to the second user equipment;

The ninth transmission module is configured to send the control instruction of the operation camera to the drone after receiving the control instruction of the operation camera of the second user equipment.
The drone remote controller of claim 32, further comprising:

The tenth transmission module is configured to: after receiving the request for acquiring the media description of the drone flight control of the first user equipment, send the media description information of the drone flight control to the first user equipment;

The eleventh transmission module is configured to, after receiving the control instruction of the third user equipment to operate the drone, send a control signal to the drone according to the control instruction.