CN108650494B

CN108650494B - Live broadcast system capable of instantly obtaining high-definition photos based on voice control

Info

Publication number: CN108650494B
Application number: CN201810533699.6A
Authority: CN
Inventors: 朱磊; 李建英; 韩琦
Original assignee: Qingdao Yispace Technology Co ltd
Current assignee: Qingdao Yispace Technology Co ltd
Priority date: 2018-05-29
Filing date: 2018-05-29
Publication date: 2020-08-04
Anticipated expiration: 2038-05-29
Also published as: CN108650494A

Abstract

The invention provides a voice control-based live broadcast system capable of acquiring high-definition photos immediately, which comprises an upper computer end, a lower computer end, a server end and a user end, wherein the upper computer end is connected with the lower computer end; the upper computer end comprises a movable bearing device which is provided with a holder, an image acquisition device and a first transmission module; the image acquisition equipment comprises one or more camera modules; the lower computer end comprises a remote control device, a second transmission module and a voice module. According to the system, the two cameras of each camera module respectively output the picture data and the video data which have the time corresponding relation and the visual angles of which are close to the same, and the double data streams are output, so that the requirement of a user for shooting high-definition pictures in real time when watching videos is met; when watching live broadcast, a user sends voice or non-voice control information, the server obtains corresponding voice and forwards the voice to ground staff, the server is instructed to carry out corresponding remote control operation to control the movable bearing equipment and/or the holder, the watching visual angle of the user can be changed, and the entertainment interactivity of the user is improved.

Description

Live broadcast system capable of instantly obtaining high-definition photos based on voice control

Technical Field

The invention relates to an information processing technology, in particular to a live broadcast system capable of instantly obtaining high-definition pictures based on voice control.

Background

Aerial photography is also called aerial photography or aerial photography, and is used for shooting and shooting the earth landform, urban landscape, people and the like in the air. At present, unmanned aerial vehicles are widely used in military affairs, traffic construction, hydraulic engineering, ecological research, television columns, artistic photography and the like.

However, no matter the existing unmanned aerial vehicle aerial photography technology or other technologies such as vehicle-mounted shooting, these existing technologies can only output videos or output pictures in a single mode, and correspond to a single user, and cannot meet the requirements of multiple users and multiple aspects, so that inconvenience is brought to the user.

Disclosure of Invention

The following presents a simplified summary of the invention in order to provide a basic understanding of some aspects of the invention. It should be understood that this summary is not an exhaustive overview of the invention. It is not intended to determine the key or critical elements of the present invention, nor is it intended to limit the scope of the present invention. Its sole purpose is to present some concepts in a simplified form as a prelude to the more detailed description that is discussed later.

In view of the above, the present invention provides a voice control-based live broadcast system capable of acquiring high definition photos in real time, so as to at least solve the problem that the existing technologies such as unmanned aerial vehicle aerial photography or vehicle-mounted shooting cannot meet the requirements of multiple users and multiple aspects because only video or pictures can be output in a single mode and correspond to a single user.

According to one aspect of the invention, a live broadcast system capable of acquiring high-definition photos instantly based on voice control is provided, and comprises an upper computer end, a lower computer end, a server end and a user end; the upper computer end comprises movable bearing equipment, and the movable bearing equipment is provided with a holder, image acquisition equipment connected with the holder and a first transmission module; the holder is used for adjusting the position and/or the direction of the image acquisition equipment connected with the holder; the image acquisition equipment comprises one or more camera modules, wherein each camera module comprises two cameras, and the corresponding acquisition visual angles of the two cameras in the same camera module at the same moment are approximately the same; the first transmission module comprises one or more wireless image transmission modules for communicating with the lower computer end; the two cameras in each camera module are respectively used for outputting picture data and video data, the picture data comprises one or more groups of pictures, and the video data comprises one or more groups of video frame sequences; the lower computer end comprises remote control equipment, a second transmission module and a voice module; the second transmission module comprises one or more wireless image transmission modules for communicating with the upper computer end and a communication module for communicating with the server end; the lower computer end is used for receiving and forwarding an acquisition instruction from the server end to the upper computer end so as to enable the image acquisition equipment at the upper computer end to start or stop acquiring image/video data in the visual field of the image acquisition equipment, and receiving the image/video data from the upper computer end to forward the image/video data to the server end in real time; the remote control equipment is suitable for receiving control instructions of ground workers, and the control instructions comprise a first instruction for controlling the movable bearing equipment and a second instruction for controlling the holder; the remote control equipment is suitable for sending a first instruction in the control instruction to a first control module in the movable bearing equipment so as to control the movable bearing equipment to perform corresponding operation action, and sending a second instruction in the control instruction to a second control module of the holder so as to control the holder to perform corresponding operation action; the voice module is suitable for receiving voice control information from the server side, generating corresponding voice according to the voice control information and playing the voice to the ground staff so as to instruct the ground staff to input corresponding control instructions to the remote control equipment; the server is used for carrying out instant and corresponding processing and storage on the picture/video data received from the lower computer end, acquiring the picture/video data matched with the request instruction and sending the picture/video data to the user end sending the request instruction when receiving the request instruction from the user end, receiving the control information from the user end, generating corresponding voice control information and sending the voice control information to the lower computer end; wherein the request instruction comprises a request video transmission instruction and/or a photographing instruction.

Further, the movable carrying device is any one of the following: unmanned aerial vehicle, mobile robotic arm, cable car, sightseeing vehicle, train, plane or ship.

Further, when the movable bearing equipment is an unmanned aerial vehicle, the first instruction comprises an instruction for controlling any one or more operation actions of self-check, take-off, landing, return voyage, hovering, pitching, rolling and yawing of the unmanned aerial vehicle.

Further, the first command comprises a flight direction control command and/or a flight height control command.

Further, the second instruction includes a control instruction for controlling the pan/tilt head to perform horizontal and/or pitch movement.

Further, if the control information sent by the user side is voice control information input through voice, the server side directly forwards the voice control information to the corresponding lower computer side in real time.

Further, if the control information sent by the user side is manually input non-voice control information, the server side converts the non-voice control information into corresponding voice control information so as to send the voice control information to the corresponding lower computer side in real time.

Furthermore, the cradle head is a multi-degree-of-freedom cradle head and is suitable for controlling the image acquisition equipment connected with the cradle head to perform multi-directional adjustment.

Further, the cradle head can control the image acquisition equipment connected with the cradle head to perform any one or more of the following movements: horizontal rotational movement, vertical tilting movement and rotational movement about the optical axis of the image acquisition device.

Furthermore, the cloud deck is provided with a telescopic device, and the image acquisition equipment connected with the cloud deck is arranged on the corresponding telescopic device so that the image acquisition equipment can move along with the telescopic device.

Further, the cradle head comprises a cradle head control module and a driving module, wherein the cradle head control module is used as the second control module; the holder control module is used for receiving a control instruction from the remote control equipment, sending an instruction to the driving module, and sending a control signal and a driving pulse signal to the corresponding motor through the driving module so as to control and change the motion attitude of each rotating shaft of the holder.

Further, the driving module further comprises a limit switch, so that the limit switch controls the rotation range of the corresponding holder.

Furthermore, the camera of each camera module is a wide-angle camera.

Furthermore, the image data and the video data collected by the two cameras in each camera module have a time corresponding relationship.

Furthermore, the movable bearing device, the holder on the movable bearing device and the image acquisition device are all communicated with the lower computer end through an image transmission module.

Further, the movable bearing device, the cradle head on the movable bearing device and the image acquisition device are respectively communicated with the lower computer end through the corresponding image transmission modules.

Furthermore, two cameras in each camera module respectively transmit picture data and video data through one image transmission module.

Furthermore, one or more cloud platforms are arranged on the movable bearing equipment, and the single cloud platform is used for controlling two cameras in the corresponding single camera module to perform synchronous adjustment.

Furthermore, one or more cloud platforms are arranged on the movable bearing equipment, and a single cloud platform is used for controlling cameras of at least two camera modules in the one or more camera modules to perform synchronous adjustment.

Furthermore, a plurality of cloud platforms are arranged on the movable bearing equipment, each camera module corresponds to two cloud platforms, and two cameras in a single camera module are asynchronously fine-tuned through the two cloud platforms respectively.

Furthermore, the number of the upper computer ends is one or more, and the number of the lower computer ends is consistent with that of the upper computer ends; and each upper computer end communicates with the server end through the lower computer end corresponding to the upper computer end.

Further, when the request instruction includes a video transmission instruction, the server is adapted to transmit video data matching the request video transmission instruction by: judging whether the request instruction is a pull stream request or an on-demand video request: if the request is a pull request, the pull address is sent to the corresponding user side; and if the request is a video-on-demand request, sending the stored virtual path address of the video corresponding to the identifier carried in the request instruction to the corresponding user side.

Further, when the request instruction includes a photographing instruction, the server is adapted to obtain the picture data matched with the photographing instruction in the following manner: and selecting picture data corresponding to the sending time from the stored picture data according to the corresponding relation between the plurality of groups of pictures and the video stream and the sending time carried in the photographing instruction, and storing the picture data in the user account sending the photographing instruction so as to respond to the subsequent downloading of the user.

Further, in a case where the image capturing apparatus includes a plurality of camera modules, the server side is adapted to: and determining the camera module to which the picture/video data matched with the request instruction belongs from the plurality of camera modules as a target camera module so as to select the picture/video data collected by the target camera module as the picture/video data matched with the request instruction of the user side.

Further, the server side is suitable for allowing the control information sent by the authorized user to be received and refusing the control information sent by the unauthorized user to be received.

Further, when one user side controls the holder to change the viewing angle, other user sides can share the viewing angle.

Further, after any user side sends the control information to the server side, the server side does not receive other control information of the same type as the control information sent by other user sides before the upper computer side does not implement the control instruction corresponding to the control information and coming from the remote control device.

Further, when receiving a request instruction or control information from a user side, the server side determines whether the instruction or the information contains paid information, and establishes a data connection with the user side when the instruction contains paid information.

Further, the server comprises a processing module, a storage module and a third transmission module; the processing module is suitable for carrying out real-time and corresponding processing on the picture/video data received from the lower computer end, the storage module is suitable for storing the picture/video data processed by the processing module, and the third transmission module is suitable for communicating with the lower computer and the user end.

Further, the server side further includes a user management module, configured to receive an identity authentication request from each user side, perform identity authentication on the corresponding user equipment based on the identity authentication request, and send authentication success information to the corresponding user side after the authentication is successful so as to establish data connection with the user equipment.

Further, each user terminal includes: the sending module is used for sending an access request to the server side to acquire video data, picture data and/or a request for controlling the holder; the receiving module is used for receiving video data, picture data and/or related information data responding to a control cloud deck request sent by the server; the display module is used for displaying the video data and the picture data sent by the server side and/or displaying the operation of a real-time control holder; and the fourth transmission module is used for carrying out data communication with the server side.

Further, the voice module comprises an earphone and/or a loudspeaker.

Furthermore, each camera module is provided with a corresponding sensor group, wherein each sensor group comprises a position sensor suitable for acquiring the position of the corresponding camera module in real time and a posture sensor suitable for acquiring the posture of the corresponding camera module in real time; the server end receives an interest target instruction from a user, selects one of a plurality of preset interest target positions as a current view target position according to the interest point instruction, and adjusts a cloud deck corresponding to the target camera module according to a connecting line between the current view target position and a real-time position of the target camera module so that the main direction of an optical axis of each camera in the target camera module is consistent with the direction of the connecting line; wherein the plurality of preset target locations of interest comprise a plurality of known location points preset within a predetermined area.

Further, when the image acquisition device comprises a camera module, the camera module is used as the target camera module; when the image acquisition equipment comprises a plurality of camera modules, the target camera module is the camera module which is closest to the current viewing target position in the plurality of camera modules.

Furthermore, each camera module is provided with a corresponding sensor group, wherein each sensor group comprises a plurality of position sensors suitable for acquiring the positions of the cameras in the corresponding camera module in real time and a plurality of attitude sensors suitable for acquiring the attitudes of the cameras in the corresponding camera module in real time; the server end receives an interest target instruction from a user, selects one of a plurality of preset interest target positions as a current view target position according to the interest point instruction, and adjusts a cloud deck corresponding to the target camera module according to a connecting line between the current view target position and the real-time position of each camera of the target camera module so that the main direction of the optical axis of each camera in the target camera module is consistent with the connecting line direction; wherein the plurality of preset target locations of interest comprise a plurality of known location points preset within a predetermined area.

Further, when the image acquisition device comprises a camera module, the camera module is used as the target camera module; when the image acquisition equipment comprises a plurality of camera modules, the target camera module is a camera module to which a camera closest to the current viewing target position belongs in each camera of the plurality of camera modules.

And further, two cameras in a single camera module respectively carry out gesture synchronous adjustment or gesture asynchronous adjustment according to the connecting line direction through two holders in real time.

The upper computer end of the live broadcast system capable of acquiring high-definition photos instantly based on voice control is provided with image acquisition equipment comprising one or more camera modules, picture data and video data with time corresponding relations and approximately consistent visual angles are respectively output through two cameras in each camera module, and the live broadcast system is different from a single output mode (video output or picture output) in the existing aerial photography, and pictures can not be downloaded instantly while live broadcast is watched; unlike the prior art that the video is captured in a video stream, the resolution of the video in this form is the same as that of the picture, and if a high-resolution picture is to be obtained, a high-resolution (e.g. 4K) video needs to be taken, which has a strict requirement on the bandwidth during the live broadcasting process; the invention meets the requirement that a user acquires high-resolution (such as 4K) pictures in real time when watching a low-resolution (such as 1080P) live video by setting a double-data-stream output mode of two cameras corresponding to time. The voice control-based live broadcast system capable of acquiring the high-definition photos immediately sends data to the lower computer end through the upper computer end, the lower computer end sends the data to the server end, and the data are sent to at least one user end through the server end, so that a plurality of users can share the system to watch live videos and take photos at the same time.

In addition, by using the live broadcast system capable of instantly obtaining high-definition photos based on voice control of the present invention, a user can not only watch live videos and take photos, but also send control information (voice or non-voice) to the server through the user terminal when watching live broadcasts, and then obtain corresponding voice control information through the server and send the corresponding voice control information (if the user terminal sends the voice control information through voice input, the voice control information is directly forwarded to the voice module in the lower computer terminal (received through the second transmission module in the lower computer terminal), and play corresponding audio content to the ground staff through the voice module to instruct the ground staff to correspondingly operate the remote control equipment, thereby realizing control of corresponding movable bearing equipment and/or a cradle head, realizing real-time control of the actions and/or postures of the movable bearing equipment and/or the cradle head, and further, the watching visual angle of the user can be changed, and the entertainment interactivity of the user is improved.

In addition, the invention can also enable the user to switch the concerned interest target in real time when watching the live video in a mode of allowing the user to set the interest target, thereby providing comfortable viewing effect for the user.

These and other advantages of the present invention will become more apparent from the following detailed description of the preferred embodiments of the present invention, taken in conjunction with the accompanying drawings.

Drawings

The invention may be better understood by referring to the following description in conjunction with the accompanying drawings, in which like reference numerals are used throughout the figures to indicate like or similar parts. The accompanying drawings, which are incorporated in and form a part of this specification, illustrate preferred embodiments of the present invention and, together with the detailed description, serve to further explain the principles and advantages of the invention. In the drawings:

FIG. 1 is a block diagram schematically illustrating one example of a voice control based live system with instant access to high definition photos of the present invention;

fig. 2 is a view schematically showing one possible structure of the server side in fig. 1;

fig. 3 is a view schematically showing one possible structure of a single user terminal in fig. 1.

Skilled artisans appreciate that elements in the figures are illustrated for simplicity and clarity and have not necessarily been drawn to scale. For example, the dimensions of some of the elements in the figures may be exaggerated relative to other elements to help improve the understanding of the embodiments of the present invention.

Detailed Description

Exemplary embodiments of the present invention will be described hereinafter with reference to the accompanying drawings. In the interest of clarity and conciseness, not all features of an actual implementation are described in the specification. It will of course be appreciated that in the development of any such actual embodiment, numerous implementation-specific decisions must be made to achieve the developers' specific goals, such as compliance with system-related and business-related constraints, which will vary from one implementation to another. Moreover, it will be appreciated that such a development effort might be complex and time-consuming, but would nevertheless be a routine undertaking for those of ordinary skill in the art having the benefit of this disclosure.

It should be noted that, in order to avoid obscuring the present invention with unnecessary details, only the device structures and/or processing steps closely related to the solution according to the present invention are shown in the drawings, and other details not so relevant to the present invention are omitted.

The embodiment of the invention provides a voice control-based live broadcast system capable of acquiring high-definition photos immediately, which comprises an upper computer end, a lower computer end, a server end and a user end; the upper computer end comprises movable bearing equipment, and the movable bearing equipment is provided with a holder, image acquisition equipment connected with the holder and a first transmission module; the holder is used for adjusting the position and/or the direction of the image acquisition equipment connected with the holder; the image acquisition equipment comprises one or more camera modules, wherein each camera module comprises two cameras, and the corresponding acquisition visual angles of the two cameras in the same camera module at the same moment are approximately the same; the first transmission module comprises one or more wireless image transmission modules for communicating with the lower computer end; the two cameras in each camera module are respectively used for outputting picture data and video data, the picture data comprises one or more groups of pictures, and the video data comprises one or more groups of video frame sequences; the lower computer end comprises remote control equipment, a second transmission module and a voice module; the second transmission module comprises one or more wireless image transmission modules for communicating with the upper computer end and a communication module for communicating with the server end; the lower computer end is used for receiving and forwarding an acquisition instruction from the server end to the upper computer end so as to enable the image acquisition equipment at the upper computer end to start or stop acquiring image/video data in the visual field of the image acquisition equipment, and receiving the image/video data from the upper computer end to forward the image/video data to the server end in real time; the remote control equipment is suitable for receiving control instructions of ground workers, and the control instructions comprise a first instruction for controlling the movable bearing equipment and a second instruction for controlling the holder; the remote control equipment is suitable for sending a first instruction in the control instruction to a first control module in the movable bearing equipment so as to control the movable bearing equipment to perform corresponding operation action, and sending a second instruction in the control instruction to a second control module of the holder so as to control the holder to perform corresponding operation action; the voice module is suitable for receiving voice control information from the server side, generating corresponding voice according to the voice control information and playing the voice to the ground staff so as to instruct the ground staff to input corresponding control instructions to the remote control equipment; the server is used for carrying out instant and corresponding processing and storage on the picture/video data received from the lower computer end, acquiring the picture/video data matched with the request instruction and sending the picture/video data to the user end sending the request instruction when receiving the request instruction from the user end, receiving the control information from the user end, generating corresponding voice control information and sending the voice control information to the lower computer end; wherein the request instruction comprises a request video transmission instruction and/or a photographing instruction.

Fig. 1 is a block diagram showing an example of a live system capable of instantly obtaining high-definition pictures based on voice control according to the present invention.

As shown in fig. 1, in this example, a live broadcast system capable of instantly obtaining high definition photos based on voice control includes an upper computer end 10, a lower computer end 20, a server end 30 and a user end 40.

The upper computer end 10 comprises a movable bearing device 11, and the movable bearing device 11 is provided with a cradle head 11-1, an image acquisition device 11-2 connected with the cradle head 11-1, and a first transmission module 11-3.

For example, the movable carrying device 11 may be, but is not limited to, any of the following: unmanned aerial vehicles, mobile robotic arms, cable cars, sightseeing vehicles, trains, airplanes or boats, and the like.

The pan/tilt head 11-1 is used to adjust the position and/or orientation of the image capturing device 11-2 connected thereto.

The image capturing device 11-2 includes one or more camera modules, wherein each camera module includes two cameras, and the capturing angles of the two cameras in the same camera module are substantially the same at the same time.

Wherein, two cameras in same camera module are roughly the same at the collection visual angle that corresponds simultaneously, and the collection visual angle that these two cameras are the same basically with the time promptly, for example can be completely unanimous, also can be that the relative contained angle of two is in predetermined range (if two relative contained angles are less than 5 degrees or 8 degrees etc.).

According to the embodiment of the invention, the camera of each camera module can be a wide-angle zoom camera or a non-zoom camera. When two cameras of the camera module are zooming cameras, the two zooming cameras can zoom synchronously, for example, and the relative included angle meets the condition that the picture visual angles acquired by the two cameras at the same time are close to the same. Wherein the relative angle can be adjusted, for example, manually/automatically.

According to the embodiment of the present invention, the arrangement of the two cameras in the single camera module includes, but is not limited to, horizontal or vertical arrangement.

According to the embodiment of the invention, each camera module group can be respectively provided with a corresponding sensor group, wherein each sensor group comprises a position sensor suitable for acquiring the position of the corresponding camera module group in real time and a posture sensor suitable for acquiring the posture of the corresponding camera module group in real time; the server receives an interest target instruction from a user, selects one of a plurality of preset interest target positions as a current view target position according to the interest point instruction, and adjusts a pan-tilt corresponding to the target camera module according to a connecting line between the current view target position and a real-time position of the target camera module, so that the main direction of an optical axis of each camera in the target camera module is consistent with the direction of the connecting line, and the current view target position is located at a shooting center of each camera of the target camera module. In this example, the positions and postures of the two cameras in the same camera module can be regarded as the same, so that the positions of the two cameras in the same camera module are obtained by using the same position sensor, and the postures of the two cameras in the same camera module are obtained by using the same posture sensor. Wherein the plurality of preset target of interest locations comprises a plurality of known location points preset within a predetermined area. When the image acquisition equipment comprises a camera module, the camera module is used as a target camera module; when the image capturing apparatus includes a plurality of camera modules, the target camera module is, for example, a camera module closest to the current viewing target position among the plurality of camera modules.

According to the embodiment of the invention, each camera module group can be respectively provided with a corresponding sensor group, wherein each sensor group comprises a plurality of position sensors suitable for acquiring the positions of the cameras in the corresponding camera module group in real time and a plurality of attitude sensors suitable for acquiring the attitudes of the cameras in the corresponding camera module group in real time; the server end receives an interest target instruction from a user, selects one of a plurality of preset interest target positions as a current view target position according to the interest point instruction, and adjusts a cloud deck corresponding to the target camera module according to a connecting line between the current view target position and the real-time position of each camera of the target camera module, so that the main direction of the optical axis of each camera in the target camera module is consistent with the direction of the connecting line. Wherein the plurality of preset target of interest locations comprises a plurality of known location points preset within a predetermined area. When the image acquisition equipment comprises a camera module, the camera module is used as a target camera module; when the image acquisition equipment comprises a plurality of camera modules, the target camera module is a camera module to which a camera closest to the current view target position belongs in each camera of the plurality of camera modules. In this example, two cameras in a single camera module can perform gesture synchronous adjustment or gesture asynchronous adjustment according to respective corresponding connection directions through two holders respectively in real time. That is to say, in the same camera module, the respective connecting line directions between the two cameras and the current view target position are different, and then the respective corresponding holders of the two cameras can be respectively adjusted (out of synchronization), so that the two cameras can enable the current view target position to be located at the respective shooting center

For example, an unmanned aerial vehicle (as an example of a movable carrying device, the unmanned aerial vehicle described below may also be a movable carrying device of another type) continuously moves in an image acquisition process, and under a condition that a focus is changed, in order to make a coincidence degree of two images shot by two cameras high, it is necessary to adjust postures of the two cameras all the time, a flight trajectory of the unmanned aerial vehicle is preset, for example, the flight trajectory is a straight line or a circle, and an interest target is set, for example, to a certain natural or artificial building at a different position inside or outside the straight line or the circle in a scenic spot, and different flight trajectories and interest targets may be set according to different scenic spots; and the geographic location of the object of interest has been previously acquired by the sight workers.

Therefore, by means of allowing a user to set an interest target, the user can switch the interest target concerned in real time when watching live videos, because the unmanned aerial vehicle moves and the interest target does not move, the position and posture data (position and posture sensors need to be placed on each camera) of two cameras in a single camera module on the unmanned aerial vehicle need to be acquired in real time, the main direction of an optical axis of the camera is determined according to a connecting line between the camera and the interest target (known geographic position), and then the two cameras in the single camera module respectively perform posture synchronous adjustment according to the connecting line data through two cloud platforms in real time, so that the interest target is always positioned in an image center. For example, at the initial flying time, the connection distance between the first interest target and the camera (which are separately calculated by the two cameras) is smaller than or equal to D (D is a set distance), then the focusing object of the camera (which are separately calculated by the two cameras) is the first interest target, after the unmanned aerial vehicle flies for a period of time, the connection distance between the first interest target and the camera (which are separately calculated by the two cameras) is greater than D, and the connection distance between the second interest target and the camera (which are separately calculated by the two cameras) is smaller than or equal to D, then the camera (which are separately calculated) automatically changes the focusing object into the second interest target, if the user still selects the first interest target as the main viewing target, then only the first interest target identifier needs to be clicked, and then the pan-tilt head still controls the image acquisition device to turn to the first interest target.

Therefore, after recognizing the acquisition command, each camera module acquires the image/video data in the field of view and transmits the image/video data to the lower computer 20 in real time.

The first transmission module 11-3 includes one or more wireless map transmission modules for communicating with the lower computer terminal 20. The two cameras in each camera module are respectively used for outputting picture data and video data, the picture data comprises one or more groups of pictures, and the video data comprises one or more groups of video frame sequences. That is, in a single camera module, one of the cameras is used to output picture data and the other camera is used to output video data.

Wherein, the picture data and the video data that two cameras including in every camera module gathered have time corresponding relation, and two cameras receive simultaneously and gather the instruction, gather picture data and video data respectively, and the corresponding relation is as follows: the video data takes n frames as a frame sequence, the video stream is divided into a plurality of frame sequences according to the playing time sequence, the r-th frame sequence is represented by Ir, and the n-frame images in the r-th video frame sequence are represented by Ir1, Ir2 and … … Irn; the picture data acquired at the same time in the same camera module can be denoted by Dr, and Dr1, Dr2 and … … Drn denote the picture data corresponding to the n-frame images in the r-th video frame sequence.

According to the embodiment of the present invention, under the condition that the image data transmission speed is ensured, the movable carrying device 11, the pan-tilt 11-1 on the movable carrying device 11, and the image capturing device 11-2 can communicate with the lower computer terminal 20 through the same mapping module, for example.

According to the embodiment of the present invention, the movable carrying device 11, the cradle head 11-1 on the movable carrying device 11, and the image capturing device 11-2 may also communicate with the lower computer 20 through the corresponding mapping modules respectively. In this way, the data transmission of the corresponding device is transmitted through the multiple image transmission modules, for example, the movable carrying device 11 transmits data through the first image transmission module, the pan-tilt 11-1 transmits data through the second image transmission module (if there are multiple pan-tilt 11-1, for example, data may be transmitted through the multiple second image transmission modules), and the image acquisition device 11-2 transmits data through the third image transmission module.

In one example, in the image capturing apparatus 11-2, when a plurality of camera modules are included, the plurality of camera modules may transmit data through a plurality of image transfer modules, for example, or may transmit data through only the same image transfer module.

In addition, for the same group of camera modules, the two cameras can transmit data through the same image transmission module, and also can transmit data through the two image transmission modules respectively. For example, in order to meet the requirement of fast transmission of image data, two cameras in each camera module group can respectively transmit picture data and video data through one image transmission module, that is, the two cameras respectively transmit the picture data and the video data through the two image transmission modules, so that the picture data and the video data are respectively transmitted.

As shown in fig. 1, the lower computer terminal 20 includes a second transmission module 21, a remote control device 22 and a voice module 23, and the second transmission module 21 includes one or more wireless map transmission modules for communicating with the upper computer terminal 10 and a communication module for communicating with the server terminal, which may be a wired communication module or a wireless communication module, for example.

The lower computer terminal 20 is configured to receive and forward the acquisition instruction from the server terminal 30 to the upper computer terminal 10 (for example, forward the acquisition instruction to a pan/tilt or an image acquisition device), so that the image acquisition device 11-2 of the upper computer terminal 10 starts or stops acquiring image/video data in the field of view thereof, and receive the image/video data from the upper computer terminal 10 to forward the image/video data to the server terminal 30 in real time.

In addition, the remote control device 22 is adapted to receive manipulation instructions from ground personnel. The remote control device 22 is, for example, a remote controller operated by a ground worker, and is used to control the movable carrying device 11 and/or the pan/tilt head 11-1, etc. in the upper computer terminal 10.

The control command received by the remote control device 22 may include a first command for controlling the movable carrying device 11 and a second command for controlling the pan/tilt head 11-1.

The remote control device 22 may, for example, send a first command of the control commands to a first control module (such as a flight control module in the unmanned aerial vehicle) in the movable carrying device 11 to control the movable carrying device to perform a corresponding operation action, and send a second command of the control commands to a second control module (such as a cradle head control module mentioned below) of the cradle head 11-1 to control the cradle head 11-1 to perform a corresponding operation action.

The control instructions may include various types of instructions, among others. For example, the control instructions include a first instruction for controlling the movable carrying apparatus 11 and a second instruction for controlling the pan/tilt head 11-1.

Different types of control instructions (e.g., first and second instructions) may be distinguished by specific fields in the control instructions. For example, a first field in the control instruction corresponds to a first instruction, and a second field in the control instruction corresponds to a second instruction.

For example, when the control instruction from the remote control device 22 received by the upper computer terminal 10 only contains the first instruction, a corresponding control operation will be performed on the movable carrying device 11, such as flying an unmanned aerial vehicle (as an example of the movable carrying device 11) to a predetermined height, or the like.

For another example, when the control instruction from the remote control device 22 received by the upper computer terminal 10 only includes the second instruction, assuming that the serial number of the cradle head corresponding to the second instruction is 2 (assuming that there are multiple cradle heads currently), the cradle head with the serial number of 2 is subjected to corresponding control operation, for example, the pitch angle of the cradle head is adjusted downward by 10 degrees.

For another example, when the control instruction received by the upper computer terminal 10 from the remote control device 22 includes both the first instruction and the second instruction, the corresponding control operation may be performed on the corresponding movable carrying device 11 and the cradle head 11-1.

According to an embodiment of the present invention, when the movable carrying device 11 is an unmanned aerial vehicle, the first instruction may include, for example, an instruction for controlling the unmanned aerial vehicle to perform any one or more of various flight control actions such as self-check, take-off, landing, return, hovering, pitching, rolling, and yawing.

For example, the first instructions may include a flight direction control instruction and/or a flight height control instruction.

The first command (e.g. flight direction control command and/or flight altitude control command) may control the movable carrier 11 (e.g. drone) within a preset range (e.g. a preset geographical location range and/or a preset altitude range, etc.), for example, and when the movable carrier 11 reaches the boundary of the preset range, the first command is no longer valid. Therefore, the unsafe problem caused by the over control of the movable carrying equipment 11 by the first instruction can be avoided.

According to an embodiment of the present invention, the second instruction may include, for example, a control instruction (such as a pan/tilt angle control instruction and/or a pan/tilt angle control instruction, etc.) for controlling the pan/tilt head 11-1 to perform horizontal and/or pitch movements.

Various specific instructions (such as a flight direction control instruction and a flight altitude control instruction, or a pan-tilt azimuth angle control instruction and a pan-tilt pitch angle control instruction and the like) contained in the first instruction or the second instruction can be further distinguished through different fields. For example, in a first field corresponding to the first command, a first one of the sub-fields corresponds to a direction of flight control command, and a first two of the sub-fields in the first field corresponds to a height of flight control command; for another example, in a second field corresponding to the second instruction, the second sub-field corresponds to represent the pan/tilt angle control instruction, and the second sub-field corresponds to represent the pan/tilt angle control instruction; and so on.

Thus, different types of control over different components in the host-side 10 are implemented by different control instructions (e.g., by different fields in the control instructions), and the control over different components in the host-side 10 may be performed simultaneously or non-simultaneously.

According to the embodiment of the present invention, if the control information sent by the user terminal 40 is the voice control information input by voice, the server terminal 30 directly forwards the voice control information to the lower computer terminal in real time, and after receiving the voice control information, the lower computer terminal generates the voice corresponding to the voice control information through the voice module and plays the voice. The voice module may be, for example, an earphone and/or a speaker. In this way, the ground staff can listen to the audio content in the voice module by wearing the voice module or by setting the voice module at a preset position (such as a fixed or movable position), so as to perform corresponding operation on the remote control device.

In addition, if the control information sent by the user terminal 40 is manually input non-voice control information, the server terminal 30 may convert the non-voice control information into corresponding voice control information, and then send the converted voice control information to the lower computer terminal in real time, and after receiving the voice control information, the lower computer terminal generates a voice corresponding to the voice control information through the voice module and plays the voice. In this way, the ground staff can listen to the audio content in the voice module by wearing the voice module or by setting the voice module at a preset position (such as a fixed or movable position), so as to perform corresponding operation on the remote control device.

The number of the upper computer terminals 10 may be one or more, and the number of the lower computer terminals 20 is the same as the number of the upper computer terminals 10. Each upper computer terminal 10 can communicate with the server terminal 30 through the corresponding lower computer terminal 20 of the upper computer terminal 10. It should be understood that fig. 1 only shows an example of one upper computer end and one lower computer end, and in other examples, the number of the upper computer end and the lower computer end may be multiple.

According to an embodiment of the present invention, the pan/tilt head 11-1 is, for example, a multi-degree-of-freedom pan/tilt head, and is adapted to control the image capturing device 11-2 connected thereto to perform multi-directional adjustment.

For example, the pan/tilt head 11-1 can control the image capturing device 11-2 connected thereto to perform any one or more of the following movements: horizontal rotational movement, vertical tilting movement, and rotational movement about the optical axis of the image pickup device 11-2, and the like. The pan/tilt head 11-1 may, for example, integrate functions of motor driving, servo control, etc., and may adjust the image capturing device connected to the pan/tilt head to perform multi-directional motion according to the instruction, for example, a three-degree-of-freedom pan/tilt head, and then may adjust the image capturing device to perform the above-mentioned horizontal rotational motion, vertical pitching motion, and rotational motion around the optical axis of the camera according to the instruction

According to the embodiment of the invention, the cloud deck 11-1 can be provided with a telescopic device, and the image acquisition equipment 11-2 connected with the cloud deck 11-1 is arranged on the corresponding telescopic device, so that the image acquisition equipment 11-2 can move along with the telescopic device.

According to an embodiment of the present invention, the pan/tilt head 11-1 may include a pan/tilt head control module (as a second control module) and a driving module; the pan/tilt control module is configured to receive a control instruction from the remote control device 22 in the lower computer terminal 20, send an instruction to the driving module, and send a control signal and a driving pulse signal to the corresponding motor through the driving module to control and change a motion attitude of each rotating shaft of the pan/tilt 11-1.

In addition, the driving module can further comprise a limit switch, so that the rotation range of the corresponding holder 11-1 is controlled through the limit switch.

According to an embodiment of the present invention, one or more holders 11-1 are provided on the movable carrying device 11, for example, and a single holder 11-1 is used for controlling two cameras in a corresponding single camera module to perform synchronous adjustment, for example.

For example, the movable carrying device 11 is only provided with one pan/tilt head 11-1 and one camera module, and the pan/tilt head 11-1 can control two cameras in the camera module to perform synchronous adjustment. The camera module lens can face to the right front of the movable carrying equipment for flying or traveling.

For another example, the movable carrying device 11 is provided with a plurality of holders 11-1 and a plurality of camera modules, the holders 11-1 correspond to the camera modules one to one, and each holder 11-1 controls two cameras in one camera module to perform synchronous adjustment, but the adjustment between the camera modules is not necessarily synchronous.

According to an embodiment of the present invention, the movable carrying device 11 is provided with, for example, one or more holders 11-1, and a single holder 11-1 is used, for example, for controlling cameras of at least two camera modules of the one or more camera modules to perform synchronous adjustment.

For example, the movable carrying device 11 is provided with a cradle head 11-1 and a plurality of camera modules, and the cradle head 11-1 can control all cameras in the plurality of camera modules to perform synchronous adjustment.

For another example, a cradle head 11-1 and a plurality of camera modules are disposed on the movable carrying device 11, and the cradle head 11-1 can control part of cameras in the plurality of camera modules to perform synchronous adjustment.

For another example, the movable carrying device 11 is provided with a plurality of holders 11-1 and a plurality of camera modules, and a single holder 11-1 can control cameras in at least two camera modules of the plurality of camera modules to perform synchronous adjustment. For example, assume that there are 2 pan/tilt/zoom stages and 5 camera modules, the first pan/tilt stage controls 6 cameras of 3 camera modules to perform synchronous adjustment, and the second pan/tilt stage controls 4 cameras of the other 2 camera modules to perform synchronous adjustment.

As an example of controlling multiple camera modules by a single pan/tilt/zoom lens, taking two sets as an example, if a lens of one camera module is facing to the front of the movable carrier, and a lens of the other camera module is facing to the back of the movable carrier, the pan/tilt lens controls the two camera modules to move synchronously, for example, if the two camera modules are connected by a mechanical shaft and kept relatively still, the pan/tilt lens controls one camera module to move while the other camera module moves coaxially, and the viewing angles are the front and the back of the relatively movable carrier.

For example, when the pan/tilt head receives a command of rotating to the left, the camera module in front of the pan/tilt head rotates to the left in real time, and the camera module in the back of the pan/tilt head rotates to the right; for another example, the two camera modules are not connected by a mechanical shaft, but the pan-tilt control module sends the same rotation instruction to the driving module, for example, when the pan-tilt control module rotates upwards, the driving module sends a control signal and a driving pulse signal to the two corresponding pitching motors, so that the two camera modules rotate upwards at the same time.

As an example of controlling multiple camera modules by a single pan/tilt unit, in order to obtain viewing angles in more directions, a single pan/tilt unit may also be used to control four camera modules, where one camera module lens faces the front of the movable carrier device, and the other camera module lens faces the rear of the movable carrier device, and the two camera modules are connected by a mechanical shaft to serve as a camera in group a; one group of camera module lenses face to the right left of the movable bearing equipment for flying or driving, one group of camera module lenses face to the right of the movable bearing equipment for flying or driving, the two groups of camera modules are connected by a mechanical shaft to serve as a group B camera, and the tripod head controls the group A camera and the group B camera to move synchronously, for example, the middle part of the mechanical shaft of the group A camera and the middle part of the mechanical shaft of the group B camera are connected by another mechanical shaft and keep relatively static, the group A camera and the group B camera are in a cross shape which is not in a plane, and the tripod head controls one camera module to move while the other three camera modules move coaxially.

For example, when the pan/tilt head receives a command of rotating to the left, the camera module right in front rotates to the left in real time, and the other three camera modules also rotate to the left; for another example, each set of camera modules is not connected by a mechanical shaft, but the pan-tilt control module sends the same rotation instruction to the driving module, for example, when the pan-tilt control module rotates upwards, the driving module sends a control signal and a driving pulse signal to the four corresponding sets of tilt motors, so that the four sets of camera modules rotate upwards at the same time.

Furthermore, according to the embodiment of the present invention, a plurality of holders may be disposed on the movable carrying device 11, and each camera module corresponds to two holders, wherein two cameras in a single camera module are asynchronously fine-tuned through two holders respectively. The fine adjustment may be, for example, an adjustment in the range of 0 to 15 degrees, or other predetermined range.

Furthermore, optionally, for a single camera module, in addition to two pan/tilt heads for controlling two cameras respectively, another pan/tilt head may be used to control the synchronous adjustment of the two cameras.

For example, assume that a certain camera module corresponds to 3 pan heads, namely, a pan head a, a pan head B, and a pan head C, the camera module includes a first camera and a second camera, wherein the pan head a is used for fine-tuning the first camera (assuming that the adjustment range of each degree of freedom direction is 0-10 degrees), the pan head B is used for fine-tuning the second camera (assuming that the adjustment range of each degree of freedom direction is 0-10 degrees), and the pan head C is used for synchronously adjusting the first camera and the second camera (assuming that the adjustment range of each degree of freedom direction is 0-360 degrees). Assuming that at the time t1, the two cameras are directed toward the front of the movable carrying device 11 (such as the unmanned aerial vehicle), and assuming that the distance between the shooting target and the movable carrying device 11 is 1000 meters at this time, the two cameras are substantially aligned (both facing the shooting target); at the time t2, assuming that the movable carrying device 11 is 500 meters away from the shooting target and still far away, and assuming that the orientation of the movable carrying device 11 is rotated 30 degrees to the right compared with the time t1, the first and second cameras can be integrally and synchronously adjusted by the pan-tilt head C, that is, the first and second cameras are synchronously rotated 30 degrees or other degrees to the left (the target may have slight movement); at time t3, assume that the movable carrying device 11 is 2 meters away from the shooting target, the distance is very close, at this moment, the first camera, the second camera, the three vertex positions of the shooting target three positions are similar to the triangle, therefore, it can be known that the directions of the first camera and the second camera towards the shooting target are different, and therefore asynchronous adjustment (fine adjustment) needs to be performed on the directions of the first camera and the second camera respectively, rather than synchronous adjustment, so that the direction of the first camera can be finely adjusted (for example, finely adjusted by 5 degrees to the right) through the pan-tilt A, and the direction of the second camera can be finely adjusted (for example, finely adjusted by 5 degrees to the left) through the pan-tilt B, so that the collection directions of the first camera and the second camera can. In addition, for a single camera module, the respective position adjustment of two cameras in the camera module can be realized through two independent telescopic devices respectively.

The server 30 is configured to perform instant and corresponding processing and storage on the picture/video data received from the lower computer 20, encode and compress video data (multiple sets of video frame sequences) into a live video stream in, for example, h.264 format, format-encapsulate and store the compressed live video stream (for example, store the compressed live video stream in F L V format), and perform stream pushing and distribution through a video transmission protocol (for example, RTSP), where a corresponding relationship between the live video stream and original video data is established in the encoding process, and a video file in F L V format is stored in the server as an on-demand video source, and a unique identifier is determined.

The server 30 may select, for example, part or all of a plurality of groups of pictures corresponding to a plurality of groups of video frame sequences acquired by each camera module to be stored, and establish a correspondence between the plurality of groups of pictures and the live video stream according to a time correspondence principle.

Or, part or all of the picture data corresponding to the r-th video frame sequence may also be set at the upper computer terminal 10, the upper computer terminal 10 transmits part or all of the picture data and the video data with the established corresponding relationship to the lower computer terminal 20 through the first transmission module 11-3 and forwards the picture data and the video data to the server terminal 30 in real time, and the server terminal 30 performs the above processing and then stores the received part or all of the picture data and all of the video data.

In addition, when receiving a request instruction from the user terminal 40, the server terminal 30 sends the picture/video data matched with the request instruction to the user terminal 40 that issued the request instruction. The request command from the user side 40 may be a request command from one user side 40, or may be multiple request commands from multiple user sides 40.

Wherein the request instruction comprises a request video transmission instruction and/or a photographing instruction.

According to the embodiment of the invention, when the request instruction comprises a request video transmission instruction, the server 30 can send the video data matched with the request video transmission instruction by judging whether the instruction (namely the request instruction) of the user is a pull stream request or a request video request, if the instruction is the pull stream request, sending a pull stream address (UR L) to the corresponding user side, and if the instruction is the request video request, sending a virtual path address (UR L) of the stored video which is consistent with the identification carried in the request instruction to the corresponding user side;

according to the embodiment of the present invention, when the request instruction includes a photographing instruction, the server 30 may obtain the picture data matched with the photographing instruction, for example, as follows: according to the corresponding relationship between the multiple groups of pictures and the video stream and the sending time carried in the photographing instruction, picture data corresponding to the sending time is selected from the picture data stored (for example, in a storage module of the server 30), and is stored in the user account sending the photographing instruction so as to respond to the subsequent downloading of the user.

When the image capturing apparatus 11-2 includes a plurality of camera modules, the server 30 may determine, for example, a camera module to which the picture/video data matching the request instruction belongs among the plurality of camera modules, as the target camera module, so as to select the picture/video data captured by the target camera module as the picture/video data matching the request instruction of the client 40. That is, under the condition that the image capturing device 11-2 includes a plurality of camera modules, before the server 30 obtains the picture/video data matching the request command of the user terminal 40, it determines the camera module to which the picture/video data matching the request command belongs, and then selects to obtain the picture/video data captured by the corresponding camera module (for example, from the storage module of the server 30).

For example, the camera module to which the picture/video data matched with the request instruction belongs may be determined as follows: and acquiring transmission parameters in the request instruction, wherein the transmission parameters comprise video frame parameters to be transmitted (such as the sequence number of which frame or frames, or the sequence number of the camera module), and determining the camera module corresponding to the transmission parameters according to the transmission parameters.

Therefore, the user side can not only watch the live broadcast/on-demand video, take pictures to obtain high-definition pictures in the process of watching the live broadcast/on-demand video, but also send out control instructions to control the cradle head when watching the live broadcast. The user of real-time control cloud platform can watch the scenery of different visual angles and take high definition picture according to oneself wish.

In addition, when one user terminal 40 controls the pan/tilt head to change the viewing angle, other user terminals 40 can share the viewing angle, for example. Thus, when one user controls the holder to change the viewing angle, other users can share the viewing angle, so as to view videos or take pictures.

According to the embodiment of the present invention, the server 30 may allow receiving the control information sent from the authorized user and refuse receiving the control information sent from the unauthorized user. Wherein, the authorized user is the user with the authority, and the unauthorized user is the user without the authority.

For example, after any user terminal 40 sends information to the server terminal 30, the server terminal 30 does not receive other control information of the same type as the control information sent by other user terminals 40 until the upper computer terminal 10 (such as the movable carrying device 11 and/or the cradle head 11-1) does not implement the control instruction from the remote control device 22 corresponding to the control information.

For example, when the server 30 receives control information sent from a certain user 40, assuming that the control information is voice control information input by voice, and performing voice character recognition and semantic recognition on the voice control information, so as to know the name of the corresponding device (such as an unmanned aerial vehicle, and/or the serial number of the pan/tilt head 11-1), when the name of the movable carrier device 11 (such as an unmanned aerial vehicle, and the like) is included, the "other control information having the same type as the control information" refers to other information whose content also corresponds to the name of the movable carrier device 11 (such as an unmanned aerial vehicle, and the like); and when the control information corresponds to the cloud platform, the other control information with the same type as the control information refers to other information with the same content corresponding to the cloud platform. Similarly, when the control information sent by the ue 40 is non-voice control information, the corresponding processing is similar and will not be described again.

Thus, when a user sends a control message to the server 30 through the user terminal 40, the control message is assumed to be a voice control message, and the content is a voice message "flying to 200 meters". The server 30 forwards the voice to the voice module 23 in the lower computer 20, and the voice module 23 plays the voice "fly to 200 meters", so that the ground staff will operate the remote control device 22 to make the unmanned aerial vehicle (as an example of the movable carrying device 11) fly to 200 meters in height after hearing the voice. Meanwhile, the server 30 recognizes that the content "flying to 200 meters" corresponds to the "drone" (as an example of the movable carrier 11) through a recognition technology (such as existing voice character recognition, semantic recognition, or the like), and thus, when there is a new control information transmitted to the server by another user side before the drone has not executed the voice control information "flying to 200 meters", if the new control information also corresponds to the drone, the new control information is rejected.

In addition, according to the embodiment of the present invention, the server 30 may further determine whether the command or the information includes the paid information when receiving a request command (e.g. download) or control information of the user 40, and establish a data connection with the user 40 if the command includes the paid information. The paid information may be obtained, for example, by means of an online payment.

Fig. 2 shows one possible configuration of the server side 30. As shown in fig. 2, the server 30 may include, for example, a processing module 31, a storage module 32, and a third transmission module 33.

The processing module 31 may, for example, perform an instant and corresponding process on the picture/video data received from the lower computer terminal 20, the storage module 32 may, for example, store the picture/video data processed by the processing module, and the third transmission module 33 may, for example, communicate with the lower computer terminal 20 and the user terminal 40.

In addition, according to the embodiment of the present invention, the server 30 may further optionally include a user management module shown in fig. 2. Wherein the dashed box of the user management module in fig. 2 indicates that this module is optional, but not necessary.

The user management module may receive an identity authentication request from each user terminal 40, perform identity authentication on corresponding user equipment (i.e., equipment corresponding to the user terminal) based on the identity authentication request, and send authentication success information to the corresponding user terminal 40 after the authentication is successful so as to establish data connection with the user equipment.

Fig. 3 shows one possible structure of the user terminal 40. As shown in fig. 3, the user terminal 40 may include, for example, a sending module 41, a receiving module 42, a displaying module 43, and a fourth transmitting module 44. The user device corresponding to the user terminal 40 may be, for example, a smart phone or a tablet computer.

The sending module 41 can send an access request to the server 30, and obtain video data, picture data and/or a request for controlling the pan/tilt head.

The receiving module 42 can receive video data, picture data and/or related information data in response to the control pan/tilt/zoom request sent by the server 30.

The display module 43 may display the video data, the picture data and/or the operation of the real-time control pan/tilt head sent by the server 30.

Furthermore, the fourth transmission module 44 may be used for data communication with the server 30.

In summary, the present application provides the following solutions:

the live broadcast system is characterized in that the live broadcast system which is based on the voice control and can obtain the high-definition photos immediately comprises an upper computer end, a lower computer end, a server end and a user end;

the upper computer end comprises movable bearing equipment, and the movable bearing equipment is provided with a holder, image acquisition equipment connected with the holder and a first transmission module; the holder is used for adjusting the position and/or the direction of the image acquisition equipment connected with the holder; the image acquisition equipment comprises one or more camera modules, wherein each camera module comprises two cameras, and the corresponding acquisition visual angles of the two cameras in the same camera module at the same moment are approximately the same; the first transmission module comprises one or more wireless image transmission modules for communicating with the lower computer end; the two cameras in each camera module are respectively used for outputting picture data and video data, the picture data comprises one or more groups of pictures, and the video data comprises one or more groups of video frame sequences;

the lower computer end comprises remote control equipment, a second transmission module and a voice module;

the second transmission module comprises one or more wireless image transmission modules for communicating with the upper computer end and a communication module for communicating with the server end; the lower computer end is used for receiving and forwarding an acquisition instruction from the server end to the upper computer end so as to enable the image acquisition equipment at the upper computer end to start or stop acquiring image/video data in the visual field of the image acquisition equipment, and receiving the image/video data from the upper computer end to forward the image/video data to the server end in real time;

the remote control equipment is suitable for receiving control instructions of ground workers, and the control instructions comprise a first instruction for controlling the movable bearing equipment and a second instruction for controlling the holder; the remote control equipment is suitable for sending a first instruction in the control instruction to a first control module in the movable bearing equipment so as to control the movable bearing equipment to perform corresponding operation action, and sending a second instruction in the control instruction to a second control module of the holder so as to control the holder to perform corresponding operation action;

the voice module is suitable for receiving voice control information from the server side, generating corresponding voice according to the voice control information and playing the voice to the ground staff so as to instruct the ground staff to input corresponding control instructions to the remote control equipment;

the server is used for carrying out instant and corresponding processing and storage on the picture/video data received from the lower computer end, acquiring the picture/video data matched with the request instruction and sending the picture/video data to the user end sending the request instruction when receiving the request instruction from the user end, receiving the control information from the user end, generating corresponding voice control information and sending the voice control information to the lower computer end; wherein the request instruction comprises a request video transmission instruction and/or a photographing instruction.

Scheme 2. according to scheme 1 the live broadcast system based on voice control and capable of instantly obtaining high definition photos is characterized in that the movable bearing equipment is any one of the following:

unmanned aerial vehicle, mobile robotic arm, cable car, sightseeing vehicle, train, plane or ship.

Scheme 3, according to scheme 1 or 2, the live broadcast system capable of acquiring high-definition pictures in real time based on voice control is characterized in that when the movable carrying device is an unmanned aerial vehicle, the first instruction comprises an instruction for controlling any one or more operation actions of self-checking, take-off, landing, returning, hovering, pitching, rolling and yawing of the unmanned aerial vehicle.

Scheme 4. the live broadcast system capable of instantly obtaining high-definition pictures based on voice control according to scheme 3 is characterized in that the first instruction comprises a flight direction control instruction and/or a flight altitude control instruction.

Scheme 5. the live broadcast system capable of instantly obtaining high-definition photos based on voice control according to any one of the schemes 1 to 4, wherein the second instruction comprises a control instruction for controlling the horizontal and/or pitching movement of the holder.

Scheme 6. the live broadcast system capable of instantly obtaining high-definition pictures based on voice control according to the schemes 1 to 5 is characterized in that the voice module comprises earphones and/or a loudspeaker.

Scheme 7. the live broadcast system capable of instantly obtaining high-definition photos based on voice control according to any one of the schemes 1 to 6, wherein if the control information sent by the user side is voice control information input through voice, the server side directly forwards the voice control information to the lower computer side in real time.

Scheme 8, the live broadcast system capable of acquiring high-definition photos in real time based on voice control according to any one of the schemes 1 to 7, wherein if the control information sent by the user side is manually input non-voice control information, the server side converts the non-voice control information into corresponding voice control information so as to send the voice control information to the lower computer side in real time.

Scheme 9. the live broadcast system capable of instantly obtaining high-definition pictures based on voice control according to any one of the schemes 1 to 8 is characterized in that the holder is a multi-degree-of-freedom holder and is suitable for controlling image acquisition equipment connected with the holder to perform multi-directional adjustment.

Scheme 10, according to the live broadcast system that can obtain high definition photo immediately of scheme 9 that voice control based on, its characterized in that, the cloud platform can control the image acquisition equipment who connects rather than carry out any one or more following motion:

horizontal rotational movement, vertical tilting movement and rotational movement about the optical axis of the image acquisition device.

Scheme 11, according to the live broadcast system that can obtain high definition photo immediately of scheme 9 or 10, characterized in that, be equipped with the telescoping device on the cloud platform, the image acquisition equipment that is connected with the cloud platform sets up on the telescoping device that corresponds to make this image acquisition equipment can move along with this telescoping device's flexible.

Scheme 12. the live broadcast system capable of instantly obtaining high-definition pictures based on voice control according to any one of the schemes 9 to 11 is characterized in that the cradle head comprises a cradle head control module and a driving module, wherein the cradle head control module is used as the second control module;

the holder control module is used for receiving a control instruction from the remote control equipment, sending an instruction to the driving module, and sending a control signal and a driving pulse signal to the corresponding motor through the driving module so as to control and change the motion attitude of each rotating shaft of the holder.

Scheme 13. the live broadcast system based on voice control and capable of instantly obtaining high-definition pictures according to scheme 12 is characterized in that the driving module further comprises a limit switch, so that the limit switch controls the rotation range of the corresponding holder.

Scheme 14. the live broadcast system capable of instantly obtaining high-definition pictures based on voice control according to any one of the schemes 1 to 13, wherein the camera of each camera module is a wide-angle zoom camera or a non-zoom camera.

Scheme 15, according to any one of schemes 1 to 14, the live broadcast system capable of acquiring high-definition photos immediately based on voice control is characterized in that picture data and video data acquired by two cameras included in each camera module have a time correspondence relationship.

Scheme 16. the live broadcast system capable of instantly obtaining high-definition photos based on voice control according to any one of the schemes 1 to 15, wherein the movable carrying device, the cradle head on the movable carrying device and the image acquisition device are all communicated with the lower computer end through an image transmission module.

Scheme 17. the live broadcast system capable of instantly obtaining high-definition photos based on voice control according to any one of schemes 1 to 15, wherein the movable carrying device, the cradle head on the movable carrying device, and the image acquisition device communicate with the lower computer terminal through respective corresponding image transmission modules.

Scheme 18. according to the live broadcast system capable of acquiring high-definition pictures instantly based on voice control of scheme 17, two cameras in each camera module respectively transmit picture data and video data through one picture transmission module.

Scheme 19. according to any one of schemes 1 to 18, the live broadcast system capable of instantly obtaining high-definition pictures based on voice control is characterized in that one or more holders are arranged on the movable carrying device, and a single holder is used for controlling two cameras in a corresponding single camera module to synchronously adjust.

Scheme 20. the live broadcast system capable of instantly obtaining high-definition photos based on voice control according to any one of schemes 1 to 18 is characterized in that one or more holders are arranged on the movable carrying device, and a single holder is used for controlling cameras of at least two camera modules in the one or more camera modules to synchronously adjust.

Scheme 21, according to any one of schemes 1-18, the live broadcast system capable of obtaining high-definition pictures in real time based on voice control is characterized in that a plurality of holders are arranged on the movable bearing device, each camera module corresponds to two holders, and two cameras in a single camera module are subjected to asynchronous fine adjustment through the two holders respectively.

The live broadcast system capable of instantly obtaining high-definition photos based on voice control according to any one of the schemes 1 to 21, wherein the number of the upper computer terminals is one or more, and the number of the lower computer terminals is consistent with the number of the upper computer terminals; and each upper computer end communicates with the server end through the lower computer end corresponding to the upper computer end.

Scheme 23, the voice control-based live broadcast system capable of instantly obtaining high-definition pictures according to any one of the schemes 1 to 22, wherein when the request instruction includes a video transmission instruction, the server is adapted to send video data matching the request video transmission instruction by:

judging whether the request instruction is a pull stream request or an on-demand video request: if the request is a pull request, the pull address is sent to the corresponding user side; and if the request is a video-on-demand request, sending the stored virtual path address of the video corresponding to the identifier carried in the request instruction to the corresponding user side.

Scheme 24, the voice-control-based live broadcast system capable of instantly obtaining high-definition photos according to any one of the schemes 1 to 23, wherein when the request instruction includes a photographing instruction, the server is adapted to obtain picture data matched with the photographing instruction in the following manner:

and selecting picture data corresponding to the sending time from the stored picture data according to the corresponding relation between the plurality of groups of pictures and the video stream and the sending time carried in the photographing instruction, and storing the picture data in the user account sending the photographing instruction so as to respond to the subsequent downloading of the user.

Scheme 25 according to scheme 23 or 24, the voice-control-based live broadcast system capable of instantly obtaining high-definition photos is characterized in that, when the image acquisition device comprises a plurality of camera modules, the server is adapted to:

and determining the camera module to which the picture/video data matched with the request instruction belongs from the plurality of camera modules as a target camera module so as to select the picture/video data collected by the target camera module as the picture/video data matched with the request instruction of the user side.

Scheme 26, the live system capable of instantly obtaining high-definition pictures based on voice control according to any one of the schemes 1 to 25, characterized in that the server is adapted to allow receiving control information sent by an authorized user and refuse receiving control information sent by an unauthorized user.

Scheme 27, the live broadcast system capable of instantly obtaining high-definition photos based on voice control according to any one of the schemes 1 to 26, wherein when one user side controls the pan/tilt head to change the viewing angle, other user sides can share the viewing angle.

Scheme 28. the live broadcast system capable of acquiring high-definition photos immediately based on voice control according to scheme 27 is characterized in that after any user side sends control information to the server side, the server side does not receive other control information of the same type as the control information sent by other user sides before the upper computer side pan-tilt does not implement the control instruction from the remote control device corresponding to the control information.

Scheme 29, the voice-control-based live broadcast system capable of instantly obtaining high-definition photos according to any one of the schemes 1 to 28, wherein the server determines whether the command or the information contains paid information when receiving a request command or control information from the user side, and establishes a data connection with the user side when the command contains paid information.

Scheme 30. the live broadcast system capable of instantly obtaining high-definition pictures based on voice control according to any one of the schemes 1 to 29, wherein the server side comprises a processing module, a storage module and a third transmission module; the processing module is suitable for carrying out real-time and corresponding processing on the picture/video data received from the lower computer end, the storage module is suitable for storing the picture/video data processed by the processing module, and the third transmission module is suitable for communicating with the lower computer and the user end.

Scheme 31, the voice control-based live broadcast system capable of instantly obtaining high definition photos according to scheme 30, wherein the server further includes a user management module, configured to receive an identity authentication request from each user terminal, perform identity authentication on corresponding user equipment based on the identity authentication request, and send authentication success information to the corresponding user terminal after the authentication is successful so as to establish data connection with the user equipment.

Scheme 32. the live broadcast system capable of instantly obtaining high definition photos based on voice control according to any one of the schemes 1 to 31, wherein each user side includes:

the sending module is used for sending an access request to the server side to acquire video data, picture data and/or a request for controlling the holder;

the receiving module is used for receiving video data, picture data and/or related information data responding to a control cloud deck request sent by the server;

the display module is used for displaying the video data and the picture data sent by the server side and/or displaying the operation of a real-time control holder; and

and the fourth transmission module is used for carrying out data communication with the server side.

Scheme 33. the live broadcasting system capable of instantly obtaining high-definition photos based on voice control according to any one of the schemes 1 to 32,

each camera module is provided with a corresponding sensor group, wherein each sensor group comprises a position sensor suitable for acquiring the position of the corresponding camera module in real time and a posture sensor suitable for acquiring the posture of the corresponding camera module in real time;

the server end receives an interest target instruction from a user, selects one of a plurality of preset interest target positions as a current view target position according to the interest point instruction, and adjusts a cloud deck corresponding to the target camera module according to a connecting line between the current view target position and a real-time position of the target camera module so that the main direction of an optical axis of each camera in the target camera module is consistent with the direction of the connecting line; wherein the plurality of preset target locations of interest comprise a plurality of known location points preset within a predetermined area.

Scheme 34. the live broadcasting system capable of instantly obtaining high definition pictures based on voice control according to scheme 33 is characterized in that,

when the image acquisition equipment comprises a camera module, the camera module is used as the target camera module;

when the image acquisition equipment comprises a plurality of camera modules, the target camera module is the camera module which is closest to the current viewing target position in the plurality of camera modules.

Scheme 35. the live broadcasting system capable of instantly obtaining high-definition photos based on voice control according to any one of the schemes 1 to 34,

each camera module is provided with a corresponding sensor group, wherein each sensor group comprises a plurality of position sensors suitable for acquiring the positions of the cameras in the corresponding camera module in real time and a plurality of attitude sensors suitable for acquiring the attitudes of the cameras in the corresponding camera module in real time;

the server end receives an interest target instruction from a user, selects one of a plurality of preset interest target positions as a current view target position according to the interest point instruction, and adjusts a cloud deck corresponding to the target camera module according to a connecting line between the current view target position and the real-time position of each camera of the target camera module so that the main direction of the optical axis of each camera in the target camera module is consistent with the connecting line direction; wherein the plurality of preset target locations of interest comprise a plurality of known location points preset within a predetermined area.

Scheme 36. the live broadcasting system capable of instantly obtaining high definition photos based on voice control according to scheme 35, wherein,

when the image acquisition equipment comprises a plurality of camera modules, the target camera module is a camera module to which a camera closest to the current viewing target position belongs in each camera of the plurality of camera modules.

Scheme 37. the live broadcasting system capable of instantly obtaining high-definition pictures based on voice control according to the scheme 35 or 36,

and two cameras in a single camera module respectively carry out attitude synchronous adjustment or attitude asynchronous adjustment according to the connecting line direction through two holders in real time.

While the invention has been described with respect to a limited number of embodiments, those skilled in the art, having benefit of this description, will appreciate that other embodiments can be devised which do not depart from the scope of the invention as described herein. Furthermore, it should be noted that the language used in the specification has been principally selected for readability and instructional purposes, and may not have been selected to delineate or circumscribe the inventive subject matter. Accordingly, many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the appended claims. The present invention has been disclosed in an illustrative rather than a restrictive sense, and the scope of the present invention is defined by the appended claims.

Claims

1. The live broadcast system capable of acquiring the high-definition photos immediately based on the voice control is characterized by comprising an upper computer end, a lower computer end, a server end and a user end;

the upper computer end comprises movable bearing equipment, and the movable bearing equipment is provided with a holder, image acquisition equipment connected with the holder and a first transmission module; the holder is used for adjusting the position and/or the direction of the image acquisition equipment connected with the holder; the image acquisition equipment comprises one or more camera modules, wherein each camera module comprises two cameras, and the corresponding acquisition visual angles of the two cameras in the same camera module at the same moment are approximately the same; the first transmission module comprises one or more wireless image transmission modules for communicating with the lower computer end; the two cameras in each camera module are respectively used for outputting picture data and video data, the picture data comprises one or more groups of pictures, and the video data comprises one or more groups of video frame sequences; the camera module comprises a camera module, a camera module and a camera module, wherein the camera module comprises a plurality of cameras, the cameras are arranged in the camera module, the camera module comprises a camera module, image data and video data, the image data and the video data acquired by the two cameras in the camera module have time corresponding relations;

the server is used for carrying out instant and corresponding processing and storage on the picture/video data received from the lower computer end, acquiring the picture/video data matched with the request instruction and sending the picture/video data to the user end sending the request instruction when receiving the request instruction from the user end, receiving the control information from the user end, generating corresponding voice control information and sending the voice control information to the lower computer end; the request instruction comprises a request video transmission instruction and/or a photographing instruction;

the server end receives an interest target instruction from a user, selects one of a plurality of preset interest target positions as a current view target position according to the interest target instruction, and adjusts a cloud deck corresponding to the target camera module according to a connecting line between the current view target position and a real-time position of the target camera module so that the main direction of an optical axis of each camera in the target camera module is consistent with the direction of the connecting line; wherein the plurality of preset target locations of interest comprise a plurality of known location points preset within a predetermined area.

2. The live system capable of instantly obtaining high definition pictures based on the voice control as claimed in claim 1, wherein the movable carrying device is any one of the following devices:

3. The live broadcast system capable of instantly obtaining high definition pictures based on the voice control as claimed in claim 1 or 2, wherein when the movable carrying device is an unmanned aerial vehicle, the first instruction comprises an instruction for controlling any one or more of self-check, take-off, landing, return, hovering, pitching, rolling and yawing actions of the unmanned aerial vehicle.

4. The live system capable of instantly obtaining high-definition pictures based on the voice control as claimed in claim 3, wherein the first command comprises a flight direction control command and/or a flight altitude control command.

5. The live system capable of instantly obtaining high-definition photos based on voice control as claimed in claim 1, wherein the second instruction comprises a control instruction for controlling the pan/tilt head to move in horizontal and/or pitch directions.

6. The live system capable of instantly obtaining high-definition photos based on voice control as claimed in claim 5, wherein the voice module comprises a headset and/or a speaker.

7. The live broadcast system capable of instantly obtaining high definition pictures based on voice control as claimed in claim 1, wherein if the control information sent by the user terminal is voice control information inputted by voice, the server terminal directly forwards the voice control information to the lower computer terminal in real time.

8. The live broadcast system capable of instantly obtaining high definition pictures based on voice control as claimed in claim 1, wherein if the control information sent by the user terminal is manually input non-voice control information, the server terminal converts the non-voice control information into corresponding voice control information so as to send the voice control information to the lower computer terminal in real time.

9. The live broadcast system capable of instantly obtaining high definition pictures based on voice control as claimed in claim 1, wherein the pan-tilt is a multi-degree of freedom pan-tilt suitable for controlling an image acquisition device connected thereto to perform multi-directional adjustment.

10. The live system capable of instantly obtaining high-definition pictures based on voice control as claimed in claim 9, wherein the pan-tilt can control the image capturing device connected thereto to perform any one or more of the following movements:

11. The live broadcast system capable of acquiring high-definition photos instantly based on voice control as claimed in claim 9 or 10, wherein a telescopic device is arranged on the pan/tilt, and the image acquisition equipment connected with the pan/tilt is arranged on the corresponding telescopic device, so that the image acquisition equipment can move along with the telescopic device.

12. The live broadcast system capable of instantly obtaining high-definition photos based on the voice control as claimed in claim 9, wherein the pan-tilt comprises a pan-tilt control module and a driving module, wherein the pan-tilt control module is used as the second control module;

13. The live system capable of instantly obtaining high-definition pictures based on voice control as claimed in claim 12, wherein the driving module further comprises a limit switch, so as to control a rotation range of the corresponding cradle head through the limit switch.

14. The live system capable of acquiring high-definition photos instantly based on the voice control as claimed in claim 1, wherein the camera of each camera module is a wide-angle zoom camera or a non-zoom camera.

15. The live broadcasting system capable of instantly obtaining high definition photos based on voice control as claimed in claim 1, wherein the movable carrying device, the cradle head on the movable carrying device and the image capturing device are all communicated with the lower computer end through a picture transmission module.

16. The live broadcasting system capable of instantly obtaining high definition photos based on voice control as claimed in claim 1, wherein the movable carrying device, the cradle head on the movable carrying device and the image capturing device are respectively communicated with the lower computer end through respective corresponding image transmission modules.

17. The live broadcasting system for instantly obtaining high definition pictures based on voice control as claimed in claim 16, wherein two cameras in each camera module respectively transmit picture data and video data through a picture transmission module.

18. The live broadcast system capable of acquiring high-definition photos instantly based on the voice control as claimed in claim 1, wherein one or more holders are provided on the movable carrying device, and a single holder is used for controlling two cameras in a corresponding single camera module to perform synchronous adjustment.

19. The live broadcast system capable of acquiring high-definition photos instantly based on the voice control as claimed in claim 1, wherein one or more holders are provided on the movable carrying device, and a single holder is used for controlling cameras of at least two camera modules of the one or more camera modules to perform synchronous adjustment.

20. The live broadcasting system capable of acquiring high-definition photos instantly based on voice control as claimed in claim 1, wherein a plurality of holders are provided on the movable carrying device, and each camera module corresponds to two holders, wherein two cameras in a single camera module are asynchronously fine-tuned through two holders respectively.

21. The live broadcast system capable of instantly obtaining high definition photos based on voice control as claimed in claim 1, wherein the number of the upper computer terminals is one or more, and the number of the lower computer terminals is consistent with the number of the upper computer terminals; and each upper computer end communicates with the server end through the lower computer end corresponding to the upper computer end.

22. The live system capable of instantly obtaining high definition photos based on the voice control as claimed in claim 1, wherein when the request command includes a video transmission command, the server is adapted to send video data matching the request video transmission command by:

23. The live broadcast system capable of instantly obtaining high definition photos based on voice control as claimed in claim 1, wherein when the request command includes a photo command, the server is adapted to obtain the picture data matching with the photo command by:

24. The live system capable of instantly obtaining high definition pictures based on the voice control as claimed in claim 22 or 23, wherein in case that the image capturing device comprises a plurality of camera modules, the server is adapted to:

25. The live system capable of instantly obtaining high definition pictures based on voice control as claimed in claim 1, wherein the server is adapted to allow receiving the control information sent by the authorized user and refuse receiving the control information sent by the unauthorized user.

26. The live system capable of instantly obtaining high definition photos based on voice control as claimed in claim 1, wherein when one user controls the pan/tilt head to change the viewing angle, other users can share the viewing angle.

27. The live broadcast system capable of instantly obtaining high definition pictures based on voice control as claimed in claim 26, wherein after any user end sends control information to the server end, the server end does not receive other control information of the same type as the control information sent by other user ends until the upper computer end pan-tilt does not implement the control command from the remote control device corresponding to the control information.

28. The live system based on voice control and capable of obtaining high definition photos instantly as claimed in claim 1, wherein the server side determines whether the command or the information contains paid information or not when receiving a request command or control information from the user side, and establishes a data connection with the user side if the command contains paid information.

29. The live system capable of instantly obtaining high-definition pictures based on the voice control as claimed in claim 1, wherein the server comprises a processing module, a storage module and a third transmission module; the processing module is suitable for carrying out real-time and corresponding processing on the picture/video data received from the lower computer end, the storage module is suitable for storing the picture/video data processed by the processing module, and the third transmission module is suitable for communicating with the lower computer and the user end.

30. The live broadcasting system capable of instantly obtaining high definition photos based on voice control as claimed in claim 29, wherein the server further comprises a user management module for receiving an identity authentication request from each user terminal, performing identity authentication on the corresponding user equipment based on the identity authentication request, and sending an authentication success message to the corresponding user terminal after the authentication is successful so as to establish data connection with the user equipment.

31. The live system capable of instantly obtaining high-definition photos based on voice control as claimed in claim 1, wherein each user end comprises:

32. The live system capable of instantly obtaining high-definition photos based on voice control as claimed in claim 1,

33. The live system capable of instantly obtaining high-definition photos based on voice control as claimed in claim 1,

the server end receives an interest target instruction from a user, selects one of a plurality of preset interest target positions as a current view target position according to the interest target instruction, and adjusts a cloud deck corresponding to the target camera module according to a connecting line between the current view target position and the real-time position of each camera of the target camera module so that the main direction of the optical axis of each camera in the target camera module is consistent with the connecting line direction; wherein the plurality of preset target locations of interest comprise a plurality of known location points preset within a predetermined area.

34. The live system capable of instantly obtaining high definition photos based on voice control as claimed in claim 33,

35. The live system capable of instantly obtaining high definition photos based on voice control as claimed in claim 33 or 34,