WO2019000420A1

WO2019000420A1 - Method and apparatus for video and audio synchronization, and unmanned aerial vehicle

Info

Publication number: WO2019000420A1
Application number: PCT/CN2017/091233
Authority: WO
Inventors: 张志鹏; 尹小俊; 王乃博; 马宁
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2017-06-30
Filing date: 2017-06-30
Publication date: 2019-01-03
Also published as: CN108496372A

Abstract

Provided in an embodiment of the present invention are a method and apparatus for video and audio synchronization, and an unmanned aerial vehicle. The method comprises: acquiring video information photographed by a video photographing apparatus installed at an unmanned aerial vehicle, and acquiring audio information recorded by a recording apparatus, wherein the video photographing apparatus and the recording apparatus are synchronized in time; and performing alignment of the video information and the audio information according to timestamp information included in the video information and timestamp information included in the audio information. The invention realizes automatic alignment and synchronization of video information and audio information, thereby increasing synchronization efficiency.

Description

Video and audio synchronization method, device and drone

Technical field

The present application relates to synchronization technology, and in particular, to a method and device for synchronizing video and audio, and a drone.

Background technique

The existing aerial drones, due to the large noise of the propeller and the distance from the target, are basically silent video, and the sound information is the background music or live recording added by post processing. Especially when adding live recording, in order to achieve audio and video synchronization, manual and manual synchronization of audio and video is required, the operation is complicated, and the efficiency is low.

Summary of the invention

Embodiments of the present invention provide a method and a device for synchronizing video and audio, and a drone to realize automatic synchronization of audio and video, and improve synchronization efficiency of audio and video.

A first aspect of the embodiments of the present invention provides a method for synchronizing video and audio, including:

Obtaining video information captured by a video capture device carried by the drone, and acquiring audio information recorded by the recording device, wherein time synchronization of the video capture device and the recording device is performed;

And aligning the video information and the audio information according to timestamp information included in the video information and timestamp information included in the audio information.

A second aspect of the embodiments of the present invention provides a method for synchronizing video and audio, including:

Obtaining video information captured by a video capture device carried by the drone;

Generating timestamp information in the video information;

Sending the video information with the timestamp information to the synchronization device, so that the synchronization device according to the timestamp information in the video information, and the timestamp information in the audio information recorded by the recording device acquired by the synchronization device, Aligning the video information with the audio information, wherein the video capture device and the recording device are time synchronized.

A third aspect of the embodiments of the present invention provides a method for synchronizing video and audio, including:

Obtaining audio information recorded by the recording device on the remote control of the drone;

Generating time stamp information in the audio information;

Transmitting the audio information with the timestamp information to the synchronization device, so that the synchronization device according to the timestamp information in the audio information, and the timestamp in the video information captured by the video capture device acquired by the synchronization device Information, aligning the audio information with the video information, wherein the video capture device and the recording device are time synchronized.

A fourth aspect of the embodiments of the present invention provides a synchronization device, including:

a communication interface, one or more processors; the one or more processors operating separately or in cooperation, the communication interface being coupled to the processor;

The communication interface is configured to: acquire video information captured by a video capture device mounted on the drone, and acquire audio information recorded by the recording device, wherein time synchronization of the video capture device and the recording device;

The processor is configured to: align the video information and the audio information according to timestamp information included in the video information and timestamp information included in the audio information.

A fifth aspect of the embodiments of the present invention provides a control device, including:

a communication interface, one or more processors; the one or more processors operating separately or in cooperation, the communication interface being in communication with the processor;

The communication interface is configured to: acquire video information captured by a video capture device carried by the drone;

The processor is configured to: generate timestamp information in the video information;

The communication interface is configured to: send the video information that generates the timestamp information to the synchronization device, so that the synchronization device: according to the timestamp information in the video information, and the audio recorded by the recording device acquired by the synchronization device Time stamp information in the information, the video information and the audio information are aligned, wherein the video capture device and the recording device are time synchronized.

A sixth aspect of the embodiments of the present invention provides a control device, including:

The communication interface is configured to: acquire audio information recorded by a recording device on the remote control of the drone;

The processor is configured to: generate timestamp information in the audio information;

The communication interface is configured to: send audio information that generates timestamp information to a synchronization device And the synchronization device, according to the timestamp information in the audio information, and the timestamp information in the video information captured by the video capturing device acquired by the synchronization device, to the audio information and the video information. Alignment is performed in which the time of the video capture device and the recording device are synchronized.

A seventh aspect of the embodiments of the present invention provides a drone, including:

body;

a power system mounted to the fuselage for providing flight power;

a video capture device mounted on the body for capturing video;

And a control device as described in the fifth aspect above.

An eighth aspect of the embodiments of the present invention provides a remote controller, including:

Body part

a rocker mounted to the body portion;

a recording device mounted on the body portion for recording audio information;

And a control device according to any of the sixth aspects of the invention.

The video and audio synchronization method, device and drone provided by the embodiment obtain the video information captured by the video capture device carried by the drone through the synchronization device, and acquire the audio information recorded by the recording device, and are included according to the video information. The time stamp information and the time stamp information included in the audio information align the video information and the audio information, thereby realizing automatic synchronization of the audio information and the video information, without manual manual operation, and the synchronization accuracy and efficiency are high.

DRAWINGS

1 is a flowchart of a method for synchronizing video and audio according to an embodiment of the present invention;

2 is a flowchart of a method for synchronizing video and audio according to another embodiment of the present invention;

FIG. 3 is a flowchart of a method for synchronizing video and audio according to another embodiment of the present invention; FIG.

4 is a structural diagram of a synchronization device according to an embodiment of the present invention;

FIG. 5 is a structural diagram of a control device according to an embodiment of the present invention;

FIG. 6 is a structural diagram of a control device according to an embodiment of the present invention;

FIG. 7 is a structural diagram of a drone according to an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly described with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

It should be noted that when a component is referred to as being "fixed" to another component, it can be directly on the other component or the component can be in the middle. When a component is considered to "connect" another component, it can be directly connected to another component or possibly a central component.

All technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs, unless otherwise defined. The terminology used in the description of the present invention is for the purpose of describing particular embodiments and is not intended to limit the invention. The term "and/or" used herein includes any and all combinations of one or more of the associated listed items.

Some embodiments of the present invention are described in detail below with reference to the accompanying drawings. The features of the embodiments and examples described below can be combined with each other without conflict.

Embodiments of the present invention provide a method for synchronizing video and audio. FIG. 1 is a flowchart of a method for synchronizing video and audio according to an embodiment of the present invention. As shown in FIG. 1, the method in this embodiment may include:

Step 101: Obtain video information captured by a video capture device carried by the drone, and acquire audio information recorded by the recording device, wherein the video capture device and the recording device are synchronized in time.

The execution body of this embodiment may be a synchronization device, and the synchronization device may be disposed on a drone such as an unmanned aerial vehicle, an unmanned ship or an unmanned vehicle, or a remote controller of the drone, or may be independently set. At the ground end. When the sync device is placed on the remote of the drone or drone, the sync device is a device with computing and/or processing functions. When the control device is independently disposed at the ground end, the synchronization device may be a smartphone, a tablet, a ground control station, a laptop, a watch, or the like.

The synchronization device and the flight controller on the drone are connected by an Application Programming Interface (API). Specifically, when the synchronization device is set on the drone, the flight controller of the synchronization device and the drone can pass Wired party Connection, for example, by at least one of the following: Universal Asynchronous Receiver/Transmitter (UART), Universal Serial Bus (USB), and Flight Controller Area Network (Controller Area Network) , referred to as CAN) bus connection.

When the synchronization device is independently set on the ground end, the synchronization device and the flight controller of the drone can be connected by wireless, for example, by at least one of the following methods: Wireless-Fidelity (WI-FI) ), Bluetooth, software defined radio (SDR) or other custom protocols.

In this embodiment, the video information is obtained by a video capturing device mounted on the drone, and the audio information is obtained by recording through a recording device or other independent recording device set on the remote control of the drone, the video capturing device and the recording. The device is synchronized in time. The UAV real-time or periodic video information obtained by the video shooting device is sent to the synchronization device in units of video files or in units of video frames, and the synchronization device obtains audio information in a similar manner. I will not repeat them here.

It should be noted that the video information and the audio information in the embodiment all include timestamp information. The following describes the location of the timestamp information in the video information:

In one possible manner, the synchronization device acquires video information in units of video files. At this time, the file header of the video file corresponding to the video information includes timestamp information.

In another possible manner, the synchronization device acquires video information in units of frames. In this case, time stamp information may be carried in a frame header of each video frame of the video information, for example, the video information includes a video frame. 1 and video frame 2, the timestamp information when the video frame 1 is captured may be carried in the frame header of the video frame 1, and the timestamp information when the video frame 2 is captured is carried in the frame header of the video frame 2. Optionally, the time stamp information may be carried in the frame header of the video frame with the preset number of frames in the video information. For example, if the video information includes video frame 1, video frame 2, video frame 3, and video frame 4, The timestamp information when the video frame 1 is captured is carried in the frame header of the video frame 1, and the timestamp information when the video frame 3 is captured is carried in the frame header of the video frame 3. Of course, this is merely an illustration and not a limitation of the invention.

In this embodiment, the location of the timestamp information in the audio information is the same as the video information, and details are not described herein again.

Optionally, in this embodiment, the recording device and the video capturing device carried by the drone are used. The frame number (including the superframe number) is used for communication. Since the frame number can be rolled back for hours or even days, it can be used to align video information and audio information. In addition, since the video capture device and the recording device are accurately synchronized in time, the time of radio wave propagation is negligible, so the distance between the drone and the subject is large (recording device and subject) The distance between them is relatively close, and there is no problem that the sound and picture are not synchronized due to the propagation delay of the video information and the audio information. Therefore, preferably, the timestamp information in this embodiment may be a frame number used by the video capturing device and the recording device for communication. Of course, the actual timestamp information may also be other information, such as time information of a video frame or a video file, and time information of an audio frame or an audio file.

Step 102: Align the video information and the audio information according to timestamp information included in the video information and timestamp information included in the audio information. Optionally, when the synchronization device acquires video information in units of video files and audio information in units of audio files, according to timestamp information of the video files in the video information, and timestamp information of the audio files in the audio information, Align the audio files in the video files and audio information in the video information. For example, the video information includes a video file 1, a video file 2, and a video file 3. The file header of the video file 1 includes a frame number a, the file header of the video file 2 includes a frame number b, and the file header of the video file 3 includes a frame. No. c; the audio information includes an audio file 1, an audio file 2, and an audio file 3. The file header of the audio file 1 includes a frame number a, and the file header of the audio file 2 includes a frame number b, and the file header of the audio file 3 Including the frame number c, the video file 1 is aligned with the audio file 1, the video file 2 is aligned with the audio file 2, and the video file 3 is aligned with the audio file 3. Of course, the description herein is merely illustrative and not a limitation of the invention.

Optionally, when the synchronization device acquires video information in units of frames and audio information in units of frames, if time stamp information is included in a frame header of each video frame in the video information, each of the audio information The time stamp information is included in the frame header of each audio frame, and the video frame and the audio frame including the same time stamp information are aligned. For example, the video information includes a video frame 1, a video frame 2, and a video frame 3. The frame header of the video frame 1 includes a frame number a, and the frame header of the video frame 2 includes a frame number b. The frame header of the video frame 3 includes Frame number c; audio information includes audio frame 1, audio frame 2 and audio frame 3, frame header of audio frame 1 includes frame number a, frame header of audio frame 2 includes frame number b, frame header of audio frame 3 Including frame number c, video frame 1 is aligned with audio frame 1, video frame 2 is aligned with audio frame 2, and video frame 3 is aligned with audio frame 3. Of course, this is just for illustration. The description is not intended to be limiting of the invention. If the video frame with the preset number of frames in the video information includes the timestamp information, and the audio frame with the same preset number of frames in the audio information includes the timestamp information, the timestamp of the video frame according to the preset preset number of frames The information, and the time stamp information included in the audio frame of the preset preset frame number, are paired with the video frame of the preset preset frame number and the audio frame of the preset preset frame number. For example, the video information includes a video frame 1, a video frame 2, and a video frame 3. The frame header of the video frame 1 includes a frame number a, and the frame header of the video frame 3 includes a frame number c. The audio information includes an audio frame 1. Audio frame 2 and audio frame 3, the frame header of the audio frame 1 includes the frame number a, and the frame header of the audio frame 3 includes the frame number c, respectively, the video frame 1 is aligned with the audio frame 1, and the video frame 3 and the audio are respectively Frame 3 is aligned. Of course, the description herein is merely illustrative and not a limitation of the invention.

In this embodiment, the video information captured by the video capture device carried by the drone is acquired by the synchronization device, and the audio information recorded by the recording device is obtained, and according to the timestamp information included in the video information and the timestamp information included in the audio information, The video information and the audio information are aligned, thereby realizing the automatic synchronization of the audio information and the video information, without manual manual operation, and the synchronization accuracy and efficiency are high.

Embodiments of the present invention provide a method for synchronizing video and audio. FIG. 2 is a flowchart of a method for synchronizing video and audio according to another embodiment of the present invention. As shown in FIG. 2, the method in this embodiment may include:

Step 201: Obtain video information captured by a video capture device carried by the drone.

The execution body of this embodiment may be a control device provided on the drone, and the control device is a device having a computing function and/or a processing function. The control device and the flight controller on the drone, as well as the video capture device, are connected via an API. Specifically, the flight controller of the control device and the drone, and the video capture device of the drone can be connected by wire, for example, by at least one of the following methods: Universal Asynchronous Receiver/Transmitter , referred to as UART), Universal Serial Bus (USB), and Flight Controller Area Network (CAN) bus connection.

Optionally, the video information obtained by the control device from the video capturing device may be video information in units of video files, video information in frame units, or other information. The video information in the form is not limited in this embodiment.

Step 202: Generate timestamp information in the video information.

Optionally, when the obtained video information is the video information in the video file, the generating the timestamp information in the video information, including generating the timestamp information in the file header of the video file corresponding to the video information. . In this embodiment, the timestamp information includes a frame number for communication between the recording device and the video capturing device, which is consistent with the foregoing embodiment, and details are not described herein again.

Optionally, when the acquired video information is video information in units of frames, the generating timestamp information in the video information includes: generating a timestamp in a frame header of each video frame in the video information. information. Or generating the timestamp information in the video information, including: generating, in the video information, a timestamp information by using a video frame of a preset number of frames. This is consistent with the above embodiment, and will not be described again here.

Step 203: Send the video information with the timestamp information to the synchronization device, so that the synchronization device according to the timestamp information in the video information and the time in the audio information recorded by the recording device acquired by the synchronization device Stamping information, aligning the video information with the audio information, wherein the video capture device and the recording device are time synchronized.

In this embodiment, a method for transmitting video information to a synchronization device, and a synchronization device according to the timestamp information in the acquired video information, and the timestamp information in the audio information recorded by the recording device acquired by the synchronization device, The method for aligning the audio information is consistent with the foregoing embodiment, and details are not described herein again.

In this embodiment, the video information captured by the video capture device carried by the drone is obtained by the control device, and the time stamp information is generated by the control device in the acquired video information, and the video information generating the time stamp information is sent to the synchronization device. The synchronization device is enabled to align the video information and the audio information according to the timestamp information in the received video information and the timestamp information in the audio information acquired from the recording device, thereby realizing the video information and the audio information. Automatic alignment, no manual manual operation, and high efficiency of synchronization.

Embodiments of the present invention provide a method for synchronizing video and audio. FIG. 3 is a flowchart of a method for synchronizing video and audio according to another embodiment of the present invention. As shown in FIG. 3, the method in this embodiment may include:

Step 301: Acquire audio information recorded by a recording device on the remote control of the drone.

The execution body of this embodiment may be a control device disposed on the remote controller of the drone, and the recording device is disposed on the remote controller of the drone. The control device is a device having computing functions and/or processing functions. The control device is connected to the recording device via an API. Specifically, the control device and the recording device can be connected by wire, for example, by at least one of the following methods: Universal Asynchronous Receiver/Transmitter (UART), Universal Serial Bus (Universal Serial Bus) USB), Flight Controller Area Network (Controller Area Network, CAN) bus connection.

Optionally, the audio information obtained by the control device from the recording device may be audio information in units of audio files, audio information in units of frames, and other forms of audio information. This is not limited.

Step 302: Generate timestamp information in the audio information.

Optionally, when the obtained audio information is audio information in an audio file unit, the generating time stamp information in the audio information includes generating timestamp information in a file header of an audio file corresponding to the audio information. . In this embodiment, the timestamp information includes a frame number for communication between the recording device and the video capturing device, which is consistent with the foregoing embodiment, and details are not described herein again.

Optionally, when the obtained audio information is audio information in units of frames, the generating time stamp information in the audio information includes: generating a time stamp in a frame header of each audio frame in the audio information. information. Alternatively, the generating the timestamp information in the audio information includes: generating, in the audio information, an audio frame separated by a preset number of frames to generate a timestamp information. This is consistent with the above embodiment, and will not be described again here.

Step 303: Send the audio information that generates the timestamp information to the synchronization device, so that the synchronization device is based on the timestamp information in the audio information, and the video information captured by the video capture device acquired by the synchronization device. Time stamp information for aligning the audio information with the video information, wherein the video capture device and the recording device are time synchronized.

In this embodiment, the method for transmitting audio information to the synchronization device, and the synchronization device according to the timestamp information in the acquired video information, and the timestamp information in the audio information recorded by the recording device acquired by the synchronization device, The method for aligning the audio information is consistent with the foregoing embodiment, and details are not described herein again.

In this embodiment, the audio information recorded by the recording device on the remote control of the UAV is obtained by the control device, and the timestamp information is generated in the obtained audio information by the control device, and the audio information of the timestamp information is sent to the synchronization device. So that the synchronization device can align the audio information and the video information according to the timestamp information in the received audio information and the timestamp information in the video information acquired from the video capturing device, thereby realizing the video information and audio. Automatic alignment of information, no manual manual operation, and high efficiency of synchronization.

The embodiment of the invention provides a synchronization device. The synchronization device may specifically be the synchronization device described in the foregoing embodiment. 4 is a structural diagram of a synchronization device according to an embodiment of the present invention. As shown in FIG. 4, the synchronization device 40 includes: a communication interface 41, one or more processors 42; and the one or more processors 42 are separated or coordinated. Working, the communication interface 41 is connected to the processor 42;

Specifically, the communication interface 41 is configured to: acquire video information captured by a video capture device mounted on the drone, and acquire audio information recorded by the recording device, wherein the video capture device and the recording device are time synchronized;

The processor 42 is configured to: align the video information and the audio information according to timestamp information included in the video information and timestamp information included in the audio information.

The location of the timestamp information in the video information and the audio information may include the following possible ways:

In a possible manner, time stamp information is included in a file header of a video file corresponding to the video information; and time stamp information is included in a file header of the audio file corresponding to the audio information.

In this possible manner, the processor 42 is configured to: according to the timestamp information of the video file in the video information, and the timestamp information of the audio file in the audio information, in the video file and the audio information in the video information. The audio files are aligned.

In a possible manner, the communication interface 41 is configured to: receive video information in units of frames sent by the video capturing device, and receive audio information in units of frames sent by the recording device. In this manner, if time stamp information is included in the frame header of each video frame of the video information, time stamp information is included in the frame header of each audio frame of the audio information, and the processor 42 is configured to: Video frames and audio frames including the same timestamp information are aligned. If the video information The video frame of the preset preset frame number includes timestamp information, and the audio frame that is separated by the preset number of frames in the audio information includes timestamp information, and the processor 42 is configured to: time included in the video frame of the preset preset number of frames according to the interval The stamp information, and the time stamp information included in the audio frame of the preset preset frame number, are aligned with the video frame of the preset preset frame number and the audio frame of the preset preset frame number.

Optionally, the video information and the timestamp information of the audio information in the embodiment include a frame number used by the video capturing device and the recording device to communicate.

The processor 42 is configured to: align the video information and the audio information according to a frame number included in the video information and a frame number included in the audio information.

The specific principles and implementations of the synchronization device provided by the embodiment of the present invention are similar to the embodiment shown in FIG. 1 and will not be further described herein.

Embodiments of the present invention provide a control device. FIG. 5 is a structural diagram of a control device according to an embodiment of the present invention. As shown in FIG. 5, the control device 50 includes: a communication interface 51, one or more processors 52; and the one or more processors 52 alone or Working in cooperation, the communication interface 51 and the processor 52 are communicatively connected;

The communication interface 51 is configured to: acquire video information captured by a video capture device carried by the drone;

The processor 52 is configured to: generate timestamp information in the video information;

The communication interface 51 is configured to: send the video information that generates the timestamp information to the synchronization device, so that the synchronization device records according to the timestamp information in the video information and the recording device acquired by the synchronization device. Time stamp information in the audio information, the video information and the audio information being aligned, wherein the video capture device and the recording device are time synchronized.

Optionally, the processor 52 is configured to: generate timestamp information in a file header of a video file corresponding to the video information.

Optionally, the processor 52 is configured to: generate timestamp information in a frame header of each video frame in the video information.

Optionally, the processor 52 is configured to: in the video information, generate a timestamp information by using a video frame of a preset number of frames.

Optionally, the video information and the timestamp information of the audio information both include a frame number used when the video capturing device and the recording device communicate.

The specific principles and implementations of the control device provided by the embodiment of the present invention are similar to the embodiment shown in FIG. 2, and details are not described herein again.

Embodiments of the present invention provide a control device. FIG. 6 is a structural diagram of a control device according to an embodiment of the present invention. As shown in FIG. 6, the control device 60 includes: a communication interface 61, one or more processors 62; and the one or more processors 62 alone or Working in cooperation, the communication interface 61 and the processor 62 are communicatively connected;

The communication interface 61 is configured to: acquire audio information recorded by a recording device on the remote control of the drone;

The processor 62 is configured to: generate timestamp information in the audio information;

The communication interface 61 is configured to: send the audio information generated by the timestamp information to the synchronization device, so that the synchronization device according to the timestamp information in the audio information, and the video capture device acquired by the synchronization device Time stamp information in the captured video information, the audio information and the video information being aligned, wherein the video capture device and the recording device are time synchronized.

Optionally, the processor 52 is configured to: generate timestamp information in a file header of the audio file corresponding to the audio information.

Optionally, the processor 52 is configured to: generate timestamp information in a frame header of each frame of the audio information.

Optionally, the processor 52 is configured to generate, in the audio information, a timestamp information by using an audio frame of a preset number of frames.

The specific principles and implementations of the control device provided by the embodiment of the present invention are similar to the embodiment shown in FIG. 3, and details are not described herein again.

Embodiments of the present invention provide a drone. FIG. 7 is a structural diagram of a drone according to an embodiment of the present invention, and the drone may be an unmanned aerial vehicle, an unmanned vehicle, or an unmanned vehicle. As shown in FIG. 7, the drone 700 includes a fuselage, a power system, and a flight controller 718, the power system including at least one of a motor 707, a propeller 706, and an electronic governor 717. a fuselage for providing flight power; a flight controller 718 is in communication with the power system for controlling the flight of the drone; wherein the flight controller 718 includes an Inertial Measurement Unit (IMU) The inertial measurement unit generally includes a gyroscope and an accelerometer. The inertial measurement unit is configured to detect a pitch angle, a roll angle, a yaw angle, an acceleration, and the like of the drone.

In addition, as shown in FIG. 7, the drone 700 further includes: a control device 711, a sensing system 708, a communication system 710, a supporting device 702, and a video capturing device 704. The supporting device 702 may specifically be a cloud platform, and the control device 711 is configured to acquire video information from the video capturing device 704 and generate timestamp information in the video information. The communication system 710 may specifically include a transmitter, and the transmitter is configured to send the video information that generates the timestamp information to the synchronization device 712. The

antennas

714, 716 represent electromagnetic waves generated during communication between the transmitter and antenna 714.

The specific principles, implementation manners, and beneficial effects of the control device 711 provided by the embodiments of the present invention are similar to the foregoing embodiments, and are not described herein again.

An embodiment of the present invention provides a UAV remote controller, where the remote controller includes a body portion;

a rocker mounted to the body portion;

a recording device mounted on the body portion for recording audio information;

And the control device as described in the embodiment of Fig. 3 above.

The specific principles, implementation manners, and beneficial effects of the control device provided by the embodiments of the present invention are similar to the foregoing embodiments, and are not described herein again.

In the several embodiments provided by the present invention, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the device embodiments described above are merely illustrative. For example, the division of the unit is only a logical function division. In actual implementation, there may be another division manner, for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional units.

The above-described integrated unit implemented in the form of a software functional unit can be stored in a computer readable storage medium. The above software functional unit is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to perform the methods of the various embodiments of the present invention. Part of the steps. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like, which can store program codes. .

A person skilled in the art can clearly understand that for the convenience and brevity of the description, only the division of each functional module described above is exemplified. In practical applications, the above function assignment can be completed by different functional modules as needed, that is, the device is installed. The internal structure is divided into different functional modules to perform all or part of the functions described above. For the specific working process of the device described above, reference may be made to the corresponding process in the foregoing method embodiments, and details are not described herein again.

Finally, it should be noted that the above embodiments are merely illustrative of the technical solutions of the present invention, and are not intended to be limiting; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art will understand that The technical solutions described in the foregoing embodiments may be modified, or some or all of the technical features may be equivalently replaced; and the modifications or substitutions do not deviate from the technical solutions of the embodiments of the present invention. range.

Claims

A method for synchronizing video and audio, characterized in that it comprises:

Obtaining video information captured by a video capture device carried by the drone, and acquiring audio information recorded by the recording device, wherein time synchronization of the video capture device and the recording device is performed;

And aligning the video information and the audio information according to timestamp information included in the video information and timestamp information included in the audio information.
The method according to claim 1, wherein the header of the video file corresponding to the video information includes timestamp information; and the header of the audio file corresponding to the audio information includes timestamp information.
The method according to claim 2, wherein said aligning said video information and said audio information according to time stamp information included in said video information and time stamp information included in said audio information ,include:

And aligning the video file in the video information and the audio file in the audio information according to timestamp information of the video file in the video information and timestamp information of the audio file in the audio information.
The method according to claim 1, wherein the acquiring the video information captured by the video capturing device mounted on the drone and acquiring the audio information recorded by the recording device comprises:

Receiving video information in units of frames transmitted by the video capturing device, and receiving audio information in units of frames transmitted by the recording device.
The method according to claim 4, wherein time stamp information is included in a frame header of each video frame of the video information, and time stamp information is included in a frame header of each audio frame of the audio information. .
The method according to claim 5, wherein the aligning the video information and the audio information according to time stamp information included in the video information and time stamp information included in the audio information ,include:

Align video frames and audio frames that include the same timestamp information.
The method according to claim 4, wherein the video frame with the preset number of frames in the video information includes timestamp information, and the audio frame separated by the preset number of frames in the audio information includes timestamp information. .
The method according to claim 7, wherein said aligning said video information and said audio information according to time stamp information included in said video information and time stamp information included in said audio information ,include:

The time frame information included in the video frame of the preset preset frame number and the time stamp information included in the audio frame of the preset preset frame number are aligned with the video frame of the preset preset frame number and the audio frame of the preset preset frame number. .
The method according to any one of claims 1 to 8, wherein the video information and the time stamp information of the audio information each include a frame number used when the video capturing device and the recording device communicate .
The method according to claim 9, wherein the aligning the video information and the audio information according to time stamp information included in the video information and time stamp information included in the audio information ,include:

And aligning the video information and the audio information according to a frame number included in the video information and a frame number included in the audio information.
A method for synchronizing video and audio, characterized in that it comprises:

Obtaining video information captured by a video capture device carried by the drone;

Generating timestamp information in the video information;

Sending the video information with the timestamp information to the synchronization device, so that the synchronization device according to the timestamp information in the video information, and the timestamp information in the audio information recorded by the recording device acquired by the synchronization device, Aligning the video information with the audio information, wherein the video capture device and the recording device are time synchronized.
The method according to claim 11, wherein the generating time stamp information in the video information comprises:

The timestamp information is generated in the file header of the video file corresponding to the video information.
The method according to claim 11, wherein the generating time stamp information in the video information comprises:

Time stamp information is generated in a frame header of each video frame in the video information.
The method according to claim 11, wherein the generating time stamp information in the video information comprises:

In the video information, a video frame separated by a preset number of frames generates a time stamp information.
The method according to any one of claims 11 to 14, wherein the video information and the time stamp information of the audio information each include a frame number used when the video capturing device and the recording device communicate .
A method for synchronizing video and audio, characterized in that it comprises:

Obtaining audio information recorded by the recording device on the remote control of the drone;

Generating time stamp information in the audio information;

Transmitting the audio information with the timestamp information to the synchronization device, so that the synchronization device according to the timestamp information in the audio information, and the timestamp in the video information captured by the video capture device acquired by the synchronization device Information, aligning the audio information with the video information, wherein the video capture device and the recording device are time synchronized.
The method according to claim 16, wherein the generating time stamp information in the audio information comprises:

Time stamp information is generated in a file header of an audio file corresponding to the audio information.
The method according to claim 16, wherein the generating time stamp information in the audio information comprises:

Time stamp information is generated in a frame header of each frame of the audio information.
The method according to claim 16, wherein the generating time stamp information in the audio information comprises:

In the audio information, an audio frame separated by a preset number of frames generates a time stamp information.
The method according to any one of claims 16 to 19, wherein the video information and the time stamp information of the audio information each include a frame number used when the video capturing device and the recording device communicate .
A synchronization device, comprising: a communication interface, one or more processors; the one or more processors operating separately or in cooperation, the communication interface being coupled to the processor;

The communication interface is configured to: acquire video information captured by a video capture device mounted on the drone, and acquire audio information recorded by the recording device, wherein time synchronization of the video capture device and the recording device;

The processor is configured to: align the video information and the audio information according to timestamp information included in the video information and timestamp information included in the audio information.
The synchronization device according to claim 21, wherein the file header of the video file corresponding to the video information includes timestamp information; and the file header of the audio file corresponding to the audio information includes timestamp information.
The synchronization device according to claim 22, wherein the processor is configured to: according to timestamp information of a video file in the video information, and timestamp information of an audio file in the audio information, to the video The video files in the information are aligned with the audio files in the audio information.
The synchronization device according to claim 21, wherein the communication interface is configured to: receive video information in units of frames sent by the video capturing device, and receive a frame-based unit sent by the recording device Audio information.
The synchronization device according to claim 24, wherein time stamp information is included in a frame header of each video frame of the video information, and a time stamp is included in a frame header of each audio frame of the audio information. information.
The synchronization device according to claim 25, wherein said processor is configured to align a video frame and an audio frame including the same time stamp information.
The synchronization device according to claim 24, wherein the video frame with the preset number of frames in the video information includes timestamp information, and the audio frame separated by the preset number of frames in the audio information includes a timestamp information.
The synchronization device according to claim 27, wherein the processor is configured to: timestamp information included in a video frame of a predetermined number of frames according to an interval, and timestamp information included in an audio frame of a preset preset number of frames, Align the video frames with the preset number of frames and the audio frames with the preset number of frames.
The synchronization device according to any one of claims 21 to 28, wherein the video information and the time stamp information of the audio information each comprise a frame used when the video capturing device and the recording device communicate number.
The synchronization device according to claim 29, wherein the processor is configured to: the video information and the audio according to a frame number included in the video information and a frame number included in the audio information The information is aligned.
A control device, comprising: a communication interface, one or more processors; the one or more processors operating separately or in cooperation, the communication interface and the processor Communication connection

The communication interface is configured to: acquire video information captured by a video capture device carried by the drone;

The processor is configured to: generate timestamp information in the video information;

The communication interface is configured to: send the video information that generates the timestamp information to the synchronization device, so that the synchronization device: according to the timestamp information in the video information, and the audio recorded by the recording device acquired by the synchronization device Time stamp information in the information, the video information and the audio information are aligned, wherein the video capture device and the recording device are time synchronized.
The control device according to claim 31, wherein the processor is configured to: generate timestamp information in a file header of a video file corresponding to the video information.
The control device according to claim 31, wherein said processor is configured to: generate time stamp information in a frame header of each video frame in said video information.
The control device according to claim 31, wherein the processor is configured to generate, in the video information, a time stamp information by a video frame of a predetermined number of frames.
The control device according to any one of claims 31 to 34, wherein the video information and the time stamp information of the audio information each include a frame used when the video capturing device and the recording device communicate number.
A control device, comprising: a communication interface, one or more processors; the one or more processors operating separately or in cooperation, the communication interface being in communication with the processor;

The communication interface is configured to: acquire audio information recorded by a recording device on the remote control of the drone;

The processor is configured to: generate timestamp information in the audio information;

The communication interface is configured to: send the audio information that generates the timestamp information to the synchronization device, so that the synchronization device shoots according to the timestamp information in the audio information, and the video capture device acquired by the synchronization device The time stamp information in the video information is aligned with the audio information and the video information, wherein the video capture device and the recording device are time synchronized.
A control device according to claim 36, wherein said processor is And generating timestamp information in a file header of the audio file corresponding to the audio information.
The control device according to claim 36, wherein said processor is configured to: generate time stamp information in a frame header of each frame of the audio information.
The control device according to claim 36, wherein said processor is configured to:

In the audio information, an audio frame separated by a preset number of frames generates a time stamp information.
The control device according to any one of claims 36 to 39, wherein the video information and the time stamp information of the audio information each comprise a frame used when the video capturing device and the recording device communicate number.
A drone, characterized in that it comprises:

body;

a power system mounted to the fuselage for providing flight power;

a video capture device mounted on the body for capturing video;

And a control device according to any of claims 31-35.
The drone according to claim 41, wherein the drone is an unmanned aerial vehicle, an unmanned aerial vehicle or an unmanned vehicle.
A remote controller, comprising:

Body part

a rocker mounted to the body portion;

a recording device mounted on the body portion for recording audio information;

And a control device according to any of claims 36-40.