WO2022040875A1

WO2022040875A1 - Data transmission and processing methods, mobile platform, terminal device and chip

Info

Publication number: WO2022040875A1
Application number: PCT/CN2020/110864
Authority: WO
Inventors: 耿超; 梁耀; 王景正
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2020-08-24
Filing date: 2020-08-24
Publication date: 2022-03-03
Also published as: CN113950825A

Abstract

Data transmission and processing methods, a mobile platform, a terminal device and a chip. Said method comprises: generating a code stream on the basis of a video data coding protocol, the code stream comprising data units, at least one of the data units comprising metadata, and the metadata being used for indicating motion state information of a mobile platform; and sending the code stream to a terminal device. According to the technical solutions provided in the embodiments of the present application, data is transmitted by being placed in a video transmission protocol-based code stream, increasing the data transmission efficiency.

Description

Data transmission, processing method, movable platform, terminal device and chip

technical field

The present application relates to the field of communication technologies, and in particular, to a data transmission method, a data processing method, a movable platform, a terminal device, and a chip.

Background technique

With the development of video shooting equipment (such as drones, PTZs, etc.), video shooting equipment needs to perform data backhaul to realize video data-based services. At present, there are various solutions in the industry for the return of video data. However, for the return of non-audio and video data (such as the attitude, speed, position of the drone, the attitude of the gimbal of the drone equipped with the camera, parameters, similar parameters of other equipment, etc.), there are few related solutions in the industry, and there is no unified standard. .

SUMMARY OF THE INVENTION

The present application provides a data transmission and processing method, a movable platform, a terminal device and a chip, which can improve the transmission efficiency of metadata.

In a first aspect, an embodiment of the present application provides a data transmission method, the method is applied to a movable platform, and the movable platform can communicate with a terminal device, and the method includes:

Generate a code stream based on a video data encoding protocol, the code stream includes data units, at least one of the data units includes metadata, and the metadata is used to indicate motion state information of the movable platform;

Send the code stream to the terminal device.

In a second aspect, an embodiment of the present application further provides a data processing method, the method is applied to a terminal device, the terminal device and a movable platform, and the method includes:

receiving the code stream sent by the movable platform;

A data unit is parsed from the code stream based on a video data encoding protocol; at least one data unit in the data unit includes metadata, and the metadata is used to indicate motion state information of the movable platform;

Motion state information of the movable platform is determined based on the metadata.

In a third aspect, the embodiments of the present application also provide a movable platform, including:

a processor, configured to generate a code stream based on a video data encoding protocol, the code stream includes data units, at least one of the data units contains metadata, and the metadata is used to indicate the movement of the movable platform status information;

The communication interface is used for sending the code stream to the terminal device.

In a fourth aspect, an embodiment of the present application further provides a terminal device, including:

The communication interface is used to receive the code stream sent by the mobile platform;

a processor, configured to parse a data unit from the code stream based on a video data encoding protocol; at least one data unit in the data unit includes metadata, and the metadata is used to indicate the motion state of the movable platform information; determining motion state information of the movable platform based on the metadata.

In a fifth aspect, an embodiment of the present application further provides a chip, where the chip includes a memory and a processor, and the processor executes a computer program stored in the memory to implement the above-mentioned data transmission method.

In a sixth aspect, an embodiment of the present application further provides a chip, where the chip includes a memory and a processor, and the processor executes a computer program stored in the memory to implement the above data processing method.

In a seventh aspect, the embodiments of the present application provide instructions that, when executed on a computer, cause the computer to execute the data transmission method as described above.

In an eighth aspect, an embodiment of the present application provides a computer-readable storage medium, including instructions, which, when executed on a computer, cause the computer to execute the data processing method as described above.

In the data transmission method provided by the embodiment of the present application, the mobile platform generates a code stream based on a video data encoding protocol, the code stream includes data units, and at least one data unit in the data units includes metadata, and the metadata is used for Indicate the motion state information of the movable platform, and send the code stream to the terminal device. The technical solutions provided by the embodiments of the present application improve data transmission efficiency by placing data in a video transmission protocol-based code stream for transmission.

Description of drawings

1 is a flowchart of a data transmission method provided by an embodiment of the present application;

2 is a flowchart of another data transmission method provided by an embodiment of the present application;

3 is a schematic diagram of a network abstraction layer unit provided by an embodiment of the present application;

4 is a schematic diagram of a code stream provided by an embodiment of the present application;

5 is a schematic diagram of a data transmission process provided by an embodiment of the present application;

6 is a schematic diagram of another data transmission process provided by an embodiment of the present application;

FIG. 7 is a flowchart of a UAV data return process provided by an embodiment of the present application;

8 is a flowchart of a data processing method provided by an embodiment of the present application;

9 is a flowchart of another data processing method provided by an embodiment of the present application;

10 is a block diagram of a movable platform provided by an embodiment of the present application;

FIG. 11 is a block diagram of a terminal device provided by an embodiment of the present application.

specific embodiment

The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application.

For a device with metadata return requirement, an embodiment of the present application provides a data transmission method based on a video data encoding protocol.

Referring to FIG. 1, FIG. 1 shows a flowchart of a data transmission method provided by an embodiment of the present application. The method can be applied to a movable platform, wherein the movable platform can communicate with a terminal device, and the method includes:

101. Generate a code stream based on a video data encoding protocol, where the code stream includes data units, and at least one of the data units includes metadata, where the metadata is used to indicate motion state information of the movable platform.

The embodiments of the present application may be applied to a movable platform, and the movable platform may be an aircraft, such as a multi-rotor UAV and a fixed-wing UAV. It can also be a mobile platform such as an unmanned vehicle, ship, submarine, etc. It can also be a handheld camera device.

In a movable platform, metadata may include data indicating the motion state of the movable platform. For example, the attitude, speed, and position of the drone; the attitude and parameters of the gimbal of the camera mounted on the drone. Metadata can also include information for various smart functions. For example, the metadata may include return point information of the automatic return-to-home function of the drone, cruise route information of the automatic cruise function of the drone, and the like.

Metadata can be data of the mobile platform itself, or data generated by other devices connected to the mobile platform.

Terminal devices may include terminals or servers. The terminal device can communicate with multiple mobile platforms, receive code streams sent by multiple mobile platforms, and perform various business processing according to the code streams. For example, the terminal device may be a video service client, a command and dispatch client, a command and dispatch platform, an image management platform, and the like.

The video data encoding protocol refers to a protocol for specifying encoding and compressing video data, for example, H264, H265, and possibly derived video data encoding protocols that support the use of custom fields. The video data encoding protocol specifies that in the protocol layer, data needs to be encapsulated into data units of a specified format. The embodiment of the present application transmits metadata by using a video data encoding protocol, and does not require an additional development protocol, which can lower development thresholds and lower development costs.

102. Send the code stream to the terminal device.

Data transmission is implemented at the transport layer. The transmission mode of the transport layer can be based on code stream transmission, and the data unit can be encoded into the code stream and sent to the terminal device.

In the embodiment of the present application, since the metadata is transmitted in-band along with the code stream, if the code stream is encrypted, the metadata is also encrypted, which can improve security, and does not need to develop an additional encryption scheme.

Referring to FIG. 2, FIG. 2 shows a flowchart of another data transmission method provided by an embodiment of the present application. The method can be applied to a movable platform, where the movable platform communicates with a terminal device, and the method includes:

201. Acquire video data, and encapsulate the video data into a data unit based on a video data encoding protocol.

The video data may be video data collected by the movable platform itself, or may be video data collected by a device connected to the movable platform. The removable platform can encapsulate video data into data units based on a video data encoding protocol.

In one example, the video data may be divided by frame numbers, and the movable platform may encapsulate the video data into individual data units frame by frame.

202. Extract metadata from a preset buffer queue, and encapsulate the metadata into a data unit based on a video data encoding protocol.

In this embodiment of the present application, the movable platform may acquire metadata of a preset device, and store the metadata in a preset cache queue. In the process of encapsulating the frequency data into the data unit, metadata is extracted from the preset buffer queue, and the metadata is encapsulated into a separate data unit, so that the time error between the metadata and the video data can be reduced.

The preset device may be a device possessed by the movable platform itself, or may be a device connected to the movable platform. Different preset devices can send metadata to the mobile platform at different frequencies, and by setting a cache queue, the mobile platform can collect data at different frequencies.

The time stamp that the data unit corresponding to the metadata may have, and the time stamp may enable the terminal device receiving the metadata to globally synchronize the metadata.

In this embodiment of the present application, the video data encoding protocol supports the use of the Supplemental Enhancement Information (SEI) field to add information to the code stream. The data unit may be a network abstraction layer unit, and the type of the network abstraction layer unit corresponding to the metadata is the supplementary enhancement information SEI type. For example, the data unit may be a network abstraction layer unit of the H264 protocol.

The step of encapsulating the metadata into a data unit based on a video data encoding protocol may include encapsulating the metadata into a network abstraction layer unit of the SEI type. For example, a mobile platform may encapsulate metadata into a separate SEI-type Network Abstraction Layer unit.

The Network Abstract Layer is referred to as NAL. In the H.264/AVC video coding standard, the entire system framework is divided into two layers: the video coding layer (Video Coding Layer, VCL) and the network abstraction layer (Network Abstraction Layer, NAL). VCL is responsible for representing the content of valid video data, and NAL is responsible for formatting the data and providing header information to ensure that the data is suitable for transmission on various channels and storage media. The network abstraction layer unit (NAL unit) is the basic syntax structure of NAL, which contains a byte of header information (NAL header) and a series of raw data byte streams (RBSP) from VCL.

Referring to FIG. 3, FIG. 3 shows a schematic diagram of a network abstraction layer unit. Metadata is encapsulated into a separate SEI type network abstraction layer unit, which can be attached to the end of the H264 common frame, for example, it can be attached to the PPS (image). Parameter set, the end of the network abstraction layer unit of type Picture Parameter Set or the end of the network abstraction layer unit of type IDR (Instantaneous Decoding Refresh).

Referring to FIG. 4, FIG. 4 shows a schematic diagram of a code stream. Each network abstraction layer unit may include a 4bytes start code startcode: 0x00000001. The first byte after the start code is the header information. The header information can store the NAL unit type. In the H.264/AVC standard, there are 17 available NAL unit types, of which the value of 0x06 represents the SEI type. The next byte after the header information is the payload type SEI payload type. When the value is 0x05, it means that the SEI payload analysis follows the user_data_unregistered() processing method, which literally means unregistered user data. The payload type is followed by the payload length SEI payload size, the length of the payload length is 1 or Nbytes, the protocol does not specify the length of the payload length, it can be defined by the user, and theoretically it can support infinite length. After the payload length is the Universal Unique Identifier UUID (Universally Unique Identifier). When setting the SEI payload to 5, 128bits (16 bytes) are allowed to specify the UUID. Here UUID can represent the role ID written to the SEI payload, or other business purposes. The remaining payloadSize-16 bytes is the specific content passed by the business layer. After the Universal Unique Identifier is metadata, the length of which is Nbytes. After the metadata is the end code endcode, which is 0x80. In the embodiment of the present application, the value of the SEI payload type can be set to 5, so that the network abstraction layer unit corresponding to the metadata is globally unique, does not conflict with the network abstraction layer unit of other SEI types, and improves compatibility.

In the embodiment of the present application, the data unit corresponding to the metadata includes: encoded data obtained by serializing the metadata, or encoded data obtained by serializing a target file in a predefined format ; the target file includes the metadata.

In an example, the step of encapsulating the metadata into the network abstraction layer unit of the SEI type may include: serializing the metadata to obtain encoded data; encapsulating the encoded data into The network abstraction layer unit of the SEI type.

Serialization is the process of converting an object or data structure into a sequence of bytes. By serializing the metadata, the metadata can be converted into binary encoded data, and this process can compress the metadata. The opposite of serialization is deserialization, which refers to the process of converting the byte sequence generated after serialization into an object or data structure. During the communication process, the data can be serialized by the sender and deserialized by the receiver to restore the data.

In another example, the step of encapsulating the metadata into the network abstraction layer unit of the SEI type may include:

Based on the metadata, a target file in a predefined format is generated; the target file is serialized to obtain encoded data; the encoded data is encapsulated into the SEI type network abstraction layer unit.

Among them, the target file in the predefined format can be a protobuf file, and Protobuf is an efficient and lightweight structured data storage method, which can be used for communication protocols, data storage, and the like. A protobuf file in a predefined format can be generated based on the metadata; the protobuf file is serialized by using a protobuf tool to obtain encoded data.

In an implementation manner, the protobuf file may include two, a base class file and a product class file. The basic class file is used to define and give some common messages (message) to achieve maximum reuse. The product file is a complete file, and each message in it is globally unique, independent of each other and not coupled to ensure that the messages are not redundant. In each basic class file, there can be a coordinated world UTC timestamp and a video frame number field, and the video frame number field is used to achieve global time synchronization of video frames and metadata.

The messages in the protobuf file can be defined according to actual business scenarios. For example, for drones, the messages may include: device identification messages, drone module messages, global messages, flight control messages, and the like. The content in the message can be configured according to the metadata.

203. Encode the data unit corresponding to the video data and the data unit corresponding to the metadata into a code stream and send to the terminal device.

By encoding the data unit corresponding to the video data and the data unit corresponding to the metadata into the code stream and sending it to the terminal device, the video data and the metadata can be transmitted synchronously, and no additional data synchronization processing is required for the metadata and the video data. . In the code stream, the data unit corresponding to the metadata is adjacent to the data unit corresponding to the video data. In an example, the data unit corresponding to each frame of video data and the data unit corresponding to the metadata obtained in the process of obtaining the frame of video data may be encoded into a code stream and sent to the terminal device, Each frame of video data can be naturally synchronized with the metadata, so that the terminal device receiving the code stream can perform other operations based on the synchronized video data and metadata. For example, flight teaching reproduction: after the drone completes the flight, the parameters of the drone can be obtained according to the returned code stream, and the parameters of the drone can be displayed to guide the newcomer to learn this action. Like a classic camera lens. Simulator flight simulation: The parameters of the drone in various environments can be stored, and playback or go-around can be performed in the simulator. Automatic flight go-around: After the drone completes the flight, it obtains video data and metadata according to the returned code stream, and performs accurate automatic flight and go-around based on the video data and metadata.

In this embodiment of the present application, the movable platform can encapsulate video data into data units based on video data encoding protocols, encapsulate metadata into data units based on video data encoding protocols, and encapsulate data units corresponding to video data and metadata corresponding to The data unit is encoded into the code stream and sent to the terminal device. The embodiments of the present application can enable the synchronous transmission of video data and metadata, improve data transmission efficiency, and do not need to perform additional data synchronization processing on metadata and video data.

This embodiment of the present application can support data return for code streams generated by different devices, and data return for multiple code streams. At present, the transmission process of audio and video data is generally: data acquisition equipment → YUV encoding → H264/H265 encoding → streaming media transmission based on H264/H265 encoding → user.

Referring to FIG. 5, FIG. 5 shows a schematic diagram of a data transmission process. Among them, the movable platform can be connected with an external video output device, and the external video output device can collect its own original data and generate metadata based on the original data, encode the metadata into binary data, and then convert the metadata in binary form to the collected data. The video data is encoded into the code stream and sent to the mobile platform. The movable platform can decode the code stream of the video output device to obtain YUV data. The mobile platform can collect its own original data and generate metadata based on the original data, encode the metadata into binary data, and encode its own binary metadata and decoded YUV data into code stream output.

Referring to FIG. 6, FIG. 6 shows a schematic diagram of another data transmission process. The movable platform can receive the YUV data output by the video output device. The mobile platform can collect its own original data and generate metadata based on the original data, encode the metadata into binary data, and encode its own binary metadata and YUV data output by the video output device into a stream output.

Referring to FIG. 7 , FIG. 7 shows a flowchart of a UAV data return process in an embodiment of the present application. 1. The client can communicate with the server, and the server can respond to the client's request to perform a video-on-demand process. The video-on-demand process can be the process of playing the video shot by the drone. 2. The drone can send the captured video data and real-time drone position to the remote controller; 3. The remote controller can obtain its own remote controller position, and can use the real-time drone position and remote controller position as metadata, Video data and metadata are streamed to the server. 5. The client can obtain video data and metadata from the server, and perform preset processing based on the video data and metadata.

Referring to FIG. 8, FIG. 8 shows a flowchart of a data processing method according to an embodiment of the present application, applied to a terminal device, wherein the terminal device and a movable platform, the method includes:

801. Receive a code stream sent by the movable platform.

In this embodiment of the present application, the terminal device may be a terminal or a server. For example, the terminal device may be a video service client, a command and dispatch client, a command and dispatch platform, and an image management platform.

The terminal device can communicate with multiple mobile platforms, can receive code streams sent by multiple mobile platforms at the transport layer, and perform various service processing according to the code streams.

802. Parse data units from the code stream based on a video data encoding protocol; at least one data unit in the data units includes metadata, where the metadata is used to indicate motion state information of the movable platform.

The mobile platform can encapsulate the metadata into the data unit based on the video data coding protocol, and the terminal device can parse the data unit obtained from the code stream, wherein at least one data unit contains the metadata.

803. Determine motion state information of the movable platform based on the metadata.

In this embodiment of the present application, a terminal device may receive a code stream sent by a mobile platform, and parse a data unit from the code stream based on a video data encoding protocol; at least one data unit in the data unit contains metadata, so The metadata is used to indicate motion state information of the movable platform; and the motion state information of the movable platform is determined based on the metadata. The technical solutions provided by the embodiments of the present application improve data transmission efficiency by placing data in a video transmission protocol-based code stream for transmission.

Referring to FIG. 9, FIG. 9 shows a flowchart of another data processing method provided by an embodiment of the present application, applied to a terminal device, wherein the terminal device and a movable platform, the method includes:

901. Receive a code stream sent by the movable platform.

902. Parse a data unit from the code stream based on a video data encoding protocol; the data unit includes a data unit corresponding to metadata and a data unit corresponding to video data.

In the embodiment of the present application, the movable platform can encode the data unit corresponding to the video data and the data unit corresponding to the metadata together into a code stream and send it to the terminal device, so that the video data and the metadata can be transmitted synchronously. After the terminal device parses and obtains the data unit corresponding to the video data and the data unit corresponding to the metadata from the code stream, it does not need to perform additional data synchronization processing on the metadata and the video data. The data unit corresponding to the metadata may have a time stamp, and the terminal device may globally synchronize the metadata according to the time stamp.

In this embodiment of the present application, the data unit may be a network abstraction layer unit, the type of the network abstraction layer unit includes the supplementary enhancement information SEI type, and the data unit based on the video data coding protocol is parsed from the code stream The step may include: parsing the network abstraction layer unit of the SEI type from the code stream to obtain the network abstraction layer unit corresponding to the metadata.

In one example, the mobile platform can serialize the metadata to obtain encoded data; and encapsulate the encoded data into a network abstraction layer unit of the SEI type. In this case, the step of extracting metadata from the data unit corresponding to the metadata may include: extracting target encoded data from the SEI type network abstraction layer unit; extracting target encoded data; Perform deserialization to get metadata.

In another example, the mobile platform may generate an object file in a predefined format based on the metadata; serialize the object file to obtain encoded data; and encapsulate the encoded data into an SEI-type network abstraction layer unit. In this case, the step of extracting metadata from the data unit corresponding to the metadata may include: extracting target encoded data from the SEI type network abstraction layer unit; extracting target encoded data; Deserialization is performed to obtain a target file in a predefined format; metadata is obtained by parsing the target file in the predefined format.

In an implementation manner, the target file in the predefined format may be a protobuf file, and the mobile platform may use a protobuf tool to serialize the protobuf file to obtain encoded data. The terminal device can use the protobuf tool to deserialize the target encoded data to obtain a protobuf file in a predefined format.

903. Perform preset processing based on the metadata and the video data.

The movable platform may extract metadata from data units corresponding to metadata, extract video data from data units corresponding to video data, and perform preset processing based on metadata and video data.

The preset processing may be to perform other operations based on the synchronized video data and metadata, for example, flight teaching reproduction, simulator flight simulation, automatic flight go-around. Since the video data and metadata are transmitted together in the code stream, the time error between the two is very small, and the terminal device can directly use the video data and metadata to perform preset processing without the need for time synchronization processing.

In this embodiment of the present application, the terminal device may receive the code stream sent by the mobile platform, parse the code stream to obtain the data unit storing metadata and the data unit storing video data, and extract the data unit corresponding to the metadata from the data unit. Metadata, extracts video data from data units corresponding to video data, thereby completing data return.

FIG. 10 shows a block diagram of a movable platform provided by an embodiment of the present application, and the movable platform may include:

The processor 1001 is configured to generate a code stream based on a video data encoding protocol, the code stream includes data units, and at least one data unit in the data units includes metadata, and the metadata is used to indicate the mobile platform. motion status information;

The communication interface 1002 is used for sending the code stream to the terminal device.

Optionally, the data unit is a network abstraction layer unit, and the type of the network abstraction layer unit corresponding to the metadata is a supplemental enhancement information SEI type.

Optionally, the data unit corresponding to the metadata includes: encoded data obtained by serializing the metadata, or encoded data obtained by serializing a target file in a predefined format; the The object file includes the metadata.

Optionally, the target file in the predefined format is a protobuf file in the predefined format.

Optionally, at least one of the data units includes video data.

Optionally, in the code stream, the data unit corresponding to the metadata is adjacent to the data unit corresponding to the video data.

Optionally, the processor is configured to encapsulate the video data into a data unit based on a video data encoding protocol; extract metadata from a preset buffer queue, and encapsulate the metadata into a data unit based on the video data encoding protocol; The data unit corresponding to the video data and the data unit corresponding to the metadata are encoded into the code stream.

Optionally, the data unit corresponding to the metadata has a time stamp.

FIG. 11 is a block diagram of a terminal device provided by an embodiment of the present application. The terminal device may include:

The communication interface 1101 is used to receive the code stream sent by the movable platform;

The processor 1102 is configured to parse out data units from the code stream based on a video data encoding protocol; at least one data unit in the data units includes metadata, and the metadata is used to indicate the movement of the movable platform state information; determining motion state information of the movable platform based on the metadata.

Optionally, at least one of the data units includes video data.

Optionally, the data unit corresponding to the metadata is adjacent to the data unit corresponding to the video data.

Optionally, the processor is further configured to perform preset processing based on the metadata and the video data.

Further, an embodiment of the present application also provides a chip, the chip includes a memory and a processor, and the processor executes a computer program stored in the memory to implement each step in the above data transmission method, and can achieve The same technical effect, in order to avoid repetition, will not be repeated here.

Further, an embodiment of the present application also provides a chip, the chip includes a memory and a processor, and the processor executes a computer program stored in the memory to implement the various steps in the above data processing method, and can achieve The same technical effect, in order to avoid repetition, will not be repeated here.

Further, the embodiments of the present application further provide a computer-readable storage medium, including instructions, which, when executed on a computer, cause the computer to execute the above-mentioned data transmission method. In order to avoid repetition, details are not repeated here.

Further, an embodiment of the present application further provides a computer-readable storage medium, which includes instructions, which, when executed on a computer, cause the computer to execute the above-mentioned data processing method. In order to avoid repetition, details are not repeated here.

The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the various embodiments may be referred to each other. Reference herein to "one embodiment," "an embodiment," or "one or more embodiments" means that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment of the present application. Also, please note that instances of the phrase "in one embodiment" herein are not necessarily all referring to the same embodiment. In the description provided herein, numerous specific details are set forth. It will be understood, however, that the embodiments of the present application may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, and third, etc. do not denote any order. These words can be interpreted as names. Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still be Modifications are made to the technical solutions described in the foregoing embodiments, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.

Claims

A data transmission method, applied to a movable platform capable of communicating with a terminal device, characterized in that the method comprises:

Generate a code stream based on a video data encoding protocol, the code stream includes data units, at least one of the data units includes metadata, and the metadata is used to indicate motion state information of the movable platform;

Send the code stream to the terminal device.
The method according to claim 1, wherein the data unit is a network abstraction layer unit, and the type of the network abstraction layer unit corresponding to the metadata is a supplemental enhancement information SEI type.
The method according to claim 2, wherein the data unit corresponding to the metadata comprises: encoded data obtained by serializing the metadata; or serializing a target file in a predefined format The encoded data obtained after processing, the target file includes the metadata.
The method according to claim 3, wherein the target file in the predefined format is a protobuf file in the predefined format.
The method of claim 1, wherein at least one of the data units includes video data.
The method according to claim 5, wherein, in the code stream, a data unit corresponding to the metadata is adjacent to a data unit corresponding to the video data.
The method according to claim 5, wherein the generating a code stream based on a video data encoding protocol comprises:

Encapsulate the video data into data units based on the video data coding protocol;

Extract metadata from a preset buffer queue, and encapsulate the metadata into a data unit based on a video data encoding protocol;

The data unit corresponding to the video data and the data unit corresponding to the metadata are encoded into the code stream.
The method according to claim 1, wherein the data unit corresponding to the metadata has a time stamp.
A data processing method, applied to a terminal device capable of communicating with a movable platform, characterized in that the method comprises:

receiving the code stream sent by the movable platform;

A data unit is parsed from the code stream based on a video data encoding protocol; at least one data unit in the data unit includes metadata, and the metadata is used to indicate motion state information of the movable platform;

Motion state information of the movable platform is determined based on the metadata.
The method according to claim 9, wherein the data unit is a network abstraction layer unit, and the type of the network abstraction layer unit corresponding to the metadata is a supplemental enhancement information SEI type.
The method according to claim 10, wherein the data unit corresponding to the metadata comprises: encoded data obtained by serializing the metadata, or serializing a target file in a predefined format The encoded data obtained after processing; the target file includes the metadata.
10. The method of claim 9, wherein at least one of the data units includes video data.
The method according to claim 12, wherein the data unit corresponding to the metadata is adjacent to the data unit corresponding to the video data.
A movable platform, characterized in that, comprising:

a processor, configured to generate a code stream based on a video data encoding protocol, the code stream includes data units, at least one of the data units contains metadata, and the metadata is used to indicate the movement of the movable platform status information;

The communication interface is used for sending the code stream to the terminal device.
The mobile platform according to claim 14, wherein the data unit is a network abstraction layer unit, and the type of the network abstraction layer unit corresponding to the metadata is a supplemental enhancement information SEI type.
The mobile platform according to claim 15, wherein the data unit corresponding to the metadata comprises: encoded data obtained by serializing the metadata, or a target file in a predefined format. The encoded data obtained after serialization; the target file includes the metadata.
The mobile platform according to claim 16, wherein the target file in the predefined format is a protobuf file in the predefined format.
15. The movable platform of claim 14, wherein at least one of the data units includes video data.
The movable platform according to claim 18, wherein, in the code stream, a data unit corresponding to the metadata is adjacent to a data unit corresponding to the video data.
The mobile platform according to claim 18, wherein the processor is configured to encapsulate the video data into a data unit based on a video data coding protocol; extract metadata from a preset buffer queue, and store the metadata Encapsulate into a data unit based on a video data encoding protocol; encode the data unit corresponding to the video data and the data unit corresponding to the metadata into a code stream.
The mobile platform according to claim 14, wherein the data unit corresponding to the metadata has a time stamp.
A terminal device, characterized in that it includes:

The communication interface is used to receive the code stream sent by the mobile platform;

a processor, configured to parse out data units from the code stream based on a video data encoding protocol; at least one data unit in the data units contains metadata, and the metadata is used to indicate the motion state of the movable platform information; determining motion state information of the movable platform based on the metadata.
The terminal device according to claim 22, wherein the data unit is a network abstraction layer unit, and the type of the network abstraction layer unit corresponding to the metadata is a supplementary enhancement information SEI type.
The terminal device according to claim 23, wherein the data unit corresponding to the metadata comprises: encoded data obtained by serializing the metadata, or serializing a target file in a predefined format The encoded data obtained after processing; the target file includes the metadata.
23. The terminal device of claim 22, wherein at least one of the data units includes video data.
The terminal device according to claim 25, wherein the data unit corresponding to the metadata is adjacent to the data unit corresponding to the video data.
A chip, characterized in that the chip includes a memory and a processor, and the processor executes a computer program stored in the memory to implement the data transmission method according to any one of claims 1-8.
A chip, characterized in that the chip includes a memory and a processor, and the processor executes a computer program stored in the memory to implement the data processing method according to any one of claims 9-13.
A computer-readable storage medium, characterized by comprising instructions, which, when executed on a computer, cause the computer to execute the data transmission method according to any one of claims 1-8.
A computer-readable storage medium, characterized by comprising instructions, which, when executed on a computer, cause the computer to execute the data processing method according to any one of claims 9-13.