WO2018107346A1 - Usb-based data transmission method and system thereof - Google Patents

Usb-based data transmission method and system thereof Download PDF

Info

Publication number
WO2018107346A1
WO2018107346A1 PCT/CN2016/109550 CN2016109550W WO2018107346A1 WO 2018107346 A1 WO2018107346 A1 WO 2018107346A1 CN 2016109550 W CN2016109550 W CN 2016109550W WO 2018107346 A1 WO2018107346 A1 WO 2018107346A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
transmitted
transmission
priority data
priority
Prior art date
Application number
PCT/CN2016/109550
Other languages
French (fr)
Chinese (zh)
Inventor
韩俊鹏
强飞莉
Original Assignee
天彩电子(深圳)有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 天彩电子(深圳)有限公司 filed Critical 天彩电子(深圳)有限公司
Priority to PCT/CN2016/109550 priority Critical patent/WO2018107346A1/en
Publication of WO2018107346A1 publication Critical patent/WO2018107346A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/38Information transfer, e.g. on bus

Definitions

  • the present invention relates to the technical field of video transmission, and more particularly to a USB-based data transmission method and system thereof.
  • the connection between various consumer Cameras and smart phones mainly uses Wi-Fi for video image data transmission, and mainly uses Wi-Fi/BLE/BT interfaces to implement an interactive control protocol framework between Camera and smart phones.
  • Wireless interfaces such as Wi-Fi/BLE/BT are commonly used in smartphones and consumer Camera products, and the excellent user experience and compatibility of wireless network access methods make these interfaces a device in the consumer industry.
  • the USB interface and UVC protocol are generally adopted to realize the application requirements of real-time reading of Camera video stream.
  • UVC is very popular in low-resolution and low-cost cameras. It can support the transmission of Camera image YUV uncompressed data, and in the 1.5 version of its protocol. It is already supported to support the transmission of Camera's H.264 compressed images.
  • Low-power communication methods such as BLE/BT are not suitable for large-volume communication such as audio and video streaming because of the low data transmission speed. They are only suitable for simple command interaction and inter-device transmission of small files. Therefore, the communication and interaction control protocol framework implemented by the interface based on BLE/BT does not include the function of transmitting audio and video streams.
  • Wi-Fi wireless communication technology has developed rapidly in recent years, communication speed has been greatly improved compared with the previous ones, but in the actual application of audio and video transmission, because of its transmission bandwidth limitation (generally reliable communication bandwidth is the highest 10Mbps or so), in actual use, it is generally used to transmit video with up to 720P and optimized 1080P resolution, for ultra-high definition video (such as 4K) and multi-channel HD video that require higher resolution and image quality. Transmission is still powerless.
  • the UVC protocol widely used in PCs and industrial applications is still mainly for low-resolution (generally no more than 640x480 resolution) low-cost Camera applications, although its 1.5 protocol can already support the transmission of H.264 data, but still not With universal application, the high version protocol has poor compatibility on various operating systems.
  • the biggest problem is that the UVC protocol was originally designed to be convenient and fast from a variety of low-cost USB.
  • the real-time image is read on the Camera, and there is no option to control the Camera and other information exchanges with the Camera.
  • the user of the protocol cannot customize the protocol.
  • USB-based data transmission method to realize real-time transmission of high-definition audio and video data, and simultaneously transmit multi-channel high-definition audio and video data, and realize communication and communication between the two parties while transmitting audio and video data.
  • the object of the present invention is to overcome the deficiencies of the prior art and to provide a data transmission method based on USB and a system thereof.
  • a USB-based data transmission method including the following steps:
  • Video data is sequentially transmitted in the order of defined high priority, medium priority, and low priority;
  • a further technical solution is: the step of defining a transmission priority of different data, where the transmission priority includes high priority data, medium priority data, and low priority data, where the high priority data is user operation and control Command-related exchange protocol information; medium-priority data is audio and video data transmitted in real time; low-priority data is file information uploaded and downloaded in the background.
  • a further technical solution is: the step of sequentially transmitting video data according to the defined high priority, medium priority, and low priority order, including the following specific steps:
  • the high priority data is transmitted preferentially, and the next step is entered, or if there is no high priority data to be transmitted, the process proceeds directly to the next step;
  • the medium priority data is transmitted, and the next step is entered, or if there is no medium priority data to be transmitted, the process proceeds directly to the next step;
  • the low priority data is transmitted, or if there is no low priority data to be transmitted, the end step is entered.
  • a further technical solution is: the step of correspondingly adjusting the transmission load and bandwidth of different channels, including the following specific steps:
  • a further technical solution is: if the medium priority data needs to be transmitted, the medium priority data is transmitted, and the next step is entered, or if there is no medium priority data to be transmitted, the process proceeds directly to the next step. If the medium priority data needs to be transmitted, before the step of transmitting the medium priority data, the method further includes:
  • Time stamping of input and output real-time audio data and video data is
  • a further technical solution is: the step of managing the time stamp of the input and output real-time audio data and the video data, including the following specific steps:
  • the timestamp is sent to the downstream link, and the downstream link uses the timestamp for data synchronization.
  • the invention also provides a USB-based data transmission system, comprising a defining unit, a scheduling unit, a management unit and an adjusting unit;
  • the definition unit is configured to define a transmission priority of different data
  • the scheduling unit is configured to sequentially transmit video data according to a defined high priority, medium priority, and low priority order;
  • the management unit is configured to perform statistics on data currently being transmitted, and calculate an instantaneous bandwidth requirement of each different channel;
  • the adjusting unit is configured to perform corresponding adjustment on transmission load and bandwidth of different channels.
  • the scheduling unit includes an advanced judging module, an advanced data transmission module, an intermediate judging module, an intermediate data transmission module, a low-level judging module, and a low-level data transmission module;
  • the advanced determining module is configured to determine whether high priority data needs to be transmitted
  • the advanced data transmission module is configured to preferentially transmit high priority data
  • the intermediate level determining module is configured to determine whether medium priority data needs to be transmitted
  • the intermediate data transmission module is configured to transmit medium priority data
  • the low-level judging module is configured to determine whether low-priority data needs to be transmitted
  • the low-level data transmission module is configured to transmit low-priority data.
  • the adjustment unit includes a quantity determination module and a processing module
  • the quantity determining module is configured to determine whether the quantity of the currently transmitted high priority data is greater than the set high priority data value
  • the processing module is configured to reduce the sending frequency and the packet size of the low priority data, or to increase the sending frequency and the packet size of the low priority data.
  • the adjustment unit further comprises an audio and video synchronization block for managing time stamps of the input and output real-time audio data and video data.
  • a USB-based data transmission method of the present invention by pre-defining the transmission priority of different data, and performing different types of data transmission according to different transmission priorities, and simultaneously Different channels are used to adjust the load and bandwidth to realize real-time transmission of high-definition audio and video data, and simultaneously transmit multi-channel high-definition audio and video data.
  • control communication, status information exchange, file upload and download operations are realized. It is convenient for users to customize the communication function.
  • FIG. 1 is a flow chart of a USB-based data transmission method according to an embodiment of the present invention.
  • FIG. 2 is a specific process block diagram for sequentially transmitting video data according to a defined high priority, medium priority, and low priority sequence according to a specific embodiment of the present invention
  • FIG. 3 is a specific process block diagram of adjusting corresponding transmission load and bandwidth of different channels according to an embodiment of the present invention
  • FIG. 5 is a structural block diagram of a USB transmission connection between a Camera device and a PC device according to an embodiment of the present invention
  • FIG. 6 is a structural block diagram of a USB transmission connection between a Camera device and an Android device according to an embodiment of the present invention
  • FIG. 7 is a structural block diagram of a USB transmission connection between a Camera device and an iPhone/iPad device according to an embodiment of the present invention
  • FIG. 8 is a structural block diagram of a USB-based data transmission system according to an embodiment of the present invention.
  • FIG. 9 is a structural block diagram of a scheduling unit according to an embodiment of the present invention.
  • FIG. 10 is a structural block diagram of an adjustment unit according to an embodiment of the present invention.
  • FIG. 11 is a structural block diagram of an audio and video synchronization module according to an embodiment of the present invention.
  • the USB-based data transmission method provided in this embodiment can be used for real-time transmission of different types of data between the Camera and the PC/smartphone, including but not limited to various consumptions.
  • Class Camera, VR Camera, industrial camera, etc. realize real-time transmission of high-definition audio and video data, and transmit multiple high-definition audio and video data at the same time, realize the control communication, status information exchange, file upload and download operations of both parties while transmitting audio and video data.
  • a USB-based data transmission method includes the following steps:
  • the step of defining a transmission priority of different data includes high priority data, medium priority data, and low priority data.
  • the high priority data here is the exchange protocol information related to the user operation control command; the medium priority data is the audio and video data transmitted in real time; the low priority data is the file information uploaded and downloaded in the background.
  • logical channels of different priorities are defined for transmitting the type of data, for example, three priorities of high, medium, and low on the logical channel, and if there is high priority data, the first High priority data is transmitted on high priority channels, and so on.
  • step S2 the steps of sequentially transmitting video data in the order of defined high priority, medium priority, and low priority include the following specific steps:
  • step S22 If high priority data needs to be transmitted, the high priority data is preferentially transmitted, and the process proceeds to step S23, or if there is no high priority data to be transmitted, the process directly proceeds to step S23.
  • step S25 transmitting the priority data, and proceeding to step S26, or if there is no medium priority data to be transmitted, proceeding directly to step S26;
  • step S24 the time stamps of the input and output real-time audio data and video data are managed to ensure accurate synchronization of audio and video data throughout the transmission process, and the S24 steps include the following steps. Specific steps:
  • S242 Send the timestamp to the downstream link, and use the timestamp to perform data synchronization processing on the downstream link.
  • step S241 the step of sorting the sequential transmission of the audio and video data according to the timestamp for encoding the audio and video data, encoding the data by using the audio and video coding system, and generating a timestamp according to the audio and video data.
  • the timestamp of the encoding where the timestamp is the start time and the end time of the data encoding, and the sequential transmission of the audio and video data is sorted according to the timestamp during transmission, thereby ensuring the orderly transmission of the data and avoiding the load during the transmission. Unbalanced, resulting in a smooth synchronization process.
  • step S242 the time stamp is sent to the downstream link, and the downstream link uses the time stamp to perform the process of data synchronization, where the downstream link includes transmitting the data.
  • step S21, S22, S23, S24, S25, S26 and S27 in the management and control of the USB data transmission channel, it is first determined whether there is high priority data to be transmitted, and if so, the high priority data is preferentially transmitted, such as If no, continue to judge whether there is medium priority data to be transmitted, determine low priority data in turn, and transmit low priority large file data without higher priority data transmission, which can flexibly adapt to various types of Communication requirements can not only respond to the user interaction type communication requirements, but also meet the stability of low priority large data volume transmission.
  • a single input and output channel is used to transmit and receive all data, and the data types sent and received are different, and the priorities are different.
  • high-priority data such as data related to user interaction
  • low-priority data such as background upload and download files
  • the step of calculating the instantaneous bandwidth requirement of each different channel for the data currently being transmitted is mainly for balancing management of the transmission load and bandwidth of each channel endpoint in the transmission process, so as to Each channel endpoint is scheduled to maximize bandwidth support and stability for USB transmission.
  • step S4 the steps of adjusting the transmission load and bandwidth of different channels include the following specific steps:
  • the step S41 is for determining whether the current high priority data is more, and adjusting according to the result of the judgment.
  • step S42 when the current high priority data is large, the transmission frequency and the packet size of the low priority data are reduced, thereby improving the real-time performance of the high priority data transmission and reception.
  • the transmission frequency and packet size of the low priority data are increased, that is, the high priority data is first transmitted first, and the low priority data can be transmitted in the background using a reasonable bandwidth as much as possible.
  • control and status information contains many different types: control and status information, real-time Live Streaming data, file upload and download, etc.
  • real-time Live Streaming data contains many different types: control and status information, real-time Live Streaming data, file upload and download, etc.
  • different ways of processing and response can be made according to different types of data.
  • the above data transmission method is completed based on USB, and considering the unidirectionality of USB endpoint communication, at least two sets of endpoints are needed for two-way communication to realize two-way full-duplex communication, that is, Host and USB communication.
  • the Device side needs at least one input endpoint and one output endpoint to communicate with the output endpoint and the input endpoint of the other end.
  • the specific communication protocol design and communication data organization can be based on different types of USB endpoints (Interrupt, Bulk, Control). , Isochronous) to select and match to adapt to different communication data transmission requirements.
  • the Camera is used as the Device end (drive end) of the USB communication
  • the PC/smartphone accesses the Camera device as the Host end (active end) of the USB communication
  • USB The communication and protocol implementation on the Device side is mainly carried out on the Camera
  • USB The communication and protocol implementation on the Host side is mainly placed on the PC, smart phone APP (Android, iOS), etc. Both of them interact with other parts of the whole system through control information and audio and video input and output.
  • the audio and video streaming media data and the transmission protocol framework for controlling the interactive information are encapsulated in the interface API library of the respective platforms, which is convenient for the third-party application.
  • the manufacturer implements streaming media data reading and control of the Camera based on the interface API.
  • the USB of the Camera device when the Camera device is connected to the USB device of the PC device, the USB of the Camera device The device is connected to the USB Host interface of the PC device, and the interface with the camera is implemented in the application by integrating the API API library of the PC end on the PC side.
  • the USB interface included in the ordinary Android phone is USB.
  • the Device interface so the Camera device needs to be connected to the USB interface of the Android smartphone via a USB (OTG) cable.
  • OTG USB
  • the interface API library of the Android side is integrated into the APP application to implement the interface with the Camera device.
  • the Lightning of the iPhone/iPad itself has a built-in USB.
  • Host's interface configuration mode designed and configured for the Lightning interface of iPhone/iPad on the interface of the Camera device, using lightning
  • the cable can be connected to the lightning interface of devices such as iPhone/iPad.
  • the interface API library of the iOS side is integrated into the APP application to implement interface control with the Camera device.
  • This embodiment performs data transmission based on USB, and has the following advantages:
  • USB transmission has the natural high transmission bandwidth capability.
  • USB2.0 has a maximum transmission bandwidth of 480Mbps, and USB3.0 can reach 5Gbps, which far exceeds the transmission speed that wireless transmission methods such as Wi-Fi can provide.
  • this embodiment only relies on the protocol framework of the USB underlying communication mechanism to be seamlessly compatible with different protocol versions of USB;
  • USB the existence of the USB interface is very extensive, whether in the emerging consumer electronics, smart phones, VR devices, or in the traditional PC, industrial cameras and other fields, USB as a convenient use to support a wide range of data transmission interfaces are basically The standard configuration of these devices makes it possible to provide audio and video data transmission capabilities in a variety of applications;
  • the smart phone APP and the Camera end the pre-defined communication protocol and the protocol extension interface are respectively implemented. Only the simple transplantation and the necessary interface implementation are needed in each communication terminal, and the entire USB video communication and control can be completed.
  • the technology integrates functions at both ends.
  • the above USB-based data transmission method performs different types of data transmission according to different transmission priorities by pre-defining transmission priority of different data, and performs load and bandwidth adjustment on different channels. Realize the transmission of real-time ultra-high-definition audio and video data, and simultaneously transmit multi-channel high-definition audio and video data, realize the control communication, status information exchange, file uploading and downloading operation of both parties while transmitting audio and video data, and facilitate users to customize the communication function. .
  • a data transmission system based on USB includes a definition unit 10, a scheduling unit 20, a management unit 30, and an adjustment unit 40;
  • the scheduling unit 20 is configured to sequentially transmit video data according to the defined high priority, medium priority, and low priority order;
  • the management unit 30 is configured to perform statistics on data currently being transmitted, and calculate an instantaneous bandwidth requirement of each different channel;
  • the adjusting unit 40 is configured to perform corresponding adjustments on transmission loads and bandwidths of different channels.
  • the scheduling unit 20 includes an advanced judging module 21, an advanced data transmission module 22, an intermediate judging module 23, an intermediate data transmission module 25, a low-level judging module 26, and a low-level data transmission module 27;
  • the advanced judging module 21 is configured to determine whether high priority data needs to be transmitted
  • the advanced data transmission module 22 is configured to preferentially transmit high priority data
  • the intermediate judging module 23 is configured to determine whether there is medium priority data to be transmitted
  • the intermediate data transmission module 25 is configured to transmit medium priority data
  • the low-level judging module 26 is configured to determine whether low-priority data needs to be transmitted
  • the low-level data transmission module 27 is configured to transmit low priority data.
  • the adjustment unit 40 includes a quantity determination module 41 and a processing module 42;
  • the quantity determining module 41 is configured to determine whether the quantity of the currently transmitted high priority data is greater than the set high priority data value
  • the processing module 42 is configured to reduce the transmission frequency and the packet size of the low priority data, or to increase the transmission frequency and the packet size of the low priority data.
  • the scheduling unit 20 includes an audio and video synchronization module 24 for managing time stamps of the input and output real-time audio data and video data.
  • the audio and video synchronization module 24 includes a sorting module 241 and a synchronization sub-module 242, and is configured to sort the sequential transmission of the audio and video data according to the timestamp of encoding the audio and video data;
  • the module 242 is configured to send the time stamp to the downstream link, and the downstream link uses the time stamp to perform the data synchronization process.
  • the transmission priority of different data is predefined by the definition unit 10, and the scheduling unit 20 performs different types of data transmission according to different transmission priorities, and the management unit 30 and the scheduling unit 20 combine.
  • Load and bandwidth adjustment for different channels real-time transmission of high-definition audio and video data, simultaneous transmission of multi-channel high-definition audio and video data, simultaneous transmission of audio and video data, control communication, status information exchange, file upload and download Operation, convenient for users to customize the communication function expansion.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Communication Control (AREA)

Abstract

The present invention relating to a USB-based data transmission method and a system thereof. The method comprises: defining transmission priorities of different data; sequentially transmitting video data in a defined order of high priority, medium priority, and low priority; counting the data which are being transmitted currently, and calculating the instantaneous bandwidth demands of various channels; and carrying out corresponding adjustments to the transmission loads and bandwidths of different channels. By means of predefining the transmission priorities of different data, different types of data transmission are performed according to the different transmission priorities; moreover, the loads and bandwidths of different channels are adjusted to achieve the transmission of real-time ultra-high-definition audio and video data, and also transmit multi-path high-definition audio and video data simultaneously. During the transmission of the audio and video data, communication control, state information exchange, file upload and download operations of both sides of the communication are achieved, thus facilitating a user expanding custom communication functions.

Description

一种基于USB的数据传输方法及其系统  USB-based data transmission method and system thereof
技术领域Technical field
本发明涉及视频传输的技术领域,更具体地说是指一种基于USB的数据传输方法及其系统。The present invention relates to the technical field of video transmission, and more particularly to a USB-based data transmission method and system thereof.
背景技术Background technique
目前各种消费类Camera与智能手机之间的连接主要采用Wi-Fi进行视频图像数据的传输,以及主要采用Wi-Fi/BLE/BT等接口来实现Camera和智能手机之间的交互控制协议框架,Wi-Fi/BLE/BT这类无线接口在智能手机和消费类Camera产品上普遍应用,而无线网络访问方式所具备的极佳的用户体验与兼容性也使得这些接口成为消费类行业设备之间对接的事实性标准。而在PC以及工业场合,为了简化Camera与类PC设备的对接,普遍采用了USB接口和UVC协议来实现实时读取Camera视频流的应用要求。作为一种USB协会所定义和推广的USB视频传输解决方案,UVC在低分辨率低成本的Camera中应用非常普遍,可以实现对Camera图像YUV无压缩数据的传输支持,而在其协议的1.5版本,已经可以支持对Camera的H.264压缩图像的传输提供支持。At present, the connection between various consumer Cameras and smart phones mainly uses Wi-Fi for video image data transmission, and mainly uses Wi-Fi/BLE/BT interfaces to implement an interactive control protocol framework between Camera and smart phones. Wireless interfaces such as Wi-Fi/BLE/BT are commonly used in smartphones and consumer Camera products, and the excellent user experience and compatibility of wireless network access methods make these interfaces a device in the consumer industry. The factual standard of interfacing. In PC and industrial applications, in order to simplify the docking of Camera and PC-like devices, the USB interface and UVC protocol are generally adopted to realize the application requirements of real-time reading of Camera video stream. As a USB video transmission solution defined and promoted by the USB Association, UVC is very popular in low-resolution and low-cost cameras. It can support the transmission of Camera image YUV uncompressed data, and in the 1.5 version of its protocol. It is already supported to support the transmission of Camera's H.264 compressed images.
BLE/BT这类低功耗的通信方式因为数据传输速度很低,不适合用于进行音视频流这类大数据量的通信,仅适合进行一些简单的命令交互和小文件的设备间传输,因此目前基于BLE/BT这种接口所实现的通信与交互控制协议框架中都不包含有传输音视频流这样的功能。Low-power communication methods such as BLE/BT are not suitable for large-volume communication such as audio and video streaming because of the low data transmission speed. They are only suitable for simple command interaction and inter-device transmission of small files. Therefore, the communication and interaction control protocol framework implemented by the interface based on BLE/BT does not include the function of transmitting audio and video streams.
Wi-Fi无线通信技术尽管在这几年发展迅速,通信速度较之前已有很大提升,但在实际的音视频传输的这类应用中,因为其传输带宽的限制(一般可靠通信带宽最高在10Mbps左右),在实际使用中一般用于传输最高720P以及码率经过优化的1080P分辨率的视频,对于需要更高分辨率和图像质量的超高清晰视频(如4K)以及多路高清视频的传输仍然无能为力。Although Wi-Fi wireless communication technology has developed rapidly in recent years, communication speed has been greatly improved compared with the previous ones, but in the actual application of audio and video transmission, because of its transmission bandwidth limitation (generally reliable communication bandwidth is the highest 10Mbps or so), in actual use, it is generally used to transmit video with up to 720P and optimized 1080P resolution, for ultra-high definition video (such as 4K) and multi-channel HD video that require higher resolution and image quality. Transmission is still powerless.
PC和工业场合所广泛使用的UVC协议主要面对的仍然是低分辨率(一般不超过640x480分辨率)低成本的Camera应用,尽管其1.5协议已经可以支持H.264数据的传输,但是仍然未能得到普遍的应用,高版本协议在各种不同操作系统上的兼容性很差。另外最大的问题是,UVC协议在最初设计时主要是考虑可以方便快捷的从各种低成本USB Camera上读取实时图像,没有提供控制Camera以及与Camera之间进行其他信息交互的选项,协议的使用者无法对该协议进行自定义的扩展。The UVC protocol widely used in PCs and industrial applications is still mainly for low-resolution (generally no more than 640x480 resolution) low-cost Camera applications, although its 1.5 protocol can already support the transmission of H.264 data, but still not With universal application, the high version protocol has poor compatibility on various operating systems. The biggest problem is that the UVC protocol was originally designed to be convenient and fast from a variety of low-cost USB. The real-time image is read on the Camera, and there is no option to control the Camera and other information exchanges with the Camera. The user of the protocol cannot customize the protocol.
综合以上存在的问题,有必要设计一种基于USB的数据传输方法,实现传输实时极高清音视频数据,同时传输多路高清音视频数据,在传输音视频数据的同时实现通信双方的控制通信、状态信息交换、文件上传下载操作。In order to solve the above problems, it is necessary to design a USB-based data transmission method to realize real-time transmission of high-definition audio and video data, and simultaneously transmit multi-channel high-definition audio and video data, and realize communication and communication between the two parties while transmitting audio and video data. Status information exchange, file upload and download operations.
发明内容Summary of the invention
本发明的目的在于克服现有技术的缺陷,提供一种基于USB的数据传输方法及其系统。The object of the present invention is to overcome the deficiencies of the prior art and to provide a data transmission method based on USB and a system thereof.
为实现上述目的,本发明采用以下技术方案:一种基于USB的数据传输方法,包括以下步骤:To achieve the above objective, the present invention adopts the following technical solution: a USB-based data transmission method, including the following steps:
定义不同数据的传输优先级;Define the transmission priority of different data;
按照定义的高优先级、中优先级以及低优先级的顺序依次传输视频数据;Video data is sequentially transmitted in the order of defined high priority, medium priority, and low priority;
对于当前正在传输的数据进行统计,计算出各个不同信道的即时带宽需求;Perform statistics on the data currently being transmitted, and calculate the instantaneous bandwidth requirements of different channels;
对不同信道的传输负荷和带宽做出对应的调节。Corresponding adjustments are made to the transmission load and bandwidth of different channels.
其进一步技术方案为:所述定义不同数据的传输优先级的步骤,所述传输优先级包括高优先级数据、中优先级数据以及低优先级数据,所述高优先级数据为用户操作与控制命令相关的交换协议信息;中优先级数据为实时传输的音视频数据;低优先级数据为后台进行上传下载的文件信息。A further technical solution is: the step of defining a transmission priority of different data, where the transmission priority includes high priority data, medium priority data, and low priority data, where the high priority data is user operation and control Command-related exchange protocol information; medium-priority data is audio and video data transmitted in real time; low-priority data is file information uploaded and downloaded in the background.
其进一步技术方案为:所述按照定义的高优先级、中优先级以及低优先级的顺序依次传输视频数据的步骤,包括以下具体步骤:A further technical solution is: the step of sequentially transmitting video data according to the defined high priority, medium priority, and low priority order, including the following specific steps:
判断是否有高优先级数据需要传输;Determine if there is high priority data to be transmitted;
若有高优先级数据需要传输,则优先传输高优先级数据,并进入下一步骤,或者,若没有高优先级数据需要传输,则直接进入下一步骤;If there is high priority data to be transmitted, the high priority data is transmitted preferentially, and the next step is entered, or if there is no high priority data to be transmitted, the process proceeds directly to the next step;
判断是否有中优先级数据需要传输;Determine whether there is medium priority data to be transmitted;
若有中优先级数据需要传输,则传输中优先级数据,并进入下一步骤,或者,若没有中优先级数据需要传输,则直接进入下一步骤;If there is medium priority data to be transmitted, the medium priority data is transmitted, and the next step is entered, or if there is no medium priority data to be transmitted, the process proceeds directly to the next step;
判断是否有低优先级数据需要传输;Determine if there is low priority data to be transmitted;
若有低优先级数据需要传输,则传输低优先级数据,或者,若没有低优先级数据需要传输,则进入结束步骤。If there is low priority data to be transmitted, the low priority data is transmitted, or if there is no low priority data to be transmitted, the end step is entered.
其进一步技术方案为:所述对不同信道的传输负荷和带宽做出对应的调节的步骤,包括以下具体步骤:A further technical solution is: the step of correspondingly adjusting the transmission load and bandwidth of different channels, including the following specific steps:
判断当前传输的高优先级数据的数量是否大于设定的高优先级数据值;Determining whether the number of currently transmitted high priority data is greater than a set high priority data value;
若是,则降低低优先级数据的发送频率和包大小;If yes, reduce the transmission frequency and packet size of the low priority data;
若不是,则提高低优先级数据的发送频率和包大小。If not, the transmission frequency and packet size of the low priority data are increased.
其进一步技术方案为:所述若有中优先级数据需要传输,则传输中优先级数据,并进入下一步骤,或者,若没有中优先级数据需要传输,则直接进入下一步骤中,所述若有中优先级数据需要传输,则传输中优先级数据的步骤之前,还包括:A further technical solution is: if the medium priority data needs to be transmitted, the medium priority data is transmitted, and the next step is entered, or if there is no medium priority data to be transmitted, the process proceeds directly to the next step. If the medium priority data needs to be transmitted, before the step of transmitting the medium priority data, the method further includes:
对输入输出的实时音频数据和视频数据的时间戳进行管理。Time stamping of input and output real-time audio data and video data.
其进一步技术方案为:所述对输入输出的实时音频数据和视频数据的时间戳进行管理的步骤,包括以下具体步骤:A further technical solution is: the step of managing the time stamp of the input and output real-time audio data and the video data, including the following specific steps:
根据对音视频数据进行编码的时间戳,对音视频数据的先后发送进行排序;Sorting the sequential transmission of audio and video data according to the timestamp of encoding the audio and video data;
将时间戳发给下游环节,下游环节使用时间戳进行数据同步的处理。The timestamp is sent to the downstream link, and the downstream link uses the timestamp for data synchronization.
本发明还提供了一种基于USB的数据传输系统,包括定义单元、调度单元、管理单元以及调节单元;The invention also provides a USB-based data transmission system, comprising a defining unit, a scheduling unit, a management unit and an adjusting unit;
所述定义单元,用于定义不同数据的传输优先级;The definition unit is configured to define a transmission priority of different data;
所述调度单元,用于按照定义的高优先级、中优先级以及低优先级的顺序依次传输视频数据;The scheduling unit is configured to sequentially transmit video data according to a defined high priority, medium priority, and low priority order;
所述管理单元,用于对于当前正在传输的数据进行统计,计算出各个不同信道的即时带宽需求;The management unit is configured to perform statistics on data currently being transmitted, and calculate an instantaneous bandwidth requirement of each different channel;
所述调节单元,用于对不同信道的传输负荷和带宽做出对应的调节。The adjusting unit is configured to perform corresponding adjustment on transmission load and bandwidth of different channels.
其进一步技术方案为:所述调度单元包括高级判断模块、高级数据传输模块、中级判断模块、中级数据传输模块、低级判断模块以及低级数据传输模块;The further technical solution is that the scheduling unit includes an advanced judging module, an advanced data transmission module, an intermediate judging module, an intermediate data transmission module, a low-level judging module, and a low-level data transmission module;
所述高级判断模块,用于判断是否有高优先级数据需要传输;The advanced determining module is configured to determine whether high priority data needs to be transmitted;
所述高级数据传输模块,用于优先传输高优先级数据; The advanced data transmission module is configured to preferentially transmit high priority data;
所述中级判断模块,用于判断是否有中优先级数据需要传输;The intermediate level determining module is configured to determine whether medium priority data needs to be transmitted;
所述中级数据传输模块,用于传输中优先级数据;The intermediate data transmission module is configured to transmit medium priority data;
所述低级判断模块,用于判断是否有低优先级数据需要传输;The low-level judging module is configured to determine whether low-priority data needs to be transmitted;
所述低级数据传输模块,用于传输低优先级数据。The low-level data transmission module is configured to transmit low-priority data.
其进一步技术方案为:所述调节单元包括数量判断模块以及处理模块;A further technical solution is that the adjustment unit includes a quantity determination module and a processing module;
所述数量判断模块,用于判断当前传输的高优先级数据的数量是否大于设定的高优先级数据值;The quantity determining module is configured to determine whether the quantity of the currently transmitted high priority data is greater than the set high priority data value;
所述处理模块,用于降低低优先级数据的发送频率和包大小,或者,提高低优先级数据的发送频率和包大小。The processing module is configured to reduce the sending frequency and the packet size of the low priority data, or to increase the sending frequency and the packet size of the low priority data.
其进一步技术方案为:所述调节单元还包括音视频同步块,用于对输入输出的实时音频数据和视频数据的时间戳进行管理。A further technical solution is that the adjustment unit further comprises an audio and video synchronization block for managing time stamps of the input and output real-time audio data and video data.
本发明与现有技术相比的有益效果是:本发明的一种基于USB的数据传输方法,通过预先定义不同数据的传输优先级,根据不同的传输优先级进行不同类型的数据传输,同时对不同的信道进行负载和带宽的调节,实现传输实时极高清音视频数据,同时传输多路高清音视频数据,在传输音视频数据的同时实现通信双方的控制通信、状态信息交换、文件上传下载操作,方便用户进行自定义通信功能扩展。The beneficial effects of the present invention compared with the prior art are: a USB-based data transmission method of the present invention, by pre-defining the transmission priority of different data, and performing different types of data transmission according to different transmission priorities, and simultaneously Different channels are used to adjust the load and bandwidth to realize real-time transmission of high-definition audio and video data, and simultaneously transmit multi-channel high-definition audio and video data. At the same time of transmitting audio and video data, control communication, status information exchange, file upload and download operations are realized. It is convenient for users to customize the communication function.
下面结合附图和具体实施例对本发明作进一步描述。The invention is further described below in conjunction with the drawings and specific embodiments.
附图说明DRAWINGS
图1为本发明具体实施例提供的一种基于USB的数据传输方法的流程框图;1 is a flow chart of a USB-based data transmission method according to an embodiment of the present invention;
图2为本发明具体实施例提供的按照定义的高优先级、中优先级以及低优先级的顺序依次传输视频数据的具体流程框图;2 is a specific process block diagram for sequentially transmitting video data according to a defined high priority, medium priority, and low priority sequence according to a specific embodiment of the present invention;
图3为本发明具体实施例提供的对不同信道的传输负荷和带宽做出对应的调节的具体流程框图;3 is a specific process block diagram of adjusting corresponding transmission load and bandwidth of different channels according to an embodiment of the present invention;
图4为本发明具体实施例提供的对输入输出的音频数据和视频数据的时间戳进行管理的具体流程框图;4 is a specific flow chart of managing time stamps of input and output audio data and video data according to an embodiment of the present invention;
图5为本发明具体实施例提供的Camera设备与PC设备的USB传输连接的结构框图;FIG. 5 is a structural block diagram of a USB transmission connection between a Camera device and a PC device according to an embodiment of the present invention; FIG.
图6为本发明具体实施例提供的Camera设备与Android设备的USB传输连接的结构框图;FIG. 6 is a structural block diagram of a USB transmission connection between a Camera device and an Android device according to an embodiment of the present invention;
图7为本发明具体实施例提供的Camera设备与iPhone/iPad设备的USB传输连接的结构框图;7 is a structural block diagram of a USB transmission connection between a Camera device and an iPhone/iPad device according to an embodiment of the present invention;
图8为本发明具体实施例提供的一种基于USB的数据传输系统的结构框图;FIG. 8 is a structural block diagram of a USB-based data transmission system according to an embodiment of the present invention; FIG.
图9为本发明具体实施例提供的调度单元的结构框图;FIG. 9 is a structural block diagram of a scheduling unit according to an embodiment of the present invention;
图10为本发明具体实施例提供的调节单元的结构框图;FIG. 10 is a structural block diagram of an adjustment unit according to an embodiment of the present invention;
图11为本发明具体实施例提供的音视频同步模块的结构框图。FIG. 11 is a structural block diagram of an audio and video synchronization module according to an embodiment of the present invention.
具体实施方式detailed description
为了更充分理解本发明的技术内容,下面结合具体实施例对本发明的技术方案进一步介绍和说明,但不局限于此。In order to more fully understand the technical content of the present invention, the technical solutions of the present invention are further described and illustrated in conjunction with the specific embodiments, but are not limited thereto.
如图1~11所示的具体实施例,本实施例提供的一种基于USB的数据传输方法,可以运用在Camera与PC/智能手机之间不同类型数据实时传输,包括但不限于各种消费类Camera、VR camera、工业相机等,实现传输实时极高清音视频数据,同时传输多路高清音视频数据,在传输音视频数据的同时实现通信双方的控制通信、状态信息交换、文件上传下载等操作。As shown in FIG. 1 to FIG. 11 , the USB-based data transmission method provided in this embodiment can be used for real-time transmission of different types of data between the Camera and the PC/smartphone, including but not limited to various consumptions. Class Camera, VR Camera, industrial camera, etc., realize real-time transmission of high-definition audio and video data, and transmit multiple high-definition audio and video data at the same time, realize the control communication, status information exchange, file upload and download operations of both parties while transmitting audio and video data.
如图1所示,一种基于USB的数据传输方法,包括以下步骤:As shown in FIG. 1, a USB-based data transmission method includes the following steps:
S1、定义不同数据的传输优先级;S1, defining a transmission priority of different data;
S2、按照定义的高优先级、中优先级以及低优先级的顺序依次传输视频数据;S2. sequentially transmitting video data according to the defined high priority, medium priority, and low priority order;
S3、对于当前正在传输的数据进行统计,计算出各个不同信道的即时带宽需求;S3. Perform statistics on data currently being transmitted, and calculate instantaneous bandwidth requirements of different channels;
S4、对不同信道的传输负荷和带宽做出对应的调节。S4. Perform corresponding adjustment on transmission load and bandwidth of different channels.
上述的S1步骤,所述定义不同数据的传输优先级的步骤,所述传输优先级包括高优先级数据、中优先级数据以及低优先级数据。In the above step S1, the step of defining a transmission priority of different data, the transmission priority includes high priority data, medium priority data, and low priority data.
这里的高优先级数据为用户操作与控制命令相关的交换协议信息;中优先级数据为实时传输的音视频数据;低优先级数据为后台进行上传下载的文件信息。The high priority data here is the exchange protocol information related to the user operation control command; the medium priority data is the audio and video data transmitted in real time; the low priority data is the file information uploaded and downloaded in the background.
因为传输数据类型的多样性,通信的数据量以及不同类型的通信数据对于传输的实时性、传输速度、传输带宽等有不同的需求,在具体的传输上面,尽管使用的是同一个通信通道,但是对于不同优先级的数据定义了各种不同优先级的逻辑信道,用于传输该种类型的数据,例如在逻辑信道上分为高中低三个优先级,如果有高优先级的数据就首先在高优先级的信道上传输高优先级数据,依次类推。Because of the diversity of transmission data types, the amount of data communicated and the different types of communication data have different requirements for real-time transmission, transmission speed, transmission bandwidth, etc., in the specific transmission, although the same communication channel is used, However, for different priority data, logical channels of different priorities are defined for transmitting the type of data, for example, three priorities of high, medium, and low on the logical channel, and if there is high priority data, the first High priority data is transmitted on high priority channels, and so on.
更进一步的,如图2所示,S2步骤,按照定义的高优先级、中优先级以及低优先级的顺序依次传输视频数据的步骤,包括以下具体步骤:Further, as shown in FIG. 2, in step S2, the steps of sequentially transmitting video data in the order of defined high priority, medium priority, and low priority include the following specific steps:
S21、判断是否有高优先级数据需要传输;S21. Determine whether high priority data needs to be transmitted;
S22、若有高优先级数据需要传输,则优先传输高优先级数据,并进入S23步骤,或者,若没有高优先级数据需要传输,则直接进入S23步骤;S22. If high priority data needs to be transmitted, the high priority data is preferentially transmitted, and the process proceeds to step S23, or if there is no high priority data to be transmitted, the process directly proceeds to step S23.
S23、判断是否有中优先级数据需要传输;S23. Determine whether there is medium priority data to be transmitted;
S24、若有中优先级数据需要传输,则对输入输出的实时音频数据和视频数据的时间戳进行管理;S24. If medium priority data needs to be transmitted, manage time stamps of input and output real-time audio data and video data;
S25、传输中优先级数据,并进入S26步骤,或者,若没有中优先级数据需要传输,则直接进入S26步骤;S25: transmitting the priority data, and proceeding to step S26, or if there is no medium priority data to be transmitted, proceeding directly to step S26;
S26、判断是否有低优先级数据需要传输;S26. Determine whether there is low priority data to be transmitted.
S27、若有低优先级数据需要传输,则传输低优先级数据或者,若没有低优先级数据需要传输,则进入结束步骤。S27. If low priority data needs to be transmitted, the low priority data is transmitted or if there is no low priority data to be transmitted, the end step is entered.
更进一步的,如图3所示,S24步骤,对输入输出的实时音频数据和视频数据的时间戳进行管理,是为了确保在整个传输过程中音频和视频数据的精确同步,该S24步骤包括以下具体步骤:Further, as shown in FIG. 3, in step S24, the time stamps of the input and output real-time audio data and video data are managed to ensure accurate synchronization of audio and video data throughout the transmission process, and the S24 steps include the following steps. Specific steps:
S241、根据对音视频数据进行编码的时间戳,对音视频数据的先后发送进行排序;S241. Sort the sequential transmission of the audio and video data according to the timestamp encoded by the audio and video data.
S242、将时间戳发给下游环节,下游环节使用时间戳进行数据同步的处理。S242: Send the timestamp to the downstream link, and use the timestamp to perform data synchronization processing on the downstream link.
上述的S241的步骤,根据对音视频数据进行编码的时间戳,对音视频数据的先后发送进行排序的步骤,使用音视频编码系统对数据进行编码,并产生时间戳,根据对音视频数据进行编码的时间戳,这里的时间戳是数据编码的开始时间以及结束时间,在传输的时候根据时间戳对音视频数据的先后发送进行排序,从而保证数据的有序传输,避免传输过程中的负载不均衡,而导致同步过程的不顺畅。In the step S241, the step of sorting the sequential transmission of the audio and video data according to the timestamp for encoding the audio and video data, encoding the data by using the audio and video coding system, and generating a timestamp according to the audio and video data. The timestamp of the encoding, where the timestamp is the start time and the end time of the data encoding, and the sequential transmission of the audio and video data is sorted according to the timestamp during transmission, thereby ensuring the orderly transmission of the data and avoiding the load during the transmission. Unbalanced, resulting in a smooth synchronization process.
上述的S242步骤,将时间戳发给下游环节,下游环节使用时间戳进行数据同步的处理的步骤,这里的下游环节包括对数据进行传输。In the above step S242, the time stamp is sent to the downstream link, and the downstream link uses the time stamp to perform the process of data synchronization, where the downstream link includes transmitting the data.
上述的S21、S22、S23、S24、S25、S26以及S27步骤,在USB数据传输通道的管理和控制上,首先判断是否有高优先级数据需要传输,如有则优先传输高优先级数据,如无则继续判断是否有中优先级数据需要传输,依次判断低优先级数据,在没有更高优先级数据传输的情况下,传输低优先级的大文件数据,可以灵活的适应各种不同类型的通信需求,既能够对用户交互类通信需求做出最快的响应,又可以满足低优先级大数据量传输的稳定性。In the above steps S21, S22, S23, S24, S25, S26 and S27, in the management and control of the USB data transmission channel, it is first determined whether there is high priority data to be transmitted, and if so, the high priority data is preferentially transmitted, such as If no, continue to judge whether there is medium priority data to be transmitted, determine low priority data in turn, and transmit low priority large file data without higher priority data transmission, which can flexibly adapt to various types of Communication requirements can not only respond to the user interaction type communication requirements, but also meet the stability of low priority large data volume transmission.
并且,使用了单个输入和输出通道对所有数据进行收发,收发的数据类型不同,优先级各自不同。对于高优先级的数据,如与用户交互相关的数据先发送,对于低优先级数据,如后台上传下载文件的数据在没有高优先级数据要发送的情况下再发送。Moreover, a single input and output channel is used to transmit and receive all data, and the data types sent and received are different, and the priorities are different. For high-priority data, such as data related to user interaction, the data for low-priority data, such as background upload and download files, is sent without high-priority data to be sent.
另外,对于S3的步骤,对于当前正在传输的数据进行统计,计算出各个不同信道的即时带宽需求的步骤,主要是为了在传输过程中对各个信道端点的传输负荷和带宽进行均衡管理,以便对各个信道端点进行调度,最大程度上提升USB传输的带宽支持和稳定性。In addition, for the step of S3, the step of calculating the instantaneous bandwidth requirement of each different channel for the data currently being transmitted is mainly for balancing management of the transmission load and bandwidth of each channel endpoint in the transmission process, so as to Each channel endpoint is scheduled to maximize bandwidth support and stability for USB transmission.
更进一步的,如图4所示,所述S4步骤,对不同信道的传输负荷和带宽做出对应的调节的步骤,包括以下具体步骤:Further, as shown in FIG. 4, in step S4, the steps of adjusting the transmission load and bandwidth of different channels include the following specific steps:
S41、判断当前传输的高优先级数据的数量是否大于设定的高优先级数据值;S41. Determine whether the quantity of the currently transmitted high priority data is greater than the set high priority data value.
S42、若是,则降低低优先级数据的发送频率和包大小;S42. If yes, reduce the sending frequency and packet size of the low priority data.
S43、若不是,则提高低优先级数据的发送频率和包大小。S43. If not, increase the transmission frequency and packet size of the low priority data.
S41步骤是用于判断当前的高优先级数据是否较多,并根据判断的结果来进行调节。The step S41 is for determining whether the current high priority data is more, and adjusting according to the result of the judgment.
S42步骤是在当前高优先级数据较多时,降低低优先级数据的发送频率和包大小,以此来为高优先级数据的收发提高实时性。In step S42, when the current high priority data is large, the transmission frequency and the packet size of the low priority data are reduced, thereby improving the real-time performance of the high priority data transmission and reception.
反之则提高低优先级数据的发送频率和包大小,即首先满足高优先级数据先发送,同时又保证低优先级数据能够在后台尽可能使用合理的带宽来传输数据。Conversely, the transmission frequency and packet size of the low priority data are increased, that is, the high priority data is first transmitted first, and the low priority data can be transmitted in the background using a reasonable bandwidth as much as possible.
若通信的两端所传输的数据包含多种不同类型:控制及状态信息、实时Live Streaming数据、文件上传下载等,通信过程中可以根据不同类型的数据做出不同方式的处理和应对。If the data transmitted at both ends of the communication contains many different types: control and status information, real-time Live Streaming data, file upload and download, etc., in the process of communication, different ways of processing and response can be made according to different types of data.
上述的数据传输方法是基于USB来完成的,而考虑到USB端点通信的单向性,对于通信的两端而言,至少需要两组端点来实现双向全双工通信,即USB通信的Host和Device端至少需要一个输入端点和一个输出端点来与对方的输出端点和输入端点进行连接通信,在具体的通信协议设计和通信数据的组织上,可基于USB端点的不同类型(Interrupt,Bulk,Control,Isochronous)来进行选择和搭配来适应不同通信数据传输的要求。The above data transmission method is completed based on USB, and considering the unidirectionality of USB endpoint communication, at least two sets of endpoints are needed for two-way communication to realize two-way full-duplex communication, that is, Host and USB communication. The Device side needs at least one input endpoint and one output endpoint to communicate with the output endpoint and the input endpoint of the other end. The specific communication protocol design and communication data organization can be based on different types of USB endpoints (Interrupt, Bulk, Control). , Isochronous) to select and match to adapt to different communication data transmission requirements.
具体的,在本实施例中,Camera作为USB通信的Device端(驱动端),PC/智能手机等访问Camera设备作为USB通信的Host端(主动端),USB Device端上的通信及协议实现主要在Camera上开展,而USB Host端上的通信及协议实现主要放在PC、智能手机APP(Android,iOS)等,以上两者均通过控制信息与音视频输入输出与整个系统中的其他部分进行交互。Specifically, in this embodiment, the Camera is used as the Device end (drive end) of the USB communication, and the PC/smartphone accesses the Camera device as the Host end (active end) of the USB communication, USB The communication and protocol implementation on the Device side is mainly carried out on the Camera, and USB The communication and protocol implementation on the Host side is mainly placed on the PC, smart phone APP (Android, iOS), etc. Both of them interact with other parts of the whole system through control information and audio and video input and output.
更进一步的,针对与PC、Android手机、iPhone等的连接,在具体方案实现上,把音视频流媒体数据以及控制交互信息的传输协议框架封装在各自平台的接口API库中,方便第三方应用厂商基于接口API实现流媒体数据的读取和对Camera进行控制。Further, for the connection with PC, Android mobile phone, iPhone, etc., in the implementation of the specific scheme, the audio and video streaming media data and the transmission protocol framework for controlling the interactive information are encapsulated in the interface API library of the respective platforms, which is convenient for the third-party application. The manufacturer implements streaming media data reading and control of the Camera based on the interface API.
如图5所示,Camera设备与PC设备的USB传输连接时,Camera设备的USB device连接到PC设备的USB Host接口上,在PC端的通过整合PC端的接口API库到应用程序中实现与camera的接口。As shown in Figure 5, when the Camera device is connected to the USB device of the PC device, the USB of the Camera device The device is connected to the USB Host interface of the PC device, and the interface with the camera is implemented in the application by integrating the API API library of the PC end on the PC side.
如图6所示,Camera设备与Android设备的USB传输连接时,由于普通的Android手机上所包含的USB接口为USB Device接口,因此Camera设备需要通过USB(OTG)线连接到Android智能手机的USB接口上。同样通过在Android App的开发中整合Android端的接口API库到APP应用程序中实现与Camera设备的接口。As shown in Figure 6, when the Camera device is connected to the USB device of the Android device, the USB interface included in the ordinary Android phone is USB. The Device interface, so the Camera device needs to be connected to the USB interface of the Android smartphone via a USB (OTG) cable. Same through on Android In the development of the app, the interface API library of the Android side is integrated into the APP application to implement the interface with the Camera device.
如图7所示,Camera设备与iPhone/iPad等设备的USB传输连接时,iPhone/iPad的Lightning本身内置了USB Host的接口配置模式,在Camera设备的接口上针对iPhone/iPad的Lightning接口进行设计和配置,使用lightning cable即可连接到iPhone/iPad等设备的lightning接口上。同样通过在iOS App的开发中整合iOS端的接口API库到APP应用程序中实现与Camera设备的接口控制。As shown in Figure 7, when the Camera device is connected to a USB device such as an iPhone/iPad, the Lightning of the iPhone/iPad itself has a built-in USB. Host's interface configuration mode, designed and configured for the Lightning interface of iPhone/iPad on the interface of the Camera device, using lightning The cable can be connected to the lightning interface of devices such as iPhone/iPad. Also by iOS In the development of the app, the interface API library of the iOS side is integrated into the APP application to implement interface control with the Camera device.
本实施例基于USB进行数据传输,具有以下的优点:This embodiment performs data transmission based on USB, and has the following advantages:
1、USB传输天然具备的高传输带宽的能力,USB2.0最高传输带宽达到480Mbps,USB3.0可以达到了5Gbps,既远远超过了Wi-Fi等无线传输方式所能够提供的传输速度,也完全能够满足多路极高分辨率压缩视频同时传输的应用要求,本实施例仅依赖于USB底层通信机制的协议框架可以无缝兼容USB的不同协议版本;1. USB transmission has the natural high transmission bandwidth capability. USB2.0 has a maximum transmission bandwidth of 480Mbps, and USB3.0 can reach 5Gbps, which far exceeds the transmission speed that wireless transmission methods such as Wi-Fi can provide. Fully capable of meeting the application requirements of simultaneous transmission of multiple extremely high resolution compressed video, this embodiment only relies on the protocol framework of the USB underlying communication mechanism to be seamlessly compatible with different protocol versions of USB;
2、基于USB标准定义的四种基本通信类型(Interupt,Bulk,Control,Isochronous),针对不同优先级的数据通信需求提供了灵活稳定的支持,在同一个通信信道上既可以传输数据量大优先级较低文件上传下载数据,也可以传输优先级较高的控制类交互协议,方便用户进行自定义通信功能扩展; 2. Based on the four basic communication types (Interupt, Bulk, Control, Isochronous) defined by the USB standard, it provides flexible and stable support for data communication needs of different priorities, and can transmit data with a large priority on the same communication channel. The lower-level file uploading and downloading data can also transmit a higher-priority control type interactive protocol, which is convenient for the user to perform custom communication function expansion;
3、USB接口的存在非常广泛,无论是在新兴的消费类电子、智能手机、VR设备,还是在传统的PC、工业相机等领域,USB作为一种使用方便支持广泛的数据传输接口基本都是这些设备的标准配置,因此可以在多种应用场合采用这种方案来提供音视频数据的传输能力;3, the existence of the USB interface is very extensive, whether in the emerging consumer electronics, smart phones, VR devices, or in the traditional PC, industrial cameras and other fields, USB as a convenient use to support a wide range of data transmission interfaces are basically The standard configuration of these devices makes it possible to provide audio and video data transmission capabilities in a variety of applications;
4、针对PC设备、智能手机APP以及Camera端分别实现了预先定义的通信协议以及协议扩展接口,在各个通信端只需要进行简单的移植和必要的接口实现,即可完成整个USB视频通信与控制的技术在两端的功能整合。4. For the PC device, the smart phone APP and the Camera end, the pre-defined communication protocol and the protocol extension interface are respectively implemented. Only the simple transplantation and the necessary interface implementation are needed in each communication terminal, and the entire USB video communication and control can be completed. The technology integrates functions at both ends.
综上所述,上述的一种基于USB的数据传输方法,通过预先定义不同数据的传输优先级,根据不同的传输优先级进行不同类型的数据传输,同时对不同的信道进行负载和带宽的调节,实现传输实时极高清音视频数据,同时传输多路高清音视频数据,在传输音视频数据的同时实现通信双方的控制通信、状态信息交换、文件上传下载操作,方便用户进行自定义通信功能扩展。In summary, the above USB-based data transmission method performs different types of data transmission according to different transmission priorities by pre-defining transmission priority of different data, and performs load and bandwidth adjustment on different channels. Realize the transmission of real-time ultra-high-definition audio and video data, and simultaneously transmit multi-channel high-definition audio and video data, realize the control communication, status information exchange, file uploading and downloading operation of both parties while transmitting audio and video data, and facilitate users to customize the communication function. .
如图8至图11所示,是本实施例提出的一种基于USB的数据传输系统,包括定义单元10、调度单元20、管理单元30以及调节单元40;As shown in FIG. 8 to FIG. 11 , a data transmission system based on USB provided in this embodiment includes a definition unit 10, a scheduling unit 20, a management unit 30, and an adjustment unit 40;
定义单元10,用于定义不同数据的传输优先级;Defining unit 10 for defining a transmission priority of different data;
调度单元20,用于按照定义的高优先级、中优先级以及低优先级的顺序依次传输视频数据;The scheduling unit 20 is configured to sequentially transmit video data according to the defined high priority, medium priority, and low priority order;
管理单元30,用于对于当前正在传输的数据进行统计,计算出各个不同信道的即时带宽需求;The management unit 30 is configured to perform statistics on data currently being transmitted, and calculate an instantaneous bandwidth requirement of each different channel;
调节单元40,用于对不同信道的传输负荷和带宽做出对应的调节。The adjusting unit 40 is configured to perform corresponding adjustments on transmission loads and bandwidths of different channels.
更进一步的,调度单元20包括高级判断模块21、高级数据传输模块22、中级判断模块23、中级数据传输模块25、低级判断模块26以及低级数据传输模块27;Further, the scheduling unit 20 includes an advanced judging module 21, an advanced data transmission module 22, an intermediate judging module 23, an intermediate data transmission module 25, a low-level judging module 26, and a low-level data transmission module 27;
高级判断模块21,用于判断是否有高优先级数据需要传输;The advanced judging module 21 is configured to determine whether high priority data needs to be transmitted;
高级数据传输模块22,用于优先传输高优先级数据; The advanced data transmission module 22 is configured to preferentially transmit high priority data;
中级判断模块23,用于判断是否有中优先级数据需要传输;The intermediate judging module 23 is configured to determine whether there is medium priority data to be transmitted;
中级数据传输模块25,用于传输中优先级数据;The intermediate data transmission module 25 is configured to transmit medium priority data;
低级判断模块26,用于判断是否有低优先级数据需要传输;The low-level judging module 26 is configured to determine whether low-priority data needs to be transmitted;
低级数据传输模块27,用于传输低优先级数据。The low-level data transmission module 27 is configured to transmit low priority data.
具体的,调节单元40包括数量判断模块41以及处理模块42;Specifically, the adjustment unit 40 includes a quantity determination module 41 and a processing module 42;
数量判断模块41,用于判断当前传输的高优先级数据的数量是否大于设定的高优先级数据值;The quantity determining module 41 is configured to determine whether the quantity of the currently transmitted high priority data is greater than the set high priority data value;
处理模块42,用于降低低优先级数据的发送频率和包大小,或者,提高低优先级数据的发送频率和包大小。The processing module 42 is configured to reduce the transmission frequency and the packet size of the low priority data, or to increase the transmission frequency and the packet size of the low priority data.
另外,调度单元20包括音视频同步模块24,用于对输入输出的实时音频数据和视频数据的时间戳进行管理。In addition, the scheduling unit 20 includes an audio and video synchronization module 24 for managing time stamps of the input and output real-time audio data and video data.
具体的,该音视频同步模块24包括排序模块241以及同步子模块242,排序子模块241,用于根据对音视频数据进行编码的时间戳,对音视频数据的先后发送进行排序;另外,同步模块242用于将时间戳发给下游环节,下游环节使用时间戳进行数据同步的处理。Specifically, the audio and video synchronization module 24 includes a sorting module 241 and a synchronization sub-module 242, and is configured to sort the sequential transmission of the audio and video data according to the timestamp of encoding the audio and video data; The module 242 is configured to send the time stamp to the downstream link, and the downstream link uses the time stamp to perform the data synchronization process.
上述的一种基于USB的数据传输系统,通过定义单元10预先定义不同数据的传输优先级,调度单元20根据不同的传输优先级进行不同类型的数据传输,同时管理单元30以及调度单元20结合,对不同的信道进行负载和带宽的调节,实现传输实时极高清音视频数据,同时传输多路高清音视频数据,在传输音视频数据的同时实现通信双方的控制通信、状态信息交换、文件上传下载操作,方便用户进行自定义通信功能扩展。In the above USB-based data transmission system, the transmission priority of different data is predefined by the definition unit 10, and the scheduling unit 20 performs different types of data transmission according to different transmission priorities, and the management unit 30 and the scheduling unit 20 combine. Load and bandwidth adjustment for different channels, real-time transmission of high-definition audio and video data, simultaneous transmission of multi-channel high-definition audio and video data, simultaneous transmission of audio and video data, control communication, status information exchange, file upload and download Operation, convenient for users to customize the communication function expansion.
上述仅以实施例来进一步说明本发明的技术内容,以便于读者更容易理解,但不代表本发明的实施方式仅限于此,任何依本发明所做的技术延伸或再创造,均受本发明的保护。本发明的保护范围以权利要求书为准。The above technical description of the present invention is further described by way of example only, and is not to be understood by the reader, but the embodiments of the present invention are not limited thereto, and any technology extending or re-creating according to the present invention is subject to the present invention. protection of. The scope of the invention is defined by the claims.

Claims (10)

  1. 一种基于USB的数据传输方法,其特征在于,包括以下步骤: A USB-based data transmission method, comprising the steps of:
    定义不同数据的传输优先级;Define the transmission priority of different data;
    按照定义的高优先级、中优先级以及低优先级的顺序依次传输视频数据;Video data is sequentially transmitted in the order of defined high priority, medium priority, and low priority;
    对于当前正在传输的数据进行统计,计算出各个不同信道的即时带宽需求;Perform statistics on the data currently being transmitted, and calculate the instantaneous bandwidth requirements of different channels;
    对不同信道的传输负荷和带宽做出对应的调节。 Corresponding adjustments are made to the transmission load and bandwidth of different channels.
  2. 根据权利要求1所述的一种基于USB的数据传输方法,其特征在于,所述定义不同数据的传输优先级的步骤,所述传输优先级包括高优先级数据、中优先级数据以及低优先级数据,所述高优先级数据为用户操作与控制命令相关的交换协议信息;中优先级数据为实时传输的音视频数据;低优先级数据为后台进行上传下载的文件信息。The USB-based data transmission method according to claim 1, wherein the step of defining a transmission priority of different data includes high priority data, medium priority data, and low priority Level data, the high priority data is the exchange protocol information related to the user operation and the control command; the medium priority data is the audio and video data transmitted in real time; and the low priority data is the file information uploaded and downloaded in the background.
  3. 根据权利要求2所述的一种基于USB的数据传输方法,其特征在于,所述按照定义的高优先级、中优先级以及低优先级的顺序依次传输视频数据的步骤,包括以下具体步骤:The USB-based data transmission method according to claim 2, wherein the step of sequentially transmitting the video data in the order of the defined high priority, medium priority, and low priority includes the following specific steps:
    判断是否有高优先级数据需要传输;Determine if there is high priority data to be transmitted;
    若有高优先级数据需要传输,则优先传输高优先级数据,并进入下一步骤,或者,若没有高优先级数据需要传输,则直接进入下一步骤;If there is high priority data to be transmitted, the high priority data is transmitted preferentially, and the next step is entered, or if there is no high priority data to be transmitted, the process proceeds directly to the next step;
    判断是否有中优先级数据需要传输;Determine whether there is medium priority data to be transmitted;
    若有中优先级数据需要传输,则传输中优先级数据,并进入下一步骤,或者,若没有中优先级数据需要传输,则直接进入下一步骤;If there is medium priority data to be transmitted, the medium priority data is transmitted, and the next step is entered, or if there is no medium priority data to be transmitted, the process proceeds directly to the next step;
    判断是否有低优先级数据需要传输;Determine if there is low priority data to be transmitted;
    若有低优先级数据需要传输,则传输低优先级数据,或者,若没有低优先级数据需要传输,则进入结束步骤。If there is low priority data to be transmitted, the low priority data is transmitted, or if there is no low priority data to be transmitted, the end step is entered.
  4. 根据权利要求3所述的一种基于USB的数据传输方法,其特征在于,所述对不同信道的传输负荷和带宽做出对应的调节的步骤,包括以下具体步骤:The USB-based data transmission method according to claim 3, wherein the step of adjusting the transmission load and bandwidth of different channels comprises the following specific steps:
    判断当前传输的高优先级数据的数量是否大于设定的高优先级数据值;Determining whether the number of currently transmitted high priority data is greater than a set high priority data value;
    若是,则降低低优先级数据的发送频率和包大小;If yes, reduce the transmission frequency and packet size of the low priority data;
    若不是,则提高低优先级数据的发送频率和包大小。If not, the transmission frequency and packet size of the low priority data are increased.
  5. 根据权利要求1至4任一项所述的一种基于USB的数据传输方法,其特征在于,所述若有中优先级数据需要传输,则传输中优先级数据,并进入下一步骤,或者,若没有中优先级数据需要传输,则直接进入下一步骤中,所述若有中优先级数据需要传输,则传输中优先级数据的步骤之前,还包括:The USB-based data transmission method according to any one of claims 1 to 4, wherein if the medium priority data needs to be transmitted, the medium priority data is transmitted, and the process proceeds to the next step, or If there is no medium priority data to be transmitted, go directly to the next step, if the medium priority data needs to be transmitted, before the step of transmitting the medium priority data, the method further includes:
    对输入输出的实时音频数据和视频数据的时间戳进行管理。Time stamping of input and output real-time audio data and video data.
  6. 根据权利要求5所述的一种基于USB的数据传输方法,其特征在于,所述对输入输出的实时音频数据和视频数据的时间戳进行管理的步骤,包括以下具体步骤:The USB-based data transmission method according to claim 5, wherein the step of managing the timestamp of the input and output real-time audio data and the video data comprises the following specific steps:
    根据对音视频数据进行编码的时间戳,对音视频数据的先后发送进行排序;Sorting the sequential transmission of audio and video data according to the timestamp of encoding the audio and video data;
    将时间戳发给下游环节,下游环节使用时间戳进行数据同步的处理。The timestamp is sent to the downstream link, and the downstream link uses the timestamp for data synchronization.
  7. 一种基于USB的数据传输系统,其特征在于,包括定义单元、调度单元、管理单元以及调节单元;A USB-based data transmission system, comprising: a definition unit, a scheduling unit, a management unit, and an adjustment unit;
    所述定义单元,用于定义不同数据的传输优先级;The definition unit is configured to define a transmission priority of different data;
    所述调度单元,用于按照定义的高优先级、中优先级以及低优先级的顺序依次传输视频数据;The scheduling unit is configured to sequentially transmit video data according to a defined high priority, medium priority, and low priority order;
    所述管理单元,用于对于当前正在传输的数据进行统计,计算出各个不同信道的即时带宽需求;The management unit is configured to perform statistics on data currently being transmitted, and calculate an instantaneous bandwidth requirement of each different channel;
    所述调节单元,用于对不同信道的传输负荷和带宽做出对应的调节。The adjusting unit is configured to perform corresponding adjustment on transmission load and bandwidth of different channels.
  8. 根据权利要求7所述的一种基于USB的数据传输系统,其特征在于,所述调度单元包括高级判断模块、高级数据传输模块、中级判断模块、中级数据传输模块、低级判断模块以及低级数据传输模块;The USB-based data transmission system according to claim 7, wherein the scheduling unit comprises an advanced judgment module, an advanced data transmission module, an intermediate judgment module, an intermediate data transmission module, a low-level judgment module, and low-level data transmission. Module
    所述高级判断模块,用于判断是否有高优先级数据需要传输;The advanced determining module is configured to determine whether high priority data needs to be transmitted;
    所述高级数据传输模块,用于优先传输高优先级数据; The advanced data transmission module is configured to preferentially transmit high priority data;
    所述中级判断模块,用于判断是否有中优先级数据需要传输;The intermediate level determining module is configured to determine whether medium priority data needs to be transmitted;
    所述中级数据传输模块,用于传输中优先级数据;The intermediate data transmission module is configured to transmit medium priority data;
    所述低级判断模块,用于判断是否有低优先级数据需要传输;The low-level judging module is configured to determine whether low-priority data needs to be transmitted;
    所述低级数据传输模块,用于传输低优先级数据。The low-level data transmission module is configured to transmit low-priority data.
  9. 根据权利要求8所述的一种基于USB的数据传输系统,其特征在于,所述调节单元包括数量判断模块以及处理模块;The USB-based data transmission system according to claim 8, wherein the adjustment unit comprises a quantity determination module and a processing module;
    所述数量判断模块,用于判断当前传输的高优先级数据的数量是否大于设定的高优先级数据值;The quantity determining module is configured to determine whether the quantity of the currently transmitted high priority data is greater than the set high priority data value;
    所述处理模块,用于降低低优先级数据的发送频率和包大小,或者,提高低优先级数据的发送频率和包大小。The processing module is configured to reduce the sending frequency and the packet size of the low priority data, or to increase the sending frequency and the packet size of the low priority data.
  10. 根据权利要求9所述的一种基于USB的数据传输系统,其特征在于,所述调节单元还包括音视频同步块,用于对输入输出的实时音频数据和视频数据的时间戳进行管理。A USB-based data transmission system according to claim 9, wherein said adjustment unit further comprises an audio and video synchronization block for managing time stamps of the input and output real-time audio data and video data.
PCT/CN2016/109550 2016-12-13 2016-12-13 Usb-based data transmission method and system thereof WO2018107346A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2016/109550 WO2018107346A1 (en) 2016-12-13 2016-12-13 Usb-based data transmission method and system thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2016/109550 WO2018107346A1 (en) 2016-12-13 2016-12-13 Usb-based data transmission method and system thereof

Publications (1)

Publication Number Publication Date
WO2018107346A1 true WO2018107346A1 (en) 2018-06-21

Family

ID=62557780

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/109550 WO2018107346A1 (en) 2016-12-13 2016-12-13 Usb-based data transmission method and system thereof

Country Status (1)

Country Link
WO (1) WO2018107346A1 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112584436A (en) * 2019-09-27 2021-03-30 深圳长城开发科技股份有限公司 Data transmission method and system of LoRa communication system
CN113115117A (en) * 2021-03-16 2021-07-13 新东方教育科技集团有限公司 Interactive video playing method and device, storage medium and electronic equipment
CN114050978A (en) * 2021-11-11 2022-02-15 安徽电信规划设计有限责任公司 Communication load big data adjusting method and system

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101242395A (en) * 2008-02-19 2008-08-13 深圳华为通信技术有限公司 A data transmission method and USB interface
CN101482854A (en) * 2009-02-10 2009-07-15 深圳华为通信技术有限公司 USB multi-channel transmission method, system and main unit
CN101685323A (en) * 2008-09-22 2010-03-31 联想(北京)有限公司 Computer docking station and switching working method thereof
CN103488449A (en) * 2012-06-08 2014-01-01 旺玖科技股份有限公司 Transmission system capable of dynamically setting transmission profile and control method thereof
CN103516789A (en) * 2013-09-16 2014-01-15 北京电视台 Method and system for managing data transmission from office network to production network
CN103516790A (en) * 2013-09-16 2014-01-15 北京电视台 Method and system for transmitting audio and video file from office network to production network
CN103685446A (en) * 2012-08-31 2014-03-26 三星电子株式会社 Data transmission apparatus and method
WO2015064838A1 (en) * 2013-10-31 2015-05-07 Samsung Electronics Co., Ltd. Scheme for supporting wireless transmission taking into account qos of usb application data

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101242395A (en) * 2008-02-19 2008-08-13 深圳华为通信技术有限公司 A data transmission method and USB interface
CN101685323A (en) * 2008-09-22 2010-03-31 联想(北京)有限公司 Computer docking station and switching working method thereof
CN101482854A (en) * 2009-02-10 2009-07-15 深圳华为通信技术有限公司 USB multi-channel transmission method, system and main unit
CN103488449A (en) * 2012-06-08 2014-01-01 旺玖科技股份有限公司 Transmission system capable of dynamically setting transmission profile and control method thereof
CN103685446A (en) * 2012-08-31 2014-03-26 三星电子株式会社 Data transmission apparatus and method
CN103516789A (en) * 2013-09-16 2014-01-15 北京电视台 Method and system for managing data transmission from office network to production network
CN103516790A (en) * 2013-09-16 2014-01-15 北京电视台 Method and system for transmitting audio and video file from office network to production network
WO2015064838A1 (en) * 2013-10-31 2015-05-07 Samsung Electronics Co., Ltd. Scheme for supporting wireless transmission taking into account qos of usb application data

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112584436A (en) * 2019-09-27 2021-03-30 深圳长城开发科技股份有限公司 Data transmission method and system of LoRa communication system
CN113115117A (en) * 2021-03-16 2021-07-13 新东方教育科技集团有限公司 Interactive video playing method and device, storage medium and electronic equipment
CN113115117B (en) * 2021-03-16 2022-12-23 新东方教育科技集团有限公司 Interactive video playing method and device, storage medium and electronic equipment
CN114050978A (en) * 2021-11-11 2022-02-15 安徽电信规划设计有限责任公司 Communication load big data adjusting method and system
CN114050978B (en) * 2021-11-11 2023-08-29 安徽电信规划设计有限责任公司 Communication load big data adjusting method and system

Similar Documents

Publication Publication Date Title
WO2016165556A1 (en) Data processing method, device and system for video stream
WO2018107346A1 (en) Usb-based data transmission method and system thereof
WO2017018793A1 (en) Antenna device and electronic device including same
WO2018192253A1 (en) Video signal transmission method and apparatus
WO2018161604A1 (en) Playback control method and device for mobile terminal, storage medium and electronic device
WO2017155251A1 (en) Video streaming apparatus and method in electronic device
WO2014187158A1 (en) Method, server, and terminal for controlling cloud sharing of terminal data
WO2016101698A1 (en) Dlna technology-based screen push method and system
WO2017020649A1 (en) Audio/video playback control method and device thereof
WO2016010319A2 (en) Cloud streaming service system, data compressing method for preventing memory bottlenecking, and device for same
WO2017088444A1 (en) Method and device for prompting electric quantity information of terminal
WO2020145631A1 (en) Content reproducing apparatus and content reproducing method
WO2018006581A1 (en) Smart-television playback method and apparatus
US20210105404A1 (en) Video photographing processing method, apparatus, and video photographing processing system
WO2019210574A1 (en) Message processing method, apparatus, device, and readable storage medium
WO2022025606A1 (en) Electronic device and method for controlling audio volume thereof
WO2022005000A1 (en) Electronic device and method of operating the same
WO2017045444A1 (en) Remote controller key allocation method and device
WO2016208988A1 (en) Method and apparatus for transmitting and receiving signal in multimedia system
WO2021025359A1 (en) Method for controlling transmission power of radio signal and electronic device therefor
WO2016101687A1 (en) Digital television terminal condition receiving method, terminal and system
WO2020171657A1 (en) Display device and image display method of the same
WO2020149722A1 (en) Working environment provisioning method and apparatus for execution of application program between electronic device and external server
WO2015192801A1 (en) Electronic device control method and system, and related device
WO2018047989A1 (en) Image display device and system thereof

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16923648

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 20.11.2019)

122 Ep: pct application non-entry in european phase

Ref document number: 16923648

Country of ref document: EP

Kind code of ref document: A1