WO2022062799A1 - Data transmission method and related apparatus - Google Patents

Data transmission method and related apparatus Download PDF

Info

Publication number
WO2022062799A1
WO2022062799A1 PCT/CN2021/114112 CN2021114112W WO2022062799A1 WO 2022062799 A1 WO2022062799 A1 WO 2022062799A1 CN 2021114112 W CN2021114112 W CN 2021114112W WO 2022062799 A1 WO2022062799 A1 WO 2022062799A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
remote device
camera
remote
local device
Prior art date
Application number
PCT/CN2021/114112
Other languages
French (fr)
Chinese (zh)
Inventor
孙伟
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2022062799A1 publication Critical patent/WO2022062799A1/en

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21Server components or server architectures
    • H04N21/218Source of audio or video content, e.g. local disk arrays
    • H04N21/21805Source of audio or video content, e.g. local disk arrays enabling multiple viewpoints, e.g. using a plurality of cameras
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21Server components or server architectures
    • H04N21/218Source of audio or video content, e.g. local disk arrays
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/236Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/236Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
    • H04N21/23608Remultiplexing multiplex streams, e.g. involving modifying time stamps or remapping the packet identifiers
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/236Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
    • H04N21/2365Multiplexing of several video streams
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/236Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
    • H04N21/2365Multiplexing of several video streams
    • H04N21/23655Statistical multiplexing, e.g. by controlling the encoder to alter its bitrate to optimize the bandwidth utilization
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/238Interfacing the downstream path of the transmission network, e.g. adapting the transmission rate of a video stream to network bandwidth; Processing of multiplex streams
    • H04N21/2385Channel allocation; Bandwidth allocation

Definitions

  • the embodiments of the present application relate to the field of communications, and in particular, to a data transmission method and a related device.
  • Distributed camera refers to the local device accessing or controlling the camera on the remote device through the network to achieve the purpose of using the camera across devices.
  • a camera is a scene that collects data and consumes data in real time. It has very high real-time requirements, but the business real-time requirements of each camera data stream are different.
  • the preview is presented to the user in real time, and the real-time requirements are very high.
  • the preview data should be presented to the user as quickly as possible, and the data frame should be stable to reduce the probability of stalling; while the large picture is generally directly stored in the file, the user Without direct perception, the real-time requirements are relatively low.
  • a transmission mechanism that adapts to the network bandwidth is required, which not only ensures the smooth and stable output of camera stream data, reduces the probability of stalling, but also meets the real-time requirements of each service stream of distributed cameras and achieves a good user experience.
  • a first aspect of the embodiments of the present application provides a data transmission method, including:
  • the remote device collects the shooting data, the shooting data is the data obtained by the camera of the remote device, and the camera of the remote device can be one or more.
  • the remote device determines the priority of the shooting data, and based on the above priority Sorting sends the shooting data to the local device, and the remote device can determine the priority in various ways. For example, the remote device identifies the data type of the shooting data and determines the priority sorting Prioritization is determined.
  • multiple camera streams are transmitted concurrently, they are sent in sequence based on the real-time requirements of the camera streams, and the data with higher priority is sent first, so as to ensure that the camera streams with high real-time requirements are transmitted first, and the camera streams with low real-time requirements are sent with a delay and lag. , to reduce the cross-device latency of high-real-time camera streams and improve user experience.
  • the priority ordering rule is that the smaller the delay threshold of the shooting data, the higher the priority of the shooting data.
  • a priority sorting solution is provided for determining priorities according to a delay threshold of shooting data.
  • the priority sorting rule is that the priority is based on the preview data, photographing thumbnail data, video call data, video recording data, photographing data
  • the big picture data decreases sequentially.
  • the remote device calculates the distance from the remote device to the local device.
  • the transmission code rate required for sending the shooting data so that the local device configures the bandwidth resources of the remote device according to the transmission code rate.
  • a second aspect of the embodiments of the present application provides a data transmission method, including:
  • the local device receives the transmission bit rate sent by the remote device, and the transmission bit rate is the bit rate required by the remote device to send shooting data to the local device, where the shooting data is the data obtained by the camera of the remote device, and the
  • the camera may include one or more local devices connected to two or more remote devices; the local device configures bandwidth resources for each remote device according to the above-mentioned transmission code rate.
  • the local device can dynamically allocate the network transmission bandwidth of each remote device and the data stream of each remote device, ensure the smooth and stable output of the data frame of the camera stream of the remote device, and achieve the distributed camera stream of each camera. business real-time requirements.
  • the configuration method for the local device to configure the bandwidth resources for each remote device according to the transmission bit rate includes:
  • BW x BW total *(B x /B total );
  • BW x is the bandwidth quota of the xth remote device, the xth remote device belongs to any of the remote devices connected to the local device, BW total is the total bandwidth that can be allocated by the local device, and Bx is the xth remote device Transmission code rate of the remote device, B total is the total transmission code rate required by all remote devices.
  • a third aspect of the embodiments of the present application provides a remote device, where the remote device executes the foregoing first aspect and the method of any implementation manner of the first aspect.
  • a fourth aspect of the embodiments of the present application provides a local device, where the local device executes the foregoing second aspect and the method of any implementation manner of the second aspect.
  • a fifth aspect of the embodiments of the present application provides a computer storage medium, where instructions are stored in the computer storage medium, and when the instructions are executed on a computer, the instructions cause the computer to execute the foregoing first aspect and any implementation manner of the first aspect or the foregoing first aspect.
  • a sixth aspect of the embodiments of the present application provides a computer software product, which, when executed on a computer, enables the computer to execute the first aspect and any implementation manner of the first aspect or the second aspect and any of the second aspect.
  • FIG. 1 is a schematic diagram of a network architecture in an embodiment of the present application.
  • FIG. 2 is a schematic flowchart of a data transmission method in an embodiment of the present application.
  • FIG. 3 is a schematic diagram of a logic diagram of a local device dynamically allocating network bandwidth resources in an embodiment of the present application
  • FIG. 4 is a schematic diagram of data transmission based on priority sorting in an embodiment of the present application.
  • FIG. 5 is a schematic diagram of a dynamic control data transmission in an embodiment of the application.
  • FIG. 6 is a schematic structural diagram of a local device in an embodiment of the present application.
  • FIG. 7 is a schematic structural diagram of a remote device in an embodiment of the present application.
  • FIG. 8 is another schematic structural diagram of a local device in an embodiment of the present application.
  • FIG. 9 is another schematic structural diagram of a remote device in an embodiment of the present application.
  • FIG. 10 is another schematic structural diagram of a local device in an embodiment of the application.
  • FIG. 11 is another schematic structural diagram of a remote device in an embodiment of the present application.
  • At least one (a) of a, b or c can represent: a, b, c, a and b, a and c, b and c or a and b and c, where a, b and c can be It can be single or multiple. It is worth noting that "at least one item(s)" can also be interpreted as "one item(s) or more(s)”.
  • a network architecture according to an embodiment of the present application includes:
  • the number of the remote devices is not limited in this embodiment of the application, and this embodiment only takes two remote devices as an example for description (remote device 1 and remote device 2).
  • the multi-camera bandwidth cooperative allocation module of the local device can dynamically allocate the network bandwidth quota of each remote device based on the real-time data flow service and real-time transmission code rate of each cross-device camera; the multi-camera stream dynamic sending module of the remote device can , based on the bandwidth quota and the real-time requirements of each camera stream, dynamically control the data transmission of each camera stream, smooth the transmission bit rate and meet the real-time requirements.
  • Both the local device and the remote device in the embodiments of the present application may be terminal devices, which are referred to as cameras in the embodiments of the present application, and are also referred to as user equipment (user equipment, UE), mobile station (mobile station, MS), A mobile terminal (mobile terminal, MT), etc., is a device that provides voice and/or data connectivity to a user.
  • terminal devices which are referred to as cameras in the embodiments of the present application, and are also referred to as user equipment (user equipment, UE), mobile station (mobile station, MS), A mobile terminal (mobile terminal, MT), etc.
  • UE user equipment
  • MS mobile station
  • a mobile terminal mobile terminal, MT
  • a device that provides voice and/or data connectivity to a user.
  • handheld devices for example, handheld devices, in-vehicle devices, etc. with wireless connectivity.
  • terminals are: mobile phone (mobile phone), tablet computer, notebook computer, PDA, mobile internet device (MID), wearable device, virtual reality (virtual reality, VR) device, augmented reality (augmented reality, AR) equipment, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, and smart grids wireless terminals, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, etc.
  • mobile phone mobile phone
  • tablet computer notebook computer
  • PDA mobile internet device
  • MID mobile internet device
  • wearable device virtual reality (virtual reality, VR) device
  • augmented reality (augmented reality, AR) equipment wireless terminals in industrial control
  • wireless terminals in self-driving wireless terminals in remote medical surgery
  • smart grids wireless terminals wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, etc.
  • the embodiment of the present application provides a data transmission method for a distributed camera usage scenario, that is, a local device controls one or more remote devices through a network, and camera data collected by the remote device needs to be transmitted back to the local device for use.
  • a local device controls one or more remote devices through a network
  • camera data collected by the remote device needs to be transmitted back to the local device for use.
  • the mobile phone uses the large-screen camera to make video calls
  • the local mobile phone controls the remote mobile phone camera/remote drone camera to achieve multi-device collaborative video recording (Vlog), live broadcast, etc.
  • Vlog multi-device collaborative video recording
  • the specific application scenarios are not limited here.
  • LTE long term evolution
  • 5th generation 5th generation
  • 5G fifth generation
  • WiFi wireless-fidelity
  • future communication system or a system integrating multiple communication systems, etc.
  • 5G can also be called new radio (NR).
  • eMBB enhanced mobile broadband
  • ultra-reliable and low-latency communication ultra-reliable and low-latency communication
  • URLLC ultra-reliable low-latency communication
  • MTC machine type communication
  • mMTC massive machine type communication
  • D2D device-to-device
  • V2X vehicle to vehicle
  • IoT Internet of things
  • a data transmission method includes:
  • Each remote device collects shooting data
  • the number of remote devices in this embodiment of the present application is not limited, and this embodiment only takes two remote devices as an example for description, that is, a first remote device and a second remote device.
  • the first remote device and the second remote device collect shooting data, the shooting data of the first remote device is the data obtained by the camera of the first remote device, and the shooting data of the second remote device is the data of the second remote device.
  • the data acquired by the camera, the data acquired by the camera may be image data, video data, etc., which are not specifically limited here.
  • the cameras of each remote device may include one or more cameras.
  • the camera of the first remote device may include two cameras, a wide-angle camera and a telephoto camera.
  • the specific camera type and number are determined according to the actual application scenario, which is not described here. limited.
  • Each remote device calculates the transmission code rate
  • the first remote device and the second remote device calculate their respective transmission bit rates, where the transmission bit rates are the bit rates required by the remote device to send the shooting data to the local device.
  • the Stream type is a continuous stream of camera data, such as a preview stream or a video stream.
  • the Image type is a data stream collected on demand, such as a large image and a thumbnail image when taking a photo. The corresponding data is generated only when the user triggers a photo-taking operation.
  • the captured data can be compressed before being transmitted across devices.
  • the dynamic bit rate (VBR) is generally used for compression, that is, the bit rate is dynamically adjusted according to the complexity of the image, so as to ensure that the quality of each frame of the compressed camera stream is within the same relatively high level. Therefore, it is necessary to dynamically estimate the stream bit rate of the Stream type camera in real time.
  • the calculation method of the stream bit rate B of a single-channel Stream type camera can be as follows:
  • B when it is less than one unit of calculation time, B takes the default code rate, and when it is greater than one unit of calculation time, the code rate is calculated by the actual data amount in 1 to n units of calculation time.
  • t is the duration of the camera stream data that has been output by the data compression module of the remote device;
  • T is the unit calculation time, that is, the I-frame interval time for an integer number of video streams compressed, generally taking an I-frame interval;
  • B is a single-channel Stream.
  • B size is the default transmission bit rate configured in xml, which is related to the resolution of the camera stream (size);
  • D n is the nth unit of calculation time, the data compression module of the remote device output The amount of data;
  • W n is the weight occupied by D n when calculating the code rate, generally W n-1 ⁇ W n , and n is a positive integer greater than or equal to 1.
  • the calculation method of the stream bit rate B of the Image type camera can be as follows:
  • Image type camera stream is not a continuous data stream, single frame compression is performed during transmission, and its code stream is calculated as follows:
  • B is the bit rate of a single-channel Image type camera stream
  • D size is the data volume of the default single-frame image compression configured in xml, which is related to the resolution of the camera stream, that is, size
  • T latency is the delay threshold of the camera stream, That is, the maximum delay of the camera stream data acceptable to the service.
  • the camera stream delay threshold supports active configuration of business applications to meet different business requirements for real-time performance.
  • the calculation method of the transmission bit rate of the xth remote device can be:
  • B x is the transmission code rate of the xth remote device. It can be understood that the value range of x is equal to the number of remote devices; B i is the transmission code of the i-th stream of the xth remote device. rate, i is a positive integer not less than 1.
  • the local device receives the transmission bit rate of each remote device
  • Data transmission between the local device and the remote device may be in a wired manner or a wireless manner, which is not specifically limited here.
  • the local device configures bandwidth resources for each remote device according to the transmission bit rate
  • the local device dynamically allocates the network bandwidth quota of each remote device according to the transmission bit rate of each remote device.
  • the configuration is as follows:
  • BW x BW total *(B x /B total );
  • BW x is the bandwidth quota of the xth remote device; BW total is the total bandwidth that can be allocated by the local device; Bx is the transmission bit rate of the xth remote device; B total is the required amount of all remote devices The total transmission bit rate.
  • the conditions for triggering the local device to update the bandwidth quota may be: a camera stream stops or a new camera stream starts, or the local device configures the bandwidth resources of each remote device according to a fixed preset period T, and the specific number of times is not required. limit.
  • steps 202 to 204 may not be executed.
  • the data transmission between the remote device and the local device may occupy all available bandwidth in the network, that is, the local device does not need to configure bandwidth.
  • Each remote device determines the priority ordering of the shooting data
  • the priority ordering rule is that the smaller the delay threshold of the captured data, the higher the priority of the captured data, and the higher priority data is sent first. Ensure that camera streams with high real-time requirements are transmitted first, and camera streams with low real-time requirements are sent with a delay, reducing the cross-device delay of camera streams with high real-time requirements, and improving user experience.
  • a common priority sorting rule may be as follows: the priority is in descending order according to preview data, photo thumbnail data, video call data, video recording data, and photo large image data. It can be understood that this rule is not the only rule, and the specific sorting rule can also be obtained through user settings and other methods.
  • Each remote device sends the shooting data to the local device, and the sending order is sorted based on the priority.
  • preview data is preferentially sent to ensure a user-visible delay experience.
  • the remaining bandwidth is used to send thumbnail data and video data preferentially. It can be seen that the large image data of the photo is divided into small pieces and sent multiple times, so as not to affect the real-time performance of preview, thumbnail and video.
  • FIG. 5 is a schematic diagram of dynamic control data transmission in the embodiment of the application, and the specific implementation of dynamic transmission of camera streams of multiple data types (such as including preview data and video recording data) is as follows:
  • the data transmission is controlled according to the unit transmission time T send , and the amount of data that can be sent within the time T send does not exceed D send :
  • D send BW x *T send ;
  • D send is the amount of data that can be sent within the unit sending time T send ;
  • BW x is the bandwidth quota of the xth remote device;
  • T send is the unit sending time, generally the preview frame interval (eg 33.3ms).
  • the data sending module fetches and sends data from the corresponding buffers according to the real-time priority of the camera stream. Only when the high-priority camera stream pending buffer is empty, the next-priority camera stream data is sent.
  • the amount of data that the remote device can send within the time T send does not exceed D send . If there is no data in all buffers, it will wait for the next sending time of T send .
  • the remote device sends data according to the unit sending time T send . For oversized camera data frames, it will be grouped into small blocks and sent multiple times.
  • the frame reassembly module of the local device reassembles the camera data into complete camera frames.
  • the stable frame rate module of the local device only controls the stable frame rate of the real-time stream, because the real-time stream is generally sent directly to the display and needs smooth and stable output.
  • the data of other camera streams, after frame reorganization, is directly sent to the consumer end of the camera stream.
  • the network transmission bandwidth of each cross-device camera and its data streams is dynamically allocated, so as to ensure smooth and stable output of data frames of camera streams, and achieve distribution.
  • the business real-time requirements of each camera stream of the type camera is dynamically allocated, so as to ensure smooth and stable output of data frames of camera streams, and achieve distribution.
  • the local device may include a processor, a display unit, a transceiver unit (such as a wireless bandwidth (WiFi) unit, one or more of a wired transmission unit), a power supply, a storage unit, a video decoder, etc.;
  • the remote device may include a processor, a camera unit, a transceiver unit (such as a wireless bandwidth (WiFi) unit, a wired transmission unit) , power supply, video encoder, picture encoder, etc. one or more.
  • an embodiment of the local device in the embodiment of the present application includes:
  • the receiving unit 801 is used for receiving the transmission code rate sent by the remote device, the transmission code rate is the code rate required by the remote device to send the shooting data to the local device, the shooting data is the data obtained by the camera of the remote device, the remote device
  • the camera consists of one or more local devices and two or more remote devices connected.
  • the configuration unit 802 is configured to configure bandwidth resources for each remote device according to the transmission code rate.
  • each unit is similar to the operations performed by the local device in the previous embodiment shown in FIG. 2 , and are not repeated here.
  • an embodiment of the remote device in the embodiment of the present application includes:
  • the collection unit 901 is configured to collect shooting data, where the shooting data is data acquired by a camera of a remote device, and the camera of the remote device includes one or more cameras.
  • the calculation unit 902 is configured to calculate the transmission code rate, so that the local device configures the bandwidth resources of the remote device according to the transmission code rate, and the transmission code rate is the code rate required by the sending unit to send the shooting data to the local device.
  • the determining unit 903 is configured to determine the priority ordering of the shooting data.
  • the sending unit 904 is configured to send the shooting data to the local device, and the sending order of the shooting data is sorted based on the priority.
  • each unit is similar to the operations performed by the remote device in the previous embodiment shown in FIG. 2 , which are not repeated here.
  • the local device 1000 may include one or more processors 1001 and a memory 1005, and the memory 1005 stores one or more application programs or data.
  • the memory 1005 may be volatile storage or persistent storage.
  • a program stored in memory 1005 may include one or more modules, each of which may include a series of instructions to operate on local device 1000 .
  • the processor 1001 may be configured to communicate with the memory 1005 to execute a series of instruction operations in the memory 1005 on the local device 1000 .
  • the local device 1000 may also include one or more power supplies 1002, one or more wired or wireless network interfaces 1003, one or more input and output interfaces 1004, and/or, one or more operating systems, such as: Microsoft operating system ( Any of Windows), Android operating system (Android), Apple operating system (Mac OS), Yonex operating system (Unix), and Linax operating system (Linux).
  • Microsoft operating system Any of Windows
  • Android operating system Android
  • Apple operating system Mac OS
  • Yonex operating system Uniix
  • Linax operating system Linux
  • the processor 1001 may perform the operations performed by the local device in any of the foregoing embodiments, and details are not described herein again.
  • the remote device 1100 may include one or more processors 1101 and a memory 1105, and one or more application programs or data are stored in the memory 1105.
  • the memory 1105 may be volatile storage or persistent storage.
  • the programs stored in the memory 1105 may include one or more modules, each of which may include a series of instructions to operate on the remote device 1100 .
  • the processor 1101 may be configured to communicate with the memory 1105 to execute a series of instruction operations in the memory 1105 on the remote device 1100 .
  • the remote device 1100 may also include one or more power supplies 1102, one or more wired or wireless network interfaces 1103, one or more input and output interfaces 1104, and/or, one or more operating systems, such as Microsoft operating systems (Windows), Android operating system (Android), Apple operating system (Mac OS), Yonex operating system (Unix), any of Linax operating system (Linux).
  • operating systems such as Microsoft operating systems (Windows), Android operating system (Android), Apple operating system (Mac OS), Yonex operating system (Unix), any of Linax operating system (Linux).
  • the processor 1101 may perform the operations performed by the remote device in any of the foregoing embodiments, and details are not described herein again.
  • the disclosed system, apparatus and method may be implemented in other manners.
  • the apparatus embodiments described above are only illustrative.
  • the division of the units is only a logical function division. In actual implementation, there may be other division methods.
  • multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented.
  • the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.
  • the above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.
  • the integrated unit if implemented in the form of a software functional unit and sold or used as an independent product, may be stored in a computer-readable storage medium.
  • the technical solutions of the present application can be embodied in the form of software products in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, and the computer software products are stored in a storage medium , including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application.
  • the aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), magnetic disk or optical disk and other media that can store program codes .

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Databases & Information Systems (AREA)
  • Telephonic Communication Services (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

Embodiments of the present application disclose a data transmission method, comprising: a remote device acquiring camera data and determining a priority order for the acquired camera data; and the remote device sending the camera data to a local device according to the priority order. In parallel transmission of multiple camera streams, the camera streams are sent in sequence on the basis of respective real-time requirements thereof, such that high-priority data is sent first, thereby ensuring that camera streams having high real-time requirements are transmitted with priority, and camera streams having low real-time requirements are delayed. In this way, the invention reduces inter-device delays for camera streams having high real-time requirements, thereby improving user experience.

Description

数据传输方法及相关装置Data transmission method and related device
本申请要求于2020年09月25日提交中国专利局、申请号为202011027401.8、发明名称为“数据传输方法及相关装置”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。This application claims the priority of the Chinese patent application with the application number 202011027401.8 and the invention titled "Data Transmission Method and Related Apparatus" filed with the China Patent Office on September 25, 2020, the entire contents of which are incorporated into this application by reference.
技术领域technical field
本申请实施例涉及通信领域,尤其涉及一种数据传输方法及相关装置。The embodiments of the present application relate to the field of communications, and in particular, to a data transmission method and a related device.
背景技术Background technique
随着多设备互联技术的快速发展和消费者需求旺盛,越来越多消费者电子设备具备分布式访问能力。分布式相机是指本地设备通过网络访问或控制远端设备上的相机,达成相机跨设备使用的目的。With the rapid development of multi-device interconnection technology and strong consumer demand, more and more consumer electronic devices have distributed access capabilities. Distributed camera refers to the local device accessing or controlling the camera on the remote device through the network to achieve the purpose of using the camera across devices.
相机是一个实时采集数据并消费数据的场景,对实时性要求非常高,但各相机数据流的业务实时性要求不一样。例如,预览是实时呈现给用户的,实时性要求很高,预览数据要尽可能快的呈现给用户,且数据帧要平稳,降低卡顿概率;而拍照大图一般是直接存文件的,用户不直接感知,实时性要求相对较低。A camera is a scene that collects data and consumes data in real time. It has very high real-time requirements, but the business real-time requirements of each camera data stream are different. For example, the preview is presented to the user in real time, and the real-time requirements are very high. The preview data should be presented to the user as quickly as possible, and the data frame should be stable to reduce the probability of stalling; while the large picture is generally directly stored in the file, the user Without direct perception, the real-time requirements are relatively low.
因此,需要一种自适应网络带宽的传输机制,既保证相机流数据的平滑稳定输出,降低卡顿概率,又要满足分布式相机各业务流的实时性要求,达成良好用户体验。Therefore, a transmission mechanism that adapts to the network bandwidth is required, which not only ensures the smooth and stable output of camera stream data, reduces the probability of stalling, but also meets the real-time requirements of each service stream of distributed cameras and achieves a good user experience.
发明内容SUMMARY OF THE INVENTION
本申请实施例第一方面提供了一种数据传输方法,包括:A first aspect of the embodiments of the present application provides a data transmission method, including:
远端设备采集拍摄数据,该拍摄数据为远端设备的摄像头获取的数据,其中远端设备的摄像头可以为一个或多个,远端设备确定拍摄数据的优先级排序,并基于上述的优先级排序向本地设备发送拍摄数据,远端设备确定优先级排序的方式有多种,如远端设备识别拍摄数据的数据类型并根据该数据类型确定优先级排序,或远端设备根据本地设备发送的优先级排序确定。The remote device collects the shooting data, the shooting data is the data obtained by the camera of the remote device, and the camera of the remote device can be one or more. The remote device determines the priority of the shooting data, and based on the above priority Sorting sends the shooting data to the local device, and the remote device can determine the priority in various ways. For example, the remote device identifies the data type of the shooting data and determines the priority sorting Prioritization is determined.
在多相机流并发传输时,基于相机流的实时性要求依次发送,优先级高的数据先发送,从而保证实时性要求高的相机流优先被传输,实时性要求低的相机流被延迟滞后发送,减少实时性高的相机流的跨设备时延,提升用户体验。When multiple camera streams are transmitted concurrently, they are sent in sequence based on the real-time requirements of the camera streams, and the data with higher priority is sent first, so as to ensure that the camera streams with high real-time requirements are transmitted first, and the camera streams with low real-time requirements are sent with a delay and lag. , to reduce the cross-device latency of high-real-time camera streams and improve user experience.
基于本申请实施例第一方面,本申请实施例第一方面的第一种实施方式中,优先级排序的规则为拍摄数据的时延阈值越小,拍摄数据的优先级越高。Based on the first aspect of the embodiments of the present application, in the first implementation of the first aspect of the embodiments of the present application, the priority ordering rule is that the smaller the delay threshold of the shooting data, the higher the priority of the shooting data.
本申请实施例中,提供了一种根据拍摄数据的时延阈值确定优先级的优先级排序方案。In the embodiment of the present application, a priority sorting solution is provided for determining priorities according to a delay threshold of shooting data.
基于本申请实施例第一方面,本申请实施例第一方面的第二种实施方式中,优先级排序的规则为优先级按照预览数据、拍照缩略图数据、视频通话数据、视频录像数据、拍照大图数据依次递减。Based on the first aspect of the embodiment of the present application, in the second implementation of the first aspect of the embodiment of the present application, the priority sorting rule is that the priority is based on the preview data, photographing thumbnail data, video call data, video recording data, photographing data The big picture data decreases sequentially.
基于本申请实施例第一方面至第一方面的第二种实施方式中任一实施方式,本申请实施例第一方面的第三种实施方式中,远端设备计算该远端设备向本地设备发送拍摄数据所需要的传输码率,以使得本地设备根据传输码率为远端设备配置带宽资源。Based on any one of the first aspect to the second implementation manner of the first aspect of the embodiments of the present application, in the third implementation manner of the first aspect of the embodiments of the present application, the remote device calculates the distance from the remote device to the local device. The transmission code rate required for sending the shooting data, so that the local device configures the bandwidth resources of the remote device according to the transmission code rate.
本申请实施例第二方面提供了一种数据传输方法,包括:A second aspect of the embodiments of the present application provides a data transmission method, including:
本地设备接收远端设备发送的传输码率,该传输码率为远端设备向本地设备发送拍摄数据所需要的码率,其中,拍摄数据为远端设备的摄像头获取的数据,远端设备的摄像头可以包括一个或多个,本地设备和两个或两个以上的远端设备连接;本地设备根据上述传输码率为各远端设备配置带宽资源。The local device receives the transmission bit rate sent by the remote device, and the transmission bit rate is the bit rate required by the remote device to send shooting data to the local device, where the shooting data is the data obtained by the camera of the remote device, and the The camera may include one or more local devices connected to two or more remote devices; the local device configures bandwidth resources for each remote device according to the above-mentioned transmission code rate.
通过本实施例,本地设备可以动态分配各远端设备以及其各远端设备的数据流的网络发送带宽,保证远端设备相机流的数据帧平滑稳定输出,并达成分布式相机各相机流的业务实时性要求。Through this embodiment, the local device can dynamically allocate the network transmission bandwidth of each remote device and the data stream of each remote device, ensure the smooth and stable output of the data frame of the camera stream of the remote device, and achieve the distributed camera stream of each camera. business real-time requirements.
基于本申请实施例第二方面,本申请实施例第二方面的第一种实施方式中,本地设备根据传输码率为各远端设备配置带宽资源的配置方式包括:Based on the second aspect of the embodiments of the present application, in the first implementation manner of the second aspect of the embodiments of the present application, the configuration method for the local device to configure the bandwidth resources for each remote device according to the transmission bit rate includes:
BW x=BW total*(B x/B total); BW x =BW total *(B x /B total );
BW x为第x个远端设备的带宽配额,第x个远端设备属于和本地设备相连的远端设备中的任一个,BW total为本地设备可分配的总带宽,B x为第x个远端设备的传输码率,B total为所有远端设备所需的总的传输码率。 BW x is the bandwidth quota of the xth remote device, the xth remote device belongs to any of the remote devices connected to the local device, BW total is the total bandwidth that can be allocated by the local device, and Bx is the xth remote device Transmission code rate of the remote device, B total is the total transmission code rate required by all remote devices.
本申请实施例第三方面提供了一种远端设备,该远端设备执行前述第一方面以及第一方面任一实施方式的方法。A third aspect of the embodiments of the present application provides a remote device, where the remote device executes the foregoing first aspect and the method of any implementation manner of the first aspect.
本申请实施例第四方面提供了一种本地设备,该本地设备执行前述第二方面以及第二方面任一实施方式的方法。A fourth aspect of the embodiments of the present application provides a local device, where the local device executes the foregoing second aspect and the method of any implementation manner of the second aspect.
本申请实施例第五方面提供了一种计算机存储介质,该计算机存储介质中存储有指令,该指令在计算机上执行时,使得计算机执行前述第一方面以及第一方面任一实施方式或前述第二方面以及第二方面任一实施方式的方法。A fifth aspect of the embodiments of the present application provides a computer storage medium, where instructions are stored in the computer storage medium, and when the instructions are executed on a computer, the instructions cause the computer to execute the foregoing first aspect and any implementation manner of the first aspect or the foregoing first aspect. The method of the second aspect and any embodiment of the second aspect.
本申请实施例第六方面提供了一种计算机软件产品,该计算机程序产品在计算机上执行时,使得计算机执行前述第一方面以及第一方面任一实施方式或前述第二方面以及第二方面任一实施方式的方法。A sixth aspect of the embodiments of the present application provides a computer software product, which, when executed on a computer, enables the computer to execute the first aspect and any implementation manner of the first aspect or the second aspect and any of the second aspect. The method of one embodiment.
附图说明Description of drawings
图1为本申请实施例中一个网络架构示意图;1 is a schematic diagram of a network architecture in an embodiment of the present application;
图2为本申请实施例中一种数据传输方法流程示意图;2 is a schematic flowchart of a data transmission method in an embodiment of the present application;
图3为本申请实施例中本地设备动态分配网络带宽资源的一个逻辑示意图;3 is a schematic diagram of a logic diagram of a local device dynamically allocating network bandwidth resources in an embodiment of the present application;
图4为本申请实施例中基于优先级排序进行数据传输的一个示意图;4 is a schematic diagram of data transmission based on priority sorting in an embodiment of the present application;
图5为本申请实施例中一个动态控制数据传输的示意图;5 is a schematic diagram of a dynamic control data transmission in an embodiment of the application;
图6为本申请实施例中本地设备的一个结构示意图;6 is a schematic structural diagram of a local device in an embodiment of the present application;
图7为本申请实施例中远端设备的一个结构示意图;FIG. 7 is a schematic structural diagram of a remote device in an embodiment of the present application;
图8为本申请实施例中本地设备的另一个结构示意图;8 is another schematic structural diagram of a local device in an embodiment of the present application;
图9为本申请实施例中远端设备的另一个结构示意图;FIG. 9 is another schematic structural diagram of a remote device in an embodiment of the present application;
图10为本申请实施例中本地设备的另一个结构示意图;10 is another schematic structural diagram of a local device in an embodiment of the application;
图11为本申请实施例中远端设备的另一个结构示意图。FIG. 11 is another schematic structural diagram of a remote device in an embodiment of the present application.
具体实施方式detailed description
下面将结合本申请实施例中的附图,对本申请实施例中的技术方案进行描述。在本申请中,“至少一个”是指一个或者多个,“多个”是指两个或两个以上。“和/或”,描述关联对象的关联关系,表示可以存在三种关系,例如,A和/或B,可以表示:单独存在A,同时存在A和B,单独存在B的情况,其中A,B可以是单数或者复数。字符“/”一般表示前后关联对象是一种“或”的关系。“以下至少一项(个)”或其类似表达,是指的这些项中的任意组合,包括单项(个)或复数项(个)的任意组合。例如,a,b或c中的至少一项(个),可以表示:a,b,c,a和b,a和c,b和c或a和b和c,其中a、b和c可以是单个,也可以是多个。值得注意的是,“至少一项(个)”还可以解释成“一项(个)或多项(个)”。The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application. In this application, "at least one" means one or more, and "plurality" means two or more. "And/or", which describes the association relationship of the associated objects, indicates that there can be three kinds of relationships, for example, A and/or B, which can indicate: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A, B can be singular or plural. The character "/" generally indicates that the associated objects are an "or" relationship. "At least one item(s) below" or similar expressions thereof refer to any combination of these items, including any combination of single item(s) or plural items(s). For example, at least one (a) of a, b or c, can represent: a, b, c, a and b, a and c, b and c or a and b and c, where a, b and c can be It can be single or multiple. It is worth noting that "at least one item(s)" can also be interpreted as "one item(s) or more(s)".
需要说明的是,本申请中,“示例性的”或者“例如”等词用于表示作例子、例证或说明。本申请中被描述为“示例性的”或者“例如”的任何实施例或设计方案不应被解释为比其他实施例或设计方案更优选或更具优势。确切而言,使用“示例性的”或者“例如”等词旨在以具体方式呈现相关概念。It should be noted that, in this application, words such as "exemplary" or "for example" are used to represent examples, illustrations or illustrations. Any embodiment or design described in this application as "exemplary" or "such as" should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "such as" is intended to present the related concepts in a specific manner.
本申请实施例中出现的第一、第二等描述,仅作示意与区分描述对象之用,没有次序之分,也不表示本申请实施例中对设备个数的特别限定,不能构成对本申请实施例的任何限制。The descriptions of the first, second, etc. appearing in the embodiments of the present application are only used for illustration and distinguishing the description objects, and have no order. any limitations of the examples.
参阅图1,本申请实施例一个网络架构包括:Referring to FIG. 1, a network architecture according to an embodiment of the present application includes:
本地设备和远端设备,本申请实施例中远端设备的个数不做限制,本实施例仅以两个远端设备为例进行说明(远端设备1和远端设备2)。For the local device and the remote device, the number of the remote devices is not limited in this embodiment of the application, and this embodiment only takes two remote devices as an example for description (remote device 1 and remote device 2).
本地设备的多相机带宽协同分配模块可以,基于各跨设备相机的数据流业务实时性和实时传输码率协同动态分配各远端设备的网络带宽配额;远端设备的多相机流动态发送模块可以,基于带宽配额和各相机流的业务实时性要求,动态控制各相机流数据的发送,平滑传输码率并达成实时性要求。The multi-camera bandwidth cooperative allocation module of the local device can dynamically allocate the network bandwidth quota of each remote device based on the real-time data flow service and real-time transmission code rate of each cross-device camera; the multi-camera stream dynamic sending module of the remote device can , based on the bandwidth quota and the real-time requirements of each camera stream, dynamically control the data transmission of each camera stream, smooth the transmission bit rate and meet the real-time requirements.
本申请实施例中的本地设备和远端设备均可为终端设备,本申请实施例中称之为相机,又称之为用户设备(user equipment,UE)、移动台(mobile station,MS)、移动终端(mobile terminal,MT)等,是指向用户提供语音和/或数据连通性的设备。例如,具有无线连接功能的手持式设备、车载设备等。目前,一些终端的举例为:手机(mobile phone)、平板电脑、笔记本电脑、掌上电脑、移动互联网设备(mobile internet device,MID)、可穿戴设备,虚拟现实(virtual reality,VR)设备、增强现实(augmented reality,AR)设备、工业控制(industrial control)中的无线终端、无人驾驶(self driving)中的无线终端、远程手术(remote medical surgery)中的无线终端、智能电网(smart grid)中的无线终端、运输安全(transportation safety)中的无线终端、智慧城市(smart city)中的无线终端、智慧家庭(smart home)中的无线终端等。Both the local device and the remote device in the embodiments of the present application may be terminal devices, which are referred to as cameras in the embodiments of the present application, and are also referred to as user equipment (user equipment, UE), mobile station (mobile station, MS), A mobile terminal (mobile terminal, MT), etc., is a device that provides voice and/or data connectivity to a user. For example, handheld devices, in-vehicle devices, etc. with wireless connectivity. At present, some examples of terminals are: mobile phone (mobile phone), tablet computer, notebook computer, PDA, mobile internet device (MID), wearable device, virtual reality (virtual reality, VR) device, augmented reality (augmented reality, AR) equipment, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, and smart grids wireless terminals, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, etc.
本申请实施例提供了一种数据传输方法,用于分布式相机使用场景,即本地设备通过网络控制一个或多个远端设备,远端设备采集的相机数据需要传回到本地设备上使用。例 如手机借助大屏相机进行视频通话,本地手机控制远端手机相机/远端无人机相机实现多设备协同拍摄微录(Vlog)、直播等,具体应用场景此处不做限定。The embodiment of the present application provides a data transmission method for a distributed camera usage scenario, that is, a local device controls one or more remote devices through a network, and camera data collected by the remote device needs to be transmitted back to the local device for use. For example, the mobile phone uses the large-screen camera to make video calls, and the local mobile phone controls the remote mobile phone camera/remote drone camera to achieve multi-device collaborative video recording (Vlog), live broadcast, etc. The specific application scenarios are not limited here.
本申请实施例提供的技术方案可以应用于各种通信系统,例如:长期演进(long term evolution,LTE)系统、第五代(5th generation,5G)移动通信系统、无线保真(wireless-fidelity,WiFi)系统、未来的通信系统、或者多种通信系统融合的系统等,本申请实施例不做限定。其中,5G还可以称为新无线(new radio,NR)。The technical solutions provided in the embodiments of the present application can be applied to various communication systems, such as: long term evolution (LTE) system, fifth generation (5th generation, 5G) mobile communication system, wireless-fidelity (wireless-fidelity, WiFi) system, future communication system, or a system integrating multiple communication systems, etc., are not limited in the embodiments of the present application. Among them, 5G can also be called new radio (NR).
本申请实施例提供的技术方案可以应用于各种通信场景,例如可以应用于以下通信场景中的一种或多种:增强移动宽带(enhanced mobile broadband,eMBB)、超可靠低时延通信(ultra-reliable low-latency communication,URLLC)、机器类型通信(machine type communication,MTC)、大规模机器类型通信(massive machine type communications,mMTC)、设备到设备(device-to-device,D2D)、车辆外联(vehicle to everything,V2X)、车辆到车辆(vehicle to vehicle,V2V)、和物联网(internet of things,IoT)等。The technical solutions provided in the embodiments of the present application can be applied to various communication scenarios, for example, can be applied to one or more of the following communication scenarios: enhanced mobile broadband (eMBB), ultra-reliable and low-latency communication (ultra-reliable and low-latency communication) -reliable low-latency communication (URLLC), machine type communication (MTC), massive machine type communication (mMTC), device-to-device (D2D), off-vehicle Vehicle to everything (V2X), vehicle to vehicle (V2V), and Internet of things (IoT), etc.
参阅图2,本申请实施例一个数据传输方法包括:Referring to FIG. 2, a data transmission method according to an embodiment of the present application includes:
201、各远端设备采集拍摄数据;201. Each remote device collects shooting data;
本申请实施例中远端设备的个数不做限定,本实施例仅以两个为例进行说明,即第一远端设备和第二远端设备。The number of remote devices in this embodiment of the present application is not limited, and this embodiment only takes two remote devices as an example for description, that is, a first remote device and a second remote device.
第一远端设备和第二远端设备采集拍摄数据,第一远端设备的拍摄数据为第一远端设备的摄像头获取的数据,第二远端设备的拍摄数据为第二远端设备的摄像头获取的数据,摄像头获取的数据可以为图像数据、视屏数据等,具体此处不做限定。各远端设备的摄像头可以包括一个或多个,如第一远端设备的摄像头可以包括广角摄像头和长焦摄像头两个摄像头,具体的摄像头类型和个数根据实际应用场景确定,此处不做限定。The first remote device and the second remote device collect shooting data, the shooting data of the first remote device is the data obtained by the camera of the first remote device, and the shooting data of the second remote device is the data of the second remote device. The data acquired by the camera, the data acquired by the camera may be image data, video data, etc., which are not specifically limited here. The cameras of each remote device may include one or more cameras. For example, the camera of the first remote device may include two cameras, a wide-angle camera and a telephoto camera. The specific camera type and number are determined according to the actual application scenario, which is not described here. limited.
202、各远端设备计算传输码率;202. Each remote device calculates the transmission code rate;
第一远端设备和第二远端设备分别计算各自的传输码率,该传输码率为远端设备向本地设备发送拍摄数据所需要的码率。The first remote device and the second remote device calculate their respective transmission bit rates, where the transmission bit rates are the bit rates required by the remote device to send the shooting data to the local device.
拍摄数据有两种数据流类型,流(Stream)类型和Image类型。Stream类型为持续的相机数据流,例如预览流或视频流。Image类型为按需采集的数据流,例如拍照时的拍照大图和缩率图,每次用户触发拍照操作,才会生成对应的数据。There are two types of data streams for shooting data, Stream type and Image type. The Stream type is a continuous stream of camera data, such as a preview stream or a video stream. The Image type is a data stream collected on demand, such as a large image and a thumbnail image when taking a photo. The corresponding data is generated only when the user triggers a photo-taking operation.
1、Stream类型的码率计算:1. Stream type bit rate calculation:
考虑到带宽限制,拍摄数据可以经过压缩后再跨设备传输。为了保证压缩后的图像质量,一般采用动态比特率(variable bit rate,VBR)进行压缩,即随着图像复杂程度的不同动态调整码率,从而保证压缩后的相机流每帧画面质量都在一个比较高的水平。因此,需要动态实时预估Stream类型相机流码率。Taking into account bandwidth constraints, the captured data can be compressed before being transmitted across devices. In order to ensure the quality of the compressed image, the dynamic bit rate (VBR) is generally used for compression, that is, the bit rate is dynamically adjusted according to the complexity of the image, so as to ensure that the quality of each frame of the compressed camera stream is within the same relatively high level. Therefore, it is necessary to dynamically estimate the stream bit rate of the Stream type camera in real time.
单路Stream类型相机流码率B的计算方式可以为:The calculation method of the stream bit rate B of a single-channel Stream type camera can be as follows:
当t<T时:B=B sizeWhen t<T: B=B size ;
当t≥T时:
Figure PCTCN2021114112-appb-000001
When t≥T:
Figure PCTCN2021114112-appb-000001
即当小于一个单位计算时间时,B取默认码率,当大于一个单位计算时间时,通过1~n个单位计算时间内的实际数据量计算码率。That is, when it is less than one unit of calculation time, B takes the default code rate, and when it is greater than one unit of calculation time, the code rate is calculated by the actual data amount in 1 to n units of calculation time.
其中,t为远端设备的数据压缩模块已输出的相机流数据时长;T为单位计算时间,即 整数个视频流压缩的I帧间隔时间,一般取一个I帧间隔时间;B为单路Stream类型相机流码率;B size为xml中配置默认传输码率,与相机流的分辨率即大小(size)相关;D n为第n个单位计算时间内,远端设备的数据压缩模块输出的数据量;W n为D n在计算码率时所占的权重,一般取W n-1≥W n,n为大于等于1的正整数。 Among them, t is the duration of the camera stream data that has been output by the data compression module of the remote device; T is the unit calculation time, that is, the I-frame interval time for an integer number of video streams compressed, generally taking an I-frame interval; B is a single-channel Stream. Type camera stream bit rate; B size is the default transmission bit rate configured in xml, which is related to the resolution of the camera stream (size); D n is the nth unit of calculation time, the data compression module of the remote device output The amount of data; W n is the weight occupied by D n when calculating the code rate, generally W n-1 ≥ W n , and n is a positive integer greater than or equal to 1.
2、Image类型相机流码率B的计算方式可以为:2. The calculation method of the stream bit rate B of the Image type camera can be as follows:
Image类型相机流不是持续的数据流,传输时做单帧压缩,其码流计算方式如下:Image type camera stream is not a continuous data stream, single frame compression is performed during transmission, and its code stream is calculated as follows:
Figure PCTCN2021114112-appb-000002
Figure PCTCN2021114112-appb-000002
其中,B为单路Image类型相机流码率;D size为xml中配置默认的单帧图像压缩后的数据量,与相机流的分辨率即size相关;T latency为相机流的时延阈值,即业务可接受的该相机流数据的最大时延。 Among them, B is the bit rate of a single-channel Image type camera stream; D size is the data volume of the default single-frame image compression configured in xml, which is related to the resolution of the camera stream, that is, size; T latency is the delay threshold of the camera stream, That is, the maximum delay of the camera stream data acceptable to the service.
对于常用相机流,参阅表1,一个时延阈值配置例子如下:For common camera streams, see Table 1. An example of a delay threshold configuration is as follows:
表1Table 1
flow 流类型stream type 时延阈值Delay threshold
预览、分析流Preview, analyze flow STREAMSTREAM 0ms0ms
缩略图Thumbnail IMAGEIMAGE 0ms0ms
视频通话流video call stream STREAMSTREAM 100ms100ms
视频录像流video recording stream STREAMSTREAM 300ms300ms
拍照大图big picture IMAGEIMAGE 3000ms3000ms
相机流时延阈值支持业务应用主动配置,以满足业务对于实时性的不同要求。The camera stream delay threshold supports active configuration of business applications to meet different business requirements for real-time performance.
第x个远端设备的传输码率计算方式可以为:The calculation method of the transmission bit rate of the xth remote device can be:
B x=∑B i B x =∑B i
其中,B x为第x个远端设备的传输码率,可以理解的是x的取值范围等于远端设备的个数;B i为第x个远端设备的第i路流的传输码率,i为不小于1的正整数。 Among them, B x is the transmission code rate of the xth remote device. It can be understood that the value range of x is equal to the number of remote devices; B i is the transmission code of the i-th stream of the xth remote device. rate, i is a positive integer not less than 1.
203、本地设备接收各远端设备的传输码率;203. The local device receives the transmission bit rate of each remote device;
参阅图3,实现逻辑示意图如下,本地设备和远端设备间数据传输可以通过有线方式也可以通过无线方式,具体此处不做限定。Referring to FIG. 3 , a schematic diagram of the implementation logic is as follows. Data transmission between the local device and the remote device may be in a wired manner or a wireless manner, which is not specifically limited here.
204、本地设备根据传输码率为各远端设备配置带宽资源;204. The local device configures bandwidth resources for each remote device according to the transmission bit rate;
本地设备根据各远端设备的传输码率动态分配各远端设备的网络带宽配额。配置方式如下:The local device dynamically allocates the network bandwidth quota of each remote device according to the transmission bit rate of each remote device. The configuration is as follows:
BW x=BW total*(B x/B total); BW x =BW total *(B x /B total );
其中,BW x为第x个远端设备的带宽配额;BW total为本地设备可分配的总带宽;B x为第x个远端设备的传输码率;B total为所有远端设备所需的总的传输码率。 Among them, BW x is the bandwidth quota of the xth remote device; BW total is the total bandwidth that can be allocated by the local device; Bx is the transmission bit rate of the xth remote device; B total is the required amount of all remote devices The total transmission bit rate.
触发本地设备进行带宽配额更新的条件可以为:有相机流停止或者有新的相机流开始,或者,本地设备按照固定的预设周期T对各远端设备的带宽资源进行配置,具体次数不做限制。The conditions for triggering the local device to update the bandwidth quota may be: a camera stream stops or a new camera stream starts, or the local device configures the bandwidth resources of each remote device according to a fixed preset period T, and the specific number of times is not required. limit.
可以理解的是,步骤202至步骤204可以不执行,如当只有一个远端设备是,远端设备和本地设备进行数据传输时可以占用网络中可用的全部带宽,即不需要本地设备配置带宽。It can be understood that steps 202 to 204 may not be executed. For example, when there is only one remote device, the data transmission between the remote device and the local device may occupy all available bandwidth in the network, that is, the local device does not need to configure bandwidth.
205、各远端设备确定拍摄数据的优先级排序;205. Each remote device determines the priority ordering of the shooting data;
在多相机流并发传输时,基于相机流的实时性要求依次发送,优先级排序的规则为拍摄数据的时延阈值越小,拍摄数据的优先级越高,优先级高的数据先发送,从而保证实时性要求高的相机流优先被传输,实时性要求低的相机流被延迟滞后发送,减少实时性高的相机流的跨设备时延,提升用户体验。When multiple camera streams are transmitted concurrently, they are sent in sequence based on the real-time requirements of the camera streams. The priority ordering rule is that the smaller the delay threshold of the captured data, the higher the priority of the captured data, and the higher priority data is sent first. Ensure that camera streams with high real-time requirements are transmitted first, and camera streams with low real-time requirements are sent with a delay, reducing the cross-device delay of camera streams with high real-time requirements, and improving user experience.
常见的优先级排序规则可以是:优先级按照预览数据、拍照缩略图数据、视频通话数据、视频录像数据、拍照大图数据依次递减。可以理解的是该规则不是唯一的规则,具体排序规则可以还通过用户设置等方式获取。A common priority sorting rule may be as follows: the priority is in descending order according to preview data, photo thumbnail data, video call data, video recording data, and photo large image data. It can be understood that this rule is not the only rule, and the specific sorting rule can also be obtained through user settings and other methods.
206、各远端设备向本地设备发送拍摄数据,发送顺序基于优先级排序。206. Each remote device sends the shooting data to the local device, and the sending order is sorted based on the priority.
参阅图4,为基于优先级排序进行数据传输的一个例子,预览数据被优先发送,以保证用户可见的时延体验。剩余带宽用来优先发送缩略图数据和录像数据。可以看到拍照大图数据被拆成小块多次发送,以免影响预览、缩略图、录像的实时性。Referring to FIG. 4 , which is an example of data transmission based on priority sorting, preview data is preferentially sent to ensure a user-visible delay experience. The remaining bandwidth is used to send thumbnail data and video data preferentially. It can be seen that the large image data of the photo is divided into small pieces and sent multiple times, so as not to affect the real-time performance of preview, thumbnail and video.
图5为本申请实施例中动态控制数据传输的一个示意图,多数据类型(如包括预览数据和视频录像数据)的相机流动态发送的具体实现方式如下:5 is a schematic diagram of dynamic control data transmission in the embodiment of the application, and the specific implementation of dynamic transmission of camera streams of multiple data types (such as including preview data and video recording data) is as follows:
1、确定实时流(时延最小的Stream流,一般为预览),并取其帧率间隔T send为单位发送时间。可以有多个实时流,取帧率间隔最大的T send为单位发送时间,一般为预览。 1. Determine the real-time stream (the Stream stream with the smallest delay, generally a preview), and take its frame rate interval T send as the unit sending time. There can be multiple real-time streams, and the T send with the largest frame rate interval is taken as the unit sending time, which is generally a preview.
2、按单位发送时间T send控制数据发送,T send时间内可发送的数据量不超过D send2. The data transmission is controlled according to the unit transmission time T send , and the amount of data that can be sent within the time T send does not exceed D send :
D send=BW x*T sendD send =BW x *T send ;
其中,D send为单位发送时间T send内可发送的数据量;BW x为第x个远端设备的带宽配额;T send为单位发送时间,一般为预览帧间隔(如33.3ms)。 Among them, D send is the amount of data that can be sent within the unit sending time T send ; BW x is the bandwidth quota of the xth remote device; T send is the unit sending time, generally the preview frame interval (eg 33.3ms).
3、拍摄数据被采集后先送入各自的待发缓冲区,数据发送模块按相机流的实时性优先级从相应的缓冲区取数据并发送。只有当高优先级的相机流待发缓冲区为空时,才发送下一个优先级的相机流数据。3. After the shooting data is collected, it is first sent to the respective buffers to be sent. The data sending module fetches and sends data from the corresponding buffers according to the real-time priority of the camera stream. Only when the high-priority camera stream pending buffer is empty, the next-priority camera stream data is sent.
远端设备在T send时间内可发送的数据量不超过D send,如果所有缓冲区中均无数据,等待下一个T send发送时间。远端设备按单位发送时间T send发送数据,对于超大的相机数据帧,会被分组成小块多次发送。 The amount of data that the remote device can send within the time T send does not exceed D send . If there is no data in all buffers, it will wait for the next sending time of T send . The remote device sends data according to the unit sending time T send . For oversized camera data frames, it will be grouped into small blocks and sent multiple times.
本地设备的帧重组模块将相机数据重组成完整的相机帧。本地设备的稳帧率模块只对实时流做稳定帧率控制,因为实时流一般是直接送显的,需要平滑稳定输出。而其他相机 流的数据,待帧重组之后,直接送给相机流的消费端。The frame reassembly module of the local device reassembles the camera data into complete camera frames. The stable frame rate module of the local device only controls the stable frame rate of the real-time stream, because the real-time stream is generally sent directly to the display and needs smooth and stable output. The data of other camera streams, after frame reorganization, is directly sent to the consumer end of the camera stream.
通过本实施例,基于不同业务属性的相机流实时性要求和实时网络带宽,动态分配各个跨设备相机以及其各相机数据流的网络发送带宽,保证相机流的数据帧平滑稳定输出,并达成分布式相机各相机流的业务实时性要求。Through this embodiment, based on the real-time requirements of camera streams and real-time network bandwidth of different service attributes, the network transmission bandwidth of each cross-device camera and its data streams is dynamically allocated, so as to ensure smooth and stable output of data frames of camera streams, and achieve distribution. The business real-time requirements of each camera stream of the type camera.
上面对本申请实施例中的数据传输方法进行了描述,下面对本申请实施例中的装置进行描述,参阅图6,本地设备可以包括处理器、显示单元、收发单元(如无线带宽(WiFi)单元,有线传输单元)、电源、存储单元、视屏解码器等中一个或多个;参阅图7,远端设备可以包括处理器、相机单元、收发单元(如无线带宽(WiFi)单元,有线传输单元)、电源、视屏编码器、图片编码器等中一个或多个。The data transmission method in the embodiment of the present application is described above, and the apparatus in the embodiment of the present application is described below. Referring to FIG. 6, the local device may include a processor, a display unit, a transceiver unit (such as a wireless bandwidth (WiFi) unit, one or more of a wired transmission unit), a power supply, a storage unit, a video decoder, etc.; referring to FIG. 7, the remote device may include a processor, a camera unit, a transceiver unit (such as a wireless bandwidth (WiFi) unit, a wired transmission unit) , power supply, video encoder, picture encoder, etc. one or more.
请参阅图8,本申请实施例中本地设备一个实施例包括:Referring to FIG. 8, an embodiment of the local device in the embodiment of the present application includes:
接收单元801,用于接收远端设备发送的传输码率,传输码率为远端设备向本地设备发送拍摄数据所需要的码率,拍摄数据为远端设备的摄像头获取的数据,远端设备的摄像头包括一个或多个,本地设备和两个或两个以上的远端设备连接。The receiving unit 801 is used for receiving the transmission code rate sent by the remote device, the transmission code rate is the code rate required by the remote device to send the shooting data to the local device, the shooting data is the data obtained by the camera of the remote device, the remote device The camera consists of one or more local devices and two or more remote devices connected.
配置单元802,用于根据传输码率为各远端设备配置带宽资源。The configuration unit 802 is configured to configure bandwidth resources for each remote device according to the transmission code rate.
本实施例中,各单元所执行的操作与前图2所示实施例中,本地设备执行的操作描述的类似,此处不在赘述。In this embodiment, the operations performed by each unit are similar to the operations performed by the local device in the previous embodiment shown in FIG. 2 , and are not repeated here.
请参阅图9,本申请实施例中远端设备一个实施例包括:Referring to FIG. 9, an embodiment of the remote device in the embodiment of the present application includes:
采集单元901,用于采集拍摄数据,拍摄数据为远端设备的摄像头获取的数据,远端设备的摄像头包括一个或多个。The collection unit 901 is configured to collect shooting data, where the shooting data is data acquired by a camera of a remote device, and the camera of the remote device includes one or more cameras.
计算单元902,用于计算传输码率,以使得本地设备根据传输码率为远端设备配置带宽资源,传输码率为发送单元向本地设备发送拍摄数据所需要的码率。The calculation unit 902 is configured to calculate the transmission code rate, so that the local device configures the bandwidth resources of the remote device according to the transmission code rate, and the transmission code rate is the code rate required by the sending unit to send the shooting data to the local device.
确定单元903,用于确定拍摄数据的优先级排序。The determining unit 903 is configured to determine the priority ordering of the shooting data.
发送单元904,用于向本地设备发送拍摄数据,拍摄数据的发送顺序基于优先级排序。The sending unit 904 is configured to send the shooting data to the local device, and the sending order of the shooting data is sorted based on the priority.
本实施例中,各单元所执行的操作与前图2所示实施例中,远端设备执行的操作描述的类似,此处不在赘述。In this embodiment, the operations performed by each unit are similar to the operations performed by the remote device in the previous embodiment shown in FIG. 2 , which are not repeated here.
图10是本申请实施例提供的本地设备的结构示意图,该本地设备1000可以包括一个或一个以上处理器1001和存储器1005,该存储器1005中存储有一个或一个以上的应用程序或数据。10 is a schematic structural diagram of a local device provided by an embodiment of the present application. The local device 1000 may include one or more processors 1001 and a memory 1005, and the memory 1005 stores one or more application programs or data.
其中,存储器1005可以是易失性存储或持久存储。存储在存储器1005的程序可以包括一个或一个以上模块,每个模块可以包括对本地设备1000中的一系列指令操作。更进一步地,处理器1001可以设置为与存储器1005通信,在本地设备1000上执行存储器1005中的一系列指令操作。Among them, the memory 1005 may be volatile storage or persistent storage. A program stored in memory 1005 may include one or more modules, each of which may include a series of instructions to operate on local device 1000 . Furthermore, the processor 1001 may be configured to communicate with the memory 1005 to execute a series of instruction operations in the memory 1005 on the local device 1000 .
本地设备1000还可以包括一个或一个以上电源1002,一个或一个以上有线或无线网络接口1003,一个或一个以上输入输出接口1004,和/或,一个或一个以上操作系统,例如:微软操作系统(Windows),安卓操作系统(Android),苹果操作系统(Mac OS),尤尼克斯操作系统(Unix),里那克斯操作系统(Linux)中任一个。The local device 1000 may also include one or more power supplies 1002, one or more wired or wireless network interfaces 1003, one or more input and output interfaces 1004, and/or, one or more operating systems, such as: Microsoft operating system ( Any of Windows), Android operating system (Android), Apple operating system (Mac OS), Yonex operating system (Unix), and Linax operating system (Linux).
该处理器1001可以执行前述任一实施例中本地设备所执行的操作,具体此处不再赘述。The processor 1001 may perform the operations performed by the local device in any of the foregoing embodiments, and details are not described herein again.
图11是本申请实施例提供的远端设备的结构示意图,该远端设备1100可以包括一个 或一个以上处理器1101和存储器1105,该存储器1105中存储有一个或一个以上的应用程序或数据。11 is a schematic structural diagram of a remote device provided by an embodiment of the present application. The remote device 1100 may include one or more processors 1101 and a memory 1105, and one or more application programs or data are stored in the memory 1105.
其中,存储器1105可以是易失性存储或持久存储。存储在存储器1105的程序可以包括一个或一个以上模块,每个模块可以包括对远端设备1100中的一系列指令操作。更进一步地,处理器1101可以设置为与存储器1105通信,在远端设备1100上执行存储器1105中的一系列指令操作。Among them, the memory 1105 may be volatile storage or persistent storage. The programs stored in the memory 1105 may include one or more modules, each of which may include a series of instructions to operate on the remote device 1100 . Further, the processor 1101 may be configured to communicate with the memory 1105 to execute a series of instruction operations in the memory 1105 on the remote device 1100 .
远端设备1100还可以包括一个或一个以上电源1102,一个或一个以上有线或无线网络接口1103,一个或一个以上输入输出接口1104,和/或,一个或一个以上操作系统,例如:微软操作系统(Windows),安卓操作系统(Android),苹果操作系统(Mac OS),尤尼克斯操作系统(Unix),里那克斯操作系统(Linux)中任一个。The remote device 1100 may also include one or more power supplies 1102, one or more wired or wireless network interfaces 1103, one or more input and output interfaces 1104, and/or, one or more operating systems, such as Microsoft operating systems (Windows), Android operating system (Android), Apple operating system (Mac OS), Yonex operating system (Unix), any of Linax operating system (Linux).
该处理器1101可以执行前述任一实施例中远端设备所执行的操作,具体此处不再赘述。The processor 1101 may perform the operations performed by the remote device in any of the foregoing embodiments, and details are not described herein again.
所属领域的技术人员可以清楚地了解到,为描述的方便和简洁,上述描述的系统,装置和单元的具体工作过程,可以参考前述方法实施例中的对应过程,在此不再赘述。Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above may refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.
在本申请所提供的几个实施例中,应该理解到,所揭露的系统,装置和方法,可以通过其它的方式实现。例如,以上所描述的装置实施例仅仅是示意性的,例如,所述单元的划分,仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式,例如多个单元或组件可以结合或者可以集成到另一个系统,或一些特征可以忽略,或不执行。另一点,所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口,装置或单元的间接耦合或通信连接,可以是电性,机械或其它的形式。In the several embodiments provided in this application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的,作为单元显示的部件可以是或者也可以不是物理单元,即可以位于一个地方,或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.
另外,在本申请各个实施例中的各功能单元可以集成在一个处理单元中,也可以是各个单元单独物理存在,也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现,也可以采用软件功能单元的形式实现。In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时,可以存储在一个计算机可读取存储介质中。基于这样的理解,本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来,该计算机软件产品存储在一个存储介质中,包括若干指令用以使得一台计算机设备(可以是个人计算机,服务器,或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括:U盘、移动硬盘、只读存储器(ROM,read-only memory)、随机存取存储器(RAM,random access memory)、磁碟或者光盘等各种可以存储程序代码的介质。The integrated unit, if implemented in the form of a software functional unit and sold or used as an independent product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application can be embodied in the form of software products in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, and the computer software products are stored in a storage medium , including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), magnetic disk or optical disk and other media that can store program codes .

Claims (14)

  1. 一种数据传输方法,其特征在于,包括:A data transmission method, comprising:
    远端设备采集拍摄数据,所述拍摄数据为所述远端设备的摄像头获取的数据,所述远端设备的摄像头包括一个或多个;The remote device collects shooting data, and the shooting data is data obtained by a camera of the remote device, and the camera of the remote device includes one or more cameras;
    所述远端设备确定所述拍摄数据的优先级排序;the remote device determines the priority ordering of the shooting data;
    所述远端设备向本地设备发送所述拍摄数据,所述拍摄数据的发送顺序基于所述优先级排序。The remote device sends the shooting data to the local device, and the sending order of the shooting data is sorted based on the priority.
  2. 根据权利要求1所述的方法,其特征在于,所述优先级排序的规则为所述拍摄数据的时延阈值越小,所述拍摄数据的优先级越高。The method according to claim 1, wherein the priority ordering rule is that the smaller the delay threshold of the shooting data is, the higher the priority of the shooting data is.
  3. 根据权利要求1所述的方法,其特征在于,所述优先级排序的规则为优先级按照预览数据、拍照缩略图数据、视频通话数据、视频录像数据、拍照大图数据依次递减。The method according to claim 1, wherein the priority ordering rule is that the priority is sequentially decreased according to preview data, photographing thumbnail data, video call data, video recording data, and photographing large image data.
  4. 根据权利要求1至3中任一项所述的方法,其特征在于,所述方法还包括:The method according to any one of claims 1 to 3, wherein the method further comprises:
    所述远端设备计算传输码率,以使得所述本地设备根据所述传输码率为所述远端设备配置带宽资源,所述传输码率为所述远端设备向所述本地设备发送拍摄数据所需要的码率。The remote device calculates a transmission code rate, so that the local device configures bandwidth resources for the remote device according to the transmission code rate, which is the transmission code rate for the remote device to send a picture to the local device. The bit rate required for the data.
  5. 一种数据传输方法,其特征在于,包括:A data transmission method, comprising:
    本地设备接收远端设备发送的传输码率,所述传输码率为所述远端设备向所述本地设备发送拍摄数据所需要的码率,所述拍摄数据为所述远端设备的摄像头获取的数据,所述远端设备的摄像头包括一个或多个,本地设备和两个或两个以上的远端设备连接;The local device receives the transmission bit rate sent by the remote device, and the transmission bit rate is the bit rate required by the remote device to send shooting data to the local device, and the shooting data is obtained by the camera of the remote device The data of the remote device includes one or more cameras, and the local device is connected to two or more remote devices;
    所述本地设备根据所述传输码率为各远端设备配置带宽资源。The local device configures bandwidth resources for each remote device according to the transmission bit rate.
  6. 根据权利要求5所述的方法,其特征在于,所述本地设备根据所述传输码率为各远端设备配置带宽资源的配置方式包括:The method according to claim 5, wherein the configuration method of the local device configuring bandwidth resources for each remote device according to the transmission code rate comprises:
    BW x=BW total*(B x/B total); BW x =BW total *(B x /B total );
    所述BW x为第x个远端设备的带宽配额,所述第x个远端设备属于和所述本地设备相连的远端设备中的任一个,所述BW total为所述本地设备可分配的总带宽,所述B x为所述第x个远端设备的传输码率,所述B total为所有所有远端设备所需的总的传输码率。 The BW x is the bandwidth quota of the xth remote device, the xth remote device belongs to any one of the remote devices connected to the local device, and the BW total is allocable to the local device The total bandwidth of , the B x is the transmission code rate of the xth remote device, and the B total is the total transmission code rate required by all remote devices.
  7. 一种远端设备,其特征在于,包括:A remote device, characterized in that it includes:
    采集单元,用于采集拍摄数据,所述拍摄数据为所述远端设备的摄像头获取的数据,所述远端设备的摄像头包括一个或多个;a collection unit, configured to collect shooting data, where the shooting data is data obtained by a camera of the remote device, and the camera of the remote device includes one or more cameras;
    确定单元,用于确定所述拍摄数据的优先级排序;a determining unit, configured to determine the priority ordering of the shooting data;
    发送单元,用于向本地设备发送所述拍摄数据,所述拍摄数据的发送顺序基于所述优先级排序。A sending unit, configured to send the shooting data to the local device, where the sending order of the shooting data is sorted based on the priority.
  8. 根据权利要求7所述的设备,其特征在于,所述优先级排序的规则为所述拍摄数据的时延阈值越小,所述拍摄数据的优先级越高。The device according to claim 7, wherein the priority ordering rule is that the smaller the delay threshold of the shooting data, the higher the priority of the shooting data.
  9. 根据权利要求7所述的设备,其特征在于,所述优先级排序的规则为优先级按照预览数据、拍照缩略图数据、视频通话数据、视频录像数据、拍照大图数据依次递减。The device according to claim 7, wherein the priority ordering rule is that the priority is sequentially decreased according to preview data, photographing thumbnail data, video call data, video recording data, and photographing large image data.
  10. 根据权利要求7至9中任一项所述的设备,其特征在于,所述远端设备还包括:The device according to any one of claims 7 to 9, wherein the remote device further comprises:
    计算单元,用于计算传输码率,以使得所述本地设备根据所述传输码率为所述远端设备配置带宽资源,所述传输码率为所述发送单元向所述本地设备发送拍摄数据所需要的码率。a calculation unit, configured to calculate a transmission code rate, so that the local device configures bandwidth resources for the remote device according to the transmission code rate, and the transmission code rate is to send the shooting data to the local device by the sending unit the required bit rate.
  11. 一种本地设备,其特征在于,包括:A local device, characterized in that it includes:
    接收单元,用于接收远端设备发送的传输码率,所述传输码率为所述远端设备向所述本地设备发送拍摄数据所需要的码率,所述拍摄数据为所述远端设备的摄像头获取的数据,所述远端设备的摄像头包括一个或多个,本地设备和两个或两个以上的远端设备连接;A receiving unit, configured to receive a transmission bit rate sent by a remote device, where the transmission bit rate is a bit rate required by the remote device to send shooting data to the local device, and the shooting data is the remote device The data obtained by the camera, the camera of the remote device includes one or more, and the local device is connected to two or more remote devices;
    配置单元,用于根据所述传输码率为各远端设备配置带宽资源。A configuration unit, configured to configure bandwidth resources for each remote device according to the transmission code rate.
  12. 根据权利要求11所述的设备,其特征在于,所述配置单元的配置方式包括:The device according to claim 11, wherein the configuration mode of the configuration unit comprises:
    BW x=BW total*(B x/B total); BW x =BW total *(B x /B total );
    所述BW x为第x个远端设备的带宽配额,所述第x个远端设备属于和所述本地设备相连的远端设备中的任一个,所述BW total为所述本地设备可分配的总带宽,所述B x为所述第x个远端设备的传输码率,所述B total为所有所有远端设备所需的总的传输码率。 The BW x is the bandwidth quota of the xth remote device, the xth remote device belongs to any one of the remote devices connected to the local device, and the BW total is allocable to the local device The total bandwidth of , the B x is the transmission code rate of the xth remote device, and the B total is the total transmission code rate required by all remote devices.
  13. 一种计算机存储介质,其特征在于,所述计算机存储介质中存储有指令,所述指令在计算机上执行时,使得所述计算机执行如权利要求1至6中任一项所述的方法。A computer storage medium, characterized in that the computer storage medium stores instructions, and when executed on a computer, the instructions cause the computer to execute the method according to any one of claims 1 to 6.
  14. 一种计算机程序产品,其特征在于,所述计算机程序产品在计算机上执行时,使得所述计算机执行如权利要求1至6中任一项所述的方法。A computer program product, characterized in that, when executed on a computer, the computer program product causes the computer to execute the method according to any one of claims 1 to 6.
PCT/CN2021/114112 2020-09-25 2021-08-23 Data transmission method and related apparatus WO2022062799A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011027401.8 2020-09-25
CN202011027401.8A CN114257820A (en) 2020-09-25 2020-09-25 Data transmission method and related device

Publications (1)

Publication Number Publication Date
WO2022062799A1 true WO2022062799A1 (en) 2022-03-31

Family

ID=80789406

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/114112 WO2022062799A1 (en) 2020-09-25 2021-08-23 Data transmission method and related apparatus

Country Status (2)

Country Link
CN (1) CN114257820A (en)
WO (1) WO2022062799A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115442408A (en) * 2022-09-13 2022-12-06 脉冲视觉(北京)科技有限公司 Image data transmission processing method, device, medium and electronic equipment

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106682644A (en) * 2017-01-11 2017-05-17 同观科技(深圳)有限公司 Double dynamic vehicle monitoring management system and method based on mobile vedio shooting device
WO2017165428A1 (en) * 2016-03-22 2017-09-28 Sensormatic Electronics, LLC Method and system for surveillance camera arbitration of uplink consumption
CN107995017A (en) * 2016-10-27 2018-05-04 杭州萤石网络有限公司 A kind of uplink bandwidth allocation method, apparatus and system
CN108093209A (en) * 2016-11-21 2018-05-29 松下知识产权经营株式会社 Image transmission system and dollying machine equipment
CN109040674A (en) * 2018-07-23 2018-12-18 华为技术有限公司 A kind of data transmission method and device

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN100412832C (en) * 2003-09-02 2008-08-20 竺红卫 Non-homogeneous multi media flow transmission regulation method based on priority regulation
JP4445477B2 (en) * 2006-02-24 2010-04-07 株式会社東芝 Video surveillance system
KR100991767B1 (en) * 2007-12-24 2010-11-03 한국전자통신연구원 System for allocating uplink resource of mobile internet and its method
CN103986666A (en) * 2013-02-07 2014-08-13 华为技术有限公司 Data processing method and device and system
CN104994590A (en) * 2015-06-29 2015-10-21 上海华为技术有限公司 Data transmission method and device
CN106302254A (en) * 2016-07-29 2017-01-04 山东大学苏州研究院 A kind of internet of things data forwarding system based on SDN
CN109495326B (en) * 2018-12-28 2021-12-14 北京东土科技股份有限公司 Network bandwidth allocation method and system
CN111279690A (en) * 2019-01-30 2020-06-12 深圳市大疆创新科技有限公司 Multi-load graph transmission method, control system, control terminal, unmanned aerial vehicle and server
CN109962760B (en) * 2019-04-09 2022-08-30 湖南智领通信科技有限公司 Service scheduling method suitable for wireless TDMA ad hoc network

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017165428A1 (en) * 2016-03-22 2017-09-28 Sensormatic Electronics, LLC Method and system for surveillance camera arbitration of uplink consumption
CN107995017A (en) * 2016-10-27 2018-05-04 杭州萤石网络有限公司 A kind of uplink bandwidth allocation method, apparatus and system
CN108093209A (en) * 2016-11-21 2018-05-29 松下知识产权经营株式会社 Image transmission system and dollying machine equipment
CN106682644A (en) * 2017-01-11 2017-05-17 同观科技(深圳)有限公司 Double dynamic vehicle monitoring management system and method based on mobile vedio shooting device
CN109040674A (en) * 2018-07-23 2018-12-18 华为技术有限公司 A kind of data transmission method and device

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115442408A (en) * 2022-09-13 2022-12-06 脉冲视觉(北京)科技有限公司 Image data transmission processing method, device, medium and electronic equipment

Also Published As

Publication number Publication date
CN114257820A (en) 2022-03-29

Similar Documents

Publication Publication Date Title
CN104394486B (en) A kind of video stream processing method and device based on cloud desktop
US20230276430A1 (en) Resource scheduling method and apparatus, communication device and storage medium
CN109194984B (en) Video frame scheduling method based on edge calculation
US8005957B2 (en) Network traffic prioritization
EP2739073B1 (en) Wireless network, implementation method thereof, and terminal
WO2018218491A1 (en) Data transmission method and device
WO2019019032A1 (en) Method and apparatus for configuring downlink data packet
US20200252337A1 (en) Data transmission method, device, and computer storage medium
WO2022062799A1 (en) Data transmission method and related apparatus
CN116016556A (en) Control method of screen projection system, display device and readable storage medium
CN108141902B (en) Wireless data acknowledgment communication using frame aggregation
CN110399210B (en) Task scheduling method and device based on edge cloud
CN111669262A (en) Data transmission method and device, electronic equipment and medium
CN106686635B (en) Data transmission method and device based on control and configuration protocol of wireless access point
US11051206B2 (en) Wi-fi optimization for untethered multi-user virtual reality
US20220330074A1 (en) Communication method and apparatus
WO2020010507A1 (en) Method for adjusting communication link, communication device, and unmanned aerial vehicle
US9182941B2 (en) Flow control with buffer reclamation
CN109922318B (en) Multi-channel video transmission scheduling method and device, storage medium and electronic terminal
WO2021109141A1 (en) Communication connection configuration method and apparatus, and communication device and storage medium
CN115362672A (en) Method and apparatus for volumetric three-dimensional session service using network edge
WO2023093461A1 (en) Resource allocation method in dual wifi mode, and device and storage medium
WO2024120309A1 (en) Rendering method, apparatus and system
CN106331591B (en) The method for synchronously switching and device of monitored picture
CN114928757B (en) 360-degree VR video data transmission method and device based on millimeter waves

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21871172

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21871172

Country of ref document: EP

Kind code of ref document: A1