CN107333143B - 5G multi-access concurrent transmission control system and method - Google Patents
5G multi-access concurrent transmission control system and method Download PDFInfo
- Publication number
- CN107333143B CN107333143B CN201710493749.8A CN201710493749A CN107333143B CN 107333143 B CN107333143 B CN 107333143B CN 201710493749 A CN201710493749 A CN 201710493749A CN 107333143 B CN107333143 B CN 107333143B
- Authority
- CN
- China
- Prior art keywords
- terminal
- video
- transmission
- cooperative
- server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000005540 biological transmission Effects 0.000 title claims abstract description 113
- 238000000034 method Methods 0.000 title abstract description 39
- 239000000872 buffer Substances 0.000 claims description 99
- 238000004891 communication Methods 0.000 claims description 10
- 230000006978 adaptation Effects 0.000 claims description 9
- 238000012544 monitoring process Methods 0.000 claims description 9
- 238000013475 authorization Methods 0.000 claims description 5
- 238000005516 engineering process Methods 0.000 abstract description 10
- 238000011084 recovery Methods 0.000 abstract description 4
- 230000002265 prevention Effects 0.000 abstract 1
- 238000007726 management method Methods 0.000 description 59
- 230000006870 function Effects 0.000 description 17
- 230000008569 process Effects 0.000 description 12
- 238000012423 maintenance Methods 0.000 description 11
- 230000003139 buffering effect Effects 0.000 description 9
- 238000010586 diagram Methods 0.000 description 8
- 230000007246 mechanism Effects 0.000 description 7
- 238000010295 mobile communication Methods 0.000 description 7
- 238000004458 analytical method Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 4
- 238000013461 design Methods 0.000 description 4
- 238000011161 development Methods 0.000 description 4
- 230000018109 developmental process Effects 0.000 description 4
- 230000003993 interaction Effects 0.000 description 4
- 230000003044 adaptive effect Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 230000009467 reduction Effects 0.000 description 3
- 238000012163 sequencing technique Methods 0.000 description 3
- 238000001228 spectrum Methods 0.000 description 3
- 230000002776 aggregation Effects 0.000 description 2
- 238000004220 aggregation Methods 0.000 description 2
- 230000000977 initiatory effect Effects 0.000 description 2
- 230000002452 interceptive effect Effects 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 208000009115 Anorectal Malformations Diseases 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 150000001875 compounds Chemical class 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 238000013467 fragmentation Methods 0.000 description 1
- 238000006062 fragmentation reaction Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 230000011664 signaling Effects 0.000 description 1
- 238000002054 transplantation Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
- H04N21/234336—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by media transcoding, e.g. video is transformed into a slideshow of still pictures or audio is converted into text
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/24—Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
- H04N21/2402—Monitoring of the downstream path of the transmission network, e.g. bandwidth available
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/258—Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data
- H04N21/25808—Management of client data
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/262—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists
- H04N21/26208—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists the scheduling operation being performed under constraints
- H04N21/26216—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists the scheduling operation being performed under constraints involving the channel capacity, e.g. network bandwidth
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
- H04N21/4363—Adapting the video stream to a specific local network, e.g. a Bluetooth® network
- H04N21/43637—Adapting the video stream to a specific local network, e.g. a Bluetooth® network involving a wireless protocol, e.g. Bluetooth, RF or wireless LAN [IEEE 802.11]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440236—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by media transcoding, e.g. video is transformed into a slideshow of still pictures, audio is converted into text
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
- H04N21/44227—Monitoring of local network, e.g. connection or bandwidth variations; Detecting new devices in the local network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/631—Multimode Transmission, e.g. transmitting basic layers and enhancement layers of the content over different transmission paths or transmitting with different error corrections, different keys or with different transmission protocols
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/643—Communication protocols
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Databases & Information Systems (AREA)
- Computer Graphics (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a 5G multi-access concurrent transmission control system and a method, which are used for flexibly selecting the number of accessed terminals according to network conditions by the cooperative work of various wireless networks. If the network condition is good, the network control platform reduces the number of the cooperative terminals as much as possible; if the network condition is poor, the network control platform calls more mobile terminals to cooperatively transmit video data in a multi-stream concurrent manner to improve user experience, and also can transmit important video information (usually boundary information) through the best network, and other information can be transmitted through other networks. The key technologies designed by the invention are a packet loss prevention and recovery method for video transmission and a video caching method based on queue prediction.
Description
Technical Field
The invention belongs to the field of wireless network streaming media transmission, and relates to a 5G multi-access concurrent transmission control system and a method.
Background
For the greatly increased wireless connection requirement, the enhanced mobile broadband access service requirement, and the very high reliability and service guarantee requirement of the emerging communication mode in the future network society, 5G defines and develops the corresponding mobile communication technology. For the fifth generation mobile communication radio access networks of the future, it has now become increasingly clear that it will be a very flexible, heterogeneous access system and will integrate a large number of technological evolutions that can be used to configure simultaneous radio connections towards a large number of applications with different characteristics and requirements. In order to meet the supporting demands for the high demand, the fifth generation mobile communication systems in the future will need to have targeted access to radio spectrum resources with different characteristics.
The global mobile communication industry has envisioned a wide variety of application/service modalities for future fifth generation mobile communication systems. The method not only comprises the enhancement of the application scenarios of the existing fourth generation mobile communication system, but also comprises some emerging applications, such as: 4K ultra-high definition video, 8K ultra-high definition video, VR (Virtual Reality), AR (Augmented Reality), mobile games, IoT (internet of things), wearable device applications, emergency task applications (oriented to vertical industry and business). The ITU-R (international telecommunications union radio communication bureau) has identified three major application scenarios of 5G (1) enhanced mobile broadband; (2) ultra-high reliability and low latency communications; (3) large-scale machine-type communication.
The increase of the demand of wireless service users also puts pressure on the expansion and upgrading of mobile communication wireless access networks, and wireless network congestion needs to be relieved by adopting a mode of accessing more available spectrum resources or adopting an advanced spectrum sharing access mechanism. Therefore, the design of the 5G multi-access concurrent transmission control method has important significance for realizing the sensing and coordination capability among heterogeneous terminals and improving the utilization rate of a system network.
Disclosure of Invention
The technical problem is as follows: aiming at the fact that various terminals in a 5G multi-access wireless network (including 2G, 3G, 4G and 5G, WiFi) environment have high heterogeneity, the invention designs a 5G multi-access concurrent transmission control system and a method, a plurality of heterogeneous wireless terminals work cooperatively through various wireless networks, and if the network condition is good, a network control platform reduces the number of cooperative terminals as much as possible; if the network condition is poor, the network control platform calls more mobile terminals to cooperatively transmit video data in a multi-stream concurrent mode to improve user experience, and also can transmit important video information (usually boundary information) through the best network, and other information can be transmitted through other networks, so that service diversification is realized and user experience is enhanced.
The technical scheme is as follows: the invention relates to a 5G multi-access concurrent transmission control system and a method, wherein the system comprises the following components:
(1) the network control platform comprises a network control management server, a terminal management server and a service server. The network management control server performs transmission control in a self-adaptive manner according to the service characteristics, the network condition and the terminal capability; the terminal management server is responsible for registering the terminal, monitoring and managing the terminal and reporting the state of the terminal to the network management control server; the service server will adapt the service according to the transmission capabilities of the existing network and the terminal.
(2) A terminal controller: and discovering the cooperative terminal and reporting the cooperative terminal to a terminal management server.
(3) And (3) coordinating the terminal: the method is used for cooperative long-distance transmission from the video server to the monitoring center client.
(4) And the video adaptation server realizes the distribution of the video.
(5) The video playing client side: and receiving the video data packet of the video server, performing operations such as caching, decoding and the like, and playing.
The connection or positional relationship of the above five components is shown in fig. 1.
The following respectively describes each component and the specific implementation process:
network management control server introduction: the network control platform monitors the network condition of each terminal within the coverage of the current terminal controller, and can select a certain number of available cooperative terminals to perform cooperative transmission of a service aiming at a certain service, such as a video service, and can adaptively increase or reduce the number of terminals participating in cooperative transmission according to the network condition and the link condition of each terminal in the transmission process, so that the transmission quality of the service can be ensured, the resources of the system can be saved to the maximum extent, the service continuity of the system in the reconstruction process can be ensured by coordinating each terminal, meanwhile, if a certain link of the system is blocked, a decision can be made in time, and a standby link is adopted to ensure the stability and robustness of the system transmission.
The network management control server interface is written by adopting C # language and developed based on VS2012 platform. The interface is mainly used for displaying access information of a terminal controller, registering, logging and logging out related information of the cooperative terminal, dynamically displaying information such as network state information, real-time rate, network system, service type, transmission state, connection state and the like of the cooperative terminal, decision information of a server and the like, and dynamically displaying a dynamic topological graph formed by a current system.
The method aims to introduce a self-adaptive adaptation technology into a multi-terminal cooperative transmission mechanism to realize the unified control of a plurality of cooperative terminals, so that the whole transmission system is more intelligent, and the transmission requirements of different services can be met.
The network management control server has the main functions: 1. receiving cooperative terminal registration, login and exit messages sent by a terminal controller, performing corresponding operations such as authorization processing and the like, and displaying the information on a server interface in real time; 2, performing access permission operation on the authorized cooperative terminal, monitoring related information such as a network state of the cooperative terminal and updating data on an interface in real time; 3, when a user plays a service request initiated by a client, the server selects a certain terminal from the currently available cooperative terminals for cooperative transmission, and monitors the real-time rate of the terminal; 4 when the network fluctuates or the cooperative terminal is reconstructed, the server can dynamically change the current transmission strategy and select a proper cooperative terminal to participate in cooperative transmission.
The network management control server implementation scheme is as follows: the network management control server is written by C # language and can be operated on a PC of the windows system. And a server interface is developed by adopting winform, and dynamic information and adaptive related information of the accessed cooperative terminal can be displayed in real time. When the network management control server is initialized, a socket is defined for responding to a connection request of the cooperative terminal, and simultaneously, a thread is started to monitor the request information of the user playing client. Then a socket dictionary is established for storing the connection of a plurality of users, when a cooperative terminal initiates a registration connection request, the IP address and the port of the cooperative terminal are stored in the socket dictionary, and a new thread is started to receive the real-time information sent by the terminal. After receiving the information sent by the terminal, the server acquires corresponding data and displays the information of the terminal and the real-time data on an interface of the server in a list form. And then, at intervals, the server receives the information sent by the terminal and updates the data in the list in real time. When receiving a service request of a user playing client, the network management control server selects a proper cooperative terminal from the current list to participate in multi-stream transmission, and the service starts. When there are multiple available coordination terminals in the list, the network management control server may perform adaptive adaptation, select an appropriate number of coordination terminals for the current service type (mainly video service), and comprehensively decide which terminals to use to complete the transmission of the service in consideration of the network conditions. In the running process of the system, the network management control server periodically checks the connection state of the terminals in the list, and deletes the information of a certain terminal when the terminal is disconnected. If the user playing client is disconnected, the network management control server is restored to the initial state, and the service request of the user is monitored in a circulating mode.
The specific implementation process comprises the following steps:
(1) program initialization, load server interface.
(2) And creating three Socket sockets, starting three threads, and monitoring messages from a terminal controller, a cooperative terminal and a user playing client at different communication ports respectively.
(3) When the terminal controller accesses, the cooperative terminal registration information sent by the terminal controller is received, authorized and allowed to log in.
(4) And creating a Socket dictionary for storing Socket connections of all the cooperative terminals. When an authorized cooperative terminal sends an access request, the Socket connection is added to a dictionary, relevant parameters such as a network system, signal strength, service type and the like are read from the information sent by the dictionary, and relevant information of the cooperative terminal is added to a listview list. Thereafter, at intervals, the server reads real-time network information from a Socket connected to the cooperative terminal, and dynamically updates related information in the list.
(5) When a video service request of a user playing client is received, one of the currently logged available terminals is immediately selected (the first one is selected by default) to participate in video transmission, that is, a control command for starting transmission is sent to the terminal, and meanwhile, the transmission state and the cooperative topological graph of the cooperative terminal in the list are updated, and after the cooperative terminal receives the command, the cooperative terminal starts to participate in video transmission.
(6) When the video service starts to be transmitted, the server monitors the network state and the transmission rate of the terminal participating in the cooperative transmission in real time, namely reads real-time data from the Socket connection of the terminal, and dynamically updates the real-time data in the list, wherein the data also serves as the basis for the self-adaptive decision of the server.
(7) Meanwhile, the server also waits for the feedback information of the user playing client, wherein the feedback information is decision information which is correspondingly judged and made by the playing client according to the occupation condition of the current buffer area. When the feedback information received by the server is "add", it indicates that the buffer area of the current playing client is lower than a certain threshold, and a pause buffering phenomenon may occur, so that the number of transmission links needs to be increased to improve throughput, so that at this time, if there is an available terminal, i.e., a terminal whose transmission state is "non-transmission", in the currently logged cooperative terminals, the server immediately sends a transmission start command to the terminal and updates the state in its list, and if there is no available terminal, no operation is performed. When the feedback information received by the server is "del", it indicates that the buffer area of the current playing client is higher than a certain threshold value, and a buffer overflow phenomenon may occur, so that the number of transmission links needs to be reduced to reduce the receiving rate, so that at this time, if the number of terminals currently participating in cooperative transmission, that is, terminals whose transmission states are "transmitting" is greater than 1, the server makes a decision immediately, selects a cooperative terminal with the lowest transmission rate, sends a command to stop transmission to the terminal, and updates the states in its list. If the number of the cooperative terminals participating in the transmission is 1, no operation is performed in order to maintain the continuity of the service.
(8) And after the user plays the service stop of the client, the server sequentially sends a transmission stop command to each terminal participating in the cooperative transmission, and the cooperative terminal stops transmitting the video service after receiving the transmission stop command. And the server updates the state information and the topological graph of each terminal in the list.
Video server and client profiles: the video server and the client are written in C + +, are developed based on VS2012, and provide video service providing and playing functions supporting multi-terminal cooperative transmission. The video server can adaptively adjust the service throughput of each path of terminal according to the condition of each path of cooperative terminal. The client can combine and sort the services of the cooperative terminals for playing, and can also adaptively request to adjust the cooperative strategy according to the number of the cached video data frames.
A block diagram of a video server system is shown in fig. 4.
The video server has the following modules in concrete realization and functions:
a video source providing module:
the functions are as follows: and monitoring a new video frame request sent by the network connection management module, providing an original video picture, and supplying the original video picture to the video coding module for coding.
The realization is as follows: the module firstly receives a request of a terminal for connecting a scheduling module, reads the next video frame to be coded and transmitted after receiving a message of providing a new video frame, and transmits the video frame to a video coding module.
A video encoding module:
the functions are as follows: and receiving an original video frame from the video source providing module, carrying out H.264 coding, and sending the original video frame to the terminal connection scheduling module for ready transmission.
The realization is as follows: the module adopts an FFmpeg coding library to carry out H.264 coding, and sends an original video frame received from the video source providing module to the terminal connection scheduling module after coding.
The terminal is connected with the scheduling module:
the functions are as follows: (1) receiving the coding frame from the video coding module by the access (2) of the monitoring cooperative terminal, carrying out flow distribution decision, selecting a path of proper terminal connection, storing the terminal connection in a corresponding buffer area for waiting to be sent (3), when the frame transmission is finished, initiating a new video frame request to the video source providing module
The realization is as follows: (1) the shunt decision device (3) of the newly connected monitor (2) is connected with the maintenance and buffer manager (4) and the data transmitter
And the new connection monitor opens a port as a service data transmission port and monitors, and when a new cooperative terminal is accessed, the connected socket is sent to the connection maintenance and cache manager for management.
The shunting decision device receives the coded video frame sent by the video coding module, requests the connection maintenance and cache manager for the cached data volume of the socket of each connection, selects the connection with the minimum caching quantity, sends the video frame to the connection maintenance and cache manager, and requests the connection maintenance and cache manager to add the frame into the cache area of the connection just selected.
The connection maintenance and buffer manager manages all connections and video frame buffer areas corresponding to the connections, receives a query request from the shunt decision device, receives new connection access from a new connection monitor, and provides video data to be sent to the data transmitter.
The data transmitter is responsible for sending out the data in each connection buffer, the data transmitter regularly inquires the connection buffer status of the connection maintenance and buffer manager, and when the connected buffer is not empty, the connected buffer data is sent out through the corresponding socket.
A block diagram of a video client system is shown in fig. 5.
The video client specifically realizes and functions as follows:
the terminal is connected with the scheduling module:
the functions are as follows: this module realizes the following functions: (1) monitoring the access of a cooperative terminal; (2) receiving and caching the data of each path of connection; (3) and sequencing and splicing the received data packets, and sending the video frame to a frame management module after a complete frame is obtained.
The realization is as follows: the module consists of the following three parts: (1) the new connection monitor (2) is connected with a data transmitter of the maintenance and cache manager (3).
And the new connection monitor opens a port as a service data transmission port and monitors, and when a new cooperative terminal is accessed, the connected socket is sent to the connection maintenance and cache manager for management.
The connection maintenance and buffer manager manages all connections and video frame buffer areas corresponding to the connections, receives new connection access from a new connection monitor, receives video data of all connected cooperative terminals provided by the data transmitter, buffers the video data by adopting a queue, periodically checks the buffer queue, and provides a frame to the frame management module after all data packets of the frame of video are detected to be received.
The data transmitter is responsible for processing and receiving the video data received by each connection socket and sending the data to the connection maintenance and cache manager for sequencing and merging processing.
A frame management module:
the functions are as follows: this module realizes the following functions: (1) receiving complete video frames from a terminal connection scheduling module (2), sequencing the cached video frames according to frame time stamps, providing new video frames to a video decoding module (3) when receiving a new frame request of a video playing module, judging the throughput according to the number of the cached video frames, and initiating a request for adjusting a cooperative strategy to a network management control server
The realization is as follows: the module consists of the following three parts: (1) the frame buffer queue (2) is transmitted to the control decision device (3) and is transmitted to the frame play manager.
The frame buffer queue receives complete video frames from the terminal connection scheduling module, stores the frames in the queue according to the sequence of the time stamps, and receives frame length query requests and new video frame requests of the transmission control decision device and the frame play manager. In order to prevent the queue access overhead from being too large due to too many buffer frame numbers, the upper limit of the buffer capacity is set to be 500 for the queue, when the length of the queue reaches the upper limit, a new video frame is not received any more, and a terminal connection scheduling module is blocked to stop receiving data from each connection.
The frame play manager is used for receiving a new frame request of the video play module and providing a video frame to the video decoding module. The frame play manager sets two frame buffer queue length thresholds, a 60 and B90, to decide whether to wait for buffering. When one frame is played, the video playing module requests a new frame from the frame playing manager, and at this time, the frame playing manager requests the queue length from the frame buffer queue. When the length is less than the threshold A, the state is set to be in buffering, and the video decoding module is blocked to enable the video playing to pause and wait for buffering. When the length is larger than the threshold B, the state is set to be in playing, and the next frame which is requested to be played from the frame buffer queue is transmitted to the video decoding module for decoding and playing.
The transmission control decision device judges the length of the frame buffer queue at regular time, and sets two thresholds A to 150 and B to 300 for transmission control adjustment judgment. And when the video frame number is lower than the threshold A, judging that the throughput can not meet the service requirement, sending a request for adjusting and increasing the throughput to a network management control server, and then sleeping the transmission control decision device for 10 seconds, waiting for adaptive adjustment and then performing next judgment. And when the video frame number is higher than the threshold B, judging that the throughput exceeds the requirement, sending a request for adjusting and reducing the throughput to a network management control server, sleeping the transmission control decision device for 10 seconds, waiting for self-adaptive adjustment, and then judging the next time.
The video decoding module:
the functions are as follows: and receiving the encoded video frame from the frame management module, decoding the encoded video frame by H.264, and sending the decoded video frame to the video playing module for playing.
The realization is as follows: the module adopts an FFmpeg coding library to perform H.264 decoding, decodes the original video frame received from the frame management module and then sends the decoded original video frame to the video playing module.
The video playing module:
the functions are as follows: and requesting a new video frame from the frame management module at regular time according to the video frame rate, and receiving the video frame decoded by the video decoding module for playing.
The realization is as follows: the module sends a new frame request to the frame management module according to the frequency of the video frame rate (30 frames per second), waits for the video frame decoded from the video decoding module, and displays the video frame by adopting an OpenGL graphic interface.
Introduction of a terminal controller: the multi-terminal cooperation mechanism refers to that a plurality of mobile terminals are organically connected together and jointly complete a certain service. The whole multi-terminal cooperation mechanism comprises the discovery and registration processes of the mobile terminal. Only if the registration operation is successfully completed on the terminal controller, the terminal controller can realize the control of the mobile terminal and coordinate a plurality of mobile terminals to jointly complete the service.
The terminal controller has the main functions: (1) discovering available mobile terminals around; (2) identifying the registration information of the mobile terminal and completing an authorization function; (3) and realizing services by cooperating with each mobile terminal.
Functions of the mobile terminal: (1) sending registration information to a terminal controller; (2) detecting status information of a network; (3) receiving authorization information of a terminal controller, and sending detected network condition information to a network management control server; (4) and cooperatively completing the service under the control of the terminal controller.
The mobile terminal is an ARM development board taking Linux as a system environment, and the terminal controller is a small-sized base station and is provided with a relatively independent database for storing the equipment information and the network information of the accessed mobile terminal. The terminal controller establishes socket connection with each mobile terminal, realizes data interaction between each terminal, and completes various services cooperatively.
The terminal controller needs to actively discover surrounding mobile terminals to organize the mobile terminals to collaborate cooperatively. The discovery of the terminal is in fact simple, as long as the terminal controller periodically sends broadcast information to the surroundings. Once the mobile terminal enters the radiation range of the terminal controller, the mobile terminal receives the broadcast information from the terminal controller and then analyzes the broadcast information to obtain the relevant information of the terminal controller. And the mobile terminal establishes connection with the terminal controller according to the information obtained by analysis.
The broadcast of the terminal controller uses UDP protocol, and the connection established between the mobile terminal and the terminal controller uses TCP protocol. The TCP protocol is more reliable than the UDP protocol, so that the reliability of signaling interaction can be ensured.
After the mobile terminal and the terminal controller are connected, the mobile terminal can actively send registration information to the terminal controller. The registration information mainly includes a user name and a password of the terminal, which are written in a program of the terminal device, and the information of each mobile terminal is different. After receiving the registration information, the terminal controller calls a user form in the database to check the information. When the registration information is accurate, the terminal controller replies confirmation information to the terminal. Meanwhile, the terminal controller establishes connection with the network management control server and sends terminal access information to the terminal controller. After receiving the reply information from the terminal controller, the mobile terminal uploads the information of 3G signal intensity, transmission rate and the like to the network management control server, so that a basis is provided for a service adaptation decision of the network management control server. After the network management control server makes a service adaptation decision, the mobile terminal can be controlled by the terminal controller to transmit the video service.
The login field in the message content is used to indicate the registration information. The database in the embedded development board needs manual transplantation, and the SQLite is selected as the database of the terminal controller due to the limited memory of the development board. The migration process of the database is also relatively simple and will not be described herein.
The signal strength of the 4G network and the transmission rate of the video data are both measured by the application running in the background. The signal strength of the 4G network is obtained by writing an "AT + CSQ" command to the USB interface, which involves the initialization operation of the USB interface and the programming of serial communication. As long as the data rate, the data bit, the stop bit, the check bit and other information are set in the serial port initialization process, then a command can be written into the USB interface, the reply information is read out, and useful data information is analyzed from the reply information. The detection of the data rate mainly provides a basis for service adaptation decision so as to make a reasonable and effective adaptation decision.
The video client initiates service request information to the network management control server, the network management control server initiates a transmission command to the mobile terminal according to the service request of the video client, and the plurality of mobile terminals download video data from the service server on the public network by using the self 4G network and transmit the video data to the video client through a Wireless Local Area Network (WLAN), so that the plurality of mobile terminals cooperatively transmit video services.
The method specifically comprises the following steps:
(1) opening a terminal controller, starting a terminal controller program, establishing UDP socket connection, broadcasting information to the periphery, and establishing TCP connection with a network management control server;
(2) opening the mobile terminal equipment, opening a 4G network and a WLAN network, and starting a mobile terminal program;
(3) a terminal program creates a UDP socket, receives broadcast information of a terminal controller, analyzes the broadcast information, establishes TCP connection with the terminal controller and uploads terminal equipment registration information;
(4) the video client initiates service request information to the network management control server, and the network management control server controls the mobile terminal through the terminal controller, so that the mobile terminal cooperatively transmits video services.
Has the advantages that: compared with the prior art, the invention designs a 5G multi-access concurrent transmission control system and a method thereof, and has the following advantages:
the network control platform can dynamically determine the number of the cooperative terminals according to the network condition, and if the network condition is good, the network control platform can reduce the number of the cooperative terminals as much as possible; if the network condition is poor, the network control platform calls more mobile terminals to cooperatively transmit video data in a multi-stream concurrent mode to improve user experience. In addition, the network control platform selects a proper network (such as a GSM network) according to the characteristics of the service, and sends the sensing data acquired by the extension layer to a data center or a mobile phone terminal in a short message mode through a terminal controller. Important video information (typically border information) is transmitted over the best network and other information may be transmitted over other networks.
The method comprises the following specific steps:
1) the terminal controller finds available mobile terminals around; the terminal sends registration information to the terminal controller;
the terminal controller identifies the registration information of the mobile terminal and completes the authorization function; the terminal detects the network condition information and sends the detected network condition information to the network management control server; and the terminal is cooperated to finish the service under the control of the terminal controller.
2)、
(1) Initializing a network management control server program and loading a server interface.
(2) And creating three Socket sockets, starting three threads, and monitoring messages from a terminal controller, a cooperative terminal and a user playing client at different communication ports respectively.
(3) When the terminal controller accesses, the cooperative terminal registration information sent by the terminal controller is received, authorized and allowed to log in.
(4) And the network management control server creates a Socket dictionary for storing Socket connections of all the cooperative terminals. When an authorized cooperative terminal sends an access request, the Socket connection is added to a dictionary, relevant parameters such as a network system, signal strength, service type and the like are read from the information sent by the dictionary, and relevant information of the cooperative terminal is added to a listview list. Thereafter, at intervals, the server reads real-time network information from a Socket connected to the cooperative terminal, and dynamically updates related information in the list.
(5) When a video service request of a user playing client is received, one of the currently logged available terminals is immediately selected (the first one is selected by default) to participate in video transmission, that is, a control command for starting transmission is sent to the terminal, and meanwhile, the transmission state and the cooperative topological graph of the cooperative terminal in the list are updated, and after the cooperative terminal receives the command, the cooperative terminal starts to participate in video transmission.
(6) When the video service starts to be transmitted, the network management control server can monitor the network state and the transmission rate of the terminal participating in the cooperative transmission in real time, namely, real-time data are read from the Socket connection of the terminal, and are dynamically updated in the list, and the data are also used as the basis of the server self-adaptive decision.
(7) Meanwhile, the network management control server also waits for the feedback information of the user playing client, wherein the feedback information is decision information which is judged and made by the playing client correspondingly according to the occupation condition of the current buffer area. When the feedback information received by the server is "add", it indicates that the buffer area of the current playing client is lower than a certain threshold, and a pause buffering phenomenon may occur, so that the number of transmission links needs to be increased to improve throughput, so that at this time, if there is an available terminal, i.e., a terminal whose transmission state is "non-transmission", in the currently logged cooperative terminals, the server immediately sends a transmission start command to the terminal and updates the state in its list, and if there is no available terminal, no operation is performed. When the feedback information received by the server is "del", it indicates that the buffer area of the current playing client is higher than a certain threshold value, and a buffer overflow phenomenon may occur, so that the number of transmission links needs to be reduced to reduce the receiving rate, so that at this time, if the number of terminals currently participating in cooperative transmission, that is, terminals whose transmission states are "transmitting" is greater than 1, the network management control server makes a decision immediately, selects a cooperative terminal with the lowest transmission rate, sends a command to stop transmission to the terminal, and updates the states in the list. If the number of the cooperative terminals participating in the transmission is 1, no operation is performed in order to maintain the continuity of the service.
(8) And after the user playing client stops the service, the network management control server sequentially sends a transmission stopping command to each terminal participating in the cooperative transmission, and the cooperative terminal stops transmitting the video service after receiving the command. And the network management control server updates the state information and the topological graph of each terminal in the list.
Drawings
Fig. 15G multiple access concurrent transmission control method architecture;
FIG. 2 illustrates an interaction process between a network management control server and a terminal controller and a cooperative terminal;
fig. 3 is an interactive process of the network management control server and the playing client;
FIG. 4 is a block diagram of a video server system;
fig. 5 is a video client system block diagram;
fig. 6 is a diagram of a multi-terminal cooperation mechanism;
fig. 7 is a broadcast information format:
fig. 8 is a registration information format:
fig. 9 shows the format of information uploaded to the network management control server:
fig. 10 is a TCP based transmission method.
Detailed Description
The technical scheme of the invention is described in detail by combining the embodiment and the attached drawings of the specification. Fig. 15G multiple access concurrent transmission control method architecture; FIG. 2 illustrates an interaction process between a network management control server and a terminal controller and a cooperative terminal; fig. 3 is an interactive process of the network management control server and the playing client; FIG. 4 is a block diagram of a video server system; fig. 5 is a video client system block diagram; fig. 6 is a diagram of a multi-terminal cooperation mechanism; fig. 7 is a broadcast information format: fig. 8 is a registration information format: fig. 9 shows the format of information uploaded to the network management control server: fig. 10 is a TCP based transmission method.
The multi-terminal collaborative multi-stream concurrent transmission system is designed and realized by analyzing the application requirements of the multi-terminal collaborative multi-stream concurrent transmission system in the heterogeneous network and comprehensively utilizing a communication network programming technology, a user interface programming technology, a database technology and an embedded development technology, and mainly comprises a video acquisition end, a media distribution server, a distribution decision server, a context server and a multi-terminal aggregation five part. Firstly, a multi-stream concurrent transmission system message data packet format is designed. Second, the implementation is specifically designed from five parts. The video acquisition equipment acquires a real-time video, and compresses and codes the acquired video through an H.264 algorithm to finish video stream acquisition and transmission; the media distribution server adopts a parallel TCP transmission protocol to realize the concurrent transmission of video streams; the context server dynamically stores the context information of each terminal in the virtual terminal system and displays the context information on an interface; the distribution decision server dynamically generates a distribution strategy according to the context information, and the distribution strategy is used as a distribution basis of the media distribution server; the terminal aggregation is that the virtual terminal system is composed of a main terminal and a slave terminal, the slave terminal completes data transmission and forwarding, and the main terminal is responsible for transmitting context information in the virtual terminal system and recombining data. The five parts cooperate with each other to form the whole system. And finally, performing combined test and analysis on the whole system, wherein a large number of test results show that the system runs stably and meets the design requirements of projects. The related key technologies include a video stream packet loss resistance preprocessing technology and a buffer management technology based on queue prediction.
The method for preventing and recovering the packet loss in video transmission comprises the following steps: aiming at the serious reduction of video quality caused by serious packet loss of streaming media service in a severe wireless network environment, the invention provides a network video stream packet loss resistance preprocessing and recovery method. The method comprises the following steps:
1) the sending end divides pixels in a video picture into different sub-pictures according to a certain rule, so that the specific flow that adjacent pixels in the original image are in different sub-pictures is as follows:
dividing an original video picture into PxQ small rectangular blocks, mapping pixels (x, y) of the original picture video to pixels (floor (x/P), floor (y/Q)) in the (xmod P, y mod Q) small rectangular blocks, and finally obtaining the small rectangular blocks as sub-pictures, wherein P is the column number of the divided small rectangular blocks, Q is the row number of the divided small rectangular blocks, x is the horizontal coordinate of the pixels of the original picture, y is the vertical coordinate of the pixels of the original picture, mod is a remainder operation, and floor (×) is a downward rounding operation;
2) performing video coding on each sub-picture obtained in the step 1) to obtain a video data stream, dividing the data stream into data packets, marking the data packets with a header in which a sub-picture division mode and a pixel position field are recorded, and transmitting the data packets to a receiving end;
3) the receiving end decodes the received data packet, and then obtains the video picture pixel range of the data packet which is successfully received according to the head of the data packet, thereby obtaining the video picture pixel range in the lost data packet;
4) the receiving end synthesizes each decoded sub-picture into a final picture according to a sub-picture segmentation mode recorded in the head of the data packet, and then recovers the data of the lost pixel by adopting an interpolation mode according to the correctly decoded pixel value to obtain a recovered video picture, wherein the flow of recovering the data of the lost pixel by adopting the interpolation mode is as follows:
a) calculating distances dx1, dx2, dy1, dy2 of each lost pixel from the received pixel M, N, P, Q closest to the lost pixel in four directions of up, down, left, and right, where the pixel M, N is on the left and right sides of the lost pixel, and the pixel P, Q is on the up and down sides of the lost pixel, respectively, defining a lateral distortion distance dx (dx 1+ dx 2), a longitudinal distortion distance dy (dy) dy1+ dy2, and a distortion distance d (dx, dy), where min (dx, dy) represents the smaller of dx and dy;
b) sorting all the lost pixels according to the size of the distortion distance d, taking the lost pixel with the minimum d, interpolating according to a linear interpolation method, and if dx is less than or equal to dy, determining the color value Y of the lost pixel to be YM + dx1 (YN-YM)/dx, wherein YM is the color value of the pixel M, and YN is the color value of the pixel N; if dx > dy, the color value Y of the missing pixel is YP + dy1 (YQ-YP)/dy, where YP is the color value of pixel P and YQ is the color value of pixel Q;
c) and (c) adding the pixel after interpolation into the received pixel set, judging whether the data of the lost pixel is not recovered, if so, returning to the step (a), and otherwise, completing interpolation recovery of the lost pixel.
The video caching method based on queue prediction comprises the following steps: the buffer queue with proper length can store the arriving packet in time without losing the data packet, and the control of the play rate can provide smoother video play for the user. The buffer length cannot be set too large, which would cause large delays and low queue utilization, nor too small, which would easily cause arriving packets to be dropped due to a full queue. Therefore, the proposed buffer management method aims to balance the elimination of delay jitter and the reduction of end-to-end delay, and sets the most appropriate buffer queue length for video applications.
1) Buffer status
For better cache management, the cache queue occupancy of the video client is divided into three states, underflow, normal and overflow. 1) underflow, i.e. too few packets in the current buffer queue, is not enough for playing video frames, and in this state, if the playing rate is greater than the arrival rate of the video frames, the video playing will be interrupted. The impact of interrupted playback on video traffic is very severe, and studies have shown that if the interruption time (or referred to as buffering time) of a piece of video exceeds 2 seconds, one starts to give up watching, then every 1 second increase in buffering time, 6% more people give up watching, and as the number of interrupted playback increases, the probability of users giving up watching increases exponentially [8 ]. Therefore, for real-time video, underflow of the buffer is avoided as much as possible. 2) In this state, the number of packets temporarily stored in the buffer queue is a normal proportion of the queue length, the arrival rate and the play rate of the video frame are basically maintained in a balanced state, and the video is smoothly played. 3) overflow, a state where the buffer queue is substantially full due to a large amount of burst data arriving, or the playout rate is less than the average arrival rate of frames, and in this state the buffer queue easily reaches a full state, causing new arriving data to be dropped due to late storage, resulting in congestion of the queue and retransmission of the data [4 ]. Therefore, when the video is played, once the overflow or underflow state occurs, the buffer area can be controlled to be in the normal state as much as possible by adjusting the size of the buffer queue and the playing speed of the video.
2) Time delay analysis
The requirement of streaming media video communication in a wireless network on real-time performance is high, especially for some real-time video applications such as video conferences and live broadcasting, generally, end-to-end delay cannot exceed 250ms, otherwise, great user experience decline is caused, and the acceptable delay of a user should be controlled within 200-500 ms [5], so that UDP which does not need to establish connection is mostly adopted in general real-time video applications as a transmission protocol, end-to-end delay can be greatly reduced, and the disadvantage is that a large amount of packet loss is easily generated, especially in the wireless network. Therefore, in some services with higher requirements on video quality, and under the condition that a certain delay can be tolerated, a reliable TCP protocol is often adopted.
As shown in fig. 10, a delay distribution generated during the whole process of collecting, encoding, sending, transmitting a frame of video data from a streaming media server, and finally buffering, decoding, and playing at a receiving end after the frame of video data is transmitted through a wireless channel is shown, it can be seen that an end-to-end delay D of a video frame can be represented by the following formula (where fragmentation and recovery delay of a video frame are not discussed):
D＝Dencode+Dsend+Dchannel+Drecv+Dbuffer+Ddecode+Dplayback
in the above formula DdecodeThe delay generated for the system codec is generally a fixed value, and as the hardware of the mobile device is increased, the delay is relative to the delay D generated when the video is transmitted in the channelchannelCan be ignored. DsendThe transmission delay for the server side, including the scheduling delay of packets, the queuing transmission delay, etc., is not controllable for the user side. DrecvThe delay, D, incurred for the arrival of the video frame at the receiving end and into the bufferbufferFor the time that the video frame waits in the buffer queue, DplaybackDelay between leaving buffer and submitting video frame to application layer for playing, and DbufferCompared with, DrecvAnd DplaybackCan be ignored 9]. The above formula can be simplified as follows:
D＝Dsend+Dchannel+Dbuffer+C
wherein C is a constant and DsendGenerated by the server side, not discussed here, due to the time-varying nature of the radio channel, DchannelIt is also dynamic and unpredictable, so to reduce the end-to-end delay of video frames, the time for packet data to queue in the buffer should be minimized at the user side, which needs to be realized through reasonable buffer queue management, and the delay should also be an important parameter for buffer queue adjustment.
3) Video caching method based on queue prediction
Based on the above analysis, a cache management mechanism based on queue prediction is proposed herein under consideration of the influence of various parameters. The method jointly adjusts the video playing speed and the buffer length to ensure that the buffer queue is controlled in a normal state as much as possible, thereby effectively reducing the overflow and underflow probability. The algorithm herein is described in detail below.
Assume that the rate at which the current video stream arrives at the receiving end is RaThe total length of the current buffer queue is q, and the video playing speed is Rp. In the algorithm, the length of the buffer queue is time-varying, so that the setting of the buffer state cannot adopt a fixed threshold value, but adopts a relative value compared with the length of the buffer queue. Herein, B is usedl，BhTo indicate a relative threshold value, Bl，BhRespectively, an underflow and overflow threshold value, where Bl＝α*q,Bhβ q, α are coefficient factors to which buffer management is to be adjusted in view of the burstiness of video frames, and it is recommended to set α 0.2 and β 0.8p＝{Rl,Rn,RhWith R corresponding to lower, normal, higher rates, respectivelyl<Rn<Rh，RnThe playing speed adopted by the default normal state is the playing speed, when the frame number stored in the buffer memory reaches the overflow threshold, the playing speed is proved to be not timely to play the buffered video, so a larger playing speed R should be adoptedhTo accelerate visionThe playing of the frequency frame and the reduction of the number of the piled buffer frames in time can not cause frame loss. When the number of the buffer frames is reduced to the underflow threshold, the video frame reaching rate is lower than the playing rate, the playing rate is reduced to prevent the situation that the playing is interrupted because the buffer queue is empty, the buffer has enough time to buffer enough frames to keep the video playing smoothly, and R is used for playing the video smoothlylCorresponding to a lower play rate.
The specific idea of the algorithm is as follows: when receiving a frame of data, the receiving end calculates the corresponding overflow and underflow thresholds B according to the current buffer lengthlAnd BhThen judging the state of the cache queue, assuming that the current cache occupation length is Q, and if Q is less than or equal to BlIf Q is greater than or equal to B, the buffer queue status is underflowhThe state is overflow, if Bl＜Q＜BhThe buffer queue status is normal. If the current buffer queue state is normal, the video is normally played, and the current frame is stored in the buffer queue. If the buffer queue status is underflow, which indicates that the buffer is about to underflow, the play rate needs to be reduced, and the buffer length is reduced to maintain a higher buffer utilization, where the buffer size is set as follows:
in the formula (I), the compound is shown in the specification,
for the sign of rounding up, q (t) is the buffer size to be set currently, q (t-1) is the size value of the buffer at the last moment, the buffer size adjustment is correspondingly fine-tuned based on the buffer queue length at the last moment, k is an adjustment factor, the value of k is not suitable for being too large, when the value of k is larger, namely the step length of the buffer adjustment is larger, unnecessary excessive adjustment is occasionally generated, the utilization rate of the buffer is lower, and the situation of the rapid change of the frame arrival rate caused by the network burst can be prevented due to the smaller value of k, the real-time change situation of the buffer is simulated and predicted through the smaller adjustment, so that the method has better effectAnd (4) stability. Herein k is 0.2. In addition, if the buffer status reaches the overflow threshold, it indicates that the video arrival rate is greater than the play rate, and the buffer area is about to be filled, so the buffer needs to be expanded in time to avoid the frame loss phenomenon caused by the full buffer, and the play rate needs to be increased to play the video frames accumulated in the buffer in time to reserve a space to store the next arriving frame sequence, and the buffer size is set as:
the size of the buffer is not increased or reduced without limit, a certain limit exists, the buffer is too large, the end-to-end delay is too large, the buffer is not suitable to be taken, the buffer is too small, the video is difficult to have a certain buffering effect, and overflow and underflow are easy to cause, so that the buffer self-adaption adjustment also has a certain upper limit and a certain lower limit qhAnd q islShould remain cached at qhAnd q islTo fluctuate. The algorithm is as follows:
when each frame is played, the end-to-end delay, namely D, is judged, and if the end-to-end delay is higher than a certain threshold value, the end-to-end delay is too large, which may affect the real-time experience of a user, so that the playing speed is increased and the cache length is reduced by timely adjusting, so that the delay is controlled within a certain range. The specific calculation is as follows.
The algorithm is simple and practical, and the phenomena of cache overflow and underflow caused by data emergency caused by dynamic change of a wireless network are reduced by adjusting the length of the buffer area and the playing speed of the video in real time, so that the video at a user end can be played smoothly, and the user experience is improved to a certain extent.
Claims (1)
1.5G inserts concurrent transmission control system more, its characterized in that includes:
(1) a network control platform: the system comprises a network control management server, a terminal management server and a service server; the network management control server performs transmission control in a self-adaptive manner according to the service characteristics, the network condition and the terminal capability; the terminal management server is responsible for registering the terminal, monitoring and managing the terminal and reporting the state of the terminal to the network management control server; the service server performs service adaptation according to the transmission capabilities of the existing network and the terminal;
(2) a terminal controller: discovering the cooperative terminal and reporting to a terminal management server;
(3) and (3) coordinating the terminal: the system is used for cooperative long-distance transmission from the video server to the monitoring center client;
(4) the video adaptation server: the video shunting is realized;
(5) the video playing client side: receiving a video data packet of a video server, performing caching and decoding operations, and playing the video data packet;
the cooperation between the terminal controller (2) and the cooperative terminal (3) is as follows:
1): the terminal controller discovers available mobile terminals around; the terminal sends registration information to the terminal controller; the terminal controller identifies the registration information of the mobile terminal and completes the authorization function; the terminal detects the network condition information and sends the detected network condition information to the network management control server; the terminal is cooperated to complete the service under the control of the terminal controller;
2)：
(2-1) initializing a network management control server program and loading a server interface;
(2-2) creating three Socket sockets, starting three threads, and monitoring messages from a terminal controller, a cooperative terminal and a user playing client at different communication ports respectively;
(2-3) when the terminal controller accesses, receiving the cooperative terminal registration information sent by the terminal controller, authorizing the terminal controller and allowing the terminal controller to log in;
(2-4) the network management control server creates a Socket dictionary for storing Socket connections of all the cooperative terminals; when an authorized cooperative terminal sends an access request, the Socket connection is added into a dictionary, relevant parameters such as a network system, signal strength, service type and the like are read from information sent by the dictionary, and relevant information of the cooperative terminal is added into a listview list; then, at intervals, the server reads real-time network information from a Socket connected with the cooperative terminal and dynamically updates related information in a list;
(2-5) when a video service request of a user playing client is received, immediately selecting one of the currently logged available terminals, selecting the first terminal by default, participating in video transmission, namely sending a control command for starting transmission to the terminal, and simultaneously updating the transmission state of the cooperative terminal and a cooperative topological graph in a list, wherein the cooperative terminal starts to participate in video transmission after receiving the command;
(2-6) after the video service starts to be transmitted, the network management control server monitors the network state and the transmission rate of the terminal participating in the cooperative transmission in real time, namely reads real-time data from the Socket connection and dynamically updates the real-time data in the list, and the data also serve as the basis of the server self-adaptive decision;
(2-7) at the same time, the network management control server also waits for the feedback information of the user playing client side all the time, wherein the feedback information is decision information which is judged and made by the playing client side according to the occupation condition of the current buffer area; when the feedback information received by the server is 'add', the buffer area of the current playing client is lower than a certain threshold value, and a buffer pause phenomenon occurs, so that the number of transmission links needs to be increased to improve the throughput, at this time, if an available terminal exists in the currently logged cooperative terminals, namely, the transmission state is 'untransmitted', the server immediately sends a transmission starting command to the terminal and updates the state in the list, and if no available terminal exists currently, no operation is performed; when the feedback information received by the server is "del", it indicates that the buffer area of the current playing client is higher than a certain threshold value, and a buffer overflow phenomenon may occur, so that the number of transmission links needs to be reduced to reduce the receiving rate, so that at this time, if the number of terminals currently participating in cooperative transmission, that is, terminals whose transmission states are "transmitting" is greater than 1, the network management control server will make a decision immediately, select a cooperative terminal with the lowest transmission rate and send a command to the terminal to stop transmission, and update the states in the list; if the number of the cooperative terminals participating in the transmission is 1, no operation is performed in order to maintain the continuity of the service;
(2-8) after the user playing client stops service, the network management control server sequentially sends a transmission stop command to each terminal participating in cooperative transmission, and the cooperative terminal stops transmitting video service after receiving the command; and the network management control server updates the state information and the topological graph of each terminal in the list.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710493749.8A CN107333143B (en) | 2017-06-26 | 2017-06-26 | 5G multi-access concurrent transmission control system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710493749.8A CN107333143B (en) | 2017-06-26 | 2017-06-26 | 5G multi-access concurrent transmission control system and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107333143A CN107333143A (en) | 2017-11-07 |
| CN107333143B true CN107333143B (en) | 2020-04-17 |
Family
ID=60195659
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710493749.8A Active CN107333143B (en) | 2017-06-26 | 2017-06-26 | 5G multi-access concurrent transmission control system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107333143B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10356552B1 (en) | 2018-09-20 | 2019-07-16 | International Business Machines Corporation | Determining hardware requirements for a wireless network event using crowdsourcing |
| US10693730B2 (en) | 2018-11-20 | 2020-06-23 | International Business Machines Corporation | Communication quality management |
| CN112333476B (en) * | 2021-01-05 | 2021-05-07 | 智者四海（北京）技术有限公司 | Video loading method and device |
| CN112954385B (en) * | 2021-01-18 | 2022-07-29 | 南京邮电大学 | Self-adaptive shunt decision method based on control theory and data driving |
| CN113824723B (en) * | 2021-09-23 | 2023-12-08 | 上海安威士科技股份有限公司 | End-to-end system solution method applied to audio and video data transmission |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101404748A (en) * | 2008-10-31 | 2009-04-08 | 广东威创视讯科技股份有限公司 | Video data transmission system and method used for large-scale high-definition network video conference |
| CN102164276A (en) * | 2011-01-26 | 2011-08-24 | 广东志成冠军集团有限公司 | Method and system for processing streaming media |
| CN103237191A (en) * | 2013-04-16 | 2013-08-07 | 成都飞视美视频技术有限公司 | Method for synchronously pushing audios and videos in video conference |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7899170B2 (en) * | 2005-04-28 | 2011-03-01 | Apple Inc. | Multi-participant conference setup |
| US7949117B2 (en) * | 2005-04-28 | 2011-05-24 | Apple Inc. | Heterogeneous video conferencing |
| US8015310B2 (en) * | 2008-08-08 | 2011-09-06 | Cisco Technology, Inc. | Systems and methods of adaptive playout of delayed media streams |
-
2017
- 2017-06-26 CN CN201710493749.8A patent/CN107333143B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101404748A (en) * | 2008-10-31 | 2009-04-08 | 广东威创视讯科技股份有限公司 | Video data transmission system and method used for large-scale high-definition network video conference |
| CN102164276A (en) * | 2011-01-26 | 2011-08-24 | 广东志成冠军集团有限公司 | Method and system for processing streaming media |
| CN103237191A (en) * | 2013-04-16 | 2013-08-07 | 成都飞视美视频技术有限公司 | Method for synchronously pushing audios and videos in video conference |
Non-Patent Citations (1)
| Title |
|---|
| 无线视频通信中基于队列预测的自适应缓存管理算法;钱超;《电信科学》;20140731;第1-6页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107333143A (en) | 2017-11-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107333143B (en) | 5G multi-access concurrent transmission control system and method | |
| US11006129B2 (en) | System and method for automatic encoder adjustment based on transport data | |
| CA2737728C (en) | Low latency video encoder | |
| JP5746392B2 (en) | System and method for transmitting content from a mobile device to a wireless display | |
| US7984179B1 (en) | Adaptive media transport management for continuous media stream over LAN/WAN environment | |
| CN110290402A (en) | A kind of video code rate method of adjustment, device, server and storage medium | |
| US20130202025A1 (en) | Method and system for transmitting video frame data to reduce slice error rate | |
| KR20040069360A (en) | Targeted scalable video multicast based on client bandwidth or capability | |
| US9232244B2 (en) | Efficient frame forwarding in large scale real-time screen content sharing meetings | |
| CN109618188B (en) | Video data encoding and forwarding method, device, equipment and storage medium | |
| CN107509120A (en) | A kind of streaming media self-adapting transmission method based on buffer underflow probability Estimation | |
| AU2021200428B2 (en) | System and method for automatic encoder adjustment based on transport data | |
| CN113068001A (en) | Data processing method, device, equipment and medium based on cascade camera | |
| CN113676404A (en) | Data transmission method, device, apparatus, storage medium, and program | |
| CN109068151B (en) | Virtual reality video wireless transmission method based on millimeter wave cellular network | |
| US11553224B1 (en) | Method and device for adjusting bit rate during live streaming | |
| US9577947B2 (en) | System and architecture to optimize video traffic over internet protocol networks | |
| Chen et al. | Robust video streaming over wireless LANs using multiple description transcoding and prioritized retransmission | |
| CN115883877A (en) | Method and system for transmitting ultra-high-definition video stream | |
| CN115209189A (en) | Video stream transmission method, system, server and storage medium | |
| CN114286132A (en) | Method for video transmission fault tolerance in wireless network | |
| CN113207011A (en) | Pre-loading method for short video processing | |
| Baik et al. | Efficient MAC for real-time video streaming over wireless LAN | |
| CN108574855A (en) | Video streaming method, apparatus and system | |
| CN117255177A (en) | Client self-adaptive video playing and requesting method, device, system and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20201012 Address after: 210042, 699-22, Xuanwu Avenue, Xuanwu District, Jiangsu, Nanjing, 18 Patentee after: CERTUSNET Corp. Address before: 210003, No. 66, new exemplary Road, Nanjing, Jiangsu Patentee before: NANJING University OF POSTS AND TELECOMMUNICATIONS |