CN117579873A - Code stream data transmission method, system, electronic device and readable storage medium - Google Patents
Code stream data transmission method, system, electronic device and readable storage medium Download PDFInfo
- Publication number
- CN117579873A CN117579873A CN202311323053.2A CN202311323053A CN117579873A CN 117579873 A CN117579873 A CN 117579873A CN 202311323053 A CN202311323053 A CN 202311323053A CN 117579873 A CN117579873 A CN 117579873A
- Authority
- CN
- China
- Prior art keywords
- clients
- client
- data packet
- code stream
- server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 63
- 230000005540 biological transmission Effects 0.000 title claims abstract description 58
- 230000001360 synchronised effect Effects 0.000 claims description 13
- 230000000903 blocking effect Effects 0.000 abstract description 6
- 230000008569 process Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 3
- 230000006978 adaptation Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 241001522296 Erithacus rubecula Species 0.000 description 1
- 239000000872 buffer Substances 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Abstract
The present disclosure provides a code stream data transmission method, a system, an electronic device, and a readable storage medium, where a server sends a first code stream to each first client in a client group according to a sequence of data packets in the first code stream until an upper limit of an available bandwidth is reached, and the code stream is sent in a distributed manner through different client connection channels, so that the code stream fully reaches the client, and the current bandwidth is fully utilized to bring about redundancy as much as possible. The system has the advantages that the blocking and delay of the system are reduced, the working performance of the system is improved, the video pictures seen by the user are smooth, and the user experience is improved.
Description
Technical Field
The present disclosure relates to the field of image processing technologies, and in particular, to a code stream data transmission method, system, electronic device, and readable storage medium.
Background
In the scenes of a server, a client side, such as a remote office, a remote teaching or a video conference, the images seen by the client side are the images transmitted by the screen image codes of the source server, and the images are displayed after being decoded by the client side.
In a one-to-many scenario, i.e., where the server has one screen, the client has multiple client screens, and the server's picture is transmitted to each client screen over a wide area network. The server typically sends the code stream to each client separately by establishing a connection with each client separately.
However, a stuck phenomenon may occur in a wide area network environment.
Disclosure of Invention
The embodiment of the disclosure provides a code stream data transmission method, a system, electronic equipment and a readable storage medium, which can solve the problem that a clamping phenomenon possibly occurs in a wide area network environment. The technical scheme is as follows:
according to a first aspect of embodiments of the present disclosure, there is provided a code stream data transmission method applied to a code stream data transmission system, the code stream data transmission system including: the system comprises a server and N first clients, wherein N is an integer larger than 1, the server is respectively connected with the N first clients through a wide area network, and the N first clients are positioned in a local area network; the method comprises the following steps:
the first client receives a first code stream sequence parameter stream sent by the server;
the first client receives a first code stream sent by the server according to the sequence of data packet rotation in the first code stream;
and after the first client detects that the target data packet is absent, the first client acquires the target data packet from other first clients in the N first clients.
In one embodiment, after the first client detects that the target data packet is absent, the first client obtains the target data packet from other first clients in the N first clients, including:
after the target first client detects that the target data packet is absent, sequentially sending query messages to other first clients according to the sequence of connection between the other first clients in the N first clients and the server respectively until the target clients in the other first clients send the target data packet to the first clients or the other first clients do not store the target data packet, wherein the query messages contain identification information of the target data packet.
In one embodiment, after none of the other first clients stores the target data packet, the method further includes:
if the target data packet is not stored in the other first clients, the target first client sends a synchronous message to the server, wherein the synchronous message contains identification information of the target data packet;
the server encodes a target frame image corresponding to the target data packet in a reference frame encoding mode to obtain the target data packet;
the server sends the target data packet to the target first client so that the target first client sends the target data packet to the other first clients.
In one embodiment, the code stream data transmission system further includes a second client, the second client and the N first clients are located in a local area network, and the method further includes:
the method comprises the steps that a first client receives a basic information acquisition request sent by a second client, wherein the basic information acquisition request is generated after the second client establishes connection with a server;
and the first client sends the current first code stream sequence parameter stream and the current reference frame image to the second client.
According to a second aspect of the embodiments of the present disclosure, there is provided a code stream data transmission method applied to a code stream data transmission system, the code stream data transmission system including: the system comprises a server and N first clients, wherein N is an integer larger than 1, the server is respectively connected with the N first clients through a wide area network, and the N first clients are positioned in a local area network; the method comprises the following steps:
the server respectively sends first code stream sequence parameter streams to the N first clients;
and the server sends the first code streams to the N first clients according to the rotation sequence of the data packets in the first code streams until the first code streams are sent at least once and the upper limit of the available bandwidth is reached, so that the first clients lacking the target data packet acquire the target data packet from other first clients, wherein the other first clients are other first clients except the first client lacking the target data packet in the N first clients.
In one embodiment, the method further comprises:
the server receives a synchronization message sent by a target client in the N first clients, wherein the synchronization message comprises identification information of a target data packet, and the synchronization message is generated when none of the N first clients stores the target data packet;
the server encodes a target frame image corresponding to the target data packet in a reference frame encoding mode to obtain the target data packet;
and the server sends the target data packet to the target client so that the target client sends the target data packet to other first clients except the target client in the N first clients.
In one embodiment, the code stream data transmission system further includes a second client, the second client and the N first clients are located in a local area network, and the method further includes:
after the connection is established between the server and the second client, the second code stream is sent to the N first clients and the second client according to the rotation sequence of the data packets in the second code stream until the second code stream is sent for at least one time and the upper limit of the available bandwidth is reached.
According to a third aspect of embodiments of the present disclosure, there is provided a code stream data transmission system comprising: the system comprises a server and N first clients, wherein N is an integer larger than 1, the server is respectively connected with the N first clients through a wide area network, and the N first clients are positioned in a local area network;
the server is used for respectively sending first code stream sequence parameter streams to the N first clients; the server sends the first code streams to the N first clients according to the rotation sequence of the data packets in the first code streams until the first code streams are sent for at least one time and the upper limit of the available bandwidth is reached;
and the first clients are used for acquiring the target data packet from other first clients in the N first clients after detecting the lack of the target data packet.
According to a fourth aspect of embodiments of the present disclosure, there is provided an electronic device comprising a processor and a memory having stored therein at least one computer instruction that is loaded and executed by the processor to implement the steps performed in the code stream data transmission method described in the first aspect and any of the embodiments of the first aspect.
According to a fifth aspect of embodiments of the present disclosure, there is provided an electronic device comprising a processor and a memory having stored therein at least one computer instruction that is loaded and executed by the processor to implement the steps performed in the code stream data transmission method described in the second aspect and any embodiment of the second aspect.
According to a sixth aspect of embodiments of the present disclosure, there is provided a computer readable storage medium having stored therein at least one computer instruction that is loaded and executed by a processor to implement the steps performed in the code stream data transmission method described in the first aspect and any embodiment of the first aspect.
According to a seventh aspect of embodiments of the present disclosure, there is provided a computer readable storage medium having stored therein at least one computer instruction that is loaded and executed by a processor to implement the steps performed in the code stream data transmission method described in the second aspect and any embodiment of the second aspect.
According to the code stream data transmission method, the system, the electronic equipment and the readable storage medium, the server transmits the first code stream to each first client in the client group according to the data packet rotation sequence in the first code stream until the upper limit of the available bandwidth is reached, the code streams are transmitted in a scattered mode through different client connecting channels, the code streams fully reach the client, the server cannot cause bandwidth congestion and idle bandwidth when transmitting the first code stream to the client group, the current bandwidth is fully utilized, as much redundancy as possible is brought, the probability of receiving the code stream in the client group is high, and the extreme condition that all data packets are transmitted in all channels in a failed mode is avoided as much as possible. If the first client finds that the received code stream lacks a target data packet, the first client can acquire the data packet from other first clients in a client group in the local area network, and the special condition of one client can be prevented from negatively affecting the clients in other normal sessions because the efficiency of transmitting the data packet information in the local area network is far higher than that of transmitting the data packet information from the server through the wide area network, the data transmission speed in the local area network is faster, the situation of packet loss and the like is not easy to happen, and therefore, the mode of acquiring the missing code stream in the client group can be completed quickly. The system has the advantages of reducing the blocking and time delay of the system, improving the working performance of the system, enabling video pictures seen by users to be smooth and improving the user experience.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure.
Fig. 1 is an interaction schematic diagram of a code stream data transmission method according to an embodiment of the disclosure;
fig. 2 is a schematic structural diagram of a code stream data transmission system according to an embodiment of the present disclosure;
fig. 3 is a schematic diagram of a code stream data transmission method according to an embodiment of the present disclosure.
Detailed Description
Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as detailed in the accompanying claims.
The code stream data transmission method provided by the embodiment of the disclosure can be applied to a code stream data transmission system, and the code stream data transmission system can comprise: the system comprises a server and N first clients, wherein N is an integer larger than 1, the server is respectively connected with the N first clients through a wide area network, and the N first clients are located in a local area network. The first client may be a computer, smart phone, tablet device, server, or the like. The server encodes the video to be transmitted to obtain a code stream, the code stream is sent to a client group consisting of N first clients, and the first clients decode the code stream to obtain video content.
The server may be one server or a server cluster, which is not limited in this disclosure. Servers in the present disclosure may also be physically located remotely from the client farm.
According to the code stream data transmission method, the system, the electronic equipment and the readable storage medium, the server transmits the first code stream to each first client in the client group according to the data packet rotation sequence in the first code stream until the upper limit of the available bandwidth is reached, the code streams are transmitted in a scattered mode through different client connecting channels, the code streams fully reach the client, the server cannot cause bandwidth congestion and idle bandwidth when transmitting the first code stream to the client group, the current bandwidth is fully utilized, as much redundancy as possible is brought, the probability of receiving the code stream in the client group is high, and the extreme condition that all data packets are transmitted in all channels in a failed mode is avoided as much as possible. If the first client finds that the received code stream lacks a target data packet, the first client can acquire the data packet from other first clients in a client group in the local area network, and the special condition of one client can be prevented from negatively affecting the clients in other normal sessions because the efficiency of transmitting the data packet information in the local area network is far higher than that of transmitting the data packet information from the server through the wide area network, the data transmission speed in the local area network is faster, the situation of packet loss and the like is not easy to happen, and therefore, the mode of acquiring the missing code stream in the client group can be completed quickly. Therefore, the phenomenon of blocking is avoided, the video pictures seen by the user are smooth, and the user experience is improved.
The technical scheme provided by the present disclosure is described in detail below with specific embodiments.
Referring to fig. 1, fig. 1 is an interaction schematic diagram of a code stream data transmission method according to an embodiment of the present disclosure, where the method provided in the embodiment may be applied to a code stream data transmission system, and the code stream data transmission system may be the code stream data transmission system provided in the foregoing embodiment. The server provided in this embodiment may be a server in the above-mentioned code stream data transmission system, and the first client provided in this embodiment may be a first client in the above-mentioned code stream data transmission system. In fig. 1, 2 first clients are exemplarily shown, namely a first client 1 and a first client 2, respectively. The method provided by the embodiment may include the following steps.
Step 101, the server sends first code stream sequence parameter streams to N first clients respectively.
The first code stream sequence parameter stream is typically transmitted before the code stream is transmitted, and typically includes coding related parameters, for example, the first code stream sequence parameter stream includes information such as resolution, coding algorithm, etc. of the code stream to be transmitted.
The first code stream sequence parameter stream is usually smaller and occupies a lower bandwidth, so the server sends the first code stream sequence parameter stream to the N first clients before sending the code stream to the client group consisting of the N first clients.
Step 102, the server sends the first code stream to the N first clients according to the rotation sequence of the data packets in the first code stream until the first code stream is sent for at least one time and the upper limit of the available bandwidth is reached.
The upper limit of the available bandwidth refers to the current maximum bandwidth, and the bandwidth exceeding the current maximum bandwidth is congested and is blocked.
In this embodiment, the round robin order of the data packets refers to sending a first data packet to a first client according to the order of the data packets, sending a second data packet to a second first client, sending a third data packet to a third first client, and so on, after all the data packets are sent once, sending the first data packet to a next first client, and so on, and then sending in a cyclic manner until the server detects that the upper limit of the available bandwidth has been reached. If the server does not send the data packet of one pass to the client group formed by the N first clients, the upper limit of the available bandwidth is reached, the server continues to wait for the idle bandwidth to send the data packet of one pass.
For example, assume that the server needs to send 3 packets, packet 1, packet 2, and packet 3, respectively. The client group comprises 4 clients, namely a client 1, a client 2, a client 3 and a client 4. The mode of the server sending the data packets according to the data packet rotation sequence is specifically as follows: firstly, a data packet 1 is sent to a client 1, then a data packet 2 is sent to the client 2, then a data packet 3 is sent to the client 3, then the data packet 1 is sent to a client 4, then the data packet 2 is sent to the client 1, and the steps are sequentially rotated in turn until the upper limit of the available bandwidth is reached.
Step 103, after the first client detects that the target data packet is absent, the first client acquires the target data packet from other first clients in the N first clients.
The target data packet is a data packet in the first code stream, and the first client detects the data packet which is not stored by the first client.
In this embodiment, the data packets are sent by the server in the order of the data packet rotation, and if the first client detects that the target data packet is missing, the target data packet may be obtained from other first clients in the N first clients.
According to the code stream data transmission method provided by the embodiment of the disclosure, the server sends the first code stream to each first client in the client group according to the sequence of the data packet rotation in the first code stream until the upper limit of the available bandwidth is reached, and the code streams are sent in a scattered manner through different client connection channels, so that the code streams fully reach the clients, the server can not cause bandwidth congestion and idle bandwidth when sending the first code stream to the client group, the current bandwidth is fully utilized, redundancy is brought as much as possible, the probability of receiving the code stream in the client group is high, and the extreme situation that a certain data packet is sent on all channels to cause sending failure is avoided as much as possible. If the first client finds that the received code stream lacks a target data packet, the first client can acquire the target data packet from other first clients in a client group in the local area network, and the special condition of one client can be prevented from influencing the clients in other normal sessions because the information of the data packet transmitted in the local area network is far higher than the information of the data packet transmitted from the server through the wide area network, the data transmission speed in the local area network is faster, the situation of packet loss and the like is not easy to happen, and therefore, the mode of acquiring the missing code stream in the client group can be completed quickly. The system has the advantages that the blocking and delay of the system are reduced, the working performance of the system is improved, the video pictures seen by the user are smooth, and the user experience is improved.
In some embodiments, step 103 may have a variety of implementations, one of which is described below by way of example.
Further, step 103 may be implemented by the following step 1031, based on the above embodiment.
Step 1031, after detecting that the target first client lacks a target data packet, sequentially sending query messages to other first clients according to the sequence of establishing connection between other first clients in the N first clients and the server respectively until the target client in the other first clients sends the target data packet to the first client or none of the other first clients stores the target data packet, wherein the query messages contain identification information of the target data packet.
The inquiry message contains identification information of the target data packet. The identification information of the target data packet is used to indicate which data packet the target data packet is, and may be the sequence number of the target data packet.
If any first client in the client group detects that the first client lacks the target data packet, the first clients can send inquiry messages with identification information of the target data packet to other first clients one by one according to the sequence of establishing connection with the server, if some other first clients store the target data packet, the first clients send the target data packet to the first clients, and after the first clients receive the target data packet, the first clients stop inquiring whether the other first clients have the target data packet.
For example, assuming that the client group includes a client 1, a client 2, a client 3 and a client 4, when the client 1 detects that the data packet a is absent, an inquiry message is sent to the client 2 that first establishes a connection with the server, and if the client 2 stores the data packet a, the client 2 sends the data packet a to the client 1; if client 2 does not store data packet a, client 1 sends an inquiry message to client 3, and so on.
In this embodiment, since the N first clients are located in the same lan, the speed of sending data between them is relatively fast, and the phenomena of packet loss and the like are not easy to occur. Therefore, after any first client in the client group detects that the client lacks a target data packet, the first client can sequentially send query messages to other first clients according to the sequence of establishing connection with the server, so that the target data packet is acquired, and the whole data packet acquisition process can be completed rapidly. If all the first clients have not stored the target data packet after the inquiry, the inquiry is stopped.
In some embodiments, if the first client lacking the target data packet does not acquire the target data packet from the client group, i.e., none of the first clients in the client group store the target data packet, then the target data packet needs to be acquired from the server. Specific examples are described in detail below.
Further, step 103 may further include step 104 on the basis of the above embodiment.
Step 104, if the other first clients do not store the target data packet, the target first client sends a synchronization message to the server.
The synchronous message contains the identification information of the target data packet.
Step 105, the server encodes the target frame image corresponding to the target data packet by means of encoding the reference frame, so as to obtain the target data packet.
And step 106, the server sends the target data packet to the target first client.
And step 107, the target client sends the target data packet to other first clients.
In this embodiment, if no target data packet is stored in any other first client, it is indicated that no target data packet exists in the current client group, and at this time, a synchronization message is sent to the server, so as to request the server to send the target data packet. After receiving the synchronization message, the server encodes the target frame image corresponding to the target data packet by a reference frame encoding method, for example, the reference frame encoding method may be an intra-frame encoding method, so as to obtain the target data packet, and then send the target data packet to the target first client.
Further, after the target first client receives the target data packet, if the data packet after the target data packet has been received before, the data packets after the target data packet received before are discarded. And decoding by taking the target frame image corresponding to the target data packet received at the time as a reference frame.
Further, if the parameters in the corresponding code stream sequence parameter stream change when the server encodes the target frame image corresponding to the target data packet, the server also needs to regenerate the code stream sequence parameter stream and send the code stream sequence parameter stream to the client group.
In some scenarios, if a second client is newly added to the client group, the second client obtains a current first code stream data parameter stream and a current reference frame from N first clients. And then the second code stream sent by the server is sent to a new client group consisting of N first clients and second clients according to the rotating sequence of the data packets. Specific examples are described in detail below.
Further, the code stream data transmission system further includes a second client, where the second client and the N first clients are located in a local area network. The method provided by the embodiment further comprises the following steps.
Step 105, the second client establishes a connection with the server.
And 106, the second client acquires the current first code stream sequence parameter stream and the current reference frame image from the N first clients.
Further, in one possible implementation manner of step 106, the second client sequentially sends the basic information acquisition requests to the N first clients according to the sequence in which the N first clients establish connections with the server respectively, until the first client sends the current first code stream sequence parameter stream and the current reference frame to the second client.
In another possible implementation of step 106, the second client sends a basic information acquisition request to any of the first clients. And the first client transmits the current first code stream sequence parameter stream and the current reference frame image to the second client.
In this embodiment, for the newly added second client in the session, the server is not required to reissue the decoding information and the reference frame image, and the information and the reference frame image required for decoding of the second client may be provided by the first client. Because the transmission efficiency of the data packet information in the local area network is far higher than that of the data packet information transmitted from the server through the wide area network, the special condition of one client can be prevented from influencing the clients in other normal sessions.
The systems and methods provided by the present disclosure are described below with specific examples.
Referring to fig. 2, fig. 2 is a schematic structural diagram of a code stream data transmission system provided in an embodiment of the present disclosure, and as shown in fig. 2, the system is composed of codecs, which are respectively located at two ends of a wide area network, the encoder end collects and encodes an original image into a code stream, the code stream is transmitted to the decoder end through the network, and the decoder decodes and reconstructs the code stream into a YUV image, which is displayed on a display for a user. The server side is also called the encoding side, and has a computer whose screen content is distributed to a plurality of client screens, each client is also called a decoder side, and the clients include client 1, client 2, client 3, clients 4, … … and client N, and the clients form a client group, and the client group is located in a local area network or in a relatively similar area, and the area is generally located in the same network node. The server side is typically located very far from the client group, which are connected via a wide area network. Wide area networks are of much poorer quality than local area networks, e.g., delay, jitter, packet loss rate, etc., are relatively severe. After being compressed and coded, the content of the screen of the coding end is sent to a client group through a wide area network, and the code stream received by each client is decoded and displayed.
Referring to fig. 3, fig. 3 is a schematic diagram of a code stream data transmission method according to an embodiment of the present disclosure, and in the example of fig. 3, a server and four clients are connected. Assuming that 3 frames of data packets (data packet 1, data packet 2 and data packet 3, respectively) need to be transmitted, the sequence is that data packet 1 is transmitted to client 1, data packet 2 is transmitted to client 2, data packet 3 is transmitted to client 3, data packet 1 is transmitted to client 4 again, data packet 2 is transmitted to client 1, and the sequence is sequentially rotated until the upper limit of the available bandwidth is reached. In this way, a plurality of code streams are uniformly transmitted, and the data packets are internally transmitted by the clients in the subsequent client group, and the specific method is described in connection with each component in fig. 2. The components in fig. 2 and their functions are as follows.
Server 101: the coding end, namely the server end, is composed of a computer. When the remote conference desktop shares, it is the computer that the client group views at the same time, it contains an encoder, transmit its desktop image code to the client;
image acquisition module 102: the image acquisition module acquires a source end screen image in YUV format, which is original pixel data to be encoded;
encoding module 103: and encoding the macro block, embedding an encoder, encoding the original image data into a compressed code stream by the encoder, and transmitting the compressed code stream to the client. It should be noted that the start of each stream will generate a stream sequence parameter stream, and each client decoder must first decode the stream sequence parameter stream before decoding the subsequent image data stream. Meanwhile, the encoding module encodes a synchronous frame code stream and a reference frame code stream, the synchronous frame code stream completely adopts an intra-frame reference mode, and the code stream can be decoded without reference to the previously encoded code stream after reaching a decoding end, so that the disadvantage is that the frame code stream of the type has larger volume and excessive occupied bandwidth; in contrast, the common frame code stream is encoded by adopting an inter-frame reference technology, the encoded code stream has smaller volume, but the decoding end needs to store the previously decoded frame (called a reference frame), otherwise, the decoding end cannot decode normally. When the encoder receives the synchronous instruction sent by the synchronous module 104, the synchronous frame is triggered to be encoded, otherwise, the common frame is always encoded to save the code stream;
the synchronization module 104: and receiving information fed back by the client group, encoding the synchronous frame when necessary, and transmitting the code stream sequence parameter stream. When the client group receives this information, a new encoding process will be started, i.e. all previous data packets will not be referenced anymore. Because the code stream sequence parameter stream and the synchronization frame volume are too large, the client group can send the synchronization message to the encoding end only under the extremely special condition. For example, when all clients in the client group lack a certain data packet, and therefore all clients cannot continue to decode correctly, a synchronization message needs to be sent to the encoding end, and after the synchronization message is received by the 104 module, the synchronization process is started, that is, the stream sequence parameter stream is retransmitted, and the next frame is additionally encoded into a synchronization frame;
client 1 105 refers to one of a group of clients, here labeled client 1. The structure of other clients in the client group is the same as that of the client 1, and the client group consists of a plurality of sub-modules;
the decoding module 106 is configured to decode the code stream to generate YUV reconstructed frame pixel data;
the display module 107 is configured to display the pixel generated by decoding 106;
the data packet management module 108 is configured to receive a data packet sent from a server and forward the specific data packet to a requesting client on the premise of other client requests. The 108 module plays an important role in both cases.
First case: when the nth new client starts to establish a connection, it needs to receive the stream sequence parameter stream and receive the sync frame to decode, since it has no stream information locally. At this time, if the encoding end is notified to send the information, the code stream will be increased, and other clients will be affected. The scheme is set so that the client broadcasts a request command to other clients in the client group, the other clients can send code stream sequence parameters to the new client, and code streams of reference frames required by decoding of the new client are also sent, and in the process, the encoder does not need to influence the normal encoding process because of the newly added client. As to which client sends these data information to the new client in particular, there are many ways, for example, in the order in which the connection is established with the encoder side, if the client has these information, it is forwarded to the new client, if not, it is continued to check the next client, and so on;
second case: when a certain client, for example, the client N, loses a packet, for example, the mth frame data is not received, it broadcasts a frame loss request in a client group, and if a client cached with the frame data exists in the client group, the frame information is forwarded to the client N, so that the frame loss problem of the client N is solved. This process is very fast due to the high efficiency of data transfer within the lan. The encoder at the opposite end of the wide area network does not sense the process, and the fluctuation and the blocking phenomenon caused by the retransmission of the wide area network are avoided. Since bandwidth is fully utilized in fig. 2 to send packets to each client, there is typically always a client that buffers some frame packets. When, in an extreme case, all clients have no data packet, a synchronization request needs to be sent to the server, and the synchronization module 104 of the server responds to the request to encode a synchronization frame and send a code stream sequence parameter, so that the client group synchronizes and decodes correctly.
Client 4 109 to client N112: other clients in the client group. The status is equivalent to the client 1;
client group 113: the clients in the client group are located in the same local area network or in relatively close areas, so that the communication efficiency between the clients is high.
The scheme utilizes one characteristic in the scene, the server side and the client side group are transmitted through a wide area network, and the wide area network has larger fluctuation and delay; however, each client in the client group is generally located in a local area network, and the network communication quality in the local area network is good, and the fluctuation and delay are small. The scheme is characterized in that when the server side sends the code stream to the client side group, the server side does not send one part only, does not send one part for each client side respectively, but dispersedly sends the code stream through different client side connecting channels according to the current real-time bandwidth, fully utilizes the bandwidth, so that the probability of receiving the code stream in the client side group is maximum, bandwidth congestion is not caused, bandwidth idling is not caused, the code stream fully reaches the client side, and the system has redundancy as much as possible, and can maximally avoid the extreme condition that all data packets are failed to be sent on all channels. In the client group, for the newly added client in the session process, the server is not required to reissue decoding information and synchronous refreshing frames, and the information and reference frame information required by decoding of the newly added client can be provided by other clients; for clients transmitting lost frames, the server side does not need to brush the synchronous frames, and the lost frames can be provided by other clients. Because the transmission efficiency of the data packet information in the local area network is far higher than that of the data packet information transmitted from the server through the wide area network, the special condition of one client can be prevented from influencing the clients in other normal sessions. The scheme utilizes the characteristic that the transmission efficiency in the local area network is higher than that of the wide area network, and in a one-to-many remote image transmission scene, a method for fully utilizing the bandwidth to send the code stream and copying and sharing the data packet in the client group is designed, so that the working performance of the system is improved, and the blocking and delay of the system are reduced.
An embodiment of the present disclosure provides an electronic device, including a processor and a memory, where the memory stores at least one computer instruction, where the instruction is loaded and executed by the processor to implement a step executed by a server in a code stream data transmission method described in any one of the foregoing embodiments, which is not described herein again.
An embodiment of the present disclosure provides an electronic device, including a processor and a memory, where the memory stores at least one computer instruction, where the instruction is loaded and executed by the processor to implement a step executed by a first client in the code stream data transmission method described in any one of the foregoing embodiments, which is not described herein again.
The disclosed embodiments also provide a computer-readable storage medium, for example, a non-transitory computer-readable storage medium may be Read Only Memory (ROM), random Access Memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like. The storage medium stores computer instructions for executing steps executed by the server in the code stream data transmission method described in any of the foregoing embodiments, which are not described herein.
The disclosed embodiments also provide a computer-readable storage medium, for example, a non-transitory computer-readable storage medium may be Read Only Memory (ROM), random Access Memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like. The storage medium stores computer instructions for executing steps executed by the first client in the code stream data transmission method described in any of the foregoing embodiments, which are not described herein.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program for instructing relevant hardware, where the program may be stored in a computer readable storage medium, and the storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any adaptations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
Claims (10)
1. A code stream data transmission method applied to a code stream data transmission system, the code stream data transmission system comprising: the system comprises a server and N first clients, wherein N is an integer larger than 1, the server is respectively connected with the N first clients through a wide area network, and the N first clients are positioned in a local area network; the method comprises the following steps:
the first client receives a first code stream sequence parameter stream sent by the server;
the first client receives a first code stream sent by the server according to the sequence of data packet rotation in the first code stream;
and after the first client detects that the target data packet is absent, the first client acquires the target data packet from other first clients in the N first clients.
2. The method of claim 1, wherein the first client, after detecting the lack of the target data packet, obtains the target data packet from the other first clients of the N first clients, including:
after the target first client detects that the target data packet is absent, sequentially sending query messages to other first clients according to the sequence of connection between the other first clients in the N first clients and the server respectively until the target clients in the other first clients send the target data packet to the first clients or the other first clients do not store the target data packet, wherein the query messages contain identification information of the target data packet.
3. The method of claim 2, wherein after none of the other first clients stores the target data packet, further comprising:
if the target data packet is not stored in the other first clients, the target first client sends a synchronous message to the server, wherein the synchronous message contains identification information of the target data packet;
the server encodes a target frame image corresponding to the target data packet in a reference frame encoding mode to obtain the target data packet;
the server sends the target data packet to the target first client so that the target first client sends the target data packet to the other first clients.
4. The method of claim 1, wherein the code stream data transmission system further comprises a second client, the second client being located within a local area network with the N first clients, the method further comprising:
the method comprises the steps that a first client receives a basic information acquisition request sent by a second client, wherein the basic information acquisition request is generated after the second client establishes connection with a server;
and the first client sends the current first code stream sequence parameter stream and the current reference frame image to the second client.
5. A code stream data transmission method applied to a code stream data transmission system, the code stream data transmission system comprising: the system comprises a server and N first clients, wherein N is an integer larger than 1, the server is respectively connected with the N first clients through a wide area network, and the N first clients are positioned in a local area network; the method comprises the following steps:
the server respectively sends first code stream sequence parameter streams to the N first clients;
and the server sends the first code streams to the N first clients according to the rotation sequence of the data packets in the first code streams until the first code streams are sent at least once and the upper limit of the available bandwidth is reached, so that the first clients lacking the target data packet acquire the target data packet from other first clients, wherein the other first clients are other first clients except the first client lacking the target data packet in the N first clients.
6. The method of claim 5, wherein the method further comprises:
the server receives a synchronization message sent by a target client in the N first clients, wherein the synchronization message comprises identification information of a target data packet, and the synchronization message is generated when none of the N first clients stores the target data packet;
the server encodes a target frame image corresponding to the target data packet in a reference frame encoding mode to obtain the target data packet;
and the server sends the target data packet to the target client so that the target client sends the target data packet to other first clients except the target client in the N first clients.
7. The method of claim 5, wherein the code stream data transmission system further comprises a second client, the second client being located within a local area network with the N first clients, the method further comprising:
after the connection is established between the server and the second client, the second code stream is sent to the N first clients and the second client according to the rotation sequence of the data packets in the second code stream until the second code stream is sent for at least one time and the upper limit of the available bandwidth is reached.
8. A code stream data transmission system, comprising: the system comprises a server and N first clients, wherein N is an integer larger than 1, the server is respectively connected with the N first clients through a wide area network, and the N first clients are positioned in a local area network;
the server is used for respectively sending first code stream sequence parameter streams to the N first clients; the server sends the first code streams to the N first clients according to the rotation sequence of the data packets in the first code streams until the first code streams are sent for at least one time and the upper limit of the available bandwidth is reached;
and the first clients are used for acquiring the target data packet from other first clients in the N first clients after detecting the lack of the target data packet.
9. An electronic device comprising a processor and a memory having stored therein at least one computer instruction that is loaded and executed by the processor to implement the steps performed in the code stream data transmission method of any one of claims 1 to 4 or of claims 5 to 7.
10. A computer readable storage medium having stored therein at least one computer instruction loaded and executed by a processor to implement the steps performed in the code stream data transmission method of any one of claims 1 to 4 or of claims 5 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311323053.2A CN117579873A (en) | 2023-10-13 | 2023-10-13 | Code stream data transmission method, system, electronic device and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311323053.2A CN117579873A (en) | 2023-10-13 | 2023-10-13 | Code stream data transmission method, system, electronic device and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117579873A true CN117579873A (en) | 2024-02-20 |
Family
ID=89885140
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311323053.2A Pending CN117579873A (en) | 2023-10-13 | 2023-10-13 | Code stream data transmission method, system, electronic device and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117579873A (en) |
-
2023
- 2023-10-13 CN CN202311323053.2A patent/CN117579873A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2020192152A1 (en) | Video transmission method, root node, child node, p2p server, and system | |
| EP2863632B1 (en) | System and method for real-time adaptation of a conferencing system to current conditions of a conference session | |
| US8988486B2 (en) | Adaptive video communication channel | |
| US6986158B1 (en) | System and method for distributing video information over network | |
| CN105704580B (en) | A kind of video transmission method | |
| US20080100694A1 (en) | Distributed caching for multimedia conference calls | |
| US10681382B1 (en) | Enhanced encoding and decoding of video reference frames | |
| CN111147860B (en) | Video data decoding method and device | |
| JP2013211835A (en) | System and method for handling critical packets loss in multi-hop rtp streaming | |
| US20220329883A1 (en) | Combining Video Streams in Composite Video Stream with Metadata | |
| EP2936803B1 (en) | Method and a device for optimizing large scaled video conferences | |
| CN109618170A (en) | D2D real-time video streaming transmission method based on network code | |
| WO2010108416A1 (en) | Method, device and communication system for forwarding scalable video coding data messages | |
| US8928728B2 (en) | Systems, methods, and media for controlling a presentation of data images in a video stream | |
| CN114827669B (en) | Video data transmission method, device, medium and equipment | |
| CN114900707B (en) | Live broadcast method and system | |
| CN117579873A (en) | Code stream data transmission method, system, electronic device and readable storage medium | |
| CN112153413B (en) | Method and server for processing screen splash in one-screen broadcast | |
| Otomo et al. | Loss resilient multi-view video streaming over multiple transmission paths | |
| CN106998328A (en) | A kind of video transmission method and device | |
| CN110719435A (en) | Method and system for carrying out terminal conference | |
| CN112770077B (en) | Video conference I frame coding method and device | |
| CN115102927B (en) | SIP intercom method, system and storage device for keeping video clear | |
| CN114070827B (en) | Transmission method, equipment and medium for forwarding and pushing stream data | |
| WO2022100742A1 (en) | Video encoding and video playback method, apparatus and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication |