WO2015196457A1

WO2015196457A1 - Method for routing key frame and media server

Info

Publication number: WO2015196457A1
Application number: PCT/CN2014/080968
Authority: WO
Inventors: 刘玎
Original assignee: 北京新媒传信科技有限公司
Priority date: 2014-06-27
Filing date: 2014-06-27
Publication date: 2015-12-30
Also published as: CN105519121A; CN105519121B

Abstract

Disclosed are a method for routing a key frame, and a media server. The method of the present invention comprises: a media server caching a last effective key frame of each user terminal; and when a key frame, of which only one receiving party requests a sending party, is received, the media server routing a last effective key frame of the sending party cached by itself to the receiving party. The solution of the present invention can realize directional routing, reduce network pressure, and avoid the problem that the entire network pressure sharply increases caused by one or more certain receiving parties due to the loss of stored key frames.

Description

The present invention relates to the field of multimedia technologies, and in particular, to a method for routing key frames in a multimedia interworking and a media server. BACKGROUND OF THE INVENTION At present, with the continuous maturity of Internet services and the improvement of terminal equipment performance and network coverage, multimedia technology has developed rapidly.

Typically, the media server interacts with multiple users (e.g., user A, user B, user C, and user D) as shown in FIG. When user A requests a key frame from user B, in the prior art, user B routes the key frame requested by user A to user B, user C, and user D connected to the media server through the media server.

In the process of implementing the present invention, the inventor has found that the above prior art has at least the following problems: When user A requests a key frame from user B, in fact, user C and user D do not need the key frame sent by user B, and the media The server unconditionally forwards (non-directional routing) key frames generated by user B to other users, which will cause waste of transmission resources, and the network pressure of the entire network will increase exponentially. SUMMARY OF THE INVENTION Based on the above problems, an embodiment of the present invention provides a method for key frame routing and a media server to solve the problem of network stress caused by non-directional routing in the prior art.

According to an aspect of the present invention, an embodiment of the present invention provides a method for routing a key frame, including: a media server buffering a last valid key frame of each user terminal; when receiving a key frame of only one receiver requesting a sender The media server routes the last valid key frame of the sender that is cached by itself to the receiver.

Optionally, the method further includes: when receiving, by the at least two receivers, the key frame of the sender, the media server determines a receiving time corresponding to the request of the at least two receivers.

Optionally, after the determining, by the media server, the receiving time corresponding to the request of the at least two receiving parties, the method further includes: If the receiving time is in the preset time range, the media server notifies the sender to regenerate a new key frame, and after receiving the new key frame regenerated by the sender, routing to other recipients, and updating at the same time The last valid keyframe of the sender that is itself cached.

Optionally, after the determining, by the media server, the receiving time corresponding to the request of the at least two receivers, the method further includes:

If the receiving time exceeds the preset time range, the media server determines the reason of the at least two receiver requests;

If the reason for the request is that the key frame saved by one or more receivers is lost, the media server routes the last valid key frame of the sender that is cached by itself to the one or more receivers;

If the reason for the request is that one or more recipients have problems in decoding with the key frame, the media server notifies the sender to regenerate a new key frame and receives a new key frame regenerated by the sender It is then routed to the other recipients, while updating the last valid keyframe of the sender that is itself cached.

Optionally, the method further includes:

The media server periodically detects the last valid key frame of each user terminal that is cached by itself;

If it is detected that the last valid key frame of a user terminal that is itself cached has expired, the user terminal is notified to regenerate a new key frame, and after receiving the new key frame regenerated by the user terminal, routing is performed to The other individual user terminals simultaneously update the last valid key frame of the user terminal that is cached by itself.

According to another aspect of the present invention, an embodiment of the present invention provides a media server, including: a cache unit, configured to cache a last valid key frame of each user terminal;

And a first routing unit, configured to: when receiving a key frame of only one receiver requesting the sender, routing the last valid key frame of the sender buffered by the buffer unit to the receiver.

Optionally, the media server further includes: a time determining unit, configured to determine, when the at least two receivers request the key frame of the sender, the receiving time corresponding to the request of the at least two receivers.

Optionally, the media server further includes: a second routing unit, configured to: after the receiving time corresponding to the request of the at least two receivers is determined, if the receiving time is in a preset time range, notify the The sender regenerates a new keyframe and receives a new regenerated by the sender The key frame is then routed to the other recipients, and the last valid key frame of the sender of the cache unit cache is updated.

Optionally, the media server further includes:

a reason judging unit, configured to determine, after the media server determines a receiving time corresponding to the request of the at least two receiving parties, if the receiving time exceeds a preset time range, the reason for requesting the at least two receiving parties Make judgments;

a third routing unit, configured to: if the request is caused by loss of a key frame saved by one or more receivers, routing the last valid key frame of the sender buffered by the cache unit to the one or more receivers Party

a fourth routing unit, if the reason for the request is that one or more receivers have problems in decoding with the key frame, notifying the sender to regenerate a new key frame, and receiving the regenerated by the sender The new key frame is then routed to the other recipients, while the last valid key frame of the sender of the cache unit cache is updated.

Optionally, the media server further includes:

a period detecting unit, configured to periodically detect a last valid key frame of each user terminal buffered by the cache unit;

a processing unit, configured to notify the user terminal to regenerate a new key frame if the last valid key frame of a user terminal that is cached by the cache unit is detected to be invalid, and regenerate the user terminal after receiving the user terminal The new key frame is then routed to the other individual user terminals, and the last valid key frame of the user terminal buffered by the cache unit is updated.

The beneficial effects of the present invention are as follows: The present invention caches the last valid key frame of each user terminal through the media server, and when the key frame of only one receiver requesting the sender is received, the media server caches the last of the senders A technical solution for routing a valid key frame to the receiver, no longer regenerating a new key frame by the sender and routing to other user terminals in the multimedia interworking through the media server, thereby solving the problem in the prior art due to a certain party When the frame is lost, the problem of the entire network pressure rise caused by the non-directional routing is adopted; at the same time, the technical solution of the present invention also considers the actual scenario. In the multimedia interworking of the multi-user terminal, if only one receiving direction requests the key frame, the request is requested. The reason is very large because the single receiver loses the key frame of the sender, and only needs to request the key frame of the request to the single receiver, and other user terminals do not need the sender's Key frames, so the technical solution of the present invention reduces network stress When it is possible to a large extent Avoid problems with subsequent data that cannot be resolved.

The above description is only an overview of the technical solutions of the present invention, and the technical means of the present invention can be more clearly understood, and can be implemented in accordance with the contents of the specification, and the above and other objects, features and advantages of the present invention can be more clearly understood. Specific embodiments of the invention are set forth below. BRIEF DESCRIPTION OF THE DRAWINGS Various other advantages and benefits will become apparent to those of ordinary skill in the art. The drawings are only for the purpose of illustrating the preferred embodiments and are not to be construed as limiting. Throughout the drawings, the same reference numerals are used to refer to the same parts. In the drawing:

FIG. 1 is a schematic diagram of a network in multimedia interworking in the prior art;

2 is a flow chart showing a method for routing key frames in multimedia interworking according to an embodiment of the present invention;

3 is a flow chart showing a method for routing key frames in multimedia interworking according to another embodiment of the present invention;

FIG. 4 is a structural diagram of a media server according to another embodiment of the present invention. Mode for Carrying Out the Invention Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While the embodiments of the present invention have been shown in the drawings, the embodiments Rather, these embodiments are provided so that this disclosure will be more fully understood, and the scope of the disclosure may be fully disclosed to those skilled in the art. The main technical idea of the present invention is that the media server caches the last valid key frame of each user terminal. When receiving a key frame that only one receiver requests the sender, the media server validates the last one of the senders that are cached by itself. The key frame is directly routed to the recipient, and the requested key frame is no longer routed by the sender to the other user terminals connected to the media server through the media server. The invention can realize the directional routing, reduce the waste of the transmission resources, reduce the network pressure, and can largely ensure the analysis of the subsequent data; when there are many user terminals, the prior art can be significantly solved due to the loss of a certain key frame in the prior art. The entire network pressure caused by non-directional routing is geometrically multiplied The number of sudden rises.

According to the corresponding application scenario, each user terminal connected to the media server in the embodiment of the present invention may be used as a sender or a receiver, and is not specifically limited.

2 is a flow chart showing a method for routing key frames in multimedia interworking according to an embodiment of the invention. As shown in FIG. 2, a method for routing key frames in multimedia interworking according to an embodiment of the present invention includes the following steps:

S11. The media server caches the last valid key frame of each user terminal.

A frame is the basic unit that makes up a video image. The key frame is also called I frame (I frame ), which is a full frame compressed coded frame; when decoding, only the I frame data can be used to reconstruct the complete image; I frame is generated without reference to other pictures. An I frame is also called an intra picture and is usually the first frame of each GOP (Group of Pictures). A GOP is a set of continuous pictures, a video compression technology used by MPEG (Moving Pictures Experts Grou).

MPEG encoding divides pictures (i.e., frames) into three types: I, P, and B. I is an internally coded frame, P is a forward predictive frame, and B is a bidirectionally interpolated frame. The I frame is a complete picture, while the P and B frames record changes relative to the I frame. Since the I frame is the reference frame of the subsequent P frame and the B frame, the I frame quality directly affects the quality of the subsequent frames in the same group, so the P frame and the B frame have a dependency on the I frame, and there is no I frame, P frame and B. The frame cannot be decoded. Compared with the subsequent P frame and B frame, the I frame is a large picture, and the amount of information occupied by the I frame is relatively large.

In multimedia interworking, the media server caches the last valid I frame sent by each user terminal connected to itself for forwarding.

S12. When receiving a key frame that only one receiver requests the sender, the media server routes the last valid key frame of the sender that is cached by the media server to the receiver.

When the user terminal has only two parties, the routing scheme of the key frame of the embodiment of the present invention is compared with the prior art. Although the media server finally routes the key frame of the sender to the receiver, the difference is the prior art. The sender needs to regenerate the key frame and send it to the media server, and the media server receives the key frame regenerated by the sender and then routes it to the receiver. The embodiment of the present invention is the sender of the media server that caches itself. The last valid keyframe is routed directly to the receiver.

When there are more than two user terminals (three or more parties), in order to clearly clarify the solution of the embodiment of the present invention, four user terminals (User A, User B, User C, and User D) are described below as an example.

When only user A requests a key frame from user B, in the prior art, user B simultaneously routes the key frame requested by user A to the user B, user C, and the media server connected to the media server through the media server. User D; that is, user B's key frame needs to be transmitted 1+3=4 times in the network. In the embodiment of the present invention, the media server only needs to send the last valid key frame of the user B that is cached to the user A, that is, the key frame of the user B only needs to transmit 0+1=1 times in the network. It can be understood that the more the user terminals are, the more significant the effect of the solution of the embodiment of the present invention is to reduce the network pressure.

In fact, in multi-user terminal multimedia intercommunication, if only one receiving direction requests a key frame from the sender, it is often because the single receiver itself loses the key frame of the sender, and only needs to request it. The sender's key frame is routed to the single receiver, and other user terminals do not need the sender's key frame. The non-directional routing scheme of the prior art does not consider the actual needs of each user terminal. Once a receiver requests a key frame of the sender, the key frame regenerated by the sender is simultaneously routed to each of the media servers. The user terminal obviously causes a waste of transmission resources, which causes the network pressure to increase exponentially. In particular, the routing is the key frame I frame, the I frame is a large picture, and the amount of information occupied by the data is relatively large, which will cause the entire network pressure to rise due to the loss of key frames in a certain user terminal.

The method embodiment of the present invention uses the media server to cache the last valid key frame of each user terminal. When receiving only one key frame of the receiver requesting the sender, the media server validates the last one of the senders that are cached by itself. The key frame is directly routed to the receiver, which can implement directional routing, reduce the waste of transmission resources, and solve the problem that the entire network pressure rises due to the use of non-directional routing when a certain key frame is lost in the prior art. And while significantly reducing network pressure, it can also largely avoid the problem that subsequent data cannot be parsed. There are two types of key frame transmission. One is that the receiver requests a key frame, and the other is that the sender actively transmits the key frame. The present invention mainly deals with the case where the receiver requests a key frame. There are two main reasons why the receiver requests key frames. One is the loss of key frames. This is common. The other is that there are problems when decoding key frames. For example, P frames have errors in using I frame restoration.

Based on the above description, FIG. 3 shows a flow chart of a method for routing key frames in multimedia interworking according to another embodiment of the present invention. As shown in FIG. 3, a method for routing key frames in multimedia interworking according to another embodiment of the present invention includes the following steps:

S31. The media server caches the last valid key frame of each user terminal.

532. The media server receives a request for at least two recipients to request a key frame of the sender.

533. The media server determines, according to the receiving time corresponding to the request of the at least two receivers, if the receiving time is in the preset time range, the process goes to step S35; if the receiving time exceeds the preset time range, Go to step S34.

Specifically, the media server determines whether the receiving time is within a preset time range, wherein the preset time range is preferably [0, 500 ms], but is not limited thereto.

If the receiving time is within the preset time range, it may be considered that the request of the at least two recipients is received simultaneously.

If the receiving time exceeds the preset time range, it may be considered that the request of the at least two parties is received at different times.

For example, when the media server receives the key frame of user A, user C, and user D requesting user B, the media server first determines the receiving time of receiving three requests of user A, user C, and user D, if three requests If the receiving time is concentrated in the time range of Is, it is considered that the three requests are received at the same time; if the receiving time exceeds the time range of Is, it is considered that the three requests are not received at the same time.

534. The media server determines the cause of the request of the at least two receivers. If the reason of the request is that one or more receivers have problems in decoding by using the key frame, proceed to step S35; if the reason for the request is one Or the key frames saved by the plurality of receivers are lost, and the process proceeds to step S36.

When it is determined that the at least two receivers request the key frame of the same sender, the media server further analyzes and determines the cause of the request of the at least two receivers.

This is because when multiple receivers request a time span of the same sender's key frame, the reason for the request is likely to be multiple, for example, part of the receiver is requested due to the loss of the saved key frame, or part of it. The receiver was found to have problems using existing keyframe decoding. The embodiment of the present invention distinguishes the request reason that at least two receivers request the key frame of the same sender at the same time, can further implement directional routing, reduce network pressure, and avoid one or some receivers from saving key frames. Loss caused by the sudden increase in network pressure.

In an implementation manner of this embodiment, the reason for the request of the receiver may be included in the request, or the media server may perform the analysis according to the received request to obtain the request of the receiver, and the embodiment of the present invention is not limited thereto. , will not repeat them here.

535. The media server notifies the sender to regenerate a new key frame, and after receiving the new key frame regenerated by the sender, routes to each of the other user terminals, and simultaneously updates the last valid key of the sender that is cached by itself. frame.

536. The media server routes the last valid key frame of the sender that is cached by itself to the one or more receivers.

When multiple receivers request a key frame from a sender at the same time, it means that multiple receivers need the The key frame of the sender, at this time, the great possibility is that these receivers have problems in decoding the key frame of the sender, and it is necessary for the receiver to generate a new key frame and route it to the other by the media server. Each user terminal simultaneously updates the last valid key frame of the sender that is itself cached.

In the embodiment of the present invention, it is quite necessary for the media server to determine the reception time of more than one recipient requesting the same recipient. This is because: If User A, User C, and User D request key frames from User B at the same time, User B will receive three requests in succession at the same time.

According to the prior art, user B continuously generates three key frames and then routes them to user A, user C, and user D simultaneously by the media server; that is, using the prior art scheme, user B's key frames need to be transmitted in the network. (1+3) *3=12 times.

According to the solution of the embodiment of the present invention, if the receiving time of the media server receiving the three requests is concentrated within a certain time range, it will be regarded as receiving the three requests at the same time, and only the user B needs to be notified once to regenerate the key frame. Then, the media server simultaneously routes to user A, user C, and user D; that is, user B's key frame only needs to be transmitted in the network (1+3) *1=4 times.

In an embodiment of the present invention, in order to prevent the media server from failing due to the last valid key frame of a certain sender of the cache, the other server will cause problems when decoding the key frame, and the media server periodically periodically The last valid key frame of each user terminal that is cached is detected. If it is detected that the last valid key frame of a user terminal has expired, the user terminal is notified to regenerate a new key frame, and the user terminal is regenerated after receiving the user terminal. The new key frame is then routed to each of the other user terminals, and the last valid key frame of the user terminal that is cached by itself is updated. The embodiment of the inventive method can ensure that the last valid key frame of each user terminal buffered by the media server is always in a valid state. Based on the same inventive concept as the above method, FIG. 4 shows a structural diagram of a media server according to another embodiment of the present invention. As shown in FIG. 4, a media server 400 according to another embodiment of the present invention includes:

The buffer unit 410 is configured to cache the last valid key frame of each user terminal;

The first routing unit 420 is configured to, when receiving a key frame that only one receiver requests the sender, route the last valid key frame of the sender buffered by the buffer unit 410 to the receiver.

In an embodiment, the media server 400 provided by the embodiment of the present invention further includes:

a time judging unit, configured to: when receiving at least two recipients requesting a key frame of the sender, The receiving time corresponding to the request of the at least two recipients is determined.

Further, the media server 400 provided by the embodiment of the present invention further includes:

a second routing unit, configured to: after the receiving time corresponding to the request of the at least two receivers is determined, if the receiving time is in a preset time range, notify the sender to regenerate a new key frame, And after receiving the new key frame regenerated by the sender, routing to the other recipients, and simultaneously updating the last valid key frame of the sender of the cache unit cache.

In another embodiment, the media server 400 according to the embodiment of the present invention further includes: a reason determining unit, configured to determine, by the media server, a receiving time corresponding to the request of the at least two receiving parties, Determining, by the preset time range, the reason for the request of the at least two receivers;

Preferably, in another embodiment, the media server 400 according to the embodiment of the present invention further includes: a period detecting unit, configured to periodically detect, on a last valid key frame of each user terminal buffered by the cache unit;

For the specific working mode of each unit in the media server of the embodiment of the present invention, refer to the related content in the foregoing method embodiment of the present invention, and details are not described herein again. In summary, a media server according to the present invention caches the sender of the sender by buffering the last valid key frame of each user terminal and receiving a key frame of only one receiver requesting the sender. The last effective key frame is routed to the receiver's technical solution, and the sender no longer regenerates new key frames and routes them to other individuals in the multimedia interworking through the media server. The user terminal solves the problem that the entire network pressure rises due to the use of non-directional routing when a certain key frame is lost in the prior art; and the technical solution of the present invention also considers the actual scenario, in the multimedia intercommunication of the multi-user terminal. If there is only one receiving direction requesting a key frame from the sender, the reason for the request is very large because the single receiver loses the key frame of the sender, and only the key frame requested by it needs to be routed to the single receiver. That is, other user terminals do not need the key frame of the sender, so the technical solution of the present invention can greatly avoid the problem that the subsequent data cannot be parsed while reducing the network pressure.

Further, when multiple receivers request a key frame from one sender at the same time, it is indicated that multiple receivers need the key frame of the sender, and the possibility is that the receiver is using the key of the sender. A problem occurs in the frame decoding. In the embodiment of the present invention, the media server is used to notify the sender to regenerate a new key frame, and after receiving the new key frame regenerated by the sender, routing to the other user terminals. The technical solution of updating the last valid key frame of the sender that is cached at the same time can largely guarantee the parsing of subsequent data.

Further, when different key receivers requesting the sender's key frame are not received at the same time, the embodiment of the present invention uses the request reason for not receiving more than one receiver requesting a certain sender's key frame at the same time. The differentiated technical solution can further realize directional routing, reduce network pressure, avoid the problem that the whole network pressure rises due to the loss of saved key frames by one or some receivers, and can also largely avoid subsequent data. Unresolved issue.

Further, in the embodiment of the present invention, the media server periodically detects the last valid key frame of each user terminal that is cached by the media server, and ensures that the last valid key frame of each user terminal buffered by the media server is always in the Valid state. It should be noted:

The various component embodiments of the present invention may be implemented in hardware, or in a software module running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that a microprocessor or digital signal processor (DSP) may be used in practice to implement some or all of the functionality of some or all of the components in accordance with embodiments of the present invention. The invention can also be implemented as a device or device program (e.g., a computer program and a computer program product) for performing some or all of the methods described herein. Such a program implementing the invention may be stored on a computer readable medium or may be in the form of one or more signals. Such signals may be downloaded from an Internet website, provided on a carrier signal, or provided in any other form. "One embodiment," or "an embodiment," or "one or more embodiments" as used herein means that the particular features, structures, or characteristics described in connection with the embodiments are included in at least one embodiment of the invention. In addition, it is noted that the phrase "in one embodiment" herein does not necessarily refer to the same embodiment.

In addition, it should be noted that the language used in the specification has been selected for the purpose of reading and teaching, and is not intended to explain or limit the subject matter of the present invention. Therefore, many modifications and changes will be apparent to those skilled in the art without departing from the scope of the invention. The disclosure of the present invention is intended to be illustrative, and not restrictive, and the scope of the invention is defined by the appended claims.

Claims

Claim

A method for routing a key frame, the method comprising:

The media server caches the last valid key frame of each user terminal;

When a key frame is received that only one recipient requests the sender, the media server routes the last valid key frame of the sender that it caches to the recipient.

2. The method according to claim 1, wherein the method further comprises: when receiving at least two recipients requesting a key frame of the sender, the media server is to the at least two receivers The corresponding reception time is requested to be judged.

The method according to claim 2, wherein, after the media server determines the receiving time corresponding to the request of the at least two receivers, the method further includes:

If the receiving time is in the preset time range, the media server notifies the sender to regenerate a new key frame, and after receiving the new key frame regenerated by the sender, routing to other recipients, and updating at the same time The last valid keyframe of the sender that is itself cached.

The method according to any one of claims 1-4, wherein the method further comprises: the media server periodically detecting the last valid key frame of each user terminal that is cached by itself;

6. A media server, comprising:

a cache unit, configured to cache a last valid key frame of each user terminal;

The media server according to claim 6, further comprising:

The time judging unit is configured to determine, when the at least two receivers request the key frame of the sender, the receiving time corresponding to the request of the at least two receivers.

The media server according to claim 7, further comprising:

9. The media server according to claim 7, further comprising:

The media server according to any one of claims 6-9, further comprising: a period detecting unit, configured to periodically detect a last valid key frame of each user terminal buffered by the buffer unit;