JP2019527497A - Method of sending multimedia resources based on heterogeneous networks synchronously - Google Patents

Abstract

The method of synchronously sending multimedia resources based on heterogeneous networks according to the present invention includes obtaining delay information of an end-to-end broadband network where different users are located or delay of broadband network and available broadband information by a server end. 1 and after the server end receives the request from the user end, the multimedia resources are synchronously displayed based on the delay information of the end-to-end broadband network where the different users are located or the delay and available broadband information of the broadband network. Step 2 for calculating a possible time, and the server end transmits the multimedia resource of the time that can be synchronously displayed, and the user end is set to the start display time of the media resource so that all broadband contents of different users can be synchronously displayed. Base Including the step 3 to broadcast synchronize the content you are, a. The present invention solves the problem that broadcasting and broadband user media resources become asynchronous due to differences in network conditions in heterogeneous network transmission. [Selection] Figure 1

Description

  The present invention relates to a media transmission method in a heterogeneous network in the information technology field, and more particularly, to a method for synchronously transmitting multimedia resources based on a heterogeneous network.

  With the ever-changing network technology, the media display system has changed greatly, and new media display formats are constantly appearing. Conventional television is not already a major route for people to acquire and enjoy information, and more and more terminal equipment, such as PCs connected to the Internet, almost every mobile phone, and is becoming popular every day Mobile tablets, etc. appear in front of us, and these new products are already eroding the traditional television market. The multimedia business is maturing day by day with the development of mobile communications and broadband wireless technologies that are undergoing transformation, and large-scale media fusion is already an inevitable trend. In this process, new ideas and high quality products emerge one after another, allowing users to conveniently connect to the network and easily use rich media content and diverse services.

  At the same time, the display of media content is not only simple video, audio and subtitles, but also more and more media types. The source of the media is not limited to a specific content provider, more creators are entered, and many individual users are also content providers and producers. There are a variety of related relationships between content from these different providers, and often these related content must be displayed synchronously to satisfy different user customization requirements. In such an environment, the fusion of heterogeneous networks is an inevitable trend for the development of next-generation networks, and various access technologies coexist and cooperate in the future, rather than a specific access technology with no future communication. Fully explained.

  In a heterogeneous media network environment, media content displayed by a terminal can be simultaneously transmitted from a plurality of transmission channels, for example, a broadcast channel and a broadband channel. Broadcast channel media resources have short delay (basically neglected), wide bandwidth and stable characteristics, but different wideband channels (WIFI, 3G / 4G, etc.) have large bandwidth and network delay. Has a difference.

For this reason, in the media resource service of a heterogeneous network composed of broadcast and broadband, the following three asynchronous situations may exist due to the difference in the network status of the broadband channel.
First, when the server end transmits media content to the user by broadcast, when the user requests a broadband resource of the same media content, the client end receives the broadcast and broadband from the delay and bandwidth of the broadband network. Content becomes asynchronous in time.
Second, when the server end transmits the same media content to the user by broadcast and broadband, the media streaming is switched when the user requests to change the media stream (eg, different viewing angle, different audio, etc.) of the broadband resource. In this case, since the delays and bandwidths of different broadband networks are different, the new bandwidth content and the broadcast content become asynchronous at the terminal.
Third, when the server end transmits the same media content to the user by broadcast and broadband, the broadband network situation deteriorates at a certain time, and the broadcast of the broadband channel does not become smooth, and the video broadcasts smoothly again. Waiting for the time to be broadcast, the broadcast and broadband resources are asynchronous between different users.

  Therefore, in the transmission system of the next generation heterogeneous network, how to make the server end synchronously transmit multimedia resources according to the network situation is a problem to be solved.

  The present invention has been made in view of the above problems, and realizes a mechanism for synchronizing broadcast media at a server end and broadband media content in heterogeneous network transmission, and broadcast and broadband users due to a difference in network conditions in heterogeneous network transmission. It is an object of the present invention to provide a method for synchronously transmitting multimedia resources based on heterogeneous networks, which can solve the problem of asynchronous media resources.

In order to achieve the above object, a method for synchronously transmitting multimedia resources based on a heterogeneous network according to the present invention uses one of the following two methods:
As method 1,
Obtaining delay information of an end-to-end broadband network where different users are located by the server end; and
After the server end receives the request from the user end, calculating a time at which multimedia resources can be synchronously displayed based on delay information of an end-to-end broadband network where different users are located; and
Step 3 where the server end transmits multimedia resources after the time that can be synchronously displayed and the user end synchronously broadcasts the content based on the start display time of the media resource so that all broadband content of different users can be synchronously displayed. And including
As method 2,
Obtaining, by the server end, delays and available broadband information of broadband networks in which different users are located;
After the server end receives the request from the user end, calculating a time at which the multimedia resource can be synchronously displayed based on the delay of the broadband network in which different users are located and the available broadband information;
A step of transmitting multimedia resources after the time that can be synchronously displayed by the server end, notifying the user end of the broadcast start time of the multimedia resource content, and different user ends synchronously displaying the broadband content based on the broadcast start time 3 is included.

Further, in step 2 of the method 1, delayed △ t is different situations broadband network location of the user, in order to ensure synchronization, the server end sends a multimedia resource t _ini time in t ₀ time , Where t _ini is the start time of the first complete independent decodable media resource MU after t _tmp , where
In “Equation 1”, t ₀ is the transmission time, MU_duration is the duration of the media unit, Δt is the downlink end-to-end broadband network delay, and t _tmp is obtained by calculation This is the time when the user end receives the first independently decodable media resource MU.

  Further, in the method 1, the value range of the MU_duration is 0 to 10 s, and the value range of the Δt is 0 to ∞.

  Furthermore, in the method 1, the value of the MU_duration is 0.5 s, and the value of Δt is 2 s.

Furthermore, in step 3 of the method 1, the first transmitted content at the server end is a multimedia resource after the time _tini , and the user end is completely first in the broadband so that the broadband and the broadcast are synchronized. After receiving the MU, the multimedia resource at this time is broadcast after the time _tini .

  Further, in the method 1, the server end transmits multimedia resources to the user end, and the user end synchronously broadcasts content based on the start display time of the media resource.

Further, in step 2 of the method 2, the server end sets the time of t _{ini at} time t ₀ in order to guarantee synchronization for different situations of delay Δt and available broadband B _b of the broadband network where the user is located. Send a multimedia resource, where t _ini is the start time of the first fully independent decodable media resource MU after t _tmp , where
, And the in the "number 2", _{t 0} is the transmit time, MU_size is the size of the average MU sent, MU_duration is the duration of the media units, _{B r} is a fixed bit for broadcasting multimedia resources B _b is the available bandwidth of the wideband network, Δt is the downlink wideband network delay, and t _tmp is the calculated first user-independent decodable media resource MU Is the time when

Further, in the method 2, the range of values of the MU_size is 0～100Mbits, the range of values of the MU_duration is 0～10S, the range of values of the _{B r} is 0～50Mbps, wherein _{B b} The range of the value of is 0 to 1 Tbps, and the range of the value of Δt is 0 to ∞.

Further, in the method 2, the value of the MU_size is 30Mbits, the value of the MU_duration is 0.5 s, the value of the _{B r} is 25 Mbps, the value of the _{B b} is 1 Mbps, the △ t The value of is 2s.

Furthermore, in step 3 of the method 2, the first transmitted content at the server end is a multimedia resource after the time _tini , and the user end is completely first in the broadband to synchronize the broadband with the broadcast. After receiving the MU, the multimedia resource at this time is broadcast after the time _tini .

  Further, in the method 2, the server end transmits the multimedia resource, and simultaneously transmits one downlink signaling to notify the serial number of the first independently decodable media resource transmitted to the user end. The user is notified of the broadcast time.

  In addition, the method 1 and the method 2 may request that different users in media services of different networks request the same multimedia resource at the server end at a certain time, switch different multimedia resources, or deteriorate various situations of the network. This is applied to the case where the multimedia resource is requested again by the above and the media resource content of the part is synchronously displayed.

  In accordance with the present invention, the server end broadcast media in heterogeneous network transmission in order for the server end to take a method of sending media resources synchronously or a method of self-adaptive access to synchronized media resources for different user broadband network conditions. A mechanism to synchronize the broadcasting of media content with broadband is solved, and the problem that media resources become asynchronous due to the difference in network conditions is solved.

It is a time chart in Example 1 which concerns on this invention. It is a time chart in Example 2 which concerns on this invention.

The technical solution of the present invention will be described in detail below through preferred embodiments with reference to the drawings.
Currently, diversified terminal display methods based on heterogeneous networks have already been developed. While watching high-quality broadcast video programs, people's demand for diversified network media services is also increasing. Generally, media content from a broadcast channel is small and has a fixed delay, so it has little impact on the synchronization of media content, but media content from a wideband, such as audio / video, subtitles, multimedia applications, etc. Is easily affected by the current IP network, causing a large and fluctuating delay and causing problems in content synchronization.

  The present invention is for solving the above technical problem, and in order to make it easy for a person skilled in the art to understand, details of implementation of the technical solution will be described in detail below through two embodiments. Either one of the following two embodiments can be adopted depending on different scenes, but when the end-to-end network delay information can be acquired by the user end, the method in the embodiment 1 is adopted and the user end In the case where only the broadband network delay can be acquired by the above and the time of the process in which the media resource enters the user end is not included, the method in the second embodiment is adopted.

(Example 1)
As an overall technical means of the present embodiment, the delay time of the end-to-end broadband network where different users are located is acquired by a corresponding method in the network, and the start time at which the server end starts to transmit the broadband resource is determined. Used to change, achieve multimedia resource synchronization in heterogeneous networks.

Specifically, the method of this embodiment includes the following steps.
First, the server end can obtain delay information of an end-to-end broadband network where different users are located by some methods, for example, by estimating and measuring signaling information fed back from the user end.

  Subsequently, a different user in a media service of a different network requests the same multimedia resource at the server end at a certain time, switches between different multimedia resources, or requests a multimedia resource again due to deterioration of network conditions. On the other hand, in order to guarantee that the contents of the multimedia resource of the part are synchronously displayed, the server end can synchronously display the multimedia resource according to the delay situation of the network where the different user is located after receiving the user's request. Calculate the time.

  Finally, the server end transmits multimedia resources after the time that can be displayed synchronously, and the client end synchronously broadcasts the content based on the broadcast start time of the media content, and all broadband content of different users is synchronously displayed. Guarantee that you can.

As shown in FIG. 1, in a specific embodiment, the following is performed.
(Step 1)
The known server end receives a request from the client end at time t ₀ , the downlink end-to-end broadband network delay for different users is Δt, and the duration of the independently decodable media unit MU (media unit) Is MU_duration.

(Step 2)
To delay different situations of end-to-end broadband network location of the user, in order to ensure synchronization, the server sends the multimedia resources t _ini time in t ₀ time. t _ini is the start time of the first complete MU after t _tmp , and has a relationship as shown in equations (1) and (2) in “Equation 3”.
In equation (1) above, MU_duration is the duration of the media unit, in equation (2) above, t ₀ is the transmission time, Δt is the downlink end-to-end broadband network delay, t _tmp is the time when the client end obtained by calculation receives the first MU.

In a preferred embodiment, the parameter can be selected from the following numerical values.
MU_duration: 0 to 10 s, preferably 0.5 s.
Δt: 0 to ∞, preferably 2s.

(Step 3)
Since the first content to be transmitted is the _tini time and subsequent multimedia resources, the broadband will not broadcast immediately after receiving the complete first MU to guarantee the synchronization between the broadband and broadcast. , Until t _ini time, and thereafter, the media content at this time is transmitted.

  In the following, a specific example in which multimedia resources are synchronized will be described (the method of notifying the request at the user end and the user end at the server end is not limited, and in the following, the signaling method is taken as an example, and the signaling is the MMT protocol. based on).

  When a user watches an Olympic live broadcast program, there are many highlight scenes and viewer preferences vary, so the TV station will be able to shoot multi-angle videos for viewers to choose from .

At this time, the viewer can watch the main program and view a close-up expression of a player from a small screen, but if the player's game is about to end, the viewer immediately changes the angle and changes to another player. The request signaling should include an end-to-end network delay fixed_end_to_end_delay, which can be obtained from the HRBM_message, as shown in Table 1 below. As shown in FIG.
message_id indicates the message indicator of HRBM,
version indicates the message version of HRBM,
length indicates the message length of HRBM,
extension_fields_Byte indicates an extension field of HRBM,
max_buffer_size indicates the maximum caching size of HRBM,
fixed_end_to_end_delay indicates end-to-end network delay;
max_transmission_delay indicates the maximum transmission delay,
At t ₀ = 18: 00: 00, the server end receives the request, where fixed_end_to_end_delay = Δt = 2s. The duration of the independently decodable media unit can be obtained from the video resource and MU_duration = 0.5s.

Δt = 2s
t _tmp = t ₀ + Δt = 18: 00: 02
t _ini = 18:00: 022
The server end transmits a resource of t _ini = 18: 00: 022 at time t ₀ = 18: 00, and the time when the client end receives the first media resource is 18:00:02 The current media unit is cached according to the time stamp information timestamp = 18: 00: 022 of the first media resource, and the resource is broadcast after 18: 00: 022.

  In this way, when the broadcast and video on demand are synchronized, the user can see the appearance of one other player with a different viewing angle, and thus the broadcast is caused by the difference in network conditions during transmission of different networks. And the problem that the broadband user media resources are asynchronous.

(Example 2)
The overall technical means employed by this embodiment is to acquire information such as the broadband network delay and the available bandwidth where different users are located by a corresponding method in the network, and the server end transmits broadband resources. Used to change the start disclosure time to achieve multimedia resource synchronization in heterogeneous networks.

Specifically, the method of this embodiment includes the following steps.
First, the server end obtains information such as delay and available bandwidth of a broadband network in which different users are located by some method, for example, by estimating and measuring signaling information fed back from the user end. .

  Subsequently, a different user in a media service of a different network requests the same multimedia resource at the server end at a certain time, switches between different multimedia resources, or requests a multimedia resource again due to deterioration of network conditions. On the other hand, in order to guarantee that the contents of the multimedia resource of the part are synchronously displayed, the server end can synchronously display the multimedia resource according to the delay situation of the network where the different user is located after receiving the user's request. Calculate the time.

  Finally, the resource at the time that can be synchronously displayed by the server end is transmitted, the broadcast start time of the media content is notified to the client end, and it is ensured that all the contents of the broadband portion of different users can be synchronously displayed.

As shown in FIG. 2, in a specific embodiment, the following is performed.
(Step 1)
A known server end receives a request from the client end at time t ₀ , the downlink wideband network delay for different users is Δt, the available bandwidth is B _b , and the video bit rate is B _r The duration of the independently decodable media unit MU (media unit) is MU_duration.

(Step 2)
For different situations delay and available bandwidth of the broadband network location of the user, in order to ensure synchronization, the server sends the multimedia resources t _ini time in t ₀ time. t _ini is the start time of the first complete MU after t _tmp , and has the relationship shown in Expression (1), Expression (2), and Expression (3) in “Equation 4”.
In the above equation (3), t ₀ is the transmission time, MU_size is the size of the average MU transmitted, the detailed calculation refers to equation (2) above, and MU_duration is the duration of the media unit Yes, B _r is the fixed bit rate for broadcasting multimedia resources, B _b is the available bandwidth of the broadband network, Δt is the downlink broadband network delay, and t _tmp was obtained by calculation This is the time when the client end received the first MU.

In a preferred embodiment, the parameter can be selected from the following numerical values.
MU_size: 0 to 100 Mbits, preferably 30 Mbits.
MU_duration: 0 to 10 s, preferably 0.5 s.
B _r : 0 to 50 Mbps, preferably 25 Mbps.
B _b : 0 to 1 Tbps, preferably 1 Mbps.
Δt: 0 to ∞, preferably 2s.

(Step 3)
Since the first content to be transmitted is the _tini time and subsequent multimedia resources, to ensure synchronization between the broadband and broadcast, the broadband will not broadcast immediately after receiving the complete first MU, Wait until t _ini time, and then send the media content at this time.

  In the following, a specific embodiment in which multimedia resources are synchronized will be described. However, the method of notifying the request at the user end and the user end at the server end in the embodiment is not limited. Signaling is based on the MMT protocol. Of course, in other embodiments, the present invention is not limited to the MMT protocol, and may be applied to other media protocols.

  When a user watches an Olympic live broadcast program, there are many highlight scenes and viewer preferences vary, so the TV station will be able to shoot multi-angle videos for viewers to choose from .

At this time, the viewer can watch the main program and view a close-up expression of a player from a small screen, but if the player's game is about to end, the viewer immediately changes the angle and changes to another player. The request signaling should include the network delay network_delay and the available broadband information network_bandwidth, and the request message request_message is as shown in “Table 2” below. It is.
At t ₀ = 18: 00: 00, the server end receives the request, where Δt = 2s, B _b = 512 Kbit / s. The video bit rate and the duration of the independently decodable media unit can be obtained from the video resource, B _r = 16482 Kbit / s, MU_duration = 0.5 s.

MU_size = MU_duration * B _r = 0.5 * 16482 = 8241 bits
ΔT = MU_size / B _b + Δt = 8241/512 + 2 = 16.10 s
t _tmp = t ₀ + ΔT = 18: 00: 1610
t _ini = 18: 00: 1641
The server end transmits a resource of t _ini = 18: 00: 1641 at the time of t ₀ = 18: 00: ₀₁ : 00, and at the same time, the server end transmits one downlink signaling to the client end. The serial number of the first independently decodable media resource must be notified to notify the user of the broadcast time, and the response message response_message is as shown in “Table 3” below.

  The client end researches the table according to the order of the first known media resource in the signaling to obtain time information timestamp = 18: 00: 1641 and size MU_size = 33 Mbits, and the time when the current client end receives the resource is 18 1: 00: 1610, the current media unit is cached, and the resource is broadcast after 18: 00: 1641.

  In this way, in a situation where broadcast and video on demand are synchronized, the user can see the appearance of one other player with a different viewing angle, and thus broadcast due to differences in network conditions during heterogeneous network transmission. And the problem that the broadband user media resources are asynchronous.

  Although the technical problem, the technical solution, and the preferred effect to be solved by the present invention have been described in more detail by the above specific examples, these are only specific examples of the present invention, and the present invention. All corrections, equivalent replacements, improvements, etc. made within the scope not departing from the technical idea and principle of the present invention are all included in the protection scope of the present invention. Needless to say.

Claims (12)

A method for synchronously sending multimedia resources based on dissimilar networks,
Obtaining delay information of an end-to-end broadband network where different users are located by the server end; and
After the server end receives the request from the user end, calculating a time at which multimedia resources can be synchronously displayed based on delay information of an end-to-end broadband network where different users are located; and
Step 3 where the server end transmits multimedia resources after the time that can be synchronously displayed and the user end synchronously broadcasts the content based on the start display time of the media resource so that all broadband content of different users can be synchronously displayed. When,
Including Method 1,
Obtaining, by the server end, delay and available broadband information of broadband networks in which different users are located;
After receiving a request from the user end by the server end, calculating a time at which the multimedia resource can be synchronously displayed based on the delay of the broadband network where different users are located and the available broadband information; and
A step of transmitting multimedia resources after the time that can be synchronously displayed by the server end, notifying the user end of the broadcast start time of the multimedia resource content, and different user ends synchronously displaying the broadband content based on the broadcast start time 3 and
Including method 2, and
A method of synchronously sending multimedia resources based on heterogeneous networks, characterized in that any one of the above is used. In Step 2 of the method 1, delayed △ t is different situations broadband network location of the user, in order to ensure synchronization, the server end sends a multimedia resource t _ini time in t ₀ time, wherein Where t _ini is the start time of the first complete independent decodable media resource MU after t _tmp , where
In “Equation 1”, t ₀ is the transmission time, MU_duration is the duration of the media unit, Δt is the downlink end-to-end broadband network delay, and t _tmp is obtained by calculation It is the time when the user end received the first independently decodable media resource MU,
The method of synchronously transmitting multimedia resources based on heterogeneous networks according to claim 1.   The method according to claim 2, wherein a range of the MU_duration value is 0 to 10 s, and a range of the Δt value is 0 to ∞. .   The method of claim 3, wherein the value of the MU_duration is 0.5 s and the value of the Δt is 2 s. In step 3 of the method 1, the first transmitted content at the server end is a multimedia resource after the time _tini , and the user end is completely first through the broadband so that the broadband and the broadcast are synchronized. 3. The multimedia resource based on the heterogeneous network according to claim 2, wherein the multimedia resource at the time _{tini is} broadcasted after the MU is received, and the multimedia resource at this time is broadcasted synchronously. Method.   The heterogeneous network according to claim 1, wherein in the method 1, the server end transmits multimedia resources to the user end, and the user end synchronously broadcasts content based on a start display time of the media resource. To send multimedia resources based on the synchronization. In step 2 of method 2 above, the server end is multimedia at t ₀ time and t _ini time to ensure synchronization for different situations of delay Δt and available broadband B _b of the broadband network where the user is located. Send a resource, where t _ini is the start time of the first fully independent decodable media resource MU after t _tmp , where
, And the in the "number 2", _{t 0} is the transmit time, MU_size is the size of the average MU sent, MU_duration is the duration of the media units, _{B r} is a fixed bit for broadcasting multimedia resources B _b is the available bandwidth of the wideband network, Δt is the downlink wideband network delay, and t _tmp is the calculated first user-independent decodable media resource MU Is the time when
The method of synchronously transmitting multimedia resources based on heterogeneous networks according to claim 1. The range of values of the MU_size is 0～100Mbits, the range of values of the MU_duration is 0～10S, the range of values of the _{B r} is 0～50Mbps, the range of values of the _{B b} is 0 The method of sending multimedia resources based on heterogeneous networks synchronously according to claim 7, wherein the value of Δt is 1 Tbps and the range of the value of Δt is 0 to ∞. The value of the MU_size is 30Mbits, the value of the MU_duration is 0.5 s, the value of the _{B r} is 25 Mbps, the value of the _{B b} is 1 Mbps, the value of the △ t is 2s, 9. The method of synchronously sending multimedia resources based on heterogeneous networks according to claim 8. In step 3 of the method 2, the first transmitted content at the server end is a multimedia resource after the time _tini , and the user end completes the first first through the broadband in order to synchronize the broadband with the broadcast. 8. The multimedia resource based on the heterogeneous network according to claim 7, wherein the multimedia resource at the time _{tini is} broadcast after the MU is received, and the multimedia resource at this time is broadcasted synchronously after receiving the MU. Method.   In the method 2, the server end transmits multimedia resources, and simultaneously transmits one downlink signaling to notify the user end of the serial number of the first independently decodable media resource transmitted to the user end. The method according to claim 1, wherein the broadcast time is notified to the network.   The method 1 and the method 2 are different methods in which different users in the media service of different networks request the same multimedia resource at the server end at a certain time, or switch between different multimedia resources, or are deteriorated due to deterioration of various network conditions. The multimedia resource based on a heterogeneous network according to any one of claims 1 to 11, wherein the multimedia resource is applied to a case of requesting a media resource again and synchronously displaying the media resource content of the portion. How to send to sync.