KR102271686B1 - Multi-media resource synchronization push method based on heterogeneous networks - Google Patents

Abstract

According to the present invention, the server-side comprises the steps of: (1) obtaining delay information of an end-to-end broadband network or delay and available broadband information of a broadband network in which different users are located; After the server end receives the user's request, calculating the time for synchronously representing the multimedia resource according to the delay of the end-to-end broadband network or the delay of the broadband network in which different users are located and the available broadband information (2) ); transmitting, by the server end, the multimedia resource of the time that can be expressed synchronously, and the client synchronously playing the contents according to the expression start time of the media resource so that all broadband contents of different users are expressed synchronously ( 3); discloses a heterogeneous network-based multimedia resource synchronization push method including; The present invention solves the problem of not synchronizing broadcast and media resources of broadband users due to different network conditions in heterogeneous network transmission.

Description

Multi-media resource synchronization push method based on heterogeneous networks

The present invention relates to a heterogeneous media network transmission method in the field of information technology, and more particularly, to a heterogeneous network-based multimedia resource synchronization push method.

As network technology is getting newer day by day, there is a huge change in the way media is expressed, and new media formats are constantly appearing. Traditional TV is no longer the main means for people to obtain information and find pleasure, and numerous terminals such as PCs that connect to the Internet, mobile phones that almost every person owns, and portable tablet PCs are becoming more and more common. They are gradually taking over the traditional TV market. Multimedia services are maturing day by day with the development of advanced mobile communication and broadband wireless technology, and large-scale media convergence is becoming an inevitable trend. You can enjoy rich media content and various services comfortably.

At the same time, the expression of media content is not limited to just moving pictures, audio, and subtitles, but will become more and more diverse. The source of the media is also not limited to a specific content provider, and more and more creators are participating, among which many individual users are also participating as content providers and creators. The contents by these different providers have various correlations, and in order to satisfy the individualization needs of each user, these related contents must always be expressed synchronously. In this environment, heterogeneous network convergence is an inevitable trend of next-generation network development, and it has been sufficiently explained that future communication is not a specific access technology, but a coexistence and cooperative operation of various access technologies.

In a heterogeneous media network environment, media content expressed by the terminal may be simultaneously transmitted from a plurality of transmission channels, for example, a broadcast channel and a broadband channel. Media resources by broadcast channels have short delay time (almost negligible), high bandwidth, and stable characteristics, and different broadband channels (WIFI, 3G/4G, etc.) have large differences in both bandwidth and network delay.

Accordingly, in the heterogeneous network media resource service composed of broadcasting and broadband, there may be the following three cases of asynchronization due to a difference in network conditions of a broadband channel. If the server side pushes the media content to the user through the broadcast, and the user requests a broadband resource of the same media content, the content received by the client from the broadcast and the broadband may not be synchronized in the timeline due to the broadband network delay and bandwidth. ; When the server end pushes the same media content to the user through broadcasting and broadband, and the user requests replacement of the broadband resource media stream (eg, different time, different voice, etc.), the delay of different broadband networks when switching media streams and bandwidth are different, so the new bandwidth content of the terminal and the broadcast content may not be synchronized; If the server end pushes the same media content to users through broadcasting and broadband, and at some point the broadband network condition deteriorates and the playback of the broadband channel is not smooth, when the video is played smoothly again, the broadcast and broadband between different users Resources may be out of sync.

Therefore, in a new generation heterogeneous network transmission system, it has become an urgent problem to have a server end transmit multimedia resources synchronously according to network conditions.

The technical problem to be solved by the present invention is to provide a heterogeneous network-based multimedia resource synchronization push method that implements a mechanism for synchronizing broadcast media and broadband media content at the server end in heterogeneous network transmission, and different network conditions in heterogeneous network transmission This solves the problem that the media resources of broadcast and broadband users are not synchronized.

The present invention solves the above technical problem through the following technical solution.

a step (1) of a server end obtaining delay information of an end-to-end broadband network in which different users are located;

calculating, by the server end, a time for synchronously expressing multimedia resources according to delay information of an end-to-end broadband network in which different users are located after receiving a request from a client (2);

The server end transmits the multimedia resource after the synchronously expressible time, and the client synchronously plays the content according to the expression start time of the media resource so that the broadband content of different users is all synchronously expressed ( 3); method (1) comprising; and

a step (1) of a server end acquiring information on delay and available broadband of a broadband network in which different users are located;

calculating, by the server end, a time during which a multimedia resource can be synchronously expressed according to the delay and available broadband information of a broadband network in which different users are located after receiving a request from the client (2);

Step (3) for the server end to transmit the multimedia resource after the synchronously expressible time, and to inform the client of the playback start time of the multimedia resource content so that different clients can synchronously express the broadband content according to the playback start time (3) ); Disclosed is a heterogeneous network-based multimedia resource synchronization push method using any one of the methods (2) including.

Furthermore, in step (2) of the method (1), if the delay Δt of the broadband network in which the user is located is different, the server end transmits the multimedia resource of time t _{ini at time t 0 to ensure synchronization, t ini} is the start time of the first complete and independent interpretable media resource MU after _{t tmp.}

0=t _ini -t _tmp <MU_duration

t _tmp = t ₀ +Δt

where t ₀ is the transmission time, MU_duration is the duration of the media unit, Δt is the downlink end-to-end broadband network delay, and t _tmp is the client obtained through the calculation to obtain the first independent interpretable media resource MU. It's time to receive

Further, in the method (1),

The value of the MU_duration is 0 to 10s,

The value of Δt is 0 to ∞.

More preferably, in the method (1),

The value of MU_duration is 0.5s,

The value of Δt is 2s.

Further, in step (3) of the method (1), the content initially transmitted by the server end is the _{multimedia resource after t ini} time, and in order to synchronize the broadcast with the broadband, the client receives the complete first MU over the broadband The multimedia resource at this time is played by waiting until time _{t ini} without playing immediately.

Further, in the method (1), the server end sends the multimedia resource to the client, and the client plays the content synchronously according to the presentation start time of the media resource.

Furthermore, in step (2) of the method (2), when the delay Δt and the available bandwidth B _b of the broadband network in which the user is located are different, the server end synchronizes by transmitting the multimedia resource of time t _{ini at time t 0} , where t _ini is the starting time of the first complete and independent interpretable media resource MU after _{t tmp.}

0=t _ini -t _tmp <MU_duration

MU_size=MU_duration*B _r

t _tmp =t ₀ +MU_size/B _b +Δt

where t ₀ is the transmission time, MU_size is the average size of the transmitting MU, MU_duration is the duration of the media unit, B _r is the fixed bitrate for playing the multimedia resource, B _b is the available bandwidth of the broadband network and , Δt is the downlink broadband network delay, and t _tmp is the time the client receives the first independent interpretable media resource MU obtained through the calculation.

Further, in the method (2),

The value of the MU_size is 0 to 100 Mbits,

The value of the MU_duration is 0 to 10s,

The _{value of B r} is 0-50 Mbps,

The _{value of B b} is 0 to 1 Tbps,

The value of Δt is 0 to ∞.

More preferably, in the method (2),

The value of the MU_size is 30 Mbits,

The value of MU_duration is 0.5s,

The _{value of B r} is 25Mbps,

The _{value of B b} is 1 Mbps,

The value of Δt is 2s.

Further, in step (3) of the method (2), the content initially transmitted by the server end is the _{multimedia resource after t ini} time, and in order to synchronize the broadcast with the broadband, the client receives the complete first MU over the broadband The multimedia resource at this time is played by waiting until time _{t ini} without playing immediately.

Furthermore, in the method (2), the server end transmits the multimedia resource and at the same time transmits one downlink signaling to inform the client of the sequence number of the transmitted first independent interpretable media resource to provide the user with the playback time. notify

Further, in the method (1) and method (2), different users request the same multimedia resource to the server end at any time in the heterogeneous network media service, switch different multimedia resources, or the network condition deteriorates to restore the multimedia resource. It is applied when requesting to synchronously express the content of the media resource of the corresponding part.

The positive and progressive effects of the present invention are as follows.

According to the above method of the present invention, the server end uses a method of synchronously pushing a media resource for different situations of a user's broadband network or a method of self-adaptively accessing a synchronization media resource for the server end in heterogeneous network transmission. It implements a mechanism for synchronously playing broadcast media and broadband media content, and solves the problem of asynchronization of media resources due to different network conditions.

BRIEF DESCRIPTION OF THE DRAWINGS It is a timing schematic diagram of Example 1 of this invention.
2 is a timing schematic diagram of Example 2 of the present invention.

Hereinafter, the technical method of the present invention will be described in detail by presenting preferred embodiments of the present invention in conjunction with the accompanying drawings.

Currently, the expression method of various terminals based on heterogeneous networks has become a development trend. While people watch high-quality broadcast video programs, demand for various network media services is also increasing. In general, media content through a broadcast channel causes a small and constant delay, so it does not significantly affect the synchronization of media content, and media content over broadband, such as audio video, subtitles, multimedia applications, etc., is currently IP network It is easily affected by , causing a large jitter delay, which causes a synchronization problem of the contents.

In order to solve the above technical problem, the present invention will be described in detail with reference to two embodiments below for those of ordinary skill in the art to understand in detail. According to different environments, any one of the following two embodiments is used. If the client can obtain the delay information of the end-to-end network, the method according to Embodiment 1 is used, and the client causes the media resource to enter the client. If only the broadband network delay can be obtained without including the time consumed in the process, the method according to the second embodiment is used.

Example 1

The main technical method of this embodiment is to obtain delay information of an end-to-end broadband network in which different users are located through a corresponding method in the network and change the start time for the server end to transmit the broadband resource, thereby reducing the amount of multimedia resources in a heterogeneous network. to implement synchronization.

Specifically, the method according to this embodiment includes the following steps.

First, the server end may acquire delay information of an end-to-end broadband network in which different users are located according to signaling information fed back by a client or through a method such as estimation measurement.

Then, when different users request the same multimedia resource to the server side at any time in a heterogeneous network media service, or switch different multimedia resources or request a multimedia resource again due to the deterioration of the network condition, the content of the multimedia resource of the corresponding part In order to synchronously express a multimedia resource, the server side calculates a time for synchronously expressing a multimedia resource according to a delay situation of a network in which different users are located after receiving a user's request.

Finally, the server end transmits the multimedia resource after the synchronously expressible time, and the client synchronously plays the content according to the playback start time of the media content, so that the contents of the broadband portions of different users are all synchronously ensure it is expressed.

As shown in Fig. 1, in a specific example, the following is carried out.

The server end _{receives the client's request at time t 0} , the downlink end-to-end broadband network of different user groups delays by Δt, and the duration of an independent interpretable media unit (MU) is MU_duration. know that (step (1)).

When the delay of the end-to-end broadband network where the user is located is different, the server sends a multimedia resource of time t _{ini at time t 0 to ensure synchronization, where t ini} is the start time of the first complete MU after _{t tmp,} Reference is made to equations (1) and (2) (step (2)).

0=t _ini -t _tmp <MU_duration......(1)

t _tmp = t ₀ +Δt....(2)

In equation (1), MU_duration is the duration of the media unit, in equation (2), t ₀ is the transmission time, Δt is the delay of the downlink end-to-end broadband network, and t _tmp is the first Time to receive the MU.

In a preferred embodiment, the parameter may select a value as follows.

MU_duration is 0 to 10s, preferably 0.5s,

Δt is 0 to ∞, preferably 2s.

Since the content initially transmitted is a _{multimedia resource at time t ini} and thereafter, in order to ensure the synchronization of broadband and broadcast, the broadband does not play immediately after receiving the complete first MU, but at _{time t ini} and thereafter. Transmit the media content (step (3)).

Hereinafter, a specific case related to the synchronization of multimedia resources (the method of notifying the client request and the server end to the client is not limited, all signaling methods are exemplified below, and the signaling is based on the MMT protocol) is presented.

When a user watches the Olympic live broadcast program, there are too many exciting contents and viewers have different tastes, so the program provider shoots videos from various angles for the viewers so that the viewers can choose.

At this time, the viewer is watching the main program and at the same time seeing a close-up of an athlete on a small screen. , and signaling of this request should include an end-to-end network delay fixed_end_to_end_delay, and the field can be obtained from HRBM_message as shown in Table 1.1 below.

Syntax Values No. of bits Mnemonic HRBM ( ){ message_id 16 version 8 length 16 extension { extension_fields_Byte } message_payload{ max_buffer_size 32 fixed_end_to_end_delay 32 max_transmission_delay 32 } }

message_id - the message identifier of the HRBM

version-message version in HRBM

message length in length-HRBM

extension_fields_Byte-HRBM extension fields

max_buffer_size - HRBM maximum buffer size

fixed_end_to_end_delay - end-to-end network delay

max_transmission_delay - maximum transmission delay

The server side _{receives the request at t 0} =18:00:00, among which fixed_end_to_end_delay=Δt=2s. The duration of an independent interpretable media unit can be obtained from a video resource, and MU_duration=0.5s.

△t=2s

t _tmp =t ₀ +Δt=18:00:02

t _ini =18:00:022

The server end _{sends a resource of t ini =18:00:022 at t 0} =18:00:00, the client receives the first media resource at 18:00:02, and receives the first media resource The current media unit is cached according to timestamp information timestamp=18:00:022, and the resource is played by waiting until 18:00:022.

At this time, the user can see the appearance of different athletes from different perspectives by synchronizing the broadcast and on-demand, which solves the problem of not synchronizing the media resources of the broadcast and broadband users due to different network conditions in heterogeneous network transmission. .

Example 2

The main technical method of this embodiment is to obtain information such as delay and available bandwidth of a broadband network in which different users are located through a corresponding method in the network and change the start time for the server end to transmit the broadband resource, thereby providing multimedia resources in heterogeneous networks. to implement synchronization of

Specifically, the method according to this embodiment includes the following steps.

First, the server end may acquire information, such as delay and available bandwidth of a broadband network in which different users are located, according to signaling information fed back by the client or through a method such as estimation measurement.

Then, when different users request the same multimedia resource to the server side at any time in a heterogeneous network media service, or switch different multimedia resources or request a multimedia resource again due to the deterioration of the network condition, the content of the multimedia resource of the corresponding part In order to synchronously express a multimedia resource, the server side calculates a time for synchronously expressing a multimedia resource according to a delay situation of a network in which different users are located after receiving a user's request.

Finally, the server end transmits the resource after the synchronously expressible time, and informs the client of the playback start time of the media content, so as to ensure that the content of different users' broadband portions are all synchronously expressed.

As shown in FIG. 2 , in a specific embodiment, it is carried out as follows.

The server end _{receives the client's request at time t 0} , the downlink broadband network of different user groups delays by Δt, the available bandwidth is B _b , the video bitrate is B _r , and the independent interpretable media unit We know that the duration of (MU, media unit) is MU_duration (step (1)).

When the delay and available bandwidth of the broadband network where the user is located are different, the server transmits the multimedia resource of time t _{ini at time t 0 to ensure synchronization, where t ini} is the start time of the first complete MU after _{t tmp,} Reference is made to equations (1), (2) and (3) (step (2)).

0=t _ini -t _tmp <MU_duration....(1)

MU_size=MU_duration*B _r.... (2)

t _tmp =t ₀ +MU_size/B _b +Δt.....(3)

In Equation (3), t ₀ is the transmission time, MU_size is the average size of the transmitting MU. Specifically, it is calculated through Equation (2), MU_duration is the duration of the media unit, and B _r is the It is a fixed bitrate, B _b is the available bandwidth of the broadband network, Δt is the downlink broadband network delay, and t _tmp is the calculated time for the client to receive the first MU.

In a preferred embodiment, the parameter may select a value as follows.

MU_size is 0 to 100 Mbits, preferably 30 Mbits,

MU_duration is 0 to 10s, preferably 0.5s,

B _r is 0-50 Mbps, preferably 25 Mbps,

B _b is 0 to 1 Tbps, preferably 1 Mbps,

Δt is 0 to ∞, preferably 2s.

Since the content initially transmitted is a _{multimedia resource at time t ini} and thereafter, in order to ensure the synchronization of broadband and broadcast, the client does not play immediately after receiving the complete first MU over broadband, but at _{time t ini} and thereafter The media content at this time is transmitted (step (3)).

Hereinafter, a specific embodiment of the synchronization of multimedia resources is presented, and in this embodiment, the method of notifying the client's request and the server end to the client is not limited, and in the following, a signaling method based on the MMT protocol is used as an example. , of course, in another embodiment, other media protocols may be applied without being limited to the MMT protocol.

When a user watches the Olympic live broadcast program, there are too many exciting contents and viewers have different tastes, so the program provider shoots videos from various angles for the viewers so that the viewers can choose.

At this time, the viewer is watching the main program and at the same time seeing a close-up of an athlete on a small screen. send a request to Signaling of this request should include network delay network_delay, available broadband information network_bandwidth, and request message request_message as shown in Table 1.2 below.

Syntax Value No. of bits Mnemonic request_message() {message_id
version
length
message_payload()
{
network_delay
network_bandwidth
}
}
8
8
16


16
32

uimsbf
uimsbf
uimsbf


uimsbf
uimsbf

The server side _{receives the request at t 0} =18:00:00, among which Δt = 2s and B _b =512 Kbit/s. The duration of the interpretable media unit independent of the video bitrate can be obtained from the video resource, B _r =16482Kbit/s, MU_duration=0.5s.

MU_size=MU_duration*B _r =0.5*16482=8241bits

△T=MU_size/B _b + △t=8241/512+2=16.10s

t _tmp =t ₀ +ΔT=18:00:1610

t _ini =18:00:1641

The server end _{transmits the resource of t ini =18:00:1641 at t 0} =18:00:00, and at the same time transmits one downlink signaling and sends the sequence number of the first independent interpretable media resource transmitted to the client. It should notify the user of the playback time, and the response message response_message is as follows.

Syntax Value No. of bits Mnemonic response_message() {message_id
version
length
message_payload()
{
number_of_assets
for(i=0; i<N; i++)
{
asset_id
mpu_sequence_number
}
}
}





N




8
8
16


16
32

16
32
uimsbf
uimsbf
uimsbf


uimsbf
uimsbf

uimsbf
uimsbf

The client searches in the table according to the first known media resource sequence during signaling to obtain time information timestamp=18:00:1641 and size MU_size=33Mbits, and the current client receiving time is 18:00:1610 , caches the current media unit, and waits until 18:00:1641 to play the resource.

At this time, the user can see the appearance of different athletes from different perspectives by synchronizing the broadcast and on-demand, so that the media resources of the broadcast and broadband users are not synchronized due to different network conditions in heterogeneous network transmission .

The specific embodiments described above describe in more detail the technical problems, technical measures and beneficial effects to be solved in the present invention, and the above contents are only specific embodiments of the present invention and do not limit the present invention, It should be understood that all modifications, substitutions of equivalents, improvements, etc. made within the spirit and principle of the present invention should be included within the protection scope of the present invention.

Claims (12)

a step (1) of a server end acquiring information on delay and available broadband of a broadband network in which different users are located;
After the server side receives the client's request, the multimedia resource is synchronized according to the delay of the broadband network where different users are located, the fixed bitrate for playing the multimedia resource, and the available broadband information including the available bandwidth of the broadband network. calculating the expressible time (2);
Step (3) for the server end to transmit the multimedia resource after the synchronously expressible time, and to inform the client of the playback start time of the multimedia resource content so that different clients can synchronously express the broadband content according to the playback start time (3) ); including;
In step (3), when the delay Δt of the broadband network where the user is located and the available bandwidth B _b are different, the server end transmits the multimedia resource of time t _{ini at time t 0 to secure synchronization.} synchronously representing the wideband content based on _{t tmp} , which is the time at which the client receives the _{first independent interpretable media resource MU, and t ini} , which is the start time of the first complete and independent interpretable media resource MU after t _tmp. A heterogeneous network-based multimedia resource synchronization push method. According to claim 1,
a step (1) of a server end obtaining delay information of an end-to-end broadband network in which different users are located;
calculating, by the server end, a time for synchronously expressing multimedia resources according to delay information of an end-to-end broadband network in which different users are located after receiving a request from a client (2);
The server end transmits the multimedia resource after the synchronously expressible time, and the client synchronously plays the content according to the expression start time of the media resource so that the broadband content of different users is all synchronously expressed ( 3); method (1) comprising; further comprising,
In the method (1), when the delay Δt of the broadband network in which the user is located is different, the server end transmits the multimedia resource of time t _{ini at time t 0 to ensure synchronization, where t ini} is the first time after _{t tmp} The start time of a complete and independent interpretable media resource MU,
0=t _ini -t _tmp <MU_duration
t _tmp = t ₀ +Δt
In the equation, t ₀ is the transmission time, MU_duration is the duration of the media unit, Δt is the delay of the downlink end-to-end broadband network, and t _tmp is the first independent interpretable media resource MU by the client obtained through the calculation. As the time to receive
Based on the calculated t _{tmp and t ini} obtained through the calculation, the heterogeneous network-based multimedia resource synchronization push method in which the client synchronously reproduces the content at the presentation start time of the media resource. 3. The method of claim 2,
The value of the MU_duration is 0 to 10s,
The value of Δt is a heterogeneous network-based multimedia resource synchronization push method, characterized in that 0 to ∞. 4. The method of claim 3,
The value of MU_duration is 0.5s,
The heterogeneous network-based multimedia resource synchronization push method, characterized in that the value of Δt is 2s. 3. The method of claim 2,
In step (3) of the method (1), the content initially transmitted by the server end is the _{multimedia resource after t ini} time, and in order to synchronize the broadcast with the broadband, after the client receives the complete first MU through the broadband A heterogeneous network-based multimedia resource synchronization push method, characterized in that the multimedia resource is played at this time by waiting until _{t ini} time, not immediately playing. 3. The method of claim 2,
In the method (1), the server end transmits the multimedia resource to the client, and the client synchronously reproduces the content according to the expression start time of the media resource. According to claim 1,
0=t _ini -t _tmp <MU_duration
MU_size=MU_duration*B _r
t _tmp =t ₀ +MU_size/B _b +Δt
where t ₀ is the transmission time, MU_size is the average size of the transmitting MU, MU_duration is the duration of the media unit, B _r is the fixed bitrate for playing the multimedia resource, B _b is the available bandwidth of the broadband network and , where Δt is the delay of the downlink broadband network,
A heterogeneous network-based multimedia resource synchronization push method for synchronously expressing the broadband content based on the t _{tmp and t ini} obtained through calculation. 8. The method of claim 7,
The value of the MU_size is 0 to 100 Mbits,
The value of the MU_duration is 0 to 10s,
The _{value of B r} is 0-50 Mbps,
The _{value of B b} is 0 to 1 Tbps,
The value of Δt is a heterogeneous network-based multimedia resource synchronization push method, characterized in that 0 to ∞. 9. The method of claim 8,
The value of the MU_size is 30 Mbits,
The value of MU_duration is 0.5s,
The _{value of B r} is 25Mbps,
The _{value of B b} is 1 Mbps,
The heterogeneous network-based multimedia resource synchronization push method, characterized in that the value of Δt is 2s. 8. The method of claim 7,
In step (3), the content initially transmitted by the server end is the _{multimedia resource after t ini} time, and in order to synchronize the broadcast with the broadband, the client does not play it immediately after receiving the complete first MU through the broadband, t A heterogeneous network-based multimedia resource synchronization push method, characterized in that by waiting until _{ini time to play the multimedia resource at this time.} According to claim 1,
The server end transmits the multimedia resource and at the same time transmits one downlink signaling to notify the client of the sequence number of the transmitted first independent interpretable media resource to notify the user of the playback time. Multimedia resource synchronization push method. 12. The method according to any one of claims 2 to 11,
In a heterogeneous network media service, different users request the same multimedia resource to the server at any time, switch different multimedia resources, or request a multimedia resource again because the network situation worsens to synchronously express the content of the media resource in the corresponding part A heterogeneous network-based multimedia resource synchronization push method, characterized in that it is applied when

Images