KR20140067220A - Method and apparatus of transmitting and receiving interactive multi-stream - Google Patents

Abstract

The present invention relates to a method and an apparatus for transmitting and receiving a multi-stream in an interactive service, and provides a method for transmitting a second stream in synchronization with a first stream being transmitted by a server, and the method comprises steps of receiving, from a client, a second stream synchronization reproduction request message indicating a request for transmission of the second stream such that the second stream should be synchronized with the first stream and including an identifier of the first stream, checking the first stream corresponding to the second stream based on the identifier, and synchronizing the second stream with the first stream based on a reproduction point of the first stream; and transmitting the second stream to the client, starting from the reproduction point synchronized with the first stream. When multiple streams need to be reproduced in synchronization, the server matches a reproduction point of another stream with reference to any one stream and transmits the other stream to the client, thus, minimizing receiving of unnecessary stream data, a waste of a buffer, and the time for the client to wait for reproducing a stream.

Description

TECHNICAL FIELD [0001] The present invention relates to a method and apparatus for transmitting and receiving interactive multi-

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and apparatus for transmitting and receiving streams in an interactive service, and more particularly, to a method and apparatus for synchronously transmitting multistreams and receiving them.

Streaming services over a network are commonplace. The streaming service is provided based on a streaming protocol such as RTSP (Real Time Streaming Protocol), and a user can request a stream from a first or a specific point in time regarding a video or the like.

In recent years, a multi-stream service that simultaneously provides a plurality of streams has been activated, and a plurality of streams can be linked to provide a service. Particularly in an interactive service, there is a need to synchronize different (or other) streams at the request of the user during playback of one stream or some of the streams. In this case, another stream may be transmitted from the server in synchronism with the stream being reproduced. However, when the user requests to transmit another stream in synchronism with the stream being reproduced, in the conventional method such as RTSP, Processing time and the like must be predicted and transmitted to the server side. If the expected value is not correct, there is a problem that the reproduction of the other stream is not performed smoothly or the buffer is wasted.

SUMMARY OF THE INVENTION The present invention is directed to a method and apparatus for synchronizing multiple streams.

It is another object of the present invention to provide a method and apparatus for synchronizing and transmitting a plurality of streams constituting an interactive service.

Another aspect of the present invention is to provide a method and apparatus for transmitting a stream being reproduced and another stream in synchronization with the stream being reproduced.

According to an aspect of the present invention, there is provided a method of synchronizing a second stream with a first stream in a server. The method includes receiving from a client a second stream synchronized playback request message indicating that the second stream is requested to be synchronized with the first stream and including an identifier of the first stream, Identifying the first stream corresponding to the second stream based on the identifier and synchronizing the second stream with the first stream based on the playback point of the first stream; And transmitting to the client from a playback point in time synchronized with the first stream.

According to the present invention, when it is necessary to synchronize a plurality of streams, it is possible for a server to match a reproduction time point of another stream with reference to a stream and transmit the same to a client, It is possible to minimize the waste of the buffer and the waiting time to reproduce the stream.

FIG. 1 shows an example of a service in which two or more streams are synchronized and reproduced according to a user's request.
FIG. 2 shows an example of signaling between a server and a client when requesting playback of an interworking stream using a real time streaming protocol (RTSP) as a streaming protocol.
FIG. 3 shows another example of server-client signaling in consideration of network delay and stream processing time in reproducing requests of interworking streams.
4 illustrates a server-to-client signaling method for synchronizing a second stream according to an exemplary embodiment of the present invention with a first stream.
5 illustrates a method for synchronizing and transmitting a plurality of streams in a server according to an exemplary embodiment of the present invention.
FIG. 6 illustrates a method for synchronously receiving a plurality of streams in a client according to an exemplary embodiment of the present invention.
FIG. 7 is a block diagram illustrating a server and a client for transmitting and synchronizing a plurality of streams according to an exemplary embodiment of the present invention. Referring to FIG.

Hereinafter, some embodiments of the present invention will be described in detail with reference to exemplary drawings. It should be noted that, in adding reference numerals to the constituent elements of the drawings, the same constituent elements are denoted by the same reference symbols as possible even if they are shown in different drawings. In the following description of the embodiments of the present invention, a detailed description of known configurations or functions will be omitted if it is determined that the gist of the present specification may be obscured.

In describing the components, terms such as first, second, A, B, (a), and (b) may be used. These terms are intended to distinguish the constituent elements from other constituent elements, and the terms do not limit the nature, order or order of the constituent elements. When a component is described as being "connected", "coupled", or "connected" to another component, the component may be directly connected or connected to the other component, Quot; may be "connected," "coupled," or "connected. &Quot;

FIG. 1 shows an example of a service in which two or more streams are synchronized and reproduced according to a user's request. 1 is an example of providing lecture contents on guitar performance as a streaming service.

Referring to FIG. 1, it can be seen that during the course of a first stream 101 being reproduced initially relative to the other body part, Quot; neck view "button 110 requesting playback of a second stream 102, for example, about the other neck portion, when requesting playback of a second stream 102 about a neck portion If the user clicks, a second stream 102 with respect to the other neck portion may be played along with the first stream 101. In this case, the second stream 102 is required to be reproduced in synchronization with the first stream 101.

FIG. 2 shows an example of signaling between a server and a client when requesting playback of an interworking stream using a real time streaming protocol (RTSP) as a streaming protocol. In FIG. 2, it is assumed that a user requests playback of a second stream interlocked with the first stream 50 seconds after watching the first stream. In the initial stage of the streaming service, a step of setting up a session between the server and the client can be performed, but this is omitted because it is outside the scope of the present invention.

Referring to FIG. 2, the client transmits a first stream replay request message requesting the server to transmit the first stream at a specific playback point, here, at step S200. When the RTSP is used, the first stream reproduction request message may include, for example, the following syntax.

PLAY rtsp: //example.com/media_1.mp4 RTSP / 1.0
CSeq: 4
Range: npt = 0-
Session: 12345678

In Table 1, 'PLAY rtsp: //example.com/media_1.mp4 RTSP / 1.0' indicates a file corresponding to a stream for which a client requests playback, and in this case, 'example.com/media_1 .mp4 '. The CSeq field is a counter that is incremented by 1 each time a message is transmitted, assuming a value of 4, for example. The Range field defines what range (or portion) of the requested stream to send. If the npt (Normap Play Time) of the Range field is '0-', it means to transmit from the beginning of the related stream. The session field indicates the identifier of the session for transmitting and controlling the stream between the client and the server.

The server sends an OK response message to the client in step S210, and the server transmits the first stream from the first playback point to the client in step S220.

After a predetermined time, here, 50 seconds, the client transmits a second stream reproduction request message to the server (S230). This may be the case, for example, when the user has requested playback of the second stream interlocked with the first stream, e.g., clicking the "view neck" button in FIG. If the RTSP is used, the second stream reproduction request message may include, for example, the following syntax.

PLAY rtsp: //example.com/media_2.mp4 RTSP / 1.0
CSeq: 5
Range: npt = 50-
Session: 23456789

In Table 2, 'PLAY rtsp: //example.com/media_2.mp4 RTSP / 1.0' indicates a stream for which a client requests playback, and 'example.com/media_2.mp4' corresponds to the second stream. The CSeq field is incremented by 1 every time a message is transmitted. Since the value of the CSeq field is 2 in S200, the value of CSeq is 5 in S230. The Range field defines what range (or part) of the requested stream is to be transmitted. In this example, npt of the Range field is '50 - 'since the example is transmitted from the 50 second portion of the second stream. The Session field indicates the identifier of the session for transmitting and controlling the stream between the client and the server. Since the second stream is a stream that can be controlled independently of the first stream, the Session field of the second stream may have a different value from the Session field of the first stream of Table 1.

The server sends an OK message for the second stream replay request message to the client (S240) and transmits the second stream from the 50 second replay time (S250). However, in such a case, a network delay time for sending and receiving a message and a stream processing time of a server for transmitting a stream are required. Accordingly, if the server transmits the second stream from the 50-second playback time at S250, the playback time has already passed when the second stream arrives at the actual client. For example, assuming that the network delay and the stream processing time all take four seconds, when the second stream from the 50-second playback time transmitted from the server arrives at the actual client, the first stream is played at the playback time of 54 seconds . That is, since the first stream is reproduced at a playback time corresponding to 54 seconds in the client, while the second stream is received at the client at a playback time corresponding to 50 seconds, a difference in reproduction time occurs between the two streams. In this case, the client can not reproduce the second stream and the first stream in synchronism with each other, and after stopping and buffering the first stream which is being reproduced properly for 4 seconds, The first stream and the second stream can be reproduced in synchronization with each other.

In order to solve the above problem, there is a method of considering delays of a network and a stream processing time in advance when a stream is requested from a client to a server. FIG. 3 shows another example of server-client signaling in consideration of network delay and stream processing time in reproducing requests of interworking streams. In FIG. 3, it is assumed that a user requests playback of a second stream interlocked with the first stream 50 seconds after watching the first stream.

Referring to FIG. 3, the client transmits a first stream replay request message requesting the server to transmit the first stream to a server at a specific playback point, in this case, from the beginning (S300). In case of using RTSP, the first stream reproduction request message may include the syntax described in Table 1. The server sends an OK message for the first stream reproduction request message to the client in step S310, and the server transmits the first stream from the first reproduction time to the client in step S320.

After a predetermined time, here, 50 seconds, the client transmits a second stream reproduction request message to the server (S330). The second stream reproduction request message includes a reproduction time point reflecting a delay time of the network and an estimated delay time considering a stream processing time. For example, if the expected delay time is 4 seconds, the second stream reproduction request message may include, for example, the following syntax.

PLAY rtsp: //example.com/media_2.mp4 RTSP / 1.0
CSeq: 5
Range: npt = 54-
Session: 23456789

In Table 3, the Range field indicates a case in which the npt value is '54 -', reflecting the request time and the expected delay time of the second stream.

The server sends an OK message for the first stream replay request message to the client in step S340 and transmits the second stream to the client in response to the 54 second replay (S350).

However, even in the case of reflecting the estimated delay time considering the network delay time and the stream processing time as described above, the delay of the network can not be constant, and since the stream processing time of the server also depends on the performance of the server, Can not be calculated. For example, when the client transmits the second stream reproduction request message to the server in step S330, the network delay time and the stream processing time are estimated to be too short to estimate the expected delay time to be 2 seconds, and the reproduction time point of the second stream 52 seconds (i.e., npt of the Range field of Table 3 is set to '52 -'), the client requests the playback of the first stream at the time when the client receives and reproduces the second stream from the server If the reproduction time point of the first stream is 54 seconds, the first stream being properly reproduced due to the difference of about 2 seconds is stopped for 2 seconds and buffered, so that there is a problem that the reproduction is interrupted and the buffer is wasted. As another example, when the client transmits the second stream reproduction request message to the server in step S330, the network delay time and the stream processing time are estimated to be too long to estimate the expected delay time to be 6 seconds, 56 seconds (i.e., npt of the Range field in Table 3 is set to '56 -'), when the client receives the second stream from the server at S350, The data of the second stream for about 2 seconds should be reproduced in synchronization with the first stream after the reproduction time of the first stream reaches 56 seconds after buffering. That is, when the client expects the expected delay time longer, there is a problem that not only the buffer is wasted but also the user has to wait a longer time until the reproduction of the second stream requested.

Hereinafter, in order to solve the above-mentioned problems in FIG. 2 and FIG. 3, the present invention proposes the following method.

4 illustrates a server-to-client signaling method for synchronizing a second stream according to an exemplary embodiment of the present invention with a first stream. In the present invention, the server transmits a plurality of streams in synchronization with each other, the server may transmit the streams corresponding to the corresponding playback points of the plurality of streams. For example, synchronizing in the present invention means that if the 40-second playback time of the first stream and the 10-second playback time of the second stream correspond to each other, the playback time of 40 + n- 2 < / RTI > < RTI ID = 0.0 > + n < / RTI > Also, in the present invention, the playback time may include another term (e.g., a time stamp, etc.) that can indicate the temporal position of the stream.

Referring to FIG. 4, the client transmits a first stream replay request message requesting the server to transmit the first stream from the specific replay point, here, from the beginning (S400). In case of using RTSP, the first stream reproduction request message may include the syntax described in Table 1. The server sends an OK message for the first stream reproduction request message to the client in step S410, and the server transmits the first stream from the first reproduction time to the client in step S420.

The client transmits a second stream synchronous playback request message to the server (S430), requesting that the second stream be synchronized with the first stream. The second stream synchronous reproduction request message includes an identifier of the first stream corresponding to (or interlocking with) the second stream. The identifier of the first stream may be a text, a number, or the like, which is a value that the server can distinguish the first stream from other streams.

The second stream synchronous reproduction request message may include, for example, the following syntax.

PLAY rtsp: //example.com/media_2.mp4 RTSP / 1.0
CSeq: 5
Range: synch = rtsp: //example.com/media_1.mp4
Session: 23456789

Referring to Table 4, the Range field includes a synch parameter, and the parameter value may indicate an identifier of the first stream. In this case, the second stream synchronous reproduction request message may indicate to the server that the second stream is to be transmitted according to the reproduction time of the first stream.

In addition, the second stream synchronous reproduction request message may include the following syntax as another example.

SYNCPLAY rtsp: //example.com/media_2.mp4 example.com/media_1.mp4 RTSP / 1.0
CSeq: 5
Session: 23456789

Referring to Table 5, in 'SYNCPLAY rtsp: //example.com/media_2.mp4 example.com/media_1.mp4 RTSP / 1.0', 'SYNCPLAY' means that the playback point of one stream is transmitted according to the playback point of another stream 'Example.com/media_2.mp4' may refer to a second stream requesting transmission, and 'example.com/media_1.mp4' may refer to a request for a playback time reference 1 stream.

Table 4 and Table 5 are examples of the second stream synchronous playback request message according to the present invention, and the second stream synchronous playback request message is not limited to the syntax of Tables 4 and 5, It may contain other statements. Also, in the examples of Tables 4 and 5, RTSP is used as an example. However, the present invention is not limited to the case where the scope of the present invention uses a specific protocol.

In step S440, the server receiving the second stream synchronous reproduction request message sends an OK message for the second stream synchronous reproduction request message to the client in step S440. The second stream is synchronized with the reproduction time point and transmitted to the client (S450).

According to the present invention, since the reproduction time point of the second stream is transmitted from the server to the client in accordance with the corresponding reproduction time point of the first stream, the client minimizes the waiting time for receiving unnecessary stream data, wasting the buffer, can do.

5 illustrates a method for synchronizing and transmitting a plurality of streams in a server according to an exemplary embodiment of the present invention.

5, the server receives a first stream replay request message from the client requesting transmission from a specific replay point (including the first replay point) of the first stream (S500). The first stream reproduction request message may include the syntax described in Table 1.

The server transmits an OK message for the first stream reproduction request message to the client (S510).

The server transmits the first stream from a specific playback point of the first stream (S520).

The server receives from the client a second stream synchronous playback request message requesting to transmit the second stream in synchronism with the first stream (S530). Here, the second stream synchronous reproduction request message may include a first stream identifier. The second stream synchronous reproduction request message may include the syntax of Table 4 or Table 5 described above.

The server transmits an OK message for the second stream synchronous reproduction request message to the client (S540).

The server synchronizes the second stream with the first stream based on the reproduction time of the first stream (S550). The server can identify the first stream corresponding to the second stream based on the first stream identifier.

The server transmits the second stream to the client from a playback point in synchronization with the first stream (S560).

Figure 6 illustrates a method for receiving multiple streams in a client in accordance with an example of the present invention.

The client transmits a first stream reproduction request message requesting transmission of the first stream from a specific reproduction time to the server (S600). The first stream reproduction request message may include the syntax described in Table 1.

The client receives an OK message for the first stream reproduction request message from the server (S610).

The client receives the first stream from the specific playback time (S620). The client can reproduce the first stream from the specific reproduction time point.

The client transmits a second stream synchronous playback request message to the server, requesting to transmit the second stream synchronized with the first stream (S630). Here, the second stream synchronous reproduction request message may include an identifier of the first stream. The second stream synchronous reproduction request message may include the syntax of Table 4 or Table 5 described above.

The client receives an OK message for the second stream synchronization reproduction request message from the server (S640).

In operation S650, the client receives the second stream transmitted from the playback point in synchronization with the first stream, from the server. The client can receive the second stream transmitted from the reproduction point synchronized with the first stream and reproduce simultaneously with the first stream currently being reproduced. The client may synchronize and simultaneously play the second stream with the first stream as needed.

FIG. 7 is a block diagram illustrating a server and a client for transmitting and receiving a plurality of streams according to an exemplary embodiment of the present invention.

7, the client 700 includes a processor unit 710, a transmitting unit 720, and a receiving unit 730.

The processor unit 710 generates a first stream reproduction request message. The first stream reproduction request message may include information on a specific reproduction point (or a reproduction range) of the first stream. The specific reproduction time may include a first reproduction time. The first stream reproduction request message may include the syntax of Table 1 described above.

In addition, the processor unit 710 generates a second stream synchronous reproduction request message. The second stream synchronous reproduction request message is a message requesting to transmit the second stream in synchronization with the first stream. The second stream synchronous reproduction request message may include an identifier of the first stream. The second stream synchronous reproduction request message may include the syntax of Table 4 or Table 5 described above.

The transmission unit 720 transmits the first stream reproduction request message to the server 750. In addition, the transmitter 720 transmits the second stream synchronous reproduction request message to the server 750. [

The receiving unit 730 receives an OK message for the first stream reproduction request message from the server 750. The receiving unit 730 receives the first stream from the base station. In this case, the receiving unit 730 may receive the first stream from the server 750 at the specific playback time.

Also, the receiver 730 receives an OK message for the second stream synchronous reproduction request message from the server 750. The receiving unit 730 receives the second stream from the base station. In this case, the receiving unit 730 can receive the second stream transmitted from the playback point in time synchronized with the first stream, from the server 750. [

The server 750 includes a processor unit 760, a receiving unit 770, and a transmitting unit 780.

The receiving unit 770 receives the first stream reproduction request message from the client 700. The first stream reproduction request message may include the syntax of Table 1 described above.

The receiving unit 770 receives the second stream synchronous reproduction request message from the client 700. The second stream synchronous reproduction request message is a message requesting to transmit the second stream in synchronization with the first stream. The second stream synchronous reproduction request message may include a first stream identifier. The second stream synchronous reproduction request message may include the syntax of Table 4 or Table 5 described above.

The processor 760 may interpret the first stream reproduction request message received by the receiver 770, generate an OK message for the first stream reproduction request message, and generate the first stream. In this case, the processor unit 760 may transmit the first stream to the client 700 through the transmitting unit 780 from a specific playback time.

 The processor unit 760 may interpret the second stream synchronous reproduction request message received by the receiver 770, generate an OK message for the second stream reproduction request message, and generate the second stream . In this case, the processor unit 760 synchronizes the second stream with the second stream on the basis of the reproduction time point of the first stream, and transmits the second stream from the reproduction point in synchronization with the first stream, To the client 700 via the Internet 780. For example, the processor unit 760 may match the reproduction time point of the second stream to the reproduction time point of the first stream with reference to the reproduction time point of the first stream, To the client 700 via the transmission unit 780 from the playback point of time matched with the playback point of time.

The transmitting unit 780 transmits an OK message for the first stream reproduction request message to the client 700. The transmitting unit 780 may transmit the first stream to the client 700 from a specific playback time.

The transmitting unit 780 transmits an OK message for the second stream synchronous reproduction request message to the client 700. [ The transmitting unit 780 may transmit the second stream to the client 700 from a playback point in time synchronized with the first stream.

The foregoing description is merely illustrative of the technical idea of the present invention, and various changes and modifications may be made by those skilled in the art without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are intended to illustrate rather than limit the scope of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be construed according to the following claims, and all technical ideas falling within the scope of the same shall be construed as falling within the scope of the present invention.

Claims (1)

A method for transmitting a second stream in synchronization with a first stream in a server,
Receiving a second stream synchronous reproduction request message from a client indicating that the second stream is requested to be synchronized with the first stream and including an identifier of the first stream;
Identifying the first stream corresponding to the second stream based on the identifier and synchronizing the second stream with the first stream based on the playback point of the first stream;
And transmitting the second stream from the playback point in time synchronized with the first stream to the client.

Images