KR20020054276A - Method for providing mobile multimedia communication with multimedia streaming format - Google Patents

Abstract

PURPOSE: A method of supplying mobile multimedia communication using a multimedia streaming format is provided to synchronize audio video caption data to minimize periodically-transmitted control data by using a streaming format for multimedia broadcasting, thereby improving multimedia picture quality and sound quality. CONSTITUTION: A client starts rendering(S1), and lists-up plug-in CODECs(S2). The client downloads an RSF(Rostic Streaming Format) stream(S3), and searches control data for initializing the CODECs(S4). The client checks whether the CODECs exist(S5). If so, the client checks a version of the CODECs(S6). If the CODECs do not exist, the client displays an error message and completes the rendering(S7). The client opens the CODECs after checking the version(S8), and decodes an audio frame as decoding a video frame(S9). The client closes the CODECs(S10), and completes the rendering(S11).

Description

METHOD FOR PROVIDING MOBILE MULTIMEDIA COMMUNICATION WITH MULTIMEDIA STREAMING FORMAT}

The present invention relates to a method for providing mobile multimedia communication using a multimedia streaming format, and in particular, a control that is periodically transmitted using a streaming format (rsf format) for multimedia broadcasting through a multicast network by rationally synchronizing audio and video subtitle data. The present invention relates to a method for providing mobile multimedia communication using a multimedia streaming format capable of minimizing data and minimizing channel burden to improve multimedia quality and sound quality.

In general, most Internet broadcasters that provide Video On Demand (VOD) services on the Internet use Real Media Server of Real Networks or Windows Media Technologies of Microsoft.

1 is a block diagram of an Internet broadcasting system using a conventional streaming format (asf, rm, etc.). Microsoft's WMT technology provides video and audio services through VOD services on the Internet, and provides MPEG-4 and 2 streaming services with MPEG-4 streaming. The WMT uses a sophisticated compression and buffering technique for streaming multimedia content in a client / server manner to receive live audio / video or on-demand audio / video data from the user's Window Media Player over the Internet. To decompress and play the content in real time.

<Windows media Technologies configuration>

Windows Media Services (WMS) consists of a set of services of unicast services, program services, station services, and media monitor services for streaming audio, video, and other data files to clients. WMS supports HTTP and Microsoft Media Server (MMS) protocols and can carry unicast and multicast streams.

Windows Media Player (WMP): User software receives and streams streams from a server running WMS.

▣ Windows Media Tools

① Windows Media Encorder: Makes live media into ASF stream or .asf file.

② Windows Media ASF indexer: Synchronize asf file with other web documents (Powerpoint, Flash, etc.).

③ Windows Media Author: Create stream source using image file and audio file (.wav).

Advanced Streaming Format (ASF) is Microsoft's streaming media format that supports audio, video, slide shows, and synchronized events. ASF is used by Microsoft's NetShow, a utility for receiving audio, video and live broadcasts over the Internet. There are two file types associated with this file format. The file with the .asx extension is used to signal the web browser to call the Windows Media Player and load the .asf file containing the streaming content.

The ASF was submitted to the ISO and the IETF to request that the standard be established. It is a key part of Windows 98, and will eventually replace the AVI format.

However, examples of existing streaming formats include Microsoft's asf and real media rm, and the system formats embedded in some international standards include mpeg-2 TS format, MPEG-1 format, MPEG-4 format, etc. Can be listed. Since the international standard formats are designed for their own applications, they are not suitable for the multicast Internet environment, that is, there is not much overhead information added and a method of expressing additional information is also inappropriate. The individual company's formats, asf and rm, are owned by their own patents in the individual company's format.

These formats are formats for transmitting Internet-based multimedia data. However, these formats are also unsuitable as the format for multicast broadcasting.

For example, there is a structural problem to send control information, video or audio related information periodically, and there is no way to send additional information such as subtitle data, and there is a lot of header information and the existing streaming format. There is a problem that there is no suitable format for broadcasting in a multicast network.

The present invention has been proposed to solve the problems of the prior art, and an object of the present invention is the rsf format is a streaming format for multimedia broadcasting over a multicast network by rationally synchronizing audio and video caption data, which should be periodically transmitted. By minimizing the control data, the burden on the channel is minimized. The rsf format contains compression codec information, and only the stream is packed in the rsf format and configured independently of audio and video compression. It has the advantage of not being dependent, and can be easily changed if a good codec is released in the future, so that the network system does not have to be concerned with these problems, thereby facilitating network management, improving multimedia quality and sound quality, audio, video, and Additional data is UDP / IP, multicast network Multimedia Streaming consists of a format for synchronizing each data with general information about audio, video, and additional data using Rsf (Rostic Streaming Format) format (hereinafter referred to as multimedia streaming format) for transmission in an environment. The present invention provides a method for providing mobile multimedia communication using a multimedia streaming format that provides a format in which additional information for data transmission is minimized.

1 is a block diagram of an Internet broadcasting system using a conventional streaming format.

2 is a flow diagram illustrating the operation of a client for providing mobile multimedia communications using a multimedia streaming format in accordance with the present invention.

3A is a general configuration of a multimedia streaming format (Rsf format) in accordance with an embodiment of the present invention.

3B is a packet format of a control header of 3A.

3C is a packet format of audio data of 3A.

3D is a packet format of video data of 3A.

3E is a packet format of text data of 3A.

<Description of the code | symbol about the principal part of drawing>

In order to achieve the above object of the present invention, the present invention provides an application system using the Rostic Streaming Format (rsf): by starting the renderer (S1), plug-in codecs (plug) retrieving control data for initializing the codecs by starting downloading the rsf stream (S3) after listing -in codecs (S2); Checking whether there is a codec (S5) and if present, checking the version of the codec (S6), if not, displaying an error message and terminating the renderer (S7); And after checking the version of the codecs, open the codecs (S8), decode the audio frame, decode the video frame (S9), close the codecs (S10), and then terminate the renderer (S11). The present invention provides a method for providing mobile multimedia communication using a multimedia streaming format.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

2 is a flowchart illustrating an operation of a client for providing mobile multimedia communication using a multimedia streaming format according to the present invention.

Start the renderer (step S1), list-up plug-in codecs (step S2), and then start downloading the rsf (Rostic Streaming Format) stream (step S3). Retrieve control data to initialize them.

The presence or absence of codecs is checked (step S5). If present, the version of the codecs is checked (step S6). If not, an error message is displayed and rendering is terminated (step S7).

After checking the version of the codecs, open the codecs (step S8), decode the audio frame and decode the video frame (step S9), then close the codecs (step S10), then terminate the rendering. (Step S11).

3A illustrates a general configuration of a multimedia streaming format (Rsf format) according to an embodiment of the present invention.

The Rsf format according to the present invention is a format for transmitting audio, video, and additional data in a UDP / IP, multicast network environment. This format consists of a format that minimizes additional information for transmission. The main role of the Rsf format is composed of a format for synchronizing each data with information about general information about audio, video, and additional data. The Rsf format has information about an audio video codec but does not define the use of only a specific codec (this is similar to the avi or asf format).

In FIG. 3A, audio, video, text data, and header information are shown in what order. The first start of the stream must start with a Control Header, which may be inserted in the middle. When transmitting a stream for multicast broadcasting, the control header should be sent in the middle. In multimedia streams, synchronization is an important factor. In the Rsf stream, audio data is used as a signal for overall synchronization. Therefore, in the Rsf format, audio streams are output after the control header, and video or text data accompanying the streams is formed in principle.

FIG. 3B shows a packet format of a control header of 3A, FIG. 3C shows a packet format of audio data of 3A, FIG. 3D shows a packet format of video data of 3A, and FIG. 3E shows a packet format of text data of 3A. .

3b shows that the start of the packet format of the control header starts with the ASCII code 'CD', followed by the size information Len of the entire field of the control header, followed by the ASCII codes 'AF' and Len. An audio format structure (AUDFORMAT) including (AUDFORMAT length) and a video format structure (VIDFORMAT) including ASCII codes 'VF' and Len (length of VIDFORMAT) are arranged. The structure (CDHEADER) of the control header is as follows.

typedef struct {

char ID [2]; // 'CD': Control data

ushort Len;

} CDHEADER;

The structure following the structure of the control header (CDHEADER) is the same as the audio format structure (AUDFORMAT) and video format structure (VIDFORMAT).

The following shows the structure (AUDFORMAT) regarding the audio format.

typedef struct {

char ID [2]; // Two character code 'AF'

ushort Len; // The length of the following structure

ushort Sam Freq; // Sampling frequency of the audio data

uchar BitPerSam; // Bit / sample

uchar NoChannels; // Number of channels

ushort SamPerCodedBlk; // Samples per block for audio coder,

ushort SamPerBlk;

// Samples per audio block, SamPerBlk = SamPerCodedBlk * n

short CodedBlkLen;

// Coded block length in byte, if = -1 then variable CodedBlk

ushort NoCodedBlk; // Number of coded blocks in audio data packets

char CID [4]; // Four character code for Codec ID

ushort Ver; // The version of the codec

ushort cLen; // The length of the following structure

} AUDFORMAT;

The following shows the structure (VIDFORMAT) related to the video format. Fields longer than 2 bytes in the structure use the Little Endian method used in PC.

typedef struct {

char ID [2]; // Two character code 'VF'

ushort Len; // The length of the following structure

ushort Width; // Image width

ushort Height; // Image height

char FramePerSec; // if FramePerSec <0, SecPerFrame

uchar BitPerColor; // Bits per color pixels: 24, 16: For RGB, 8: For Gray

ushort BitRate; // Bitrate

char CID [4]; // Four character code for Codec ID

ushort Ver; // The version of the codec

ushort cLen; // The length of the following structure

} VIDFORMAT;

FIG. 3C shows a form in which audio data is configured as a packet. The start code of the audio data packet is an ASCII code 'AD' (inter frame coded audio data packet) or 'AK' (key frame or intra frame coded audio data packet). The start code of the audio data packet is followed by the audio sequence number of the audio data packet, which is used as a synchronization signal of video and text data and used for processing when an error occurs. The audio data packet number is generated by a 16-bit counter and by a modular operation returning to 0 after 65535. Next to the audio sequence number, the data size Len and the data Data are arranged. The following shows the structure (ADHEADER) related to the audio data packet header.

typedef struct {

char ID [2]; // 'AD': aud data, 'AK': Key aud data,

ushort NoSeq; // Number of audio or video sequence

uint Len;

} ADHEADER;

3D shows a configuration diagram of a video data packet. The start code of the video data packet is 'VD' (inter frame coded video data packet) or 'VK' (Key frame or intra frame coded video data packet). The video code number comes after the start code of the video data packet. In order to synchronize with an audio signal, synchronization information is given by an audio sequence number to be synchronized and offset information in the audio packet data. After the synchronization data, the data size Len is followed by information last, followed by the data Data. The following shows a structure (VDHEADER) of the video data packet header.

typedef struct {

char ID [2]; // 'VD': vid data, 'VK': Key vid data

ushort NoSeq; // Number of audio or video sequence

ushort NoAudSeq; // Audio sequence number for synchronization

ushort Offset; // Offset position in audio block for synchronization uint Len;

} VDHEADER;

3E shows a configuration diagram of a text data packet. The start code is TD, followed by the text sequence number, followed by the audio sequence number and duration of the corresponding audio packet data to synchronize. The data size Len is followed by the last information followed by the corresponding text data.

The following shows the definition of a structure (TDHEADER) for the header of the text packet data.

typedef struct {

char ID [2]; // 'TD': text data

ushort NoSeq; // Number of sequence

ushort NoAudSeq; // Audio sequence number for synchronization

ushort Duration; // Duration of the txt data to be shown

uint Len;

} TDHEADER;

Accordingly, the burden of the channel is minimized by minimizing the control data periodically transmitted using the streaming format (rsf format) for multimedia broadcasting in a multicast network by rationally synchronizing audio and video subtitle data when transmitting multimedia broadcasting data through a multicast network. By minimizing this, it is possible to improve multimedia quality and sound quality.

As described above, in the method for providing mobile multimedia communication using the multimedia streaming format according to the present invention, the rsf format is a streaming format for multimedia broadcasting through a multicast network, and this format should be periodically generated in consideration of a narrow channel environment. It is possible to use channel efficiently by minimizing audio and video information. It is a very useful format for multimedia broadcasting service in narrowband channel such as wireless network, and it can be applied to wireless VoD service or wireless video conferencing or wireless lecture. It has an effect.

Claims (3)

In an application system that uses the Rostic Streaming Format (rsf: Start the renderer (S1), list-up the plug-in codecs (S2), and then start downloading the rsf stream (S3) to initialize the control code (Control) Retrieving Data); Checking whether there is a codec (S5) and if present, checking the version of the codec (S6), if not, displaying an error message and terminating the renderer (S7); And After checking the version of the codecs, opening the codecs (S8), decoding the audio frame, decoding the video frame (S9), closing the codecs (S10), and terminating the renderer (S11). Mobile multimedia communication providing method using a multimedia streaming format, characterized in that. The method of claim 2, wherein the rsf stream is The first start of the multimedia stream must start with a control header, and when the stream is transmitted for multicast broadcasting or when necessary, the control header can be inserted in the middle. (Rsf format) is a packet format in order of the control header, audio data, video data, text data, and then continuous audio data, video data, and text data. The multimedia stream is a method for providing mobile multimedia communication using a multimedia streaming format, characterized in that the Rsf stream is used as a signal for synchronizing the overall synchronization in the Rsf stream for synchronization. The method of claim 1, wherein the Rsf format is The start code of the packet format of the control header starts with the ASCII code '' CD, followed by the size information (Len) of the entire field of the control header, followed by the ASCII codes 'AF' and Len ( A structure of a control header (CDHEADER) in which an audio format structure (AUDFORMAT) including an AUDFORMAT length) and a video format structure (VIDFORMAT) including an ASCII code 'VF' and Len (length of the VIDFORMAT) are arranged; The start code of the audio data packet starts with ASCII code 'AD' (inter frame coded audio data packet) or AK (key frame or intra frame coded audio data packet), and then as a synchronization signal of video and text data. It is used for processing in case of error and used as a 16-bit counter, followed by the audio sequence number of the audio data packet generated by the modular operation returning to 0 after 65535, followed by the data of the audio data packet. A structure ADHEADER relating to an audio data packet header in which size Len and data Data are arranged; The start code of the video data packet starts with ASCII code 'VD' (inter frame coded video data packet) or 'VK' (key frame or intra frame coded video data packet), and then the audio signal. Video sequence number for assigning synchronization information with the audio sequence number of the audio data packet to be synchronized and the offset information in the audio packet data for synchronization with the audio packet. Is a structure VDHEADER of a video data packet header in which data size Len and data Data are placed after synchronization data; And The start code of the text packet data begins with the ASCII code 'TD', followed by the text sequence number, followed by the corresponding audio packet data number and the audio sequence number for synchronization. A computer-readable recording medium on which data is recorded having a duration (Duration), followed by a structure (TDHEADER) relating to a header of text packet data in which data size (Len) information and corresponding text data are placed. .