KR100763013B1 - Method and apparatus for processing data from a plurality of sources - Google Patents

Abstract

The invention includes a method for preparing data for transmission, the method of multiplexing data from a plurality of sources (S1, ... Si, ... Sn), wherein the multiplexing step comprises at least one Including the source Si, and classifying the data from the source Si into two or more classes (C1, ... Cj, ... Cm) according to the priority of the data, the data from Mapping the data from the sources to the positions of the data structure D according to another priority assigned to the source Si and the class Cj of the data; Subdividing the data of data structure D into frames while preserving the relative priority of the data. The sources may be various multimedia sources. Also disclosed are methods for forward error correction (FEC) encoding multiplexed data for transmission while preserving the relative priority of the data and methods for decoding and demultiplexing data. Also disclosed is an apparatus for implementing these methods.

Description

Method and apparatus for processing data from a plurality of sources

The present invention generally relates to multiplexing, encoding, and transmission of data. In particular, the present invention relates to the transmission of data originating from a plurality of sources. These may be multimedia sources.

Examples of single media services are speech and data. For such single media services, forward error correction (hereinafter referred to as FEC) schemes are used, which are developed on a case-by-case basis. The FEC uses redundancy to allow the receiver of the corrupted digital signal to determine the actual signal transmitted. FEC mitigates data corruption caused by error-prone transmission paths.

In the case of voice, the development of the FEC scheme typically involves making a different FEC scheme for all systems, taking into account the different effects of bit errors during the transmission of different parameters of the bit stream. This means that it is almost always necessary to redesign FEC to introduce new voice codecs into the system.

In designing FEC schemes for certain types of media transmissions, such as voice or video, for example, some of the bits may not receive any FEC protection, while allowing greater protection for more important segments of the data stream. In order to avoid this, it is desirable to use some type of combined source / channel or non-identical guard coding techniques.

The different requirements for FEC protection of any of these parameters or bits are well known characteristics of a low bit rate source encoder that operates according to a single source, i.e., one particular type of media transmission. For example, an error of motion vectors in the context of video coding causes a perceived degradation that is greater than an error of DCT (Discrete Cosine Transform) coefficients. This requirement means that FEC is applied with detailed knowledge of the source encoding algorithm.

Current mobile system FEC schemes, for example for single services such as voice, are mostly Rate Compatible Punctured Convolutional to provide different error protection for different bits of the encoder bit stream. Always referred to as RCPC). For example, IEEE Trans Comms, vol. See the prior art publication 'Rate-Compatible Punctured Convolutional Codes (RCPC codes) and their Applictions' published in COM-36, No.4. However, in such single service applications, the number of protected bits is generally fixed, such as the position in the FEC frame of the different parameters. Alternatively, for variable bit rate sources, encoding rules are selected from a limited number of possibilities.

Considering mobile systems as well, the current approach in designing FEC schemes for these systems is to develop the FEC scheme only for single services, ie voice. Although the performance of these FEC schemes is almost optimal for these single services, applying a similar approach to multimedia will potentially waste channel resources and lead to potential mismatches in the error performance of different services.

In a multimedia environment, it is generally necessary to multiplex bit streams from multiple sources such as video, audio, and data into a single bit stream. There are many standards that define ways to achieve this, such as ITU-T Recommendation H.223 "Line Transmission of Non-Telephone Signals-Multiplexing Protocol for Low Bit Rate Multimedia Communication".

Conventional designs for multimedia services use different FEC schemes for each of the individual services that are then multiplexed. At this time, additional error protection is required to protect the multiplexed information. This is a complex and inflexible configuration. According to this design philosophy, it is also very difficult for services to have the same quality to channel bit error rate (BER) profiles, especially if new service components are added, for example a new voice codec. Thus, if the relative robustness of the plurality of service components differs, there will be a degradation in the quality of these components that occurs at different rates at different locations in the coverage area. In practice, for example, in the case of a mobile user located in a locally recessed area or at the edge of a coverage area, some of the multimedia services received before other users may experience excessive degradation. It is clearly undesirable to have one component of a service component of service more important than others, particularly text or pictures, deteriorate faster than other services of less importance, such as sound.

Especially for multimedia data transmission for future mobile systems, service providers and network operators will want to introduce new services quickly and easily. In a multimedia situation, this may include putting together different audio, voice, and video codecs to meet the specific market niches. These places will also require very different requirements in terms of quality for different components of the multimedia service.

EP-A-0171596 provides a signal concentrator configuration. Multiple user channels are concentrated on a common communication channel. To this end, information from each user channel is buffered, prioritized, and transferred to the common communication channel based on this priority. The priority assigned to the information depends on its type. For example, data has a higher priority than voice packets. To maintain a delay associated with normal voice, some voice packets from a particular source may be given a different priority than other voice packets from another source.

US published patent US-A-5,280,479 shows a multiplexing device for inserting digital packets supplied by several different sources into the same transport channel. Each source generates an insertion priority for each packet as a function of the packet type. A particular source may have a different priority than other similar sources, for example based on the number of packets that are storing idle transmissions.

1 illustrates an embodiment of the invention in which data from a plurality of sources is multiplexed, prepared, and encoded for TDMA transmission.

2 shows selecting bits from one of the sources, in which the same portion of each class of bits is taken.

3A illustrates a method for mapping bits from two sources into a data structure in accordance with the present invention.

3B illustrates a second method for mapping bits from two sources into a data structure in accordance with the present invention.

4 shows an example of adapting the FEC protection method to data throughput changes.

According to the invention, a method for preparing data from a plurality of sources for transmission comprises: multiplexing data from a plurality of sources, wherein the multiplexing step is performed for at least one source of data from the source. Classifying into two or more classes according to priority, and mapping data from sources to locations in the data structure according to both the other priority assigned to the source from which the data originated and the class of data. The multiplexing step comprising a; Sub-dividing the data in the data structure into frames, while preserving the relative prioritization of the data.

If data from the second or other sources is classified into a plurality of grades, these grades are not necessarily the same as those used for the first said source, ie the particular grades used are unique to the relevant source.

Also provided is a method of encoding data for transmission in combination with such a method for preparing data. A method of encoding data for transmission includes preparing data from a plurality of sources according to the preparation method given above, and performing forward error correction (FEC) encoding on data frames while preserving the relative prioritization of the data. Include.

Partitioning of the data into ratings and / or prioritization of the sources may be done according to the importance of the data and the importance of the source to the data receiver (s), respectively.

Alternatively, partitioning of data into classes and / or prioritization of sources may be done according to the potential impact of transmission errors on the data.

Also provided are methods for decoding and demultiplexing data.

According to the encoding method given above, a method of decoding and demultiplexing encoded data is that the data is first decoded into frames, and then the data frames are reconstructed into a data structure, and finally, the data of the sources from which the data originates. This includes demultiplexing back into the format.

The invention also includes an apparatus adapted to perform any of the methods given above.

The present invention facilitates data transmission to or from a multimedia terminal that can be moved. For this transmission, the data processing involves multiplexing the data taking into account the two priorities described above, thus dividing the generated data structure into frames, and finally using the RCPC forward error correction method. It includes encoding.

The present invention provides several advantages. In principle, the system can choose from a wide range of input sources and codecs without requiring major system redesign when new codecs are added or removed. Thus, codecs can simply be treated as 'modular' blocks.

In order to encode pre-transmission data, a single FEC encoder and decoder design is used for all services and all logical channels, so that FEC protection of either the new or existing codecs can be balanced for performance. It is simpler to add new codecs and new service choices without having to redesign.

The present invention provides for each logical channel the flexibility to change the FEC rate, i.e. the source of the multimedia call, which allows FEC protection tradeoffs between different services of the multimedia transmission to be modified. This can also occur during call (s). The relative priorities of the sources and the priorities of the different classes of data from each particular source are selectable at will.

In its most general form, the invention relates to preparations for transmitting data from several sources. Data from one source may be the sources themselves, and different priorities may be assigned. The data output by each source constitutes one "logical channel", where the data of the logical channel must be multiplexed and divided into frames in preparation for transmission.

The illustrated preferred embodiment of the present invention shows a method and apparatus for preparing the data as well as forward error correction of the data and finally sending the data to the receiver according to the above method. In its most complete form, the present invention takes data from multiple sources to optimize the processing of the data for transmission. The data preparation and encoding included in the present invention is important to maximize the usefulness of the data to the receiver and can be separated from the transmitting user by a transmission path that causes errors in the received data. The nature of the errors and the frequency of occurrence of the errors will change over time.

An embodiment of the present invention, selected for the purposes of illustration of the drawings, receives data from a plurality of sources. The sources may have different data rates and formats, and each source may provide data of different importance to the recipient of the transmission. As an example, a source such as a video codec can generate more data frames than an audio codec, for example, but fewer frames per second.

1 shows the method of the present invention. Sources of data can take various forms. Sources shown are video codec (VC), voice / audio codec (S / A C), and data sources DS1... DSn. There may be more or less video codec and / or audio codec sources than shown, and the generalized DS type data sources may be up to n. These sources may be mobile multimedia terminals, or terminals, and may provide data for transmission to these terminals. This embodiment also includes a FEC framework for mobile multimedia systems. In this case, RCPC techniques are shown. The present invention can be implemented as an extension of existing simple multiplexing methods, as known from H.223.

Considering FIG. 1 in detail:

(1) The top row of the figure shows the data sources VC, S / A C, DS1... DSn.

(2) The second line shows the data from each source divided into grades. These classes are for example class N to class 1 for the video codec and the CRC bits are separate classes.

Prioritizing bits from one source to classes is known in the art for a separate low bit rate speech codec serving only one source. The bit stream of an individual channel is in most cases divided into units relating to some logical structure of the source encoding algorithm, such as speech frames or video frames. This applies to the prior art configuration for a single individual channel and to the configuration for each channel of the present invention.

(3) The third line shows bold arrows indicating the steps for multiplexing data into a single data structure, where data structure D consists of data of grades varying from P +, P, P-1 to 1, It is shown in the fourth line of. An example of the relative priority of each source is shown overlapped in the figure part, and the distance from the right edge of the figure indicates the priority. For example, Video Priority takes precedence over Data Source 1 Priority.

The arrow just below the voice / audio codec shows that there is no limit on its width. This indicates that the voice / audio codec is a rate determination source. In practical terms, this is used for each data structure created by taking only parts of data from different sources. Only the voice / audio codec is the whole unit of data generated by a source combined into one data structure, for example data from the entire voice frame. The selection of the data portion from each source other than the rate determination source is shown in FIG. 2 and will be described in more detail with reference to the drawings. The rate determination source can be selected in various ways. For example, the rate determination source may be a source that supports a service whose delivery is maximally impaired by timing jitter during transmission.

(4) The fourth line of the figure shows the data structure D obtained from the multiplexing step.

(5) The fifth line of the figure shows RCPC frames in which data from the data structure D is divided. This partitioning allows to retain the relative priorities of the data in the data structure.

(6) The sixth line of the figure shows the encoding and puncturing of the data from the RCPC frame and the interleaving of the final data into the slots for transmission, which are shown in the seventh and final lines of the figure. It is shown.

(7) The seventh line of the figure shows data in slots for transmission, here TDMA bursts.

Conventional FEC schemes for multiple sources are designed to perform FEC encoding on each source before multiplexing. In the multiplexing phase, each service or logical channel has a fixed importance with respect to others. In contrast, embodiments of the present invention assign a relative priority to each logical channel (and thus its associated service) and perform pre-FEC multiplexing. Relative priority information may be assigned during call setup time. Priority information must be included in the channel destination table and can be changed dynamically during a call.

According to the invention, each logical channel has a finite number of priority classes with a known number of bits in each class, with each part of its bit stream aligned. The multiplex layer will select a portion of the bits from each class in each logical channel. This choice is explained in more detail with reference to FIG. 2.

Considering in detail FIG. 2:

(1) In the upper part of the figure, a frame of data from one of the sources with data divided into classes is shown. The numbers 16, 16, 40, and 16 indicate the number of bits of the data classes immediately below each of these numbers.

(2) The lower part of the figure shows the selection of bits taken for multiplexing from the classes. In this example, the selection of bits contains half of the bits present in a single original data frame. The number of bits taken from each class of the original frame is shown as numbers 8, 8, 20, and 8 under selection. The remainder in the single frame shown in the upper part of the figure will be sequentially multiplexed into later data structures, or structures.

According to the invention, the number of bits selected from each class is limited by the size of the output data unit of the multiplex. This size is determined by the particular type and multiplexing implementation, but is generally constrained by logical channels and physical data rates whose output frames cannot be divided across multiple output data units.

In conventional multiplexing methods, bits are generally selected from input data units in a sequential manner. However, in the present invention, the bits for each class Cj are selected in the same way as obtained in a data unit where the ratio of bits of each class is equal, such as an input data frame from a particular source. See FIG. 2. Another way of describing this is to consider each class in a logical channel as a virtual subchannel with its own data rate and protection.

The final grades from all logical channels are sorted in global priority order, thereby creating a data structure (D). This global priority is derived from the priorities of the classes in each channel and the relative priorities of the logical channels. The data structure D is shown in FIG. 1 as the fourth line of the figure and consists of data classes P +, P, P-1, ....

There are a number of ways to use, i.e. combine, the priority of a channel over other channels and the priority of a particular class within that channel. Two ways of performing this mapping in the full order of the channel classes are shown in FIGS. 3A and 3B.

According to the configuration shown in FIG. 3A, a single number may be used to define the priority of each logical channel. Two channels, one video and audio channel and their data ratings are shown in the upper row of the figure. The grades of each channel are again shown in the center of the figure, offset from the right edge of the figure as an amount according to the priority of the channel. Finally, the row below the figure shows the data structure D resulting from multiplexing the ratings from the two channels.

In the case shown, the ordering within the data structure is simple shuffling of the various classes. In this example, the CRC bits for each channel were added to the left edge of that channel's highest ranking data. Thus, the CRC bits of a video channel are added with grade 3 bits of that channel. Except for this combination, which includes CRC bits, in the embodiment of FIG. 3A, the left edge of each class defines that a higher priority is obtained between channels with different classes. Outside of the embodiment of FIG. 3A, the CRC bits may be of greater or less importance than the source top rank of the data, depending on the included source.

In the configuration of FIG. 3B, an alternative to FIG. 3A is shown. In this configuration, two identical channels with the same grades are shown at the top of the figure. Channel priorities are defined by two numbers, namely maximum and minimum. This technique allows classes within each channel to cross-disperse the global order between given maximum and minimum priorities. In the center of the figure again two channels are shown, offset from the right edge. At this time, the priority varies according to the grades in each channel, and the grades thus spread out between "maximum priority" and "minimum priority" for each channel.

This configuration is useful when the highest priority level of one channel is more important than that of another channel, but the lowest priority level is less important. The configuration of FIG. 3B actually divides the same classes of data from the same channels as shown in FIG. 3A, but results in a different data structure (D).

No alternative is shown in the figures, and the classes may have individual mappings of channel grade to global grade.

Considering now the data structure D, which is the result of multiplexing, the data structure receives various classes mapped from different sources. This data structure is shown in FIG. 1 as a line of classes P +, P, P-1, P-2 ..... 1. The data structure has a header that contains framing information and is placed within the top level. This grade is indicated as P + in FIG. 1. Header information is important and therefore must be protected. If this header is damaged, the structure of the multiplexed data unit is unknown, which means that data is not available. The data structure as shown falls priorities of the data from left to right.

After mapping the data bits into the data structure D, the bits from the global ordering are placed in the frame. This is preferably FEC data frames for FEC encoding. The priority given to the data in the multiplexing phase must be maintained while these frames are created. FEC data frames must support differential protection of different classes of bits. RCPC techniques are obvious candidates for this framing, although obviously other techniques may be used.

The relative priority of the data is preserved during the repartitioning of the data from the data structure D into these FEC frames. Mapping bits into FEC (RCPC) frames is the least significant bit (class 1) in the most significant bits of the global order (class P +), i.e. the most protected parts of the frame, acting left to right in FIG. We use a simple algorithm where the outermost parts of an RCPC frame are filled first. This algorithm meets the requirement that class P + bits be placed in the most protected portions of the first frame to minimize buffering requirements and delay. Some bits are left unprotected (classes 2 and 1 in FIG. 1). These bits may ultimately be bits with non-constrained delay requirements using the ARQ type of method to ensure accurate data transmission or insensitive parameters of the source coding method.

The data frames are then encoded for transmission. In a preferred embodiment, FEC coding is used. In the case of RCPC encoding, the data is punctured while preserving all of the relative priorities of the data. The data can then be transferred. Interleave is also applicable to improve performance on burst channels.

The receiver of the data can convert the data back to the originally transmitted format. The data then regains its original format and meaning when generated by the incoming source source Si. Therefore, the present invention also extends to decoding and demultiplexing data methods.

Decoding is the inverse of the encoding process. It is assumed that knowledge of multiple framing, channel definitions, and details of the FEC scheme also exist in the encoder and decoder. A method for decoding and demultiplexing data encoded according to the steps of the above described methods includes decoding the data into frames and then demultiplexing the data back into the format of the originating sources. Such demultiplexing of data includes demultiplexing of multiplexed data as described above. According to this demultiplexing, the inversion of the multiplexing steps given above is done, preferably on data received by the user after transmission and inverse encoding. Demultiplexing converts the data back into the format of the sources from which it originates.

Embodiment of the Adaptive FEC Rate Scheme

Another improvement of the basic invention is to adapt to the FEC protection scheme used. This adaptation can be done in response to changes in data throughput, quality changes in the radio channel used for transmission, or changes in available channel bandwidth (eg, the number of TDMA slots).

In the case of RCPC, the simplest approach is to apply puncturing between output multiplexed data units, and the change in puncturing pattern is communicated to the receiver. To minimize the complexity of FEC decoding of RCPC frames, under multiple circumstances, multiple headers are encoded at the same rate at the beginning of the RCPC frame, and then a puncturing pattern, typically parent cone, for bits beyond any point in the RCPC frame. It is desirable to change the 4-5 constraint lengths of the parent convolution code (see FIG. 4).

Claims (7)

In the method for preparing multimedia data for transmission, Multiplexing data from a plurality of sources (S1, ... Si, ... Sn), wherein the multiplexing step includes, for each source (Si), the data from the source (Si); Classified into two or more grades (C1, ... Cj, ... Cm) according to the priority of, and the other priority assigned to the source (Si) from which the data originated and the grade of the data ( Cj) mapping the data from the sources to locations in the data structure D in accordance with both, partitioning the data into classes C1, ... Cj, ... Cm and / or Or the prioritization of the sources (S1, ... Si, ... Sn) is performed according to the effect of transmission errors on the data; Subdividing the data in the data structure (D) into frames while preserving the relative priority of the data. 2. The method of claim 1, further comprising performing forward error correction (FEC) encoding on the data frames while preserving the relative priority of the data. The method of claim 1 or 2, wherein the partitioning of the data into the grades C1, Cj, Cm and / or of the sources S1, Si, Sn Prioritization is performed according to the importance of the data and the importance of the source to the receiver (s) of the data, respectively. 3. The method according to claim 1 or 2, wherein the data is first decoded into frames, and then the data frames are reconstructed into a data structure (D), and finally the data is obtained from the sources (S1) from which data originated. A method for preparing multimedia data for transmission, which is demultiplexed again into the format ... Si, ... Sn). In the device, A multiplexer for multiplexing data from a plurality of sources S1, ... Si, ... Sn, wherein the multiplexer is configured to retrieve data from the source Si for at least one source Si. According to the priority of the data is classified into two or more grades (C1, ... Cj, ... Cm), the priority assigned to the source (Si) from which the data originated and the grade of the data ( Cj) maps data from the sources to locations in a data structure D according to both, partitioning the data into classes C1, ... Cj, ... Cm and / or the sources Prioritizing (S1, ... Si, ... Sn) is performed according to the effect of transmission errors on the data; And a repartitioner coupled to the multiplexer for repartitioning the data in the data structure (D) into frames while preserving the relative priority of the data. The apparatus of claim 5, wherein the decoder decodes the data into frames, and the demultiplexer demultiplexes the data into a format of the sources S1,... Further comprising the device. 6. The apparatus of claim 5, further comprising a forward error correction (FEC) encoder that encodes the data frames while preserving the relative priority of the data.