WO2008139120A2

WO2008139120A2 - Multimedia flow processing architecture

Info

Publication number: WO2008139120A2
Application number: PCT/FR2008/050637
Authority: WO
Inventors: Thomas Bouton; Vincent Themereau
Original assignee: Streamwide
Priority date: 2007-04-13
Filing date: 2008-04-10
Publication date: 2008-11-20
Also published as: FR2915047A1; FR2915047B1; WO2008139120A3

Abstract

The invention relates to a system for managing a flow of digital data of the multimedia type, that comprises: a means (10) for acquiring a flow of multimedia data; a means (14) for processing said data flow adapted for carrying out least two different transcodings of said data flow; a means (12) for transmitting said processed data flow; a means (16) for storing data from said flow, characteristics thereof and results from the processing of said data flow. The processing means include an optimisation means (18) for selecting, during a given transcoding, a processing mode for minimising the resources required for said transcoding based on the information contained in the storage means.

Description

ARCHITECTURE FOR PROCESSING MULTIMEDIA STREAMS

The present invention relates to a system and method for managing multimedia digital data streams. It also relates to a computer program for implementing the method.

The digital media data stream servers, called media server, are used to record streams of images, sound or video circulating on a network and / or to broadcast these files over the network.

The routing role of multimedia streams is usually accompanied by a computation required by a filtering or transcoding operation of the content of the stream. For example, it may be necessary to change the encoding mode of the stream before distributing it by changing the type of encoding, sampling, or bit rate. Processing may also include calculating the power of an audio signal, Goertzel coefficients for detecting DTMF carriers, gaining a signal, and so on.

Depending on the type of application running on the media server, the same stream may be subjected to different processing before broadcast on several different channels. For example, a video stream intended to be broadcast in parallel on computers connected by an ADSL-type wired connection and so-called 3G mobile telephones, will have to be coded with different compression ratios to take account of the difference in speed between the ADSL link and the 3G link. Thus the media server performs as much processing as there is coding or computation particular to that creates an important constraint on the sizing of the computing units of the media servers. As a result, these servers are built around massively parallel architecture using a large number of microprocessors. In addition to the difficulty of developing such an architecture, these servers have high manufacturing costs.

It would therefore be desirable to be able to use servers that are capable of having the same capacity in terms of the number of streams supported in parallel while being of a lower cost, or which, for the same cost, are capable of processing more of flow.

Thus, to solve this problem, in one aspect of the invention, a multimedia type digital data flow management system comprises: means for acquiring a multimedia data stream; means for processing said data stream adapted to perform at least two different transcodings of said data stream, means for transmitting said processed data stream, means for storing data from said stream, its characteristics and the results of the data streams; processing of said data stream.

In addition, the processing means comprise optimization means for, during a given transcoding, select a processing mode for minimizing the resources required for said transcoding based on the information contained in the storage means. Thus, this advantageously allows the results of calculations already made to be retained for use in subsequent treatments, thus saving processing capacity by performing each operation only once. In addition, the means of treatment are adaptive insofar as the search optimization is made based on the information available at the time the transcoding is to be performed.

The characteristics of the flows as well as the results of the treatments are stored in a data structure advantageously allowing coherent and easy access to these results.

For example, for each encoding of the stream, a data structure comprises fields defining the type of coding, the frame length, the coding depth and the sampling rate.

Thus, this data structure advantageously contains, among other things, the different versions of the medium allowing quick access to the most relevant coding for the subsequent processing. The structure also contains the information characteristics of the medium, for example the signal strength as well as any key transmitted via the medium (DTMF).

The optimization means, before any transcoding, verify the content of the storage means so as to calculate only the results of the processing not available in the storage means using data available in the data structure to minimize the calculations to achieve. The processing means are adapted to transfer to the storage means all the results of the treatments performed.

According to a second aspect of the invention, a method for managing a multimedia type digital data stream comprising the steps of:

- allocation of a data structure (IDC) capable of containing the data and characteristics associated with a frame of the multimedia stream,

- reception in the data structure of a frame of the multimedia stream from the source,

performing at least one transcoding on said stream, storing the result or results of said transcoding in said data structure.

The execution of each transcoding is adapted to minimize the resources required for said transcoding based on the information stored in the data structure. According to a particular embodiment, the data structure comprises among others, for each encoding of the stream, fields defining the type of coding, the frame length, the coding depth and the sampling rate. In a third aspect of the invention, a computer program product includes instructions for implementing the above method when running on a computer.

The invention will be better understood on reading the description which follows, given solely by way of example, and with reference to the appended figures in which: FIG. 1 is a block diagram of a multimedia flow processing system comprising a multimedia type digital data stream management server according to one embodiment of the invention; Figure 2 is a flow chart of a management method according to an embodiment of the invention; and Fig. 3 is a diagram of a data structure according to one embodiment of the invention.

With reference to FIG. 1, a network 1 comprises a digital data flow management server 2 of multimedia type.

The server 2 is connected to other computer equipment 3, 4, 5 and / or telecommunications to receive multimedia type data streams, that is to say streams of video, audio or image data.

The server 2 is also connected to computer equipment 6, 7, 8 and / or telecommunications to send multimedia streams.

It should be noted that, very often, the same equipment can be both source and destination of a multimedia stream. Similarly, a stream can come, or be provided, to an internal storage unit at the server 2.

The network 1 is typically an internet type digital data network and the data streams are, for example, transported using the RTP protocol.

{Real-Time Transport Protocol - real-time transport protocol).

The server 2 thus comprises means 10 for acquiring a multimedia stream as well as means 12 for transmitting multimedia streams.

It also comprises data processing means 14 connected to storage means 16 adapted to store the results of the calculations carried out by the processing means 14 on the multimedia streams, as well as the data of the streams and their characteristics.

The processing means 14 comprise, in particular, means 18 for optimizing the processing to be performed based on the data stored in the storage means 16.

The operation of the server 2 will now be explained with reference to FIG.

In a preliminary step, the server 2 allocates in 20 a storage space dedicated to a stream. This dedicated storage space consists of a zone of addresses in RAM memory, a storage area on a hard disk mass data medium or a combination of both. This storage space is logically a data structure called IDC subsequently, {Intelligent Data Container) schematically in Figure 3.

A stream F of multimedia data is read or received, in step 22, by the server 2 in the IDC. This stream F comes from one of the equipment 3, 4, 5 or another stream, a file or a buffer internal to the server 2. In the latter case, the stream is already in the form of an IDC that can already contain multiple encodings and feed information.

Depending on the processing needs to be made on the stream, the server 2 applies at 26, a filter on the IDC. The filter is an operation on the content intended to modify or act on a particular property of the stream data such as, for example, a media overlay or the sending of an event. Filter results are stored at 28 in the IDC.

In a particular embodiment, the filter is defined as a particular result to obtain: coding according to such characteristics, signal strength, etc. This query is analyzed by a sub-structure of the IDC called computational context which, according to already known or calculated elements, defines the optimal mode of obtaining and applies the most relevant transformation.

The computational context is, for example, in the form of a matrix defining on one side the different possible results and on the other side the necessary starting data. The cell of the matrix also contains the algorithm to be used to obtain a given result from a given type of data. Some changes made by the filter may invalidate some versions or properties of the feed already contained in the IDC.

The filter usually works on a YUV encoding for images and linear for other types of media. Depending on the result to be achieved, the filter begins by analyzing the content of the IDC to determine whether it can use results contained therein, either as an intermediate result or as a final result.

Thus, the filter advantageously performs only computations strictly necessary and not already available in the IDC.

Likewise, it advantageously chooses, by the available data, those which make it possible to optimize the calculations to be made.

This is particularly advantageous when several filters are successively applied to the data stream. Indeed, with the application of each filter, the content of the IDC is enriched with new results, thus increasing the probability that a subsequent filter can use already obtained results.

A relatively common example is the use by multiple filters of the same encoding. Since this has been done for the first time and stored in the IDC, the following filters do not need to encode the stream again.

It should be noted that the initial encoding of the stream is not lost but also stored in the IDC, which makes it possible to reuse it if necessary.

Steps 26 to 30 are thus performed a number of times. Then the stream is sent, in step 32, to one or more particular destinations. This sending consists of transferring the content in a particular format to one or more of the equipment 6, 7, 8 but can also consist of writing the content in another stream, a file, a buffer, etc.

Since the stream can be sent to several destinations in different codings, the described method also includes a feedback loop at the output of the transmission step 32.

It is understandable, therefore, that any stream is transcoded from any source to any compatible destination (full audio transcoding, resampling and requantification matrix, full video transcoding, size change, color space, etc. .)

By way of example, operations related to two streams will be described: a transmission stream for sending voice signals over an IP network, and a voice signal receiving stream.

This is a typical example of VoIP that can be compared to someone talking and listening on the phone.

The reception stream comprises: a source in the form of an RTP object. The content is read from the network under U-coding (coded pulse-code based audio coding (PCM - Puise

Code Modulation)). A U-law encoding is sampled at 8 kHz on 8 bits. The incoming stream consists of 20 ms packets. (The RTP object receives an audio frame of 160 samples every 20 milliseconds); an iDTMF filter for detecting an in-band DTMF signal. The filter needs Goertzel coefficients for each DTMF frequency as well as the power of the audio signal in decibels. Those data are calculated by server 2; a destination in the form of a file stored on the hard disk of the server 2.

The file contains media content encoded in A-law, sampled at 8 kHz with 8-bit quantization.

The corresponding IDC structure evolves as shown in the table below as the flow processing steps progress.

STEP DESCRIPTION IDC CONTENT (after step)

I empty DC allocation

RTP object loads IDC with audio frames in law

U coming from the network

IDTMF filtering looks for signal strength and coefficients

from Goertzel

Storing A-coded data in a file

It should be noted that, in step 2 of the table, the audio frame is decoded in the form of a linear coding from the U-law coding. This linear coding is stored in the IDC structure and is used for calculating the power. signal and Goertzel coefficients. Thus, in step 4 of the table, the A-law coding for the file storage is carried out on the basis of the linear coding performed in step 3.

Linear coding is therefore used for the calcαl of the signal power and Goertzel coefficients as well as for the A-law coding.

The transmission stream comprises, a source in the form of a file coded in law A. This file, stored on the hard disk of the server 2, is different from that of the reception stream. This is, for example, a greeting; a destination in the form of a RTP object that sends the U-coded content on the network.

The corresponding IDC data structure is then constructed according to the following table.

STEP DESCRIPTION IDC CONTENT (after step)

I empty DC allocation

Data coded in law A are read from a law A 20ms δbits 8kHz file

RTP object sends a U-law frame on the network

It should be noted that in step 3, the U-law transcoding is linked to the fact that the original file is coded in law A and not in law U. The file would have been initially coded in U-law, no transcoding would have been achieved in step 3.

Thus, there is described a server and a method for managing the multimedia streams adapted to optimize the calculation quantities made on these streams.

The invention is not limited to the above description in the sense that many variations can be made.

For example, the data structure may contain each encoding of the data or contain only a pointer to a file or other data structure adapted to contain the encoded content.

It is also understood that the invention can be implemented in the form of a program product computer implementing the management method when it is run on a computer. But it can also be implemented in the form of a hardware structure of the FPGA (Field Programmable Gate Arrays) type. This solution is particularly suitable when many flows always having the same structure and the same types of treatment must be managed.

Claims

A multimedia type digital data flow management system comprising: means (10) for acquiring a multimedia data stream; means (14) for processing said data stream adapted to perform at least two different transcodings of said data stream, means (12) for transmitting said processed data stream, means (16) for storing data from said stream , its characteristics and the results of the processing of said data stream, characterized in that the processing means comprise optimization means (18) for, during a given transcoding, select a processing mode for minimizing the resources necessary for said transcoding according to the information contained in the storage means.

2. System according to claim 1, characterized in that the storage means (16) comprise a data structure associated with said multimedia data stream, said data structure comprising at least the data of said stream, its characteristics and the results of the processing operations. .

3. System according to claim 2, characterized in that said data structure comprises for each encoding of said stream, at least fields defining the type of coding, the frame length, the depth of coding and sampling rate.

4. System according to any one of claims 1, 2 or 3, characterized in that the optimization means, before any transcoding, verify the contents of the storage means so as to calculate only the results of said transcoding not available in the storage means, said calculation using the information available in the data structure.

5. System according to claim 4, characterized in that the processing means are adapted to transfer to the storage means all the results of the treatments performed.

6. A method of managing a multimedia type digital data stream comprising the steps of:

- allocation (20) of a data structure (IDC) capable of containing the data and characteristics associated with a frame of the multimedia stream,

receiving (22) in the data structure of a frame of the multimedia stream from the source,

execution (26) of at least one transcoding on said stream,

storage (28) of the result or results of said transcoding in said data structure, characterized in that the execution of each transcoding is adapted to minimize the resources required for said transcoding as a function of the information stored in the data structure.

7. Method according to claim 6, characterized in that the data structure comprises, for each encoding the stream, at least fields defining the type of coding, the frame length, the coding depth and the sampling rate.

8. Computer program product comprising instructions for implementing the method according to claim 6 or 7 when running on a computer.