EP0978199A2 - Mpeg-4 systems compliant architecture - Google Patents
Mpeg-4 systems compliant architectureInfo
- Publication number
- EP0978199A2 EP0978199A2 EP99900029A EP99900029A EP0978199A2 EP 0978199 A2 EP0978199 A2 EP 0978199A2 EP 99900029 A EP99900029 A EP 99900029A EP 99900029 A EP99900029 A EP 99900029A EP 0978199 A2 EP0978199 A2 EP 0978199A2
- Authority
- EP
- European Patent Office
- Prior art keywords
- mpeg
- operating system
- architecture
- buffer
- packet data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
- H04N19/423—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation characterised by memory arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
- H04N19/436—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation using parallelised computational arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
Definitions
- the present invention relates to an MPEG-4 Systems compliant architecture.
- This invention has applications in relation with the MPEG-4 standard, for the streaming of MPEG-4 multimedia data.
- MPEG-4 Systems The Systems part of the MPEG-4 standard defines the architecture to access MPEG-4 multimedia data (video, audio, 2D and 3D graphics).
- MPEG-4 Systems is, more precisely, described in the following document "MPEG-4 Systems", ISO/TEC JTC1/3C29/WG11 N 1901, 21 November 1997.
- Such an architecture is complex, since it provides access to an arbitrary number of streams, such as audio and video streams for instance, originated from a large number of sources (networks, broadcasting, storage,).
- the standard specifies, for the synchronization of audio and video data, a mechanism allowing to introduce, by means of time-stamps, a temporal information into said different streams.
- bitrate for instance, mobile networks, Internet.
- bitrate variations occur (for Internet, depending on the time in the day ; for mobile networks, depending on the distance to the nearest base station).
- MPEG-4 compliant architecture the topology of which has to be handled with respect to a so- called “configuration” that may be fixed but generally evolves dynamically during the session
- an optimal architecture implementation has to ensure an appropriate buffer management.
- the problems to be solved in this context then concern the exchange of information between the different modules of said architecture, in order to avoid drawbacks such as :
- the invention relates to an architecture characterized in that it is subdivided into key modules, a specific task managed by a multi-threading operating system being assigned to each part of said architecture, and the strategy for the buffer management being optimally based on specific synchronization primitives provided by the multi-threading operating system.
- An implementation of MPEG-4 Systems has to be able to instantiate dynamically the differents components of a configuration, whatever this one is very simple (for instance, a movie composed of an audio flow and a video flow) or very complex (for example, for an application of interactive multi-user virtual reality with teleconferencing).
- the technical solution according to the invention that defines a generic architecture for every implementation of MPEG-4 Systems based on microprocessor(s) with a multi-threading operating system, is a reply to this problem.
- the proposed architecture that uses basic primitives present in all multi-threading environments, is indeed modular, flexible, dynamic, extensible, and easy to implement and to manage.
- multi-threading operating systems are optimized for the target processor and are built to be able to efficiently manage the synchronization of numerous tasks.
- - Fig.l depicts an implementation of an MPEG-4 Systems buffer partitioning model (on the so-called Win32 operating system);
- Fig.2 illustrates the control management with semaphores, used to keep the different threads efficient.
- the proposed solution addresses the following issues.
- key modules for an optimal buffer management are defined, which implies a specific generic partitioning of MPEG-4 Systems.
- Each of these modules is then implemented as a task that is managed by a multi-threading operating system (with the use, in order to manage the filling of the buffers in an optimal way, of specific synchronization primitives provided by said multi -threading operating system).
- the strategy for the buffer management is defined.
- Fig.l TransMux, FlexMux, Elementary Stream.
- a decoder buffer (audio, video,...) is also defined (and represented in Fig.l), in order to read Access Units (AUs) and produce frames that can be played (images, sounds), but this task is not part of the one carried out by the demultiplexer.
- FMX FlexMux thread
- DEC Decoder thread
- AUs Access Units
- FIFO First-In, First-Out
- synchronization primitives which is done by means of semaphores (or : counters) that keep the different threads efficient.
- Each of these semaphores controls each respective buffer (FM-PDU, AL-PDU, AU) in the demultiplexer.
- For each decoder there is also one semaphore controlling the decoder buffer, and one more for signalling to the TransMux to have to stop if the AU buffer becomes too large.
- the first operation is done by the TransMux TMX that reads data on the local disk DK or on the network NW ;
- the regulation of the TransMux TMX is made by the AU feedback semaphore controlling that the AU buffer is not in a critical state.
- the principle of this buffer management strategy is to keep the buffer sizes as small as possible. When the number of AL-PDUs is sufficient to reconstruct an AU, it has to be done immediately. Thus there is always data present for decoding.
- TransMux Two types of TransMuxes can be differentiated : slow ones (network access), and fast ones (disk access).
- the disk accesses are quick and stable in terms of latency, and, with such types of accesses, it is then not needed to store many data in the different buffers, since no loss would occur and the maximal critical value could be rather low.
- network accesses can introduce some latency (Internet, for example), so that it is important to store more data in the buffers in order not to let the decoder starve. In the case of more than one decoder, it must however be indicated that the
- TransMux has to be stopped only when all the AU buffers have reached the critical maximum value in order not to have starving decoding threads (it is therefore assumed that the multiplexing strategy has been adapted for that purpose on the sending side).
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computing Systems (AREA)
- Theoretical Computer Science (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Communication Control (AREA)
Abstract
The invention relates to an MPEG-4 Systems compliant architecture subdivided in key modules having a specific task managed by a multi-threading operating system. The MPEG-4 demultiplexer is partitioned into three levels of process. Each process is independent of each other, but synchronization primitives are however provided by the operating system, in the form of semaphores controlling the buffers so that their sizes are kept as small as possible. Application: all kinds of industrial MPEG-4 equipment (desk-tops, set-tops boxes).
Description
MPEG-4 systems compliant architecture.
FIELD OF THE INVENTION
The present invention relates to an MPEG-4 Systems compliant architecture. This invention has applications in relation with the MPEG-4 standard, for the streaming of MPEG-4 multimedia data.
BACKGROUND OF THE INVENTION
The Systems part of the MPEG-4 standard defines the architecture to access MPEG-4 multimedia data (video, audio, 2D and 3D graphics). MPEG-4 Systems is, more precisely, described in the following document "MPEG-4 Systems", ISO/TEC JTC1/3C29/WG11 N 1901, 21 November 1997. Such an architecture is complex, since it provides access to an arbitrary number of streams, such as audio and video streams for instance, originated from a large number of sources (networks, broadcasting, storage,...). Particularly, the standard specifies, for the synchronization of audio and video data, a mechanism allowing to introduce, by means of time-stamps, a temporal information into said different streams.
Most of the concerned potential products will use network technology with any type of stability in terms of bitrate (for instance, mobile networks, Internet). On these networks, bitrate variations occur (for Internet, depending on the time in the day ; for mobile networks, depending on the distance to the nearest base station). As an MPEG-4 compliant architecture (the topology of which has to be handled with respect to a so- called "configuration" that may be fixed but generally evolves dynamically during the session) has to deal with a number of memory buffers that will be specifically organized, an optimal architecture implementation has to ensure an appropriate buffer management. The problems to be solved in this context then concern the exchange of information between the different modules of said architecture, in order to avoid drawbacks such as :
- a break of the "pipe-line" effect, for instance losses in the stream that could lead to a desynchronization (a break in the audio stream or a desynchronization of the audio signals with respect to the video stream are very perceptible by the end user and therefore unacceptable) ;
- a processing time that would be too long for interactive applications such as video conference ;
- a waste of the memory resources if over-sized buffers are used.
A compromise between the solutions to these problems is difficult to obtain, since the only way to reduce the effect of the network is to increase the size of the buffers, which however increases the delays and processing time, and so on. Moreover, it seems unrealistic to predict the size of the buffers at an early stage of the implementation of the architecture, as it was the case for previous applications (for example with the MPEG-2 standard in the field of TV broadcast).
SUMMARY OF THE INVENTION
It is therefore an object of the invention to obviate these drawbacks. To this end, the invention relates to an architecture characterized in that it is subdivided into key modules, a specific task managed by a multi-threading operating system being assigned to each part of said architecture, and the strategy for the buffer management being optimally based on specific synchronization primitives provided by the multi-threading operating system.
An implementation of MPEG-4 Systems has to be able to instantiate dynamically the differents components of a configuration, whatever this one is very simple (for instance, a movie composed of an audio flow and a video flow) or very complex (for example, for an application of interactive multi-user virtual reality with teleconferencing). The technical solution according to the invention, that defines a generic architecture for every implementation of MPEG-4 Systems based on microprocessor(s) with a multi-threading operating system, is a reply to this problem. The proposed architecture, that uses basic primitives present in all multi-threading environments, is indeed modular, flexible, dynamic, extensible, and easy to implement and to manage. Moreover, multi-threading operating systems are optimized for the target processor and are built to be able to efficiently manage the synchronization of numerous tasks.
BRIEF DESCRIPTION OF THE DRAWINGS
These and other aspects of the invention will be now explained with reference to the following description, given in connection with the accompanying drawings, in which :
- Fig.l depicts an implementation of an MPEG-4 Systems buffer partitioning model (on the so-called Win32 operating system);
- Fig.2 illustrates the control management with semaphores, used to keep the different threads efficient.
DESCRIPTION OF THE INVENTION
The proposed solution addresses the following issues. First, key modules for an optimal buffer management are defined, which implies a specific generic partitioning of MPEG-4 Systems. Each of these modules is then implemented as a task that is managed by a multi-threading operating system (with the use, in order to manage the filling of the buffers in an optimal way, of specific synchronization primitives provided by said multi -threading operating system). Finally, the strategy for the buffer management is defined.
For the definition of the MPEG-4 Systems key modules, it is proposed, in order to assign a specific task to each part of the MPEG-4 demultiplexer, to partition said demultiplexer into three different levels, shown in Fig.l : TransMux, FlexMux, Elementary Stream. A decoder buffer (audio, video,...) is also defined (and represented in Fig.l), in order to read Access Units (AUs) and produce frames that can be played (images, sounds), but this task is not part of the one carried out by the demultiplexer.
The scheme of Fig.l has been implemented for instance on the Win32 operating system (= Windows 95), which provides multi-threading functionality. One thread is assigned to each process :
- a TransMux thread (TMX), for reading data on the disk DK or on the network NW and storing a number N of FlexMux Packet Data Units (= FM-PDUs) ;
- a FlexMux thread (FMX), for reading these FM-PDUs in an FM-PDU buffer, storing N Access unit Layer Packet Data Units (= AL-PDUs) and reconstructing N Access
Units (= AUs), the FlexMux FMX and the Elementary Stream ELS being, in that implementation, organized in one single thread ;
- a Decoder thread (DEC), for building N frames with the Access Units (AUs) for the decoders. Each of these threads is independent from the others. As shown in Fig.l, the buffers are implemented as queues, i.e. as FIFO (First-In, First-Out) memories, without any fixed length (it is consequently recommended to control the state of these buffers in order not to have huge buffers nor starving tasks).
However the tasks have to watch at each other using so-called synchronization primitives, which is done by means of semaphores (or : counters) that keep the different threads efficient. Each of these semaphores, provided by the operating system, controls each respective buffer (FM-PDU, AL-PDU, AU) in the demultiplexer. For each decoder, there is also one semaphore controlling the decoder buffer, and one more for signalling to the TransMux to have to stop if the AU buffer becomes too large.
These actions are depicted in Fig.2, that illustrates the control management with the semaphores. The steps (1) and (2) correspond to the following tests : (1) means "if N greater than 0, work", and (2) means "if N greater than Max, stop". Based on the synchronization facilities of the operating system, this control function is carried out according to the following buffer management strategy :
(a) at run-time, the first operation is done by the TransMux TMX that reads data on the local disk DK or on the network NW ;
(b) the FlexMux FMX watches at the TransMux TMX ; (c) the Elementary Stream ELS watches at the FlexMux FMX;
(d) the real-time decoding process starts when AUs are present in the AU buffer ;
(e) the regulation of the TransMux TMX is made by the AU feedback semaphore controlling that the AU buffer is not in a critical state. The principle of this buffer management strategy is to keep the buffer sizes as small as possible. When the number of AL-PDUs is sufficient to reconstruct an AU, it has to be done immediately. Thus there is always data present for decoding.
Moreover, an important point is to consider the type of TransMux that is used. Two types of TransMuxes can be differentiated : slow ones (network access), and fast ones (disk access). The disk accesses are quick and stable in terms of latency, and, with such types of accesses, it is then not needed to store many data in the different buffers, since no loss would occur and the maximal critical value could be rather low. On the contrary, network accesses can introduce some latency (Internet, for example), so that it is important to store more data in the buffers in order not to let the decoder starve. In the case of more than one decoder, it must however be indicated that the
TransMux has to be stopped only when all the AU buffers have reached the critical maximum value in order not to have starving decoding threads (it is therefore assumed that the multiplexing strategy has been adapted for that purpose on the sending side).
Claims
1. An MPEG-4 Systems compliant architecture, characterized in that it is subdivided into key modules, a specific task managed by a multi-threading operating system being assigned to each part of said architecture, and the strategy for the buffer management being optimally based on specific synchronization primitives provided by the multi-threading operating system.
2. An architecture according to claim 1, wherein the MPEG-4 demultiplexer is partitioned into three different levels of process, the first one, TransMux, being provided for reading data on a local disk or a network and storing them by packet data units, the second one, FlexMux, being provided for reading said packet data units and storing access unit layer packet data units, and the third one for reconstructing access units.
3. An architecture according to claim 2, wherein each process is independent from each other, synchronisation primitives being however provided by said operating system in order to watch at each other, by means of semaphores controlling each buffer so that the buffer sizes are kept as small as possible.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| EP99900029A EP0978199A2 (en) | 1998-01-27 | 1999-01-14 | Mpeg-4 systems compliant architecture |
Applications Claiming Priority (4)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| EP98400160 | 1998-01-27 | ||
| EP98400160 | 1998-01-27 | ||
| PCT/IB1999/000038 WO1999038329A2 (en) | 1998-01-27 | 1999-01-14 | Mpeg-4 systems compliant architecture |
| EP99900029A EP0978199A2 (en) | 1998-01-27 | 1999-01-14 | Mpeg-4 systems compliant architecture |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| EP0978199A2 true EP0978199A2 (en) | 2000-02-09 |
Family
ID=8235248
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| EP99900029A Withdrawn EP0978199A2 (en) | 1998-01-27 | 1999-01-14 | Mpeg-4 systems compliant architecture |
Country Status (5)
| Country | Link |
|---|---|
| EP (1) | EP0978199A2 (en) |
| JP (1) | JP2001518272A (en) |
| KR (1) | KR100639894B1 (en) |
| CN (1) | CN1150768C (en) |
| WO (1) | WO1999038329A2 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6606329B1 (en) | 1998-07-17 | 2003-08-12 | Koninklijke Philips Electronics N.V. | Device for demultiplexing coded data |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5668599A (en) * | 1996-03-19 | 1997-09-16 | International Business Machines Corporation | Memory management for an MPEG2 compliant decoder |
| GB9704027D0 (en) * | 1997-02-26 | 1997-04-16 | Discovision Ass | Memory manager for mpeg decoder |
| EP0909091A1 (en) * | 1997-10-07 | 1999-04-14 | CANAL+ Société Anonyme | Memory manager |
-
1999
- 1999-01-14 EP EP99900029A patent/EP0978199A2/en not_active Withdrawn
- 1999-01-14 JP JP53808799A patent/JP2001518272A/en not_active Abandoned
- 1999-01-14 WO PCT/IB1999/000038 patent/WO1999038329A2/en not_active Application Discontinuation
- 1999-01-14 KR KR1019997008809A patent/KR100639894B1/en not_active IP Right Cessation
- 1999-01-14 CN CNB998000663A patent/CN1150768C/en not_active Expired - Fee Related
Non-Patent Citations (1)
| Title |
|---|
| See references of WO9938329A2 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6606329B1 (en) | 1998-07-17 | 2003-08-12 | Koninklijke Philips Electronics N.V. | Device for demultiplexing coded data |
Also Published As
| Publication number | Publication date |
|---|---|
| KR100639894B1 (en) | 2006-10-31 |
| WO1999038329A2 (en) | 1999-07-29 |
| CN1150768C (en) | 2004-05-19 |
| CN1256047A (en) | 2000-06-07 |
| WO1999038329A3 (en) | 1999-09-30 |
| JP2001518272A (en) | 2001-10-09 |
| KR20010005738A (en) | 2001-01-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US6185602B1 (en) | Multi-user interaction of multimedia communication | |
| US6584125B1 (en) | Coding/decoding apparatus, coding/decoding system and multiplexed bit stream | |
| US9264472B2 (en) | Audio-video data switching and viewing system | |
| EP1775895B1 (en) | Apparatus and method for receiving multiple streams in a mobile broadcast system | |
| WO1994018776A2 (en) | Multimedia distribution system | |
| Kalva et al. | Implementing multiplexing, streaming, and server interaction for MPEG-4 | |
| JP2003534741A (en) | Communication system with MPEG-4 remote access terminal | |
| JP4391231B2 (en) | Broadcasting multimedia signals to multiple terminals | |
| EP0978199A2 (en) | Mpeg-4 systems compliant architecture | |
| Kalva et al. | Delivering object-based audio visual services | |
| Li et al. | MPEG-2 coded-and uncoded-stream synchronization control for real-time multimedia transmission and presentation over B-ISDN | |
| Herpel et al. | MPEG-4 systems: elementary stream management and delivery | |
| Boutaba et al. | Distributed video production: Tasks, architecture and qos provisioning | |
| Li et al. | An MPEG-4 based interactive multimedia system | |
| Daami et al. | Client based synchronization control of coded data streams | |
| Katto et al. | System architecture for synthetic/natural hybrid coding and some experiments | |
| Huang et al. | Design and implementation of an efficient MPEG-4 interactive terminal on embedded devices | |
| PROVISIONING¹ et al. | DISTRIBUTED VIDEO PRODUCTION: TASKS, ARCHITECTURE AND | |
| Casalino et al. | MPEG-4 systems, concepts and implementation | |
| Oliveira et al. | An MPEG-2 distributed studio architecture based on ATM | |
| JP2001016567A (en) | Multimedia data reproducing device/transmitter | |
| KR100437168B1 (en) | communication method used in MPEG-4 system | |
| El Zarki et al. | An interactive object based multimedia system for IP networks | |
| Kalva | Object-Based Audio-Visual Services | |
| KR20030004061A (en) | Multilayer multiplexing for generating an MPEG2 transport stream from elementary MPEG2 and MPEG4 streams |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
| 17P | Request for examination filed |
Effective date: 19991027 |
|
| AK | Designated contracting states |
Kind code of ref document: A2 Designated state(s): DE ES FR GB IT SE |
|
| 17Q | First examination report despatched |
Effective date: 20061115 |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN |
|
| 18D | Application deemed to be withdrawn |
Effective date: 20070526 |