RU2351002C2 - Demultiplexer application program interface - Google Patents

Abstract

FIELD: physics; computer engineering.

SUBSTANCE: present invention pertains to processing multimedia data in a computing environment. On the data carrier, there are commands executed by the computer, providing for an interface for communication with the demultiplexer component, dividing the multiplexed multimedia stream into elementary streams. The interface contains commands for initiating the demultiplexer component and its configuration. There is dynamic installation of a descriptor of active presentation with regard to the demultiplexer component. The new input multiplexed stream is transmitted in form of input samples into the demultiplexer component. The above mentioned active presentation is extracted, in accordance with the active presentation descriptor of at least, one elementary stream, and then transmitted in form of output samples. The samples currently in the input and output queues are cleared.

EFFECT: enabling a user to consistently use data of a multiplexed stream, for example DV, for generating elementary stream data, including audio and video (compressed and uncompressed), and provision for use of demultiplexers as independent components.

21 cl, 7 dwg

Description

Field of Invention

The invention relates, in General, to electronic data processing, and more particularly to processing multimedia data in a computing environment.

BACKGROUND OF THE INVENTION

The presentation of multimedia information based on digital data processing, a combination of video and audio in digital format for viewing on a digital device is widely distributed and rapidly developing. Almost every new personal computer manufactured at present contains some form of multimedia. The sales of digital products such as cameras, VCRs, telephones and televisions are constantly growing. Multimedia is becoming more and more widespread in the field of the Internet in the same way as the constant and rapid development of the Internet continues. Along with development, user requirements for the effectiveness of such computer equipment have increased. Increased user requirements extend not only to the hardware capabilities, but also to the processing capabilities of the data itself.

To meet the growing demands for multimedia applications, a method known as streaming data has been developed. Stream data transfer provides such a data transfer in which they can be processed as a constant and continuous stream. This provides the advantage, especially for large multimedia files, that the data can be displayed or listened to until the end of the transfer of the whole file.

Initially, the streaming data structure consisted of a sequence of data processing modules (for example, capture filters, transform filters, and information presentation (visualization) filters) with very little intellectual capabilities of the sequence control program. Data processing modules, also called filters, make decisions on how to get a connection, which data formats to use, and how to manage each other. When connecting filters into a sequence, the protocols determine a predefined fixed sequence of the data stream and control the negotiation of the connection. The usual sequence of approvals is designed to coordinate the following elements: interface, storage medium, data format, distribution programs, and master oscillator. A data processing sequence provides a complete solution within a computer system.

With the increasing complexity of streaming data processing, the industry has recognized the need to optimize processing sequences, which are data processing with constraints imposed by interactivity, such as video and audio processing sequences. One solution is Microsoft® Corporation’s DirectShow®, which enables playback of media streams from local files or Internet servers, capture media streams from devices, and convert media streams. DirectShow® provides the ability to play video and audio content of file types such as: Windows Media Audio, Windows Media Video, MPEG, Apple®QuickTime®, Audio-Video Interleaved (AVI) and WAV (Windows Wave). DirectShow® includes a plug-in filter system. Filters are objects that support DirectShow® interfaces that can handle data streams by reading, copying, modifying, and writing data to a file. The main types of filters include an input filter, which takes data from some source, for example, from a disk file, satellite dish, from an Internet server or VCR tape recorder, and passes them to the input of the filter circuit, which is a filter connection. Filters in the filter circuit contain a conversion filter that performs data format conversion, a synchronization filter and an input filter that receives data and transmits them; and a visualization filter that forms a data image, for example, forms a data image on a display device. Data can also be visualized at any location that accepts multimedia. Other types of filters included with DirectShow® include target filters that add effects without changing the data type, and parser filters that “understand” the source data format and can read the correct bytes, create temporary labels, and perform searches.

During operation, all data passes from filter to filter together with a lot of control information. Each filter contains objects called “ pins ” that are used to connect to other filters. When the filters are connected using the leads, a filter circuit is formed. It should be noted that there is a difference between a "filtering scheme", which is a concept designating a group of connected filters, and a "Filtering scheme", which is an object that a user creates in DirectShow®, managing a group of connected filters, a "filtering scheme". The "Filter Schema" is more correctly called the administrator of the filter circuit. To control data flow and connections in the filtering scheme, DirectShow® contains a control program (administrator) for the filtering scheme. The administrator of the filtering scheme helps ensure that the filters are connected in the appropriate order. However, data and most of the control do not pass through the filtering circuit manager. Filters should be linked appropriately. For example, the administrator of the filtering scheme must find the configuration (structure) of the visualization, determine the types of filters available, associate the filters in the appropriate order for the given data type and provide the appropriate visualization filter.

Although filters made it possible to reuse programs, using filters also created some unforeseen problems. One of the problems created by filters is the large number of application programming interfaces (APIs) for the created filters. Each filter essentially has a separate API. As a result, this filter must allow the ability to interface with the API for each filter to which it can be attached. Also, the use of filters creates a problem when the specified filter is disabled. When you turn off a given filter in a graph, any filter that paired with a disabled filter requires another appropriate interface. Basically, it is difficult to program the filter to correctly handle the loss of pairing, since the state of the filter in case of loss of pairing may not be known. Therefore, the loss of pairing tends to lead to an unpredictable mode in the filters and, ultimately, to incorrect execution of programs. Additionally, the overall management in DirectShow® is distributed between two units. The interface between the filters controls the flow of data, while the filter manager controls the implementation and deletion of filters. This distribution of control makes software design cumbersome, as there are inevitably some control functions that “cross” the boundary between blocks. Another problem with DirectShow® is that the filters take responsibility for the coordination of the multimedia format and for the functionality of organizing data buffering. Filters interact with other filters to accomplish this task. Dependence on filters leads to susceptibility of applications built on DirectShow® to errors and shortcomings that could be programmed in the filter. Therefore, a poorly written filter can easily reduce the efficiency of the filter circuit and the application associated with the filter circuit.

There is a need to improve the DirectShow® architecture. More specifically, there is a need for improved multimedia processing control.

SUMMARY OF THE INVENTION

The invention includes a set of interfaces, data structures, and events for representing a demultiplexer (distribution device) of multimedia data. The data structure uses a number of fields, each of which contains a command element. In one embodiment, at least seven commands are generated for the demultiplexer to function correctly, including the Initialize (Set), SetPresentationDescriptor (set the presentation descriptor), GetPresentationDescriptor (retrieve the presentation descriptor), GetPendingPresentationDescriptor (retrieve the presentation descriptor pending processing), ProcessInput (process ), ProcessOutput (process output) and Flush (reset). The interfaces are collectively known as the Demux Multiplexer Application Programming Interface (Demux API). The Demux API enables consumers to uniformly use multiplexed stream data, such as digital (DV) video data , to generate elementary stream data, such as audio and video (compressed or uncompressed).

The Initialize method is used to initialize and configure the demultiplexer object and has parameters that are used to configure the demultiplexer. The parameters include a multiplexed stream descriptor object that describes the multiplexed stream, the selected medium type for the multiplexed stream descriptor, an array of the basic types of elementary streams that the user is interested in extracting as output from the demultiplexer, and the number of basic types in the array of basic types.

The SetPresentationDescriptor method is used to dynamically set the descriptor of the active representation of the demultiplexer object. The SetPresentationDescriptor method contains a pointer to a presentation descriptor object. The ProcessInput method is used to provide a new input multiplexed stream for the demultiplexer object and contains a pointer to the sample object. The ProcessInput method has a return value that has a new view flag. If the flag for the new view is set to TRUE, then in the multiplexed selection, the view has changed based on the view descriptor. The user must call the GetPendingPresentationDescriptor method to retrieve the next presentation that is waiting to be processed, select the required streams, and call the SetPresentationDescriptor method to enable processing of samples from the input demultiplexer queue.

The ProcessOutput method is used to extract at least one elementary stream from the active view. The ProcessOutput method contains a thread identifier and a pointer to a pointer to a selection object. The ProcessOutput method additionally contains an output return value. The output return value contains the end-of-stream error code and the following data missing error code.

The Flush method is used to discard the input and output samples currently queued. The Flush method does not require parameters.

The GetPresentationDescriptor method is used to retrieve the clone (absolute copy) of the descriptor of the current active representation of the demultiplexer object. The GetPresentationDescriptor method contains a presentation descriptor.

The GetPendingPresentationDescriptor method is used to retrieve the next pending presentation. The GetPendingPresentationDescriptor method contains a pending presentation descriptor.

Additional features and advantages of the invention will be apparent from the following detailed description of illustrative embodiments, taken in conjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

Although the appended claims spell out the features of the present invention, the invention, its objectives, and advantages may become more apparent from the following detailed description, taken in conjunction with the accompanying drawings.

FIG. 1 is a block diagram generally illustrating a possible computer system in which the present invention is implemented.

FIG. 2 is a block diagram essentially illustrating a possible computer environment in which the present invention operates.

FIG. 3 is a block diagram illustrating a demultiplexer according to the disclosure of the present invention.

FIG. 4 is a transition diagram of a demultiplexer provided by the present invention.

FIG. 5 is a data structure diagram illustrating a data structure model for constructing calls in accordance with the present invention.

FIG. 6 is a simplified data structure diagram illustrating a structure of a possible message according to the present invention.

FIG. 7 is a flowchart illustrating steps taken to download and operate a demultiplexer according to the methods of the present invention when the application does not agree on a multimedia receiver.

DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT

The invention provides a set of interfaces, data structures, and events for representing a multimedia data demultiplexer, collectively referred to as a demultiplexer API. The API provides the consumer the ability to uniformly use multiplexed stream data, such as DV, MPEG2, ASF, etc., to generate elementary stream data, such as audio and video (compressed or uncompressed). The demultiplexer API supports the dynamic generation of new stream descriptors based on the initialization data of the multiplexed stream and supports the dynamic generation of new stream descriptors based on samples of the multiplexed stream. Additionally, the demultiplexer API supports external initialization through multiplexed stream metadata and internal initialization through multiplexed stream samples and supports the generation of demultiplexed samples that may occupy multiple buffers. In the context of the architecture of the Basic Concepts of Multimedia, the Media Foundation Microsoft® Corporation API of the demultiplexer is primarily controlled by a multimedia processor to which data is output, as described below. The demultiplexer API is designed to well define the use of the demultiplexer for any multimedia architecture.

According to the drawings, in which like reference numerals refer to like elements, the invention is implemented in an appropriate computing environment. Although not necessary, the invention will be described in the general context of computer-executable instructions, for example, program modules, executed by a personal computer. Essentially, program modules contain routines, programs, objects, components, data structures, etc. that perform specific tasks or implement certain types of abstract data. In addition, it will be apparent to those skilled in the art that the invention can be practiced with other computer system configurations, including portable devices, multiprocessor systems, programmable, or microprocessor-based, consumer electronics, network PCs, minicomputers, universal computers etc. The invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communication system. In a distributed computing environment, program modules may be located in local and remote memory storage devices.

FIG. 1 illustrates a possible embodiment of a corresponding computing system environment 100 in which the invention may be implemented. The computing system environment 100 is only one possible variant of the corresponding computing environment and is not intended to impose any limitation on the scope or functionality of the invention. Also, computing environment 100 should not be interpreted as requiring or dependent on any component illustrated by possible environment 100, or a combination thereof.

The invention may function in conjunction with many other universal or specialized environments or computer system configurations. Possible variants of known computing systems, environments and / or configurations that may be suitable for use with the invention include: personal computers, server computers, portable or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, computer consoles to TV, programmable consumer electronics, network PCs, minicomputers, universal computers, distributed computing environments containing any of the above omyanutyh systems or devices, etc.

The invention may be described in the general context of computer-executable instructions, for example, program modules, executed by a personal computer. Essentially, program modules contain routines, programs, objects, components, data structures, etc. that perform specific tasks or implement certain types of abstract data. The invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communication system. In a distributed computing environment, program modules may be located in a storage medium of a local and / or remote computer, including storage devices.

According to FIG. 1, a possible system for implementing the invention comprises a universal computing device in the form of a computer 110. Components of a computer 110 may include a processor 120, a system memory 130, and a system bus 121 connecting various components of the system, including system memory with a processor 120, the components being not limited by them. The system bus 121 may be any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, and a local bus using any of many bus architectures. Optionally, such architectures include an industry standard (ISA) architecture bus, microchannel architecture bus (MCA), an extended standard architecture bus for industrial applications (EISA), a local VESA bus, and 32-bit system bus with the possibility of expanding to 64 bits, the interaction through which occurs without the participation of the central processor (PCI), also known as the Mezzanine bus, etc.

Computer 110 typically comprises several storage media readable by a computer. The storage medium readable by a computer can be any available storage medium that can be accessed by computer 110, and includes a volatile and non-volatile, removable and non-removable storage medium. Alternatively, a computer readable medium may include a computer storage medium, a communication medium, etc. The storage medium of a computer includes a volatile and non-volatile, removable and non-removable medium implemented by any method or technology for storing information, such as instructions read by a computer, data structures, program modules or other data. The computer’s storage medium includes random access memory (RAM), read-only memory (ROM), EEPROM (EPROM), flash memory or other memory technology, a CD-ROM, universal digital DVDs (DCD), or other optical disk drive, magnetic tapes, magnetic tape, magnetic disk drive or other magnetic storage devices, etc. or any other medium that can be used to store the necessary information and which can be accessed by computer 110. A communication medium typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal, such as a carrier or other information transfer mechanism , and includes any means of transmitting information. The term "modulated data signal" means a signal that has one or more of a set of its characteristics, modified so as to encode information in the signal. Alternatively, the communication means includes wired means, such as a wired network or a direct cable connection, and wireless means, such as acoustic, radio, infrared and other wireless means, etc. Combinations of any of the means mentioned above should also be included in the context of computer readable media.

System memory 130 includes a computer storage medium in the form of volatile and / or non-volatile memory, such as ROM 131 and RAM 132. A BIOS BIOS I / O base system 133, containing basic procedures that facilitate the transfer of information between elements within computer 110, such as used at startup, it is usually stored in ROM 131. RAM 132 typically contains data and / or program modules that can be accessed immediately and / or with which processor 120 is currently operating. As a possible option, fi . 1 shows an operating system 134, application programs 135, other program modules 136, and program data 137.

The computer 110 may also include other removable / non-removable, volatile / non-volatile options for the storage medium of the computer. Exclusively as a possible embodiment of FIG. 1 shows a hard disk drive 141 that reads from and writes to a non-removable, non-volatile magnetic storage medium, a magnetic disk drive 151 that reads from and writes to a removable non-volatile magnetic disk 152, and an optical drive 155 reads from a removable non-volatile optical disk 156, for example, a CD-ROM or other optical information medium, and writes to it. Other removable / non-removable, volatile / non-volatile computer storage media options that can be used in a possible operating environment include magnetic tape, flash memory cards, universal digital disks, tape for digital video recorder, solid state RAM, solid state ROM and etc. A hard disk drive 141 is typically connected to the system bus 121 via a non-removable memory interface, such as interface 140, and a magnetic disk drive 151 and an optical disk drive 155 are usually connected to the system bus 121 via a removable memory interface, such as interface 150.

Disk drives and their corresponding variants of the computer storage medium described above and shown in FIG. 1 provide storage for computer readable instructions, data structures, program modules, and other data for computer 110. For example, in FIG. 1 shows a hard disk drive 141 in which an operating system 144, application programs 145, other program modules 146, and program data 147 are stored. It should be noted that these components may be the same as, or may be different from, the operating system 134, application programs 135, other program modules 136, and program data 137. Here, the operating system 144, application programs 145, other program modules 146, and program data 147 are assigned different reference numbers to illustrate that, at a minimum, they are different copies. The user can enter commands and information into the computer 110 through input devices such as a keyboard 162 and a pointing device 161, usually defined as a mouse, ball pointer or touch pad, microphone 163, tablet or electronic digitizer 164. Other input devices (not shown) may include a joystick, game pad, satellite dish, etc. Often these and other input devices are connected to the processor 120 via a user interface 160 connected to the system bus, but they can be connected to the processor via another interface and other bus structures such as a parallel port, a game port, or a universal USB serial bus. To the system bus 121 also through an interface, such as a video interface 190, a monitor 191 or other type of display device is connected. It should be noted that the monitor and / or the touch screen panel can be physically connected to the housing in which the computing device 110 is integrated, for example, a tablet-type personal computer. Additionally, computers, such as computing device 110, may also include other peripheral output devices, such as speakers 197 and a printer 196, which can be connected via an interface 194 of peripheral output devices, etc.

Computer 110 may operate in a networked environment using logical connections to one or more remote computers, such as remote computer 180. Remote computer 180 may be a personal computer, server, router, network PC, peer-to-peer device, or other host on a shared network, and typically contains many or all of the elements described above with respect to computer 110, although FIG. 1 only depicts a storage device 181. These logical connections depicted in FIG. 1 include a local area network (LAN) 171 and a global network (HS) 173, but may also include other communication networks. Such options for the network environment are often used in offices, corporate computing networks, intranets (corporate local networks of increased reliability with limited access) and the Internet. For example, computing system 110 may comprise a source machine from which data is transferred, and remote computer 180 may comprise a destination machine. However, it should be noted that the connection of the source machine and the destination machine via a network or any other means is not required; instead, data can be transferred through a storage medium that can be recorded by the source platform and read by the destination platform or platforms.

When used in a LAN network environment, the computer 110 is connected to the LAN 171 via a network interface or adapter 170. When used in a LAN network environment, the computer 110 typically includes a modem 172 or other means for establishing communication via the GS 173, for example, the Internet. The modem 172, which may be internal or external, may be connected to the system bus 121 via an input user interface 160 or other appropriate mechanism. In an environment with a network structure, program modules described with respect to computer 110, or parts thereof, may be stored in a remote storage device. As a possible embodiment of FIG. 1 depicts remote application programs 185 as stored on a storage device 181. It is clear that the network connections shown are possible options, and other means of establishing a communication link between computers can be used.

In the following description, the invention will be described with respect to actions and symbolic representations of operations that are performed by one or more computers, unless otherwise indicated. It will also be understood that such operations and actions, which are sometimes referred to as computer-executable, include the manipulation by an electric processor of a computer of electrical signals representing data in a structural form. Such manipulation converts data or stores it at locations in the computer’s memory system, which reconfigures or otherwise modifies the functioning of the computer in a manner known to those skilled in the art. The data structures where the data is stored are physical memory locations having certain properties defined by the data format. However, although the invention is described in the context defined above, it is not intended to limit the invention to this context, as it is obvious to those skilled in the art that the various operations and actions described below can also be implemented in hardware.

According to FIG. 2, the demultiplexer provided by the present invention can function in the architecture of the Basic Concepts of the Multimedia Media Foundation, which is an implementation of one of the Microsoft multimedia architectures. Although FIG. 2 shows a demultiplexer in the Media Foundation architecture, it is obvious that the demultiplexer API of the invention can be used in other multimedia architectures. Before describing the demultiplexer directly, you need to describe the Media Foundation. Media Foundation is an architecture represented by components. As depicted, the Media Foundation contains core level 211 components that are responsible for some core functionality module in Media Foundation, and Management level 201 components that are responsible for more general tasks using core core components.

Kernel level 211 components include multimedia sources 210 and stream sources 214 that provide multimedia data transmission through a unified, well-defined interface. Media sources 210 describe the presentation, including the streams that should be available. There are many implementations of multimedia sources for providing multimedia data from various types of multimedia files or devices. The core level 211 further comprises conversion modules depicted by block 208, which perform some kind of multimedia data conversion operation through a unified well-defined interface. Possible conversion options are codecs, video resizing components, audio resampling modules, statistical processors, color resampling modules, etc. The demultiplexer provided by the present invention, which takes interleaved (interleaved) multimedia data as input and splits the data into multimedia streams used individually, is a transformation in the architecture of FIG. 2. Block 208 further includes multiplexers that receive individual multimedia data streams and combine them into interleaved multimedia data. Multiplexers share a unified, well-defined interface, and there are many implementations for multiplexing into different types of multimedia data. The core level 211 further includes stream receivers 212 (output) and multimedia receivers 230 (output). Multimedia receivers 230 receive multimedia input as a unified, well-defined interface. There are many implementations of multimedia receivers for performing various functions with multimedia data. For example, recording multimedia data in a given file type or displaying multimedia data on a monitor using a video card.

Management level 201 components use kernel level 211 components to perform higher level tasks in a simpler way. Typically, for a given task, a control-level component uses many different kernel-level components. For example, playing a multimedia file uses a multimedia source to read a file from disk and analyze data, one or more transducers to recover compressed multimedia data, and one or more multimedia receivers to display multimedia data. The control layer 201 comprises a special multimedia processor 260 that interacts with an application 202 for receiving and transmitting multimedia data streams, a multimedia session 240, a multimedia processor 220, and a topology downloader 250 depicted inside the multimedia session 240. Topology loader 250 is a control layer component responsible for describing the flow of data between kernel level components. The components of the control layer can be configured to avoid accessing more of the core elementary components used by the control layer. Data passes through the system, starting from the multimedia source 210, through the multimedia session 240 to the multimedia processor 220 and output to the multimedia receiver 230. The multimedia processor 220 launches a pipeline of multimedia sources and other components in a topology. A media session 240 provides guidance when an event in the topology occurs, and the topology loader 250 confirms that the prescribed events in the topology are occurring. The media session 240 also configures the media processor 220 and uses the samples returned by the media processor 210. He does this in the context of the special multimedia processor 260 and transfers samples from the multimedia processor 220 to the multimedia receivers, which he coordinated with the calling program of the special multimedia processor 260 (for example, application 202). The components in the topology include the components of the multimedia source 210 and the components of the multimedia receiver 230, as well as other elements. A system of 200 basic multimedia concepts provides interfaces and a distribution scheme for connecting multimedia streaming data objects. The system provides the user with the ability to specify the connections between unified or specialized sources, transformations, and receiver objects through symbolic abstraction using the concept of topology.

According to FIG. 3, a brief description of the demultiplexer 300 is provided below. In the following description, references to commands will be used. These commands are described below. The demultiplexer API separates the data format from the data source. The demultiplexer API accepts the multiplexed data as a data buffer in RAM, and performs the demultiplexing operation. This gives the desired result, consisting in the fact that many different data sources can use one implementation of the demultiplexer to perform the specified operation. For example, DV data can come directly from a DV camera, but can also be stored as a file on your hard drive. In this case, there are two pieces of code for generating multiplexed DV data (for example, one that matches the camera, the other that matches the file system), but the same demultiplexer implementation can be used. The demultiplexer 300 supports the IMFDemultiplexer interface and is responsible for splitting the multiplexed stream into parts of elementary streams. Demultiplexer 300 operates in synchronous mode (similar to DMO) and can process dynamic changes in available elementary streams caused by changes in the multiplexed stream. It coordinates and generates samples that are transmitted through the IMFsample interface, and places the generated samples spanning several buffers.

Multiplexed stream 302 is a single stream containing more than one elementary stream. Elementary stream 304, 306 is a stream of similar elements (e.g., audio, video, etc.). There is no need for a one-to-one correspondence between multiplexed samples (M) 308 and elementary samples (E) 310, 312. For example, there may or may not exist a complete elementary sample for each elementary stream in each multiplexed sample. Additionally, there is also no specific requirement that the elementary samples be arranged in the correct order in the multiplexed stream. Elementary samples leaving the stream may not share the same timestamps. One elementary stream can be offset relative to another. Some multiplexed streams may contain only one set of elementary streams per stream. Some multiplexed streams may have different sets of elementary streams at different times.

Each set of coordinated elementary streams is called a view 320. Each set has a corresponding view descriptor 322 (P). This view descriptor 322 has two main tasks. First, it describes the media types of each elementary stream. Secondly, it provides a mechanism for choosing which of the available streams should be extracted by the demultiplexer 300. The Current View 324 always describes the selected streams and data types of the current output streams.

Before the sample can be processed by the demultiplexer 300, a partition algorithm is used to convert the multiplexed samples to elementary samples in order to know which streams to retrieve. This information is contained in view descriptor 322. Prior to selecting threads, the view descriptor is “waiting for processing”. After selecting the threads, the view descriptor becomes "active". To make a presentation active, it is retrieved from the queue of 326 pending presentations by calling GetPendingPresentationDescriptor. The appropriate streams are selected and then the caller calls the SetPresentationDescriptor method. At this point (if certain conditions are met), the view becomes the active view 324 and is removed from the queue 326 of the views awaiting processing. A pending view can become active only if all output from the previous active view is served.

Demultiplexer 300 contains at least two sets of queues. Queues are input queues 330 and output queues 340, 342. When you call ProcessInput () on the demultiplexer 300, the input data can be processed immediately or placed in the input queue 330. Once the data is processed, they are placed in the output queue 340, 342. Data types and the threads available in the output queues always match the current active view 324.

Now, after a general description of the demultiplexer 300, the states and transitions of the demultiplexer 300 will be described. In the following description, references to commands will be made. These commands are described below. According to FIG. 4, when the demultiplexer 300 is created, before calling the Initialize command, it is in the uninitialized state 400. In this state, Initialize is the only allowed operation on the demultiplexer object 300. A call to Initialize () puts the demultiplexer 300 into a wait state 402.

A wait state 402 indicates that there are no allowed active views. A call to ProcessOutput will fail. To set an active presentation, the multimedia processor 220 calls GetPendingPresentationDescriptor (), selects the appropriate streams, and calls SetPresentationDescriptor (). If the call to GetPendingPresentationDescriptor () is unsuccessful, then ProcessInput () is called. Then GetPendingPresentationDescriptor () is called again until it becomes possible to get the presentation descriptor 322. When the PresentationDescriptor is installed on the demultiplexer 300, it will go to the neutral state 404. Calling Flush () will reset all queued input and output data, and the demultiplexer 300 will go to the idle state 402.

In the neutral state 404, calls to all functions (except for initialization) are allowed. When a new view is detected during a call to ProcessInput and the last sample from the current view from the output queue is served, the demultiplexer 300 will enter the idle state 402.

Some streams have fixed and finite durations, which can be detected based on the contents of the stream. When such a situation is detected and all the output data is served, the demultiplexer will enter the end_of_stream state 406 end of the stream. All subsequent calls will return the corresponding error code.

If an unrecoverable error occurs in the demultiplexer 300, it will transition to error state 408. Error state 408 is reachable from any other state. When you want to remove the demultiplexer 300, Release () is called, and the demultiplexer 300 transitions to the final state 410 before it is removed from the system. When the last reference to the demultiplexer 300 is released, then the demultiplexer 300 is deleted from the memory, regardless of which state it is in. Releas () can be called from any state, including an uninitialized state.

Now, after describing the states and transitions of the demultiplexer 300, the commands referenced above will be described. These commands include Initialize (), SetPresentationDescriptor (), GetPresentationDescriptor (), GetPendingPresentationDescriptor (), ProcessInput (), ProcessOutput (), and Flush ().

FIG. 5 is an exemplary data structure diagram illustrating a basic message data structure 460 used to create seven demultiplexer API messages provided by the present invention. As you can see, the message data structure 460 contains several fields 462 _1-N . In a preferred embodiment, the first field 462 _{1 is} reserved for the Title. The remaining fields are parameters.

The Initialize command is created in accordance with the data structure provided by the present invention. As can be seen from the data structure diagram of FIG. 6, the 480 Initialize command is created from several fields 482-490. These fields are the header field 482, the stream descriptor object field 484, the media type field 486, the main type number field 488 and the main type array field 490. Each of the various commands is created in the same way as illustrated in FIG. 5 and a description of each command is given below.

The Initialize () method configures and initializes the demultiplexer object 300. The multiplexed stream descriptor may contain appropriate metadata to initialize the state of the demultiplexer (including any header data, etc.). The syntax of the command is as follows:

HRESULT Initialize (

IMFStreamDescriptor * pMuxedStreamDescriptor,

IMFMediaType * pSelectedMediaType,

DWORD cMajorTypes,

GUID * aMajorTypes,

);

The pMuxedStreamDescriptor parameter is an input parameter and is a pointer to a stream descriptor object that describes the multiplexed stream. The primary objective of this parameter is to enable the demultiplexer 300 to use any metadata that may be in the stream descriptor. The pSelectedMediatype parameter is an input parameter that defines the selected media type for the pMuxedStreamDescriptor parameter. It determines the type of media that corresponds to the samples that are passed to the Demultiplexer 300 in calls to ProcessInput (). The cMajorTypes parameter is an input parameter that determines the number of basic types in the aMajorType array. This parameter may be null. The parameter aMajorTypes is an input parameter, and it is an array of the main types of elementary streams, in which the calling program is interested in extracting from the demultiplexer 300. This parameter can be zero NULL if cMajorTypes is zero. The default stream of each of the main types found in the array will be selected in the presentation descriptor returned by GetPendingPresentationDescriptor (). If the method is successful, then it returns the value S_OK. If the method is unsuccessful, then it returns an error code. If a presentation is available, then it can be retrieved using the GetPendingPresentationDescriptor method. If the view is not available after initialization, then the view may become available after the data has been submitted to the demultiplexer via ProcessInput.

The SetPresentationDescriptor method sets an active presentation descriptor on a demultiplexer 300 indicating a new selection of streams in which the calling program is interested. The presentation descriptor must be a descriptor generated using the GetPresentationDescriptor method or the GetPendingPresentationDescriptor method. The syntax of the command is as follows:

HRESULT SetPresentationDescriptor (

IMFPresentationDescriptor * pPresentationDescriptor

);

The pPresentationDescriptor parameter is a pointer to a presentation descriptor object. If the method is successful, then it returns S_OK. The method may fail with MF_E_INVALID_PRESENTATION if the view descriptor is invalid. The method may fail with MF_E_OUTPUT_PENDING if the view descriptor is a pending view and there is output from the current active view waiting to be processed. If there is output from the currently active presentation that is awaiting processing, then the SetPresentationDescriptor method cannot be called by a view that is awaiting processing. The active presentation may call SetPresentationDescriptor () to select or deselect streams. If the selection of a stream is canceled, then all samples from this stream that are in the output queue are lost. If a new stream is selected, then samples from this stream will become available at some subsequent point in time. Since different streams have different requirements for buffering at the input and output, the point in time when samples of a new stream become available will depend on the particular demultiplexer.

The GetPresentationDescriptor method retrieves the descriptor clone of the currently active presentation on demultiplexer 300. The command syntax is as follows:

HRESULT GetPresentationDescriptor (

IMFPresentationDescriptor * ppPresentationDescriptor

);

The ppPresentationDescriptor parameter is a pointer to a pointer to a presentation descriptor object. If the method is successful, then it returns S_OK. If it is unsuccessful, it returns an error code. If there is output waiting to be processed, the GetPresentationDescriptor returns a presentation descriptor corresponding to this output.

The GetPendingPresentationDescriptor method retrieves the next pending presentation. The syntax of the command is as follows:

HRESULT GetPendingPresentationDescriptor (

IMFPresentationDescriptor * ppPendingPresentationDescriptor

);

The ppPendingPresentationDescriptor parameter is a pointer to a pointer to a presentation descriptor object. If the method is successful, then it returns S_OK. If there are no pending views, the method returns MF_E_PRESENTATION_NOT_AVAILABLE. If ProcessInput is called several times, then several views can be queued waiting to be processed. This method will only return the next view that is awaiting processing. If all the output for the current active presentation has been processed, then the pending presentation can be made active by calling SetPresentation.

The ProcessInput method enables the caller to submit to the demultiplexer a sample of a new input multiplexed stream. If the demultiplexer detects the presence of new streams in the presentation, then * pfNewPresentationAvailable will be set to TRUE (true), and the caller can retrieve the descriptor of the pending presentation using :: GetPendingPresentationDescriptor. The syntax for this command is as follows:

HRESULT ProcessInput (

IMFSample * pSample

BOOL * pfNewPresentationAvailable

);

The pSample parameter is a pointer to a selection object. If the method is successful, then it returns S_OK. The pfNewPresentationAvailable parameter returns TRUE when the result of calling ProcessInput is a handle to the new presentation that is added to the queue of presentations waiting to be processed. When a ProcessInput or ProcessOutput is called by the demultiplexer, the Demultiplexing operation can be performed. In any case, when calling ProcessInput, the data will be queued until the user calls ProcessOutput or the data is reset by calling Flush (). Only one view can be active at a time. When a new presentation becomes available, the caller must clear all pending output by calling ProcessOutput, retrieve the new presentation by calling GetPendingPresentationDescriptor, and then set the active presentation by calling SetPresentation.

The ProcessOutput method enables the caller to retrieve the elementary stream or elementary streams from the active view. The syntax of the command is as follows:

HRESULT ProcessOutput (

DWORD dwStreamIdentifier,

IMFSample * ppSample

);

The DwStreamIdentifier parameter is a 32-bit value that contains the stream identifier from the active view for the required selection. The ppSample parameter is a pointer to a pointer to a selection object. If the method is successful, then it returns S_OK. If it is unsuccessful, it returns an error code. If the end of the stream has been reached, the error code MF_E_ENDOFSTREAM will be returned. If E_NO_MORE_DATA is returned, then there is no data available for processing. Calling ProcessInput () with a different selection of multiplexed data may reduce the error. When the processing of available data is blocked due to the fact that the active view is no longer allowed, the method will return MF_E_NEW_PRESENTATION. When you change the view, a new set of threads becomes available. The user must indicate to the demultiplexer 300 which streams should be retrieved. Calling GetPendingPresentationDescription, selecting the required streams, and calling SetPresentationDescriptor () will allow you to process more samples from the input queue. If necessary, the demultiplexer entity 300 will allocate sample space. Demultiplexing Operation Demultiplexing can be performed by a demultiplexer by calling ProcessInput or ProcessOutput. In any case, when calling ProcessInput, the data will be queued until the user calls ProcessOutput or the data is flushed by calling Flush ().

The Flush method enables the caller to discard all samples currently in the input and output queues in the demultiplexer 300. Flush also clears the ActivePresentationDescriptor. This can be done when the “upstream” data tends to a new location, or the caller simply prefers to flush all buffered data. The syntax of the command is as follows:

HRESULT Flush ();

The Flush command has no parameters. If the method is successful, then it returns S_OK. If it is unsuccessful, it returns an error code. The Flush call clears all data from the demultiplexer 300 in the queue and also clears the descriptor of the active view. Depending on the particular demultiplexer, a pending presentation may become available immediately by calling GetPendingPresentation, or the calling program may need to call ProcessInput repeatedly until a pending presentation is available.

Now that the commands of the demultiplexer 300 have been described, a standard operation will be described where the source (e.g., application 202) does not match the receiver receiving the multiplexed stream. According to FIG. 7, the source provides a multiplexed stream for selection by the application (for example, a video digitization source provides DV or a television source TV provides an MPEG2 code stream) (step 500). The control layer 201 determines whether the application has negotiated a receiver for the multiplexed stream (block 502). If application 202 has not matched the receiver for the multiplexed stream, then the media session 240 downloads the corresponding demultiplexer and initializes it with the multiplexed stream descriptor (step 504).

If the presentation is accessible from the demultiplexer (step 506), then the multimedia session 240 attempts to reconcile using the elementary stream descriptors (step 508). The rest of the topology is formed (block 510). The provided elementary streams are processed by the multimedia processor 220 using DMO (e.g., decoders).

If the view is not available, then the multimedia session uses the NULL multimedia receiver (empty) to complete the topology for the multiplexed stream until more information is available (step 512). Each time a session is opened and a stream of samples is created, samples are applied to the demultiplexer 300 until a presentation descriptor is available (step 514). This is accomplished by calling IMFMediaStream :: ProcessSample on the media processor node that represents the multiplexed stream. Media processor 220 calls IMFMediaStream :: ProcessSample for the underlying multiplexed stream, such as an AVI source that provides a DV stream. When the multiplex stream sample is retrieved, the media processor calls IMFDemultiplexer :: ProcessInput on the demultiplexer 300. The ProcessSample and ProcessInput calls continue until IMFDemultiplexer :: ProcessInput returns a new view flag equal to TRUE. Then, the multimedia processor 220 signals the multimedia session 240 through an event that the current topology needs to be updated due to a change in the demultiplexer. A multimedia session 240 calls IMFDemultiplexer :: GetCurrentPresentation to access elementary stream descriptors that have recently become available. When a presentation descriptor becomes available, the multimedia session may renegotiate using the elementary stream descriptors (block 516).

The demultiplexer API for multimedia data streams is described. The demultiplexer API contains a set of interfaces, data structures, and events for representing a multimedia data demultiplexer. The API enables consumers to uniformly use multiplexed stream data, such as DV, to generate elementary stream data, such as audio and video (compressed or uncompressed). The API provides the ability to use demultiplexers as independent components. The API reduces the need for a large number of APIs for filters, and a given filter is no longer required to provide an interface with an API for each filter to which it can be connected in systems where the filters used are not supported. Also, the demultiplexer can be turned off correctly, since the multimedia processor controls the demultiplexer, and not the filters in the filter graph.

All materials presented here, including patents, patent applications and publications, are fully incorporated into the description of the invention by reference.

Given the large number of possible embodiments in which the principles of this invention can be applied, it should be clear that the embodiment described herein, together with the drawings, is intended to be illustrative only and should not be construed as limiting the scope of the invention. It will be apparent to those skilled in the art that, for example, the elements of the illustrated embodiment shown as software can be implemented in hardware and vice versa, or that, without departing from the essence of the invention, the structure and details of the illustrated embodiment can be changed. Therefore, the described invention encompasses all embodiments that may be practiced in the context of the following claims and their equivalents.

Claims (21)

1. An information carrier readable by a computer on which computer-executable instructions for performing the interface providing step for providing communication with a demultiplexer component performing separation of the multiplexed multimedia stream into elementary streams are recorded, wherein the interface contains instructions for:
initialization of the demultiplexer component and its configuration,
dynamically setting the active view descriptor with respect to the demultiplexer component,
supplying a new input multiplexed stream in the form of input samples to the demultiplexer component,
extracting from said active representation, according to the descriptor of the active representation, at least one elementary stream and issuing it in the form of output samples, and
reset samples currently in the input and output queues. 2. The computer readable storage medium according to claim 1, wherein the interface further comprises instructions for retrieving a clone of the currently active presentation descriptor with respect to the demultiplexer component. 3. The computer readable storage medium according to claim 2, wherein said clone retrieving instructions comprise a presentation descriptor. 4. The storage medium readable by a computer according to claim 1, wherein the interface further comprises instructions for retrieving the next view waiting to be processed. 5. The storage medium readable by a computer according to claim 4, wherein the instructions for retrieving the next view waiting to be processed comprise a handle to the view waiting to be processed. 6. The storage medium readable by a computer according to claim 1, wherein the commands for initializing the multiplexer component contain parameters including:
multiplex stream descriptor
The selected media type for the multiplexed stream descriptor,
an array of basic types of elementary streams, and
the number of basic types in the array of basic types. 7. The storage medium readable by a computer according to claim 1, wherein the instructions for dynamically setting an active view descriptor comprise a pointer to a presentation descriptor object. 8. The storage medium readable by a computer according to claim 1, wherein the instructions for supplying a new input multiplexed stream comprise a pointer to a sample object. 9. The storage medium readable by a computer according to claim 8, wherein the instructions for supplying a new input multiplexed stream further comprise a return value, in which there is a flag of the new view. 10. The storage medium readable by a computer according to claim 9, wherein additional computer-executable instructions are recorded for performing steps including:
if the flag of the new view is TRUE, then:
retrieving the following view, pending processing,
select the desired flows, and
call the component of the dynamic installation of the descriptor of the active view in relation to the component of the demultiplexer to enable processing of samples from the input queue of the demultiplexer. 11. The storage medium readable by a computer according to claim 1, wherein the instructions for extracting from the active view, according to the descriptor of the active view, at least one elementary stream and issuing it in the form of output samples contain a stream identifier and a pointer to an object pointer sampling. 12. The storage medium readable by a computer according to claim 11, wherein the instructions for extracting from the active representation, according to the descriptor of the active representation, at least one elementary stream and issuing it in the form of output samples additionally have a return value at the output. 13. The storage medium readable by a computer according to claim 12, in which the value returned at the output includes an error code for the end of the stream or an error code for the absence of the following data. 14. The storage medium readable by a computer according to claim 1, wherein the interface receives the multiplexed data as a data buffer in memory. 15. The computer readable storage medium of claim 14, wherein the multiplexed data is in a format including at least one of Digital Video, MPEG2, and ASF formats. 16. The storage medium readable by a computer according to claim 1, wherein the instructions for initializing the demultiplexer component refer to a data structure for use in a multiplexer, said data structure comprising:
The first field containing the title
a second field containing a handle to the multiplexed stream,
a third field containing the selected media type of the multiplexed stream descriptor,
a fourth field containing an array of basic types of elementary streams, and
the fifth field containing the number of basic types in the array of basic types. 17. The storage medium readable by a computer according to claim 1, wherein the commands for dynamically setting the active view descriptor for the component access a data structure for use in a demultiplexer, said data structure comprising:
the first field containing the title, and
the second field containing the view descriptor. 18. The computer readable storage medium of claim 2, wherein the instructions for retrieving a clone of the currently active view descriptor in relation to the demultiplexer component refer to a data structure for use in the demultiplexer, said data structure comprising:
the first field containing the header and the second field containing the view descriptor. 19. The storage medium readable by a computer according to claim 4, in which the team to cure the next view, waiting to be processed, refer to a data structure for use in a demultiplexer, said data structure comprising:
the first field containing the title, and
a second field containing a handle to the view waiting to be processed. 20. The computer readable storage medium according to claim 1, wherein the instructions for supplying a new input multiplexed stream as input samples to a demultiplexer component refer to a data structure for use in a demultiplexer, said data structure comprising:
the first field containing the title, and
second field containing a pointer to the selection object. 21. The computer readable medium according to claim 1, wherein the instructions for retrieving, according to the descriptor of the active representation, at least one elementary stream from said active representation and issuing it in the form of output samples, refer to a data structure for use in a demultiplexer, wherein said data structure contains:
The first field containing the title
a second field containing a stream identifier, and
the third field containing a pointer to a pointer to a selection object.

Images