KR20010098904A - Multiprocessor object control - Google Patents

Abstract

A client-server system is disclosed that performs two-stage server task scheduling in which information of a client deadline step is used for subtask server scheduling in a second step. The object broker of the system also collapses client requests and responses in order to maintain data in the coprocessor, and multitasking and data flows through the shared memory of multiple coprocessors to prevent main processor bus congestion. A server memory management method is provided.

Description

MULTIPROCESSOR OBJECT CONTROL}

The present invention relates to an electronic device, and more particularly, to a multiprocessor and digital signal processor and methods distributed in an object.

The growth of the Internet, combined with high-speed network access, has made distributed computing mainstream. Common object request broker archirecture (CORBA) and distributed component object model (DCOM) standards simplify object-oriented network programming and component software methods. Thus, a client application can call a remote server object to provide data or functionality to simplify application programming. 24 shows the call structure of a general remote method. As a result, object-oriented programming conceals specificity, representing only object interfaces for queries or interactions with other objects, enabling this distributed computation.

The core of CORBA provides a "bus" for conversations between objects, both local and remote. CORBA objects are a set of plus methods and interfaces. A client of a CORBA object uses the object's reference as the handler for the method invocation, even if the object is located in the client's address space. The ORB discovers an implementation of the object (possibly on a remote server), prepares the object to receive a call request from the client application, sends a request (eg, parameters) from the client to the object, and sends the client to the object. Can return any response. The object implementation interacts with the ORB by its ORB interface or object adapter (OA). 25 shows all CORBA structures.

Interface definition language (IDL) typically defines the interface of an object, including methods to be called by the client, while concealing specifics (data, implementations). IDL typically provides data encapsulation, polymorphism, and inheritance. As shown in Fig. 24, a client first calls an object function with a client stub (proxy) by making a call. The stub organizes the call parameters into messages. The wire protocol sends a message to the server stub (skeleton). Server stubs do not clean up call parameters from messages, but call object functions. The upper layer of FIG. 25 is the basic programming structure, the middle layer is the remote architecture, and the lower layer is the wire protocol architecture. Developers of client programs and server object programs target the underlying programming architecture, and the remote architecture creates interface pointers, object references, and handles meaningfully between client and server processing. The wire protocol effectively extends the remote architecture between various hardware devices.

As described in Cheung et al., DCOM and CORBA Side by Side, Step by Step, and Layer by Layer, a simple application that uses remote objects on CORBA-enabled client and server processors uses five files: Can be generated. (1) IDL file to define the interface (s) for the object. The IDL compiler generates client stub and object skeleton code and interface header files used by both client and server. (2) Execution header file to derive the server implementation class for the object from the interface. In essence, an implementation class can be associated (by inheritance) with an interface class generated by the IDL compiler. (3) the implementation of the server class method. (4) the main program for the server. This program creates an instance of the server class. (5) A client application that will invoke the object's method by calling the client stub.

In static object invocation, CORBA registers an association between an executable interface name and a path name within the implementation repository (see FIG. 25) before execution. In dynamic object invocation, the IDL compiler generates type information for each method in the interface and stores it in the interface repository. The client queries the interface repository to get runtime information about a particular interface, and uses it to dynamically create and invoke methods on the object via the dynamic invocation interface. Similarly, on the server side, the dynamic skeleton interface allows clients to invoke operations on objects that do not have compile-time knowledge of the type of object being implemented.

FIG. 26A illustrates the CORBA upper layer (basic programming architecture) operation of client requests of objects and invocation methods, and server creation of object instances and usability for clients. In particular, the object behavior follows: (1) The client calls the static function of the client stub for the object interface. (2) The ORB starts up a server containing objects that support the object interface. (3) The server program creates an object and registers an object reference. (4) The ORB returns the object reference back to the client application. Then, for object method invocation [1], [2], the client calls the method of the object interface that actually invokes the methods in the server. When the method returns the values, the server sends them back to the client.

Figure 26B shows a CORBA middle layer (remote architecture) with the following object behaviors. (1) Upon receiving a call, the client stub delegates the task to the ORB. (2) The ORB queries the implementation repository, maps the call to its server path name, and activates the server program. (3) The server creates an object and also generates a unique reference ID to get an object reference. Register an object reference with the ORB. (4) The constructor of the server also creates an instance of the skeleton class. (5) The ORB sends an object reference tack to the client and also creates an instance of the client stub class and registers it with the corresponding object reference in the client stub object table. (6) The client stub sends an object reference back to the client. The client call of the object method then proceeds [1] upon receipt of the client call, and the client stub generates the request pseudo object, the marshall parameters of the call to the server in the pseudo object, and includes the pseudo object in a message in the channel. It asks for it and waits for a response. When the message reaches the server, the ORB finds the target skeleton, rebuilds the request pseudo-object, and sends it to the skeleton. [3] The skeleton does not clean up parameters from the request pseudo-object, calls the server object's method, cleans up the return value, and returns from the skeleton method. The ORB builds a response message and places it in the transmit buffer. [4] When the response reaches the client side, the ORB returns the ORB call after reading the response message from the receive buffer. Next, the client stub does not clean up the return values, but sends them back to the client to terminate the call.

As described in Figure 26c, the lower layer (wire protocol architecture) for object activation selects a machine that (1) upon receipt of a request, the client-side ORB supports the object and sends the request to the server-side ORB via TCP / IP. do. (2) When the server is started by the server side ORB, the object is created by the server, the ORB constructor is called, and the creation function is performed. The create function creates a socket endpoint, an object is assigned an object identity, and an object reference is created that includes an interface and implementation name, a reference identity, and an endpoint address. Object references are registered by the ORB. (3) When the object reference returns to the client side, the client stub extracts the endpoint address and establishes a socket connection to the server. The method call then proceeds upon receipt of the [1] call, and the client stub cleans up the parameters in the Common Data Indication (CDR) format. [2] The request is sent to the target server over the established socket connection. [3] The target skeleton is identified by either a reference identity or an interface instance identity. [4] After calling the actual method on the server object, the return value in the skeleton CDR format is summarized.

CORBA's real-time expansion typically provides quality of service features (QoS) such as predictable performance, security behavior, and resource allocation. For example, Gill et al., Applying Adaptive Middleware to Manahe End-to-End QoS for Next-generation Distributed Applications.

CORBA components were introduced as metatypes, and a related component implementation definition language (CIDL) is available to describe the implementation. 27 shows a programming step.

DCOM has similarly three layers and an architecture somewhat similar to CORBA.

Notenboom, U. S. Patent No. 5,748, 468 and Equator Technologies PCT Publication Application WO 99/12097 each describe a method for allocating processor resources to multiple tasks. Notenboom considers coprocessors with host processors and allocates coprocessor resources to tasks according to conventional systems. Equator Technologies schedules processor resources according to the task time consumption of each task that provides at least one supported service level (processor resource consumption rate), and the resource manager allows the task if sufficient resources exist for the supported service level. .

A system or BIOS, where each processor has two or more processors with its own operating system, is integrated on the same semiconductor die, such as a system with processors that are widely separated over the Internet, and one or more DSPs with RISC CPUs. It includes a system with two or more processors.

The XDAIS standard specifies an interface to algorithms that run on DSPs, which provide reusable objects. XDAIS requires algorithms to implement extensions with the standard interface IALG for executing algorithms. XDAIS also requires that certain flexible rules and naming conventions, such as relocatable code, be followed. Client applications can manage instances of the algorithm by calling tables of function pointers. Using the XDAIS standard / guideline, algorithm developers can develop or transform algorithms to facilitate plugging into DSP application frameworks such as the iDSP Media Platform DSP framework.

The need for a quality of service (QoS) management program within a network node (client / server) comes from the real-time service requirements of all streaming media-based applications, in particular. Streaming media applications must handle heterogeneous codecs (encoders / decoders) and filters with unique rendering deadlines. These applications also need to be able to translate using cognitive characteristics to a degree that does not seriously degrade the quality of service. They must be able to handle some jitter in the processing and rendering cycles. For example, in a video application, the frame rate for rendering must be maintained at 30 frames / second (fps), which translates to a frame period of 33 ms. However, the application must be able to tolerate limited instantaneous changes in consultation with the server. Also, at 30 fps, the time of day can allow for a frame drop of about 6 frames / second. Client applications should also be able to support non-fatal degradation in performance (temporary degradation of frames) and maintain a steady state of rendering within specified tolerances negotiated with the server. The QoS management program is a mechanism that provides the necessary functions and capacity to realize such a real-time system.

As broadband communications such as DSL and cable modems are rapidly expanding into new markets and must deliver unprecedented amounts of data to consumer devices for processing and consumption, more efficient data processing, routing, and processing technologies are following. You will need to go.

20 shows a diagram of how data is processed through the processing elements of current heterogeneous systems. Each data transaction is numbered to show a time sequence. For each transaction, the data must pass through the system bus under the control of the Center Control Processor (CCP). The CCP initiates a message or trigger through a control path to various processing elements in the system.

The processing elements of FIG. 20 are separate processors (eg, DSP, ASIC, GPP, etc.) that can drive a defined set of tasks. This is why each is shown with its own memory. The processing elements can also be separate tasks running on the same processor.

In some cases, the same data must pass through the system bus multiple times (eg, transactions 1 and 2, 3 and 4, and 5 and 6). In such a system, data must pass a total of 2+ (2 × n) times, or six times, in the system bus. Each passes through the system bus and arbitration by the CCP introduces overhead in the data flow and reduces overall system throughput.

Data flow overhead negatively affects the amount of data that passes through the system in a given time frame, thereby limiting the amount of data that the system can process. Such a system would perform fewer useful tasks than the sum of the capacities of the elements would otherwise indicate.

The present invention provides a client-server system having one or more features, which features two-stage scheduling of server tasks, an object request broker for a client-server system that binds tasks on the server DSP, and internal memory. Multitasking internal memory management by dividing the processor overhead into task tasks belonging to a single execution task at the same time, adding processing elements bus-connected to the central control processor, and avoiding the central control processor bus. Each includes a data flow in a heterogeneous system, including the addition of shared memory.

1 shows a DSPORB structure of a preferred embodiment.

2 illustrates IDL translation.

3-13 is a timing diagram of QoS.

14-19 illustrate memory analysis of a preferred embodiment.

20 illustrates a known data flow diagram in a heterogeneous system.

21-23 show data flow diagrams of a preferred embodiment.

24-27 depicts CORBA.

1. Overview

The system of the preferred embodiment typically has a host processor running a client application and one or more server processors running a server algorithm, the object request broker for algorithm objects, the quality of service control for the object request broker, the memory paging for the algorithm objects. And data flow for algorithm objects. The preferred embodiment of the term iDSPOrb applies to a system having a main processor and one or more DSP coprocessors.

iDSPOrb is a high performance DSP Object Request Broker (DSPORB) that supports the creation and access of DSP objects from a General Purpose Processor (GPP) or other DSPs in a multiprocessor environment. iDSPOrb has a general structure and behavior similar to CORBA. iDSPOrb has the following DSPORB features:

(1) iDSPOrb supports calling and object binding (DSP object procedure call) across processor boundaries.

(2) iDSPOrb provides a GPP-side proxy interface consisting of compile-time headers and stubs for static call and runtime dynamic call interfaces.

(3) iDSPOrb provides DSP-side algorithm interfaces (stubs and headers) for building iDSP servers.

(4) iDSPOrb provides both synchronous and asynchronous calls.

(5) iDSPOrb provides guaranteed real time QoS.

(6) iDSPOrb provides frame based processing and stream based processing.

(7) iDSPOrb provides object binding data flow (intermediate results are held in DSP memory).

(8) iDSPOrb is implemented on the high-pass multichannel GPP / DSP I / O interface.

Figure 1 shows the iDSPOrb architecture for a GPP / DSP dual processor with GPP serving as a "client" and a DSP serving as a "server."

The quality of service (QoS) management program of the iDSP system, referred to herein as iDSP-QoSM, is a mechanism (in the server) that provides a negotiated level of service to client applications. This provides a guaranteed quality of service in accordance with certain grade down policies in communication with the client. iDSP-QoSM has the following characteristics. That is, (1) it is assumed that there is a QoS management program defined within the limited context of a node (intra-node) residing on the network and suitable for controlling inter-node (network) communication. (2) Defined as a multi-processor environment with load sharing capabilities.

Functions performed by iDSP-QoSM of the preferred embodiment include the following. That is, (1) monitoring the steady state processing load on the server in this system. (2) Distributing the load from overloaded servers to equivalent communication entities. (3) Negotiate service requirements with the client application to register any additional load on the server. (4) Estimate different loads on the server based on the specific characteristics of the individual objects served by the server. (5) Algorithm runtime estimation will be based on cycles of processor time instead of process time. In other words, the algorithm runtime prediction scheme is not combined with the processor operating frequency.

In Texas Instruments TMS320C62XXDSPs, the amount of internal memory (on-chip) data is limited. Except for the TMS320C6211 (and its derivatives), the TMS320C62XXDSPs do not have a data cache for efficient external memory (off-chip) access. Internal memory is the highest level in the data memory hierarchy of the TMS320C62XXDSP. Therefore, all algorithms run on the TMS320C62XXDSP want to use internal memory as the data work area because this is the highest efficiency level for data memory access.

Typically, algorithms for DSP have been developed assuming they own the entire DSP processor, i.e. all the internal memory of the DSP. This makes these algorithms very difficult even if several different algorithms integrated are the same (homogeneous) or different (heterogeneous). Algorithm developers need a set of rules regarding conventional methods of accessing and using system resources such as internal memory.

The preferred embodiment provides a method of increasing processor utilization when executing multiple algorithms for a DSP with a low data cache by using a data paging architecture for DSP internal memory. Flexibly developing or converting DSP algorithms according to the data paging architecture can be accomplished according to the Texas Instruments ADAIS standard.

This International Standard requires an algorithm developer to define at least one memory area to support all data memories for that algorithm. One or both of these user-defined areas are selected by the algorithm developer to operate the internal memory of the TMS320C62X DSP. Within the DSP system software portion of the application, the internal memory is divided into system support and data workspaces (pages). All algorithms in a DSP application share a workspace and own the entire workspace at run time. In the context switch between the two algorithms, the DSP system will handle each transfer between the work area of each algorithm and the external shadow memory. According to a preferred embodiment the following points are provided:

(1) In DSPs without a data cache, sharing internal data memory between two or more DSP algorithms increases processing efficiency.

(2) Operating multiple algorithms from the same shared internal memory allows each algorithm to have maximum efficiency in the TMS320C62X DSP environment when accessing data memory to support stack requests and algorithm internal variables.

(3) The architecture will function on any single processor with internal memory and a DMA utility that accesses the processor's internal memory.

(4) Performing a context switch only at the data input frame boundaries provides the best efficiency of the data paging architecture.

The data flow in the application will be from algorithm to algorithm, and the preferred embodiment allows the data to remain in one or more DSPs rather than from bus to / to GPP for each algorithm implementation.

2. DSP ORB in Dual-Processor Configuration

Figure 1 shows an ORB ("iDSPOrb") architecture, which is a preferred embodiment for a dual-processor including a general purpose processor (GPP) and a digital signal processor (DSP), where the GPP operates as a "client" and the DSP is a "server". It acts as ". Note that iDSPOrb includes a quality of service (QoS) management program. Figure 1 shows a client application calling two DSP algorithm objects "A" and "B". iDSPOrb first provides GPP with object bindings of proxy (client stub) objects "a" and "b". For example, "A" and "B" may be an extension of the DSPIDL interface to the decoder DEC as follows:

DSP-side applications (called iDSP servers) are implemented using the following algorithmic interfaces provided by the DSPIDL compiler:

GPP-side applications are implemented using the following proxy interfaces provided by the DSPIDL compiler, or using the iDSPOrb dynamic call interface.

In operation, "a" may be called from the GPP side client application to process the buffer. This data is transferred to the actual object "A" on the DSP side. Using the object chain data flow, the output of "A" can be connected to the input of "B" so that the arbitration data buffer is not sent back to GPP. "B" calls "B" which results in another processing step of returning this data to GPP. iDSPOrb's dynamic call interface supports both synchronous and asynchronous calls.

iDSPOrb need not be partitioned between GPP and a single DSP. It can also work with one consisting of multiple DSPs. In this case, the QoS management program (server side) performs load-balancing of the DSP algorithm among the available DSPs. Other configurations where an algorithmic interface is provided to the client application may include an ASIC (acting as a fixed function DSP), or an ASIC plus RISC.

2a. DSPIDL compiler

iDSPOrb supports DSPIDL and IDL (Interface Definition Language) with the following keywords:

module: a set of interface specifications

For example, the H263 module may also include decoder and encoder interfaces.

interface: Interface specification

in: Represents input argument

out: indicates output arguments

BUFFER: Indicates buffer type

STREAM: indicates the stream type

RESULT indicates the return type of the function

The rest is for memory use, real time

The general form of a DSPIDL file is:

Where method is

, Direction is in, out, or [in, out], and TYPE is BUFFER or STREAM. For example, H263 IDL may generate algorithms and proxy interfaces as shown in FIG.

2b. Frame and Stream Processing

Frame-to-stream processing has the following differences.

Keywords:

BUFFER: Functions with BUFFER as argument type handle on a frame-by-frame basis.

STREAM: Functions with STREAM as the argument type typically handle the stream of frames that cause the task.

The above function connects the object outputs to inputs (frames or streams, respectively). For a buffer, the connect operator is a memory buffer on the DSP where the output of one method call is stored for the input of another method call (object chain). Will cause DSPORB to generate E.g:

For stream processing, the following proxy call will typically result in a task created on the DSP side to manipulate the SIO stream (an implementation of H263_TIDEC_decodeStream will allow the task to do this). Streams as not connected provide I / O between the client proxy and the server.

2c. Real-time QoS Manager

iDSPOrb can provide hard real-time QoS by allocating resources necessary to perform a given operation within a set time constraint through the DSPORB_System_setTimeConstraint () and DSPORB_System_setPriority () interfaces. GPP / DSP channel I / O drivers allow multiple threads to run in parallel. The QoS manager can (1) instantiate the algorithms that the client needs, (2) update the constraints from the client application and manage the resource to satisfy these constraints (or report that the above constraints cannot be met). (3) A part of iDSPOrb of the DSP side which performs other operations.

2d. iDSPORB Registration Service

iDSPOrb provides a class registration service that allows server objects to register their services. For example, a server object can be registered by iDSPOrb, which can decode MP3 audio. The client object creates a server object by supplying the name of the desired service. The iDSPOrb Registration Service can be used for any kind of DSP object service, but the media domain is known by providing a set of monikers for audio and video services:

Audio Service Video Service

---------------------------

MP3 audio decode MPEG 1 video decode

MP3 audio encode MPEG 1 video encode

MPEG1 L2 Audio Decode MPEG2 Video Decode

MPEG1 L2 Audio Encode MPEG2 Video Encode

G. 723 decode MPEG4 video decode

G. 723 encode MPEG4 video encode

G. 729 decode H. 263 decode

G. 729 encode H. 263 encode

... ...

The iDSPOrb registration service allows iDSPOrb to dynamically create server objects during runtime. When creating a server object, iDSPOrb dynamically allocates low-level I / O channels between the microprocessor and the DSP. These low level channels can be accessed directly by the client object via the iDSPOrb streaming interface (see DSPORB_stream interface). The iDSPOrb registration service also provides information that allows the DSP to provide a particular service to be located, which allows the QoS management program to perform load balancing and projection scheduling (real-time QoS management program). For example, using the dynamic call model, call DSPORB_ALG_create ("MP3 audio decode", null) will create an instance of the MP3 audio decoder. The iDSPOrb load balances the system, and the client is shielded from the details of which DSP actually runs the decoder and which low level stream has been allocated to pass the data. The client can also enumerate the list of currently registered server classes by querying iDSPOrb. The function DSPORB_Alg * DSPORB_System_getServices () can be used to get an enumerator of the currently registered services. Then char * DSPORB_System_next (DSPORB_Alg * enum) can be called to get the name of each registered service. The enumeration can be reset initially by calling DSPORB_System_reset (DSPORB_Handle * enum).

2e. Media framework support.

iDSPOrb can be used to support media processing acceleration by providing components for specific media frameworks such as DirectShow (Windows Media): Filter objects are implemented to wrap iDSPOrb codec client objects and DirectShow It can be plugged into the framework.

RealMedia Architecture (RealSystem G2): Renderer plug-ins are implemented to wrap iDSPOrb codec client objects and can be plugged into the RealSystem G2 framework.

DSPOrb can also be plugged into JMF and QuickTime using the same method.

The API for iDSPOrb is encapsulated within the DSPORB module. The data type and function of the client (GPP) side DSPORB is described in detail below.

2f. Data type:

DSPORB_Alg: Client proxy for DSP algorithm object.

DSPORB_Fxn: Function object to be used with dynamic invocation.

DSPORB_Arg: Function discussion object to be used with dynamic invocation.

DSPORB_Buffer and DSPORB_stream are "subclasses" of DSPORB_Arg.

DSPORB_Params: Provides parameters for the algorithm that matches the IALG_Params algorithm parameter structure on the DSP side.

DSPORB_Buffer: A buffer object.

DSPORB_Stream: Stream object.

2 g. DSPORB_Buffer Interface

DSPORB_Buffer * DSPORB_Buffer_create (int size, int direction);

Create a buffer object that can reference the data length size. direction is either DSPBUFFER_INPUT or DSPBUFFER_OUTPUT. The buffer direction must match the function call signal or an iDSPOrb runtime error will occur.

Alternatively, DSPORB_Buffer * DSPORB_Buffer_create (DSP ORB_Alg *, int, int); The buffer used by the object.

-unsigned char * DSPORB_Buffer_getData ();

Takes data referenced by a buffer object. NULL is returned if the buffer is connected to another buffer.

-void DSPORB_Buffer_setData (unsigned char * data)

Set the buffer data pointer. If this buffer is connected to another buffer, this operation fails because the memory space for the data in this buffer is in the DSP memory space.

-void DSPORB_Buffer_setSize (int)

Set the actual data size.

-int DSPORB_Buffer_getSize ()

Take the size of the actual data.

-void DSPORB_Buffer_delete (DSPORB_Buffer * buffer)

-int DSPORB_Buffer_connect (DSPORB_Buffer * output, DSPORB_Buffer * input)

Connect the input buffer to the output buffer for the DSP. When these buffer objects are connected, the data is kept in the DSP and not sent back to GGP (the buffer is created by iDSPOrb in the DSP to keep intermediate results).

2h. DSPORB_Stream Interface

The stream interface has the following method.

DSPORB_Stream * DSPORB_Stream_create (int n, int direction); Create a stream that can hold n buffers. direction is either DSPSTREAM INPUT or DSPSTREAM OUTPUT.

int DSPORB_Stream_issue (DSPORB_Buffer * buf); An input buffer buf sent on the input stream, or an empty buffer placed on the queue, is filled and filled on the output stream. For concatenated streams, this operation is ineffective because the streams will be directly connected between algorithms.

DSPORB_Buffer * DSPORB_Stream_reclaim (); An output buffer from an output stream; Or take an input buffer that can be retransmitted on the input stream. For connected streams, this operation has no effect.

DSPORB_Stream_select (DSPORB_Stream array, int n_streams, int * mask, long millis); Block until the stream is ready for I / O.

DSPORB_Stream_idle (DSPORB_Stream * str); Idle the stream.

DSPORB_Stream_close (DSPORB_Stream * str); Close the stream.

DSPORB_Stream_connect (DSPORB_Stream * out, DSPORB_Stream * in); Connect the output stream to the input stream. The two streams now bisect the operation in the DSP processor space and are not accessible to GPP.

2i. DSPORB dynamic call interface

The dynamic call interface has the following method.

int DSPORB_System_int (); It must be called initially to initialize DSPOrb.

DSPORB_Alg * DSPORB_Alg_create (const char * name, DSPORB_Params * params); Create an instance of the algorithm referenced by the symbol 'name'.

void DSPORB_Alg_delete (DSPORB_Handle alg); Delete the algorithm instance.

DSPORB_Fxn * DSPORB_Alg_getFxn (DSPORB_Alg * alg, const char * fxn_name); Return the function object associated with the symbol 'fxn_name'.

int DSPORB_Fxn_setTimeConstraint (DSPORB_Fxn * fxn); Set the time boundary for running fxn. DSPOrb meets this constraint by allocating sufficient resources, or returns to zero.

int DSPORB_Fxn_setPriority (DSPORB_Fxn * fxn); Set a priority level of 1 to 15.

int DSPORB_Fxn_invoke (DSPORB_Fxn * fxn, DSPORB_Arg * args); Call functions on inputs and outputs. This call blocks until all data is available for unconnected output. For inputs and outputs connected to 'DSPORB_Buffer_connect', 'NULL' may be passed.

int DSPORB_Fxn_invokeAsync (DSPORB_Fxn * fxn, DSPORB_Arg * arg); Call functions on inputs and outputs. This call returns immediately; The application uses 'DSPORB_getData' to retrieve data from the output discussion object.

-unsigned char * DSPORB_Arg_getData (DSPORB_Arg * output, long timeout); Takes data from the output discussion object. Block until timeout occurs in nanoseconds, or indefinitely if timeout = -1.

-voidDSPORB_Arg_setCallback (DSPORB_Arg * output, unsignedchar * (* getData)

(DSPORB_Arg *)); Set the callback function for the output discussion, getData is called if data is available.

-voidDSPORB-System_close () closes DSPOrb.

2j. Example of iDSPOrb

In the first example, we show how iDSPOrb is used to connect the TI H.263 decoder to C6xxx using the dynamic call interface. The second example shows the same program recorded as a proxy stub.

3. Quality of Service (QoS)

The configuration of the preferred embodiment in which the quality management program (iDSP-QoSM) of the iDSPOrb service is defined consists of a host processor having a pool of DSPs as a peer server. The umbrella QoS-management program, which performs all the functions necessary to maintain the quality of a particular service, manages this pool of DSP servers. The host computer is usually a general purpose processor (GPP), which is connected to the DSP through a hardware interface such as shared memory or a bus type interface. The QoS management program may be part of iDSPOrb or, more generally, a separate management program on the DSP. The system is driven by hardware and software interrupts. The preferred implementation allows the main user (client) application to run in GPP and to run certain services on the DSP on a load-sharing basis. Running concurrently with the QoS management program on all processors may be a framework such as the iDSP media framework. The iDSP-QoS management program performs three main functions: (1) classification of objects, (2) scheduling of objects, and (3) prediction of execution time of objects.

These functions will be described below under a GPP / multi-DSP environment using a MIDI specific example.

3a. Classification of objects

In a media native environment, objects are translated into media codecs / filters (algorithms). Media objects may be classified based on their stream type, application type or algorithm type. Depending on the type of algorithm, the QoS management program defines metrics known as codec-cycles, filter-cycles, and the like.

3b. Scheduling of Objects (hard-duration)

iDSP-QoSM schedules algorithm objects based on a two-phase scheduler. The first step is a high level scheduler that determines whether a new media stream is scheduleable on the DSP and sets a hard-real time deadline for the codec-style. The second stage schedules individual media frames and uses the hard-real time deadline from the first stage. The first phase is executed at object negotiation time and generally on a host (GPP). The second stage is executed in the DSP (server) and executed on a per frame basis.

The first step in scheduling is when the QoS manager determines on average whether an object can be supported by an object already running at the same time. Consideration of sufficient support for objects in terms of memory is required as part of the first stage scheduling. Object memory buffers, inputs, and outputs for internal use must be statically fixed at the time of their instantiation to eliminate the uncertainty of dynamically allocating memory. The iDSP media platform only handles XDAIS compliant algorithms. The developer is required to define different processing times for the algorithm. The approximate time required to transfer data to / from the server is determined by the initialization time that is factored by the QoS management program when a deadline is set for each object.

Each DSP object is required to supply the following information about the QoS management program.

n Codec-cycle and number of frames (default: frames / sec)

T _acc average time to calculate codec-cycles to the number of target server (DSP) cycles

Display time of codec-cycles in number of T _acd target server (DSP) cycles

For video codecs, n is typically the number of frames (eg 15 frames) between successive I-frames. And, T _acc is usually the sum of the maximum amount of time required for I frames plus the average time required for P and B frames. The QoS manager maintains a track of T _ccd for all media objects. This time (in the DSP cycle period) is based on the current frame rate. For example, assume that 30 ccs video stream and n = 15, T _ccd = 125 Mcycle.

The QoS management program can now determine whether a new stream is schedulable as follows. Let S sum the codec-cycles (T _acc ) for all currently scheduled streams. If (S + T _acc ) for the new stream is less than T _acc for the new stream, this stream is schedulable, otherwise it is not schedulable. For example, suppose you have Object-A with n = 15, T _axc = 39.5 Mcycles (158 ms) and T _ccd = 125 Mcycle (500 ms), and there are no jobs scheduled on the DSP (thus S = 0). . The QoS management program will notify to schedule resources for the new stream that requested Object-A. Since S + 39.5 = 39.5 Mcycles <125 Mcycles (500ms), we can schedule the stream. When the second stream passes through the requesting object-A, it is also scheduled as S + 39.5 = 79 Mcycles (316 ms) <125 Mcycles (500 ms). The third stream may also be scheduled. However, the fourth stream cannot be scheduled by requesting 158 Mcycles (632 ms), so we cannot meet the strict deadline of 500 ms. At this point the QoS manager will decide to reduce the frame rate of the stream and will reject the stream as a whole.

The variant allows the scheduler to handle heterogeneous media objects according to differences in codec cycle times. Objects with longer T _ccds are assigned to the smallest T _ccd . For example, there is Object-B with n = 30, T _axc = 40 Mcycles (160ms) and T _ccd = 169Mcycle (675ms), and two Object-As (as defined above) scheduled on the DSP Suppose there is, therefore, S = 79 Mcycles / 316ms. We can schedule a new Object-B stream because S + 40 * (125/158) = 110.45 Mcycles (S + 160 * 500/675 = 435ms). This is clearly accurate as (79 + 40 <125) Mcycles / (316 + 160 <150) ms, so we can actually finalize all streams within a shorter codec cycle of 500ms. What happens when a second stream requesting Object-B needs to be scheduled? 110.45 + 40 * 125/158 = 139> 125 Mcycles / 435 + 160 * (500/675) = 554ms> 500ms. Thus, the scheduler rejects this stream and starts negotiation as described above.

iDSP-QoSM will negotiate with the application or its proxy to ensure sufficient processing bandwidth for the media object based on the codec-cycle. This negotiation would take into account the valid MIPS of the DSP with the requested memory of the object, the requested QoS level, and other concurrently running DSP applications. As the object selection changes, the QoS manager will perform renegotiation of the DSP processor bandwidth. The input parameters for the negotiation process of the QoS management program request an application to define the following for the object.

(1) DSP memory conditions (number and size of input / output buffers)

(2) the desired QoS level (typically expressed in frames per second)

(3) Worst case runtime for starting an object

(4) fixed real time deadline (number of frames and average execution time) for sequences of media frames, called codec cycles

The second phase scheduling of an object in an iDSP-QoS manager is based on two aspects: the first to reach due date and the one with the highest priority. If Object-A has a deadline of 10ms and Object-D has a deadline of 3ms, the iDSP QoS manager can schedule Object-D to run first even if Object-A has the highest priority. Consider the following example. Because we know the approximate runtime of an object, we can determine the "NO Later" time when the object should be started to meet its deadline. In FIG. 3, it is assumed that Object-D ends before the “No Later” start point for Object-A. In this scenario, there is no conflicting time between object-A and object-D of higher priority. Therefore, Object-A is executed after Object-D of low priority.

Another example of scheduling is prioritization on a first deadline if the "No Later" time of high-ranking Object-A is before the predicted end time of the predicted Object-D. In this case, the Object- Only if D matches the frame dropping parameter specified at object instatntiation time, it operates first because object-A is not only ranked higher, but object-D is allowed to run afterwards. See FIG. 4.

For iDSP QoS to manage deadlines for the best possible efficiency, GPP allows data input frames to pass through the DSP subsystem as quickly as possible, thus maximizing the amount of time between the incoming and deadline for the object. The larger the time for the data frame between the incoming and the due date, the more flexible the iDSP QoSM is in scheduling that object with other concurrent objects.

3c. Predictive runtime of objects (soft-due date)

The central function of the iDSP QoSM is to predict the required processing time for the next input frame of all scheduled objects. The QoS manager predicts the run time for the object by using the statistics of the previous run time to calculate the expected run time for the next input frame. The expected run time for an object is a function of the previous run time (also determined uniquely for each object) with the maximum possible positive change (object specific). For example, for video objects, the periodicity of I, P, and B frames is deterministic. That is, the future processing time can be predicted within the periodicity of the video frame based on the type of the current frame and its location. Such predictions made on all concurrent algorithms help to dynamically reassign priorities based on predicted processing time and access hard deadlines.

These examples are key enablers for managing jitter within soft-duration and processing time. In accordance with the prediction, the iDSP QoSM instantly reschedules the object for processing. This instantaneous rescheduling occurs within the codec-cycle deadline time (hard-duration defined on average) of each object. This method is unique in that individual frames are scaled according to both hard and soft deadlines. In the above example, it was assumed that all frames in Object-B required the same amount of time when the workload for 500 ms overlap was averaged by Object-A. This may not be true when the frame for Object-B requires more time during the actual overlap, or when Object-B is not given an average amount of time. Thus, the frame closest to its codec-cycle deadline receives the highest priority.

If the predicted runtime violates a user-defined time requirement, the QoS manager will take one of several possible actions.

In a single DSP configuration:

(Level 1) Simple Binary Cut Off: This causes an automatic frame-drop. The object of interest must be able to indicate whether the frame drop will cause a destructive effect.

(Level 2) General reduction in allocation runtime of lower priority objects by pre-emption of objects at the end of the allocated time. This may or may not cause frame-drop.

(Level 3) The object is required to have the ability to accept QoS commands such as scaling back quality of output data.

In a multiple DSP configuration:

(1) At the end of each QoS time-slice, a message with load-data is sent from each DSP to the GPP.

(2) GPP goes to redistribution of objects only in the event of production dead-line errors. Reassignment of this task is accomplished by the GPP (ORB layer) after receiving "load-data" from the serving DSP. However, to reduce task switching time, it is highly desirable that all DSPs operate from a common cluster of external memory space.

All objects running within the iDSP system must be critical in execution time. DSP objects can be divided into three types: compression (encoding) of data, decompression (decoding) of data, and data conversion (pre or post processing of data to objects). The data represented, these blocks are called input data frames, the object processes the input data frame and generates an output data frame, as any computational data, both the input and output data frames are bound for the size and amount of processing There may be an accurate determination of the maximum amount of processing that the DSP, or any other computer for that matter, should perform on that input frame based on the size of any given input frame.

Before this is integrated into the iDSP system, each object needs to declare the worst case runtime for that object for a single frame. This worst case runtime is used to calculate the runtime of the first input data frame so that the object can be started. The QoS manager cannot characterize the input data frame before the object is performed. Since the encoder and decoder objects perform rarely in the worst case scenario, the first input frame will be significant (because this should be predicted as the worst case). This worst case schedule is likely to generate more time than the actual runtime for the first frame. This is only a problem if the actual runtime is more than the worst case schedule.

As described above, the processing time of the algorithm object will vary between input frames. Initially, the iDSP-QoSM will begin with the worst case for the first data input frame. After the first frame, the QoS management program will predict the processing time for the next input frame based on the characteristics of the algorithm and the measured processing time for the first frame. Every subsequent frame, the QoS manager predicts an approximate processing time based on the semantics and history of the algorithm object. For example, an encoder object uses object semantics (eg, I, P, and B frame types) along with the average encoding time of previous similar input frames to predict future encoding time requirements. Encoder objects work on the same size input frame whenever they are scheduled to run. The variation in processing time comes from factors such as activity levels in frames, the degree of movement between frames, and the like. However, these deviations are bound. Therefore, the processing time between two frames will have a finite maximum difference that can be added to the predicted processing time to determine the worst case processing time for the next frame. See FIGS. 5-6.

The decoding object is typically a variable size input frame present. The processing time of the input data frame is directly proportional to its size. To determine if there is an increase in the next frame processing time, the QoS manager will check the magnitude of the difference between the current and next input frame size. Similar independent variables, such as encoders, also hold for the decoder. That is, the difference in processing between two semantically similar frames is bound. The maximum or worst case processing time for a decoder is the maximum possible buffer defined for the object. See FIG. 7.

Transform objects perform similarly to encoder objects in that they always work on the same size. Each frame always takes the same amount of processing time and passes through the input frame once. Therefore, the processing time per input frame always remains constant.

Each object will receive a relative time from the user application that the frame passed must be completed by the object. One example is that the application is characterized that this frame must be processed within the next 7mS. Due to the lack of a common software clock between the host GPP and the DSP, the deadline can only be specified within a relative period of time. Assume that the data frame between the host and the DSP is crucial. The iDSP system maintains an internal clock that receives a timestamp when the data frame arrives and calculates the next expected processing time. After calculating the expected processing time, the QoS manager now schedules data frame execution.

Before an object can be scheduled, the QoS manager determines the proper order of execution of the object compared to other concurrent objects. If no other object is processing the input frame, the object frame is immediately scheduled for execution. If there are other objects running, the QoS manager determines the order of execution taking into account the priority, expected deadline, and hard or soft real-time requirements of each required object. See FIG. 8.

When multiple objects with different runtime priorities are combined on the same DSP, the QoS manager will calculate the expected runtime for each object based on the specific runtime calculation of each object. Next, different things are scheduled based on the scheduling object TBD. Three scheduling scenarios are possible:

(1) All objects operate until complete on a given input data frame, and complete within application-specific deadlines. This scenario is shown in FIG. 9, and it is important that all objects in the drawing are completed before the deadline of each object. If all objects are completed before each deadline, the work required by the QoS manager will be minimal.

(2) The processing load increases on one or more objects (eg Object-B) but does not miss the expected deadline for the next object. It is possible for the load to increase on one or more objects, such as Object-B. Depending on the object, missed deadlines may be acceptable if sequential data frames of the same object are processed within their deadlines. One example may be in the H263 encoder which takes the longest "I" frame calculation. The frame following the "I" frame is always a "P" frame and generally has much less processing requirements. This allows "I" frame processing to cycle from the next "P" frame processing. Therefore, if there is enough processing space in the next frame, missing the deadline in one frame is not catastrophic.

After the deadline for Object-B has elapsed, the overall system effect must be determined. If missing the deadline by Object-B does not miss the expected deadline for the next objects, the overall system risk is minimal. See Figures 10-11.

(3) The processing load increases on one or more objects (e.g., Object-B), but this misses the expected deadline for the next object. See Figure 12.

In this case, missing the deadline by Object-B causes the expected deadline for the next objects to be missed. Even in this case, the overall system risk may or may not be minimal. Each of the objects working at the same time can steal cycles from the sequential frames, thereby avoiding the domino-effect of missed deadlines.

iDSP-QoSM presents a set of rules for adjusting soft-duration. This set of rules is designed to limit the snow-balling effect of missed deadlines from a single critical missed deadline. (1) Every algorithm object provides the QoS management program with the maximum number of frame-drop / second allowed. (2) Each object updates the 'missed due' number of operation counts to a moving average after each processing period. (3) If an object exceeds the missed deadline limit, change the object's priority to the highest value. The original priority is restored once when the number falls below the limit. (4) All sequential frames that miss the deadline after the limit are dropped. This temporarily causes QoS to degrade to the next immediate level. And, in QoS, this momentary drop is reported to the client. (5) A frame is dropped as a rule only if the DSP has not started the object in question until after that time has passed.

3d. Deceleration Control for Periodic Media Rendering

For a given algorithm object, iDSP-QoSM assumes that there is only one request in the wait queue at any moment. Media streams generally have a periodic deadline (e.g., 30 frames / second during the video stream) specified as the quality of service required by the QoS management program. The components of audio and video rendering in the media system can mitigate the frame to adjust for changes in arrival time while allowing the frame to arrive just before the schedule. However, this buffer is limited and the upstream component of the media system must carefully reduce the relative speed at which frames are processed.

Two mechanisms are provided by iDSP-QoSM to reduce the processing speed of algorithmic objects.

(1) The client of the DSP algorithm object controls the speed of invoking the processing function (server) of the algorithm object. This may result in a sub-optimal operation of the scheduling algorithm of the QoS management program if the request to be completed is made in a time period. For example, consider Algorithm Object A, where buffer A1 should be processed within period T1 and buffer A2 should be processed within period T2. In the diagram where T1 and T2 are two consecutive periods, [x] indicates the arrival of buffer x and {x} indicates the completion of the processing of buffer x. See FIG. 31A.

(2) The QoS management program controls the trottling of the media stream. This mechanism allows the client to invoke the processing functions of an algorithm object into the input buffer as soon as possible. The QoS manager will attach a "start-due" to the input buffer. The scheduler only schedules this buffer after the "due to" date. The client blocks until processing of the current buffer is complete. See FIG. 13B.

Thus, in both cases there is at most one request per algorithm object, in the QoS manager ready queue.

4. Memory Paging

In order to best run multiple algorithms or any processor on such a problem on the DSP, a set of rules is established so that system resources are shared equally among the algorithms. These rules specify access to the processor's peripherals such as DMA, internal memory and scheduling methods for algorithms. Once a set of rules is accepted, a system interface can be developed for the plugging algorithm, so that system resources can be accessed. The common system interface alone can concentrate on algorithm development and lacks system support, thus providing well-defined limits for algorithm developers to develop algorithms faster. An example of such an interface is Texas Instruments' iDSP media platform DSP framework. All access between the algorithm and the TMS320C62XX DSP takes place through this framework.

Texas Instruments' XDAIS standard requirements establish rules that allow plugging of one or more algorithms into the iDSP media platform and allow system integrators to quickly assemble product quality systems from one or more algorithms. The XDAIS standard requires that algorithms meet a common interface requirement called the Alg interface. Several rules are imposed by the XDAIS standard, which is most characteristic of algorithms specifying memory directly or accessing hardware peripherals directly. System services are provided through a single common interface for all algorithms. Thus, the system integrator only provides a DSP framework that supports the Alg interface with all algorithms. The Alg interface also provides algorithm developers with a means of accessing system services and calls to their algorithms.

The algorithm must exactly define the internal memory requirements. This is a need for a paging architecture that supports multiple algorithms that access the same space in internal memory. XDAIS compliant algorithms need to specify their internal and external memory requirements.

Internal (on-chip) memory is divided into two areas. The first area is the system overhead area, which supports OS data structures for specific DSP system structures. The second region is used by the algorithm only when it is scheduled to run. The second area of memory is called an algorithm on-chip work area, which may alternatively be described as a data overlay or data memory page. See FIG. 14.

To determine how much memory is available for the algorithm on-chip workspace, system developers take the total amount of available internal data memory space and subtract the amount needed to support system software such as OS support and data support for the paging architecture. . OS structures such as tasks, signaling devices, etc., must be set by the system DSP designer to a maximum size that supports the total number of algorithms the designer wants to run simultaneously at one time. This keeps OS support overhead to a minimum and increases the algorithm workspace.

For the algorithm to run in this environment, the internal memory requirements must be smaller than the size of the work area. Otherwise, the system integrator cannot integrate the algorithm, and there is a limit to having only one page per algorithm. This architecture does not support multiple pages for the algorithm.

The algorithm workspace is divided into three components: stack (forced), persistent memory and non-persistent memory. Sometimes there is a fourth component, described below, that handles only the read-only portion of persistent memory. See FIG. 15.

The algorithm only uses on-chip work area during execution. When the algorithm is scheduled to run, the DSP system software will move the algorithm's workspace from an external storage location (shadow memory) to an internal workspace on chip. When an algorithm generates control, the DSP system software will determine which algorithm will run next, and if it is the same algorithm, there is no need to move the work area. If the next algorithm is another algorithm, the current working area is stored in the external memory as a shadow position and the working area of the next algorithm is moved. See FIG. 16.

The entire working area of the algorithm is shifted in context switching. Only the used portion of the stack and persistent data memory is moved. The stack of algorithms is at the highest level (almost unused) when the algorithm is at the highest level in the call stack. In other words, the algorithm is at its entry point.

The ideal context switch for the algorithm occurs when its stack is at the highest level because it means there is less data to move the off-chip into shadow storage. See FIG. 17.

The data page architecture of the preferred embodiment requires that the context switch be the most efficient. Context switch processing overhead can be taken from the time the DSP can perform the algorithm. Since the algorithm is the best time for context switching when the algorithm is at the call boundary, the preemption algorithm must be minimized. When the stack is larger than the minimum, preempting the algorithm will degrade the overall system. It will be allowed to preempt on a necessary but very limited basis. See FIGS. 18-19.

A special case of an algorithmic workspace is when an algorithm requires read-only persistent memory. This kind of memory is used as the lookup table used by the algorithm. Since these memories are never modified, they need to be read only, not write. This asymmetric page transfer reduces overhead by context switching of the algorithm.

This data paging architecture allows a single algorithm to be instantiated more than once. Since the algorithm specifies what is required for internal memory requirements, the DSP system integrator may be one or more instances of the same algorithm. The DSP system software continues to know when to schedule multiple instances and each instance of an algorithm. The limit on the number of instances is how much external memory the DSP system has to maintain the shadow versions of the algorithm instances.

The DSP system software must manage each instance so that it is correctly matched to the algorithm data when scheduling the algorithm. Since most DSP algorithms are instantiated as tasks, DSP system software can use task environment pointers as a means of managing algorithm instances.

5. Chained Data Flow

Preferred embodiments of the data flow rely on integrating processing elements, providing them to a shared memory space, and routing data directly between processing elements without intervention by GPP. Such a system is shown in FIG. 21.

When processing element PE _a has finished processing the set of data, it writes the resulting data to a predetermined output buffer in shared memory. PE _a then informs the next processing element PE _b in the chain via the appropriate control path. The notification indicates that the shared memory buffer PE _b should be used as input. PE _b then reads data from the input buffer for further processing. In this way, data is passed between all processing elements until all data is consumed.

As described above, a set of buffers are used to communicate data between two processing elements and include an I / O channel between the elements. Multiple I / O channels may exist between any two processing elements that allow multiple data streams to be processed simultaneously (ie, in parallel) by the system. Fig. 22 shows an example of parallel processing of multiple data streams s1 and s2.

Serial processing elements connected by I / O channels constitute a channel chain. Multiple channel chains can be defined within a particular system. In the case of an intermediate chain processing element each input channel has an associated output channel. The terminal processing element has only an input channel or only an output channel.

The input channel of the processing element defines a buffer from which data is read. The output channel of the processing element defines the buffer into which the data is written as well as which the processing element later notifies. The types of control messages between the data processing element and the central control processor (CCP) are as follows.

(1) Status messages: data stream processing such as start, stop, stop, stop, resume and so on.

(2) Quality of service messages: time stamp, system load, resource free / busy, etc.

(3) Data stream control messages: start, stop, stop, resume, rewind, etc.

(4) System load messages: task driving, number of active channels, channels per processing element, etc.

In a preferred embodiment, the creation and association of an I / O channel with processing elements is statistically defined through a structure file that can be read at system initialization time. For each bitstream type to be processed, the structure file defines the channel chain (ie, data path) that connects the appropriate processing elements. Collective processing of all processing elements in the channel chain causes data to be consumed completely.

In the case where multiple data paths exist in a given bitstream, alternative or backup channel chains may be defined. The bitstream may be routed to any processing element of the main channel chain if it is not available. Determination of the bit stream type in runtime and dynamic QoS analysis selects the channel chain through which data is routed. At runtime, all legal channel chains in the system are fixed and cannot be modified.

In another preferred embodiment, channel chains for other bitstreams may be dynamically constructed when a new bitstream arrives at the communication processor. Bitstream information derived at runtime is sent to the CCP via a control message, which determines the necessary processing elements and dynamically allocates I / O channels between them. This approach allows resources to freeze or come online at run time so the system can adapt automatically.

In shared memory heterogeneous systems, data flows between processing elements through external shared memory without being interrupted by the CCP. Data never appears on the bus, so the speed of data processing is determined by the shared memory access time, not the bus transfer time. Interference with CCP is also minimized, so CCP response and processing delays are eliminated from the overall data flow time. This increases the throughput of the system by minimizing the data transfer time between processing elements.

5a. Yes

An exemplary use of the data flow techniques described herein is for media processing systems. The system initiates and controls the stream of broadband media for processing such as decoding, encoding, translation, conversion, scaling, and the like. Communication media such as cable modem, DSL or wireless can be used to process media streams originating from local disks or from remote machines / servers. 23 shows an example of such a system.

The media processing system of FIG. 23 has five processing elements as follows:

(1) DSL or cable modem I / O front-end DSP

(2) media processing DSP

(3) video / graphic overlay processor

(4) H.263 decoder task

(5) color space converter task

The H.263 stream input to the front-end I / O DSP follows the channel chain defined by the number arcs 1 to 3. Each channel connects two processing elements and consists of a set of I / O buffers used to transfer data between the elements. Control flow is shown through the shaded arc.

H.263 streams flow from the I / O front-end DSP to the channel 1 I / O buffers defined in global shared memory. The I / O front-end DSP notifies the destination processing element associated with channel 1, the H.263 decoder task on the media processing DSP, that the input buffer is full and ready to be read. The H.263 decoder task reads data from the channel 1 I / O buffer and writes the resulting YUV data to the channel 2 I / O buffer in local shared memory.

Note that the channel can be an interprocessor or an intraprocessor. Data may be transferred between processors via global shared memory (interprocessor) or local shared memory (intraprocessor) limited to a given processor. In Figure 4, channels 1 and 3 are interprocessors and channel 2 is an intraprocessor.

6. Modifications

Preferred embodiments may be modified in various ways within the scope of the appended claims.

According to the present invention, in a client-server system that performs a two-stage server task scheduling wherein information of a client deadline step is used for subtask server scheduling in a second step, the object broker of the system requests the client to maintain data in the coprocessor. The main processor bus congestion can be avoided by disrupting calls and responses and providing a method of managing server memory for multitasking and data flow through the shared memory of multiple coprocessors.

Claims (7)

(a) a first step of scheduling on a client to set a real time deadline for a task of a server coupled to the client; And (b) a second step of scheduling subtasks of the task on the server; Wherein the second scheduling step uses the real time deadline of step (a). The method of claim 1, (a) the task comprises decoding a media stream, (b) the subtask includes frame decoding for a frame of the media stream. (a) disrupting the first client request return and the second client request call; And (b) connecting the output of the first server object to the input of a second server object, wherein the first server object and the second server object correspond to first and second client requests, respectively; Object request broker method for a client-server system comprising a. The method of claim 3, (a) The connection is by creation of a buffer for intermediate results (output of the first object and input of the second object) at the server. (a) allocating a first portion of processor memory to processor overhead; And (b) allocating a second portion of said processor memory to a task workspace; And wherein the second portion can be occupied only by one task at a time. The method of claim 5, (a) the second portion of the memory comprises a stack component, a persistent memory component and a non-persistent memory component. A data flow method in a heterogeneous system wherein a bus is connected to each of a control processor and a plurality of processing elements, (a) transferring data between the processing elements using a common memory separate from the bus How to include.