TW514832B - Multiprocessor object control - Google Patents

Abstract

A client-server system having server task scheduling in two phases with client deadlines phase information used in a second phase subtask server scheduling. Also, an object broker for the system with collapsing of client request calls and returns to maintain data in coprocessors, and server memory management for multitasking and data flow through a shared memory for multiple coprocessors to avoid primary processor bus congestion.

Description

514832 A7 B7 V. Description of the Invention (1), Related Applications This patent application has priority over provisional applications filed on April 26, 2000. Case numbers 60 / 199,753; 60 / 199,755; 60 / 199,917; and 60 / 199,754. BACKGROUND OF THE INVENTION The present invention relates to electronic devices. In particular, the present invention relates to multiple processors and digital signal processor distributed objects and methods. BACKGROUND OF THE INVENTION The growth of the Internet combined with high-speed Internet access has led to the decentralized computer system becoming mainstream. Common object request broker architecture (CORBA) and distributed component object model (DCOM) standards have tended to simplify object-oriented network programming and component software methods. Therefore, the client application can call the remote server object to provide data or functions and simplify the application design; Figure 24 shows the general remote program call architecture. In fact, object-oriented programming encapsulates detailed information and thereby presents only the object interface for querying or interacting with other objects, allowing such decentralized computer systems. Printed by the Consumer Cooperatives of the Central Standards Bureau of the Ministry of Economic Affairs (please read the notes on the back before filling this page) The core of CORB A is to apply for the intermediary of objects that provide `` busbar π '' for interaction between this machine and remote objects (Object request broker; ORB). CORBA objects are a set of methods and interfaces. The client of a CORBA object uses the object's reference as a method to call the handle as if the object were in the client's address space. -4- This paper scale is applicable to the Chinese National Standard (CNS) A4 specification (210X 297 mm) 514832 Printed by the Consumer Cooperatives of the Central Standards Bureau of the Ministry of Economic Affairs A7 B7 V. Description of the invention (2) ORB is responsible (in the possible remote servo On the server) to find the implementation of the object, prepare the object to receive the call request from the client application, transfer the request (eg, parameters) from the client to the object, and pass any reply from the object back to the client. The interaction between the object implementation and the ORB is carried out through an ORB interface or an object interface or an object adapter (0A). Figure 25 shows the overall CORBA architecture. An interface definition language (IDL) defines an object interface, which includes methods to be invoked by the client while hiding detailed information (data, implementation), just like the conventions of object-oriented programming. IDL usually provides data encapsulation, polymorphism, and inheritance. As shown in Figure 24, the way in which the client invokes the object function is to call the client stub (proxy) first; the stub marshals the call parameters to a (marshal) message; the wired communication protocol sends the message to the server stub (skeleton); unmarshal unmarshal call parameters from the message, and call the function of the object. The top layer of Figure 25 is the basic programming architecture, the middle layer is the remote architecture, and the bottom layer is the wired protocol architecture. Client program and server object program developers use the basic programming framework and remote architecture to create interface pointers, object references, and handle meaning between the client and the server. Wired protocols effectively extend the remote architecture between various hardware devices. As described in "DCOM and CORBA Side by Side, Step by Step, and Layer by Layer" by Cheung et al., The following five files can be used to create a simple application to use Paper size applies Chinese National Standard (CNS) A4 specification (210X 297 mm) (Please read the precautions on the back before filling this page) Order 514832 A7 ___B7 5. Description of the invention (3) Client with CORBA function And remote objects of the server processor: (l) IDL file, which is used to define the interface of the object. The IDL compiler is used to generate the client's stub and the object's basic structure code with the interface header file for users. And server use. (2) The implementation title file is a server implementation class used to derive objects from the interface. In essence, the implementation class (inheritance) and the interface created by the IDL compiler Class associations. (3) — implementation of a servo class method. (4) the main program of the server; this program individualizes instances (objects) of the server class. And, (5) Client application, which calls the object method to the client stub by calling. Printed by the Consumer Cooperatives of the Central Standards Bureau of the Ministry of Economic Affairs. As far as the static object activation process is concerned, after editing and before execution, CORBA will temporarily store the actual For the implementation of the storage mechanism (rep0sit〇ry), the association between the interface name and the path name (see Figure 25). As far as dynamic objects are concerned, the IDL compiler also generates interfaces. Type information in each of the methods' and store the type information in the interface storage mechanism (rep0S1t〇ry). The client can query the interface storage mechanism ㈣ heart to obtain the runtime of the specific interface (Runtime ) Information, money uses the execution period information to dynamically establish financial and call object methods through the dynamic process. Similarly, on the server side, the dynamic basic architecture interface allows the client to call object operations, and implement this There is no compile-time knowledge of the object type at the time of the request. Figure 26a shows the top-level (basic programming framework) operation of the object client request and its method activation process. Status, and server establishment of the object instance (mstance) and availability to the client. Specifically, this paper size H uses the Chinese National Standard (CNS) Α4 specification (210X297 mm) 514832 Employees of the Central Standards Bureau of the Ministry of Economic Affairs Cooperative printed A7 B7 V. Description of the invention (4) The activation process of the object (Activation) is as follows: (1) The client calls the client stub static function applicable to the object interface. (2) 〇rb startup includes the interface that supports the object Server of the object. (3) The server program personalizes the object and temporarily stores the object reference. (4) 0RB sends the object reference back to the client application. Then, for the object method initiation process [UJ2], the client call will eventually invoke the object interface method of the method in the server. If the method returns packets, the server passes those packets back to the client. Figure 26b shows the c ORB A middle layer (remote architecture) that contains the activation process of the object: (1) After receiving the call, the user terminal delegates work to the ORB. (2) 〇RB consults and implements a repository to map the call to its server path name, and starts the servlet program. (3) The server uses individual objects, and also establishes a unique test ID to obtain object references. Its temporary object reference and orb. The constructor of the server class also creates instances of the basic architecture class. (5) The 0RB transmits the object reference back to the client, and also establishes an instance of the client stub type, and temporarily stores the instance in the client stub object table with the corresponding object reference. (6) The client stub sends the object reference back to the client. Then, the client-side activation process of the object method is performed as follows: [1] Upon receiving the client-side call, the client-side stub creates a request for a virtual object, marshals the call parameters to the virtual object, and calls to place the virtual object in In a message to the server ’s channel, and waiting for a reply. [2] When the message arrives at the server, the ORB is responsible for finding the target basic architecture, rebuilding the request virtual object, and forwarding the request virtual object to the basic architecture. [3] Basic structure This paper size applies to China National Standards (CNS) A4 (210X297 mm) (Please read the notes on the back before filling out this page) Printed by the Staff Consumer Cooperative of the Central Standards Bureau of the Ministry of Clothing and Commerce 514832 A7 B7 V. Description of the invention (5) Unmarshalling parameters from requesting virtual objects, calling server object methods, marshaling and returning 若 (if any), and returning from the basic framework method. The ORB builds a reply message and places it in the transmission buffer. [4] When the reply arrives at the client side, the ORB call returns after reading the reply message from the receive buffer. Then, the client stub is unblocked and sent back to the client and it is sent back to the client to complete the call. As shown in Figure 26c, the lowest layer of the object startup process (wired protocol architecture) includes: (1) After receiving the request, the client ORB then chooses a machine that supports the object, and sends the request to the server via TCP / IP. Server side ORB. (2) When the server-side ORB starts the server, the server will personalize the object, call the ORB Constructor, and call the create function. In the create function, create a Socket end point, assign an object identity to the object, and create an object reference. The object reference contains the interface and implementation name, reference identity, and end. Point (endpoint) address. Temporary Object Reference and ORB. (3) When the object reference is returned to the client, the client stub retrieves the endpoint address and establishes a socket connection to the server. Then, the method is initiated according to the following procedure: Once a call is received, the client stub marshals the parameters in a common data representation (CDR) format. [2] Send the request to the target server through the established socket connection. [3] The target basic architecture is identified by reference identification (identity) or interface instance identifier (interface instance identifier). And [4] After calling the actual method of the server object, the basic structure is marshaled back in CDR format. -8- This paper size applies to China National Standard (CNS) A4 specification (210X297 mm) (Please read the note on the back first) (Please fill in this page for matters)) Rongrong-Printed by the Consumers' Cooperative of the Central Bureau of Standards of the Ministry of Economic Affairs 514832 A7 B7 V. Description of Invention (6) 値. CORBA's real-time extensions typically provide quality of service (QoS) aspects such as predictable performance, secure operations, and resource allocation. For example, "Applying Adaptive Middleware to Manage End-te ^ End QoS for Next · Generation Distributed Applications" by Gill et al. 0 has adopted a CORBA component as a relay type, and can use the associated component to implement a component implementation language (component implementation definition language; CIDL). Figure 27 illustrates the programming steps. DCOM also has three layers, and its architecture is somewhat similar to CORBA.

Notenboom USP 5,748,468 and Equator Technologies PCT published applications wo 99/12097 both describe methods for allocating processor resources to multiple jobs. Notenboom considers the work of the host processor plus the auxiliary processor plus the auxiliary processor based on a prioritized system. Equator Technologies schedules processor resources based on the amount of time spent on the job and each job presented at a supported service level (processed as a resource consumption rate), and if there are sufficient resources available to support the service Level, the resource manager will grant permission to join the job. A system with two or more processors (each with its own operating system or BIOS) includes a system with a widely separated processor connected via the Internet, and also includes A system in which two or more processors are integrated on the same semiconductor wafer such as one or more DSP-type RISC CPUs. -9- '^ Paper size applies to Chinese National Standard (CNS) A4 specification (210X297 mm) --- (Please read the notes on the back before filling out this page) Printed by the Staff Standards Bureau of the Central Standards Bureau of the Ministry of Clothing and Economy A7 B7 V. Description of Invention (7) The XDAIS standard specifies the interface of the algorithm executed on the DSP; this provides objects that can be used multiple times. XDAIS requires the algorithm to implement a standard interface IALG with extended functions to be suitable for implementing the algorithm. XDAIS also requires compliance with specific flexible rules such as reconfigurable code and naming conventions. The client application can manage the instance of the algorithm by calling a function indicator table. In line with the XDAIS standard / criteria, algorithm developers can develop or transform algorithms so that it is easy to insert DSP application program architectures such as the iDSP Media Platform DSP Framework. Due to the real-time service requirements of all streaming-media applications, a quality of service (QoS) manager is required in the network nodes (clients / servers). Pay-as-you-go media applications must handle heterogeneous codecs (encoders / decoders) and filters with unique rendering deadlines. These applications should also be able to leverage and interpret human perception to reduce service quality non-missively. These applications should be able to handle a reasonable amount of jitter during the processing and rendering cycles. For example, in a video application, the frame rate of the rendering must be maintained at 30 frames per second (fps) to interpret a frame period of 33 milliseconds (ms). However, when the application negotiates with the server, it should be able to withstand limited transient changes. Furthermore, with 30 fp s, human visual perception can withstand a frame drop of about 6 frames / second. In addition, client applications should be able to support non- erroneous performance degradation (instantaneous frame degradation) and be able to maintain constant rendering within specific tolerances negotiated with the server. The QoS manager is a mechanism that provides the necessary functions and functions to implement such real-time systems. -10- This paper size applies to Chinese National Standard (CNS) A4 (210X297 mm) (Please read the precautions on the back before filling this page)

Order printed by the Consumer Cooperative of the Central Standards Bureau of the Ministry of Economic Affairs 514832 A7 B7 V. Description of the invention (8) With the proliferation of broadband communications such as data subscriber lines (DSL) and cable modems, new markets have been formed, and consumer devices Achieving an unprecedented amount of data for processing and consumption has prompted more efficient data processing, path selection and processing technology to grow with it. Figure 20 shows a diagram of how the data flow is processed through the current heterogeneous system. Each data transaction is coded to present the time sequence. Each data transaction must pass through the system bus under the control of the Central Control Processor (CCP). CCP initiates transactions by sending messages or triggers to various processing elements in the system via the control path. Figure 20 shows the processing elements as separate processors (e.g., DSP, ASIC, GPP, etc.) capable of performing a defined set of tasks. This is why each processor shown has its own memory. The processing elements may also be individual tasks performed on the same processor. In some cases, the same data must pass through the system bus multiple times (for example, transactions 1 and 2, 3 and 4 and 5 and 6). In such systems, the data must pass through the system bus a total of 2 + (2 X η) times, in this case 6 times. Every time through the system bus and CCP intervention, the internal operation time (overhead) of the data flow will be introduced, and the overall system processing capacity will be reduced. The internal operation time of the data flow (overhead) will have a negative impact on the amount of data moved through the system within a given time range, thereby limiting the amount of data that the system can process. Such systems are likely to perform less useful work than otherwise indicate the sum of the functions of their elements. Summary of Invention -11-This paper size is applicable to Chinese National Standard (CNS) A4g (210X297 mm) (Please read the precautions on the back before filling out this page) Threading Department of the Ministry of Economic Affairs Central Standards Bureau Consumption Cooperation Du printed A7 ~- --------- B7 _ V. Description of the invention (9) " --- The present invention discloses a master-slave, erver) system with functions above mm, which includes a two-stage scheduling of a server operation; called牛 ect 长 仲 7 丨 (obJect request broker), which is applicable to a master-slave system with a working link on the server DSP, which is used to divide the internal memory into two working areas that belong to a single execution task ( w〇 ... pace) to handle the internal operating time (overhead) for multi-tasking processors. Z It & 'Data flow in a heterogeneous system, which includes a central k-processor and additional The bus core of the processing element's shared memory is connected to the processing element to avoid the central control processor bus. Brief description of the drawings, The drawings are provided to make the invention easier to understand. Figure 1 shows the DSPORB architecture of the preferred embodiment. Figure 2 illustrates IDL compilation. Figures 3 to 13 show timing diagrams for Quality of Service (QoS). Figures 14 to 19 show the preferred embodiment memory analysis & Figure 20 shows the known heterogeneous system data flow. 21 to 23 show the data flow of the preferred embodiment. Figures 24 to 27 illustrate CORBA. Detailed description of the preferred embodiment 1. Overview The preferred embodiment system typically has a main processor executing a client application plus one or more servers processing server algorithms, and includes algorithm objects. Object request intermediary, object request intermediary service quality control, algorithm object memory paging and algorithm-12- This paper size applies to China National Standard (CNS) A4 specification (210X297 mm) '--- (Please read first Note on the back, please fill out this page again.) Order printed by the Consumers Cooperative of the Central Standards Bureau of the Ministry of Economic Affairs 514832 A7 B7 V. Description of the invention (1) Information process of the articles. The preferred embodiment, called iDSPOrb, is applied to a system with a main processor and one or more DSP processors.

iDSPOrb is a high-performance DSP object request broker (DSP) that supports the creation of DSP objects and access to DSP objects from another DSP in a general purpose processor (GPP) or multiprocessor environment.

Object Request Broker; DSPORB). iDSPOrb has a general architecture and operates similar to CORB A. iDSPOrb has the following DSPORB functions: (1) iDSPOrb supports object binding (Object binding) and activation process (DSP object program call) across processor boundaries. (2) iDSP0rb provides a GPP party agent interface composed of a compilation period header and a static boot process stub, and a run-time dynamic boot process interface. (3) iDSP0rb provides a DSP algorithm interface (stub and title) to build an iDSP server. (4) iDSPOrb provides synchronous and asynchronous triggering process. (5) iDSP0rb provides guaranteed immediate quality of service (QoS). (6) iDSP0rb provides frame type and data stream type processing. (7) iDSP0rb provides object link data flow (intermediate results left in DSP memory). (8) iDSP0rb is implemented on a high-bandwidth multi-channel GPP / DSP I / O interface. Figure 1 shows the iDSPOrb architecture of the GPP / DSP dual-processor configuration configuration, where the GPP is used as the "user end" and the DSP is used as the "server". -13- This paper standard applies to China National Standard ) A4 specification (210X297 public envy) (Please read the precautions on the back before filling this page)

514832 A7 B7 V. Description of the Invention (11) The Quality of Service (QoS) administrator in the iDSP system (herein referred to as iDSP-QoSM) is a way (in the server) to provide negotiation for client applications Mechanism of service level, which provides guaranteed service quality with pre-determined downgrade principles communicated to the client. iDSP-QoSM has the following characteristics: (l) iDSP-QoSM is defined within the limited content of nodes residing on the network (inside the nodes). It assumes that an appropriate QoS administrator is responsible for controlling the node's internal (network) communication. (2) The defined iDSP-QoSM is suitable for a multi-processor environment with a load sharing function. The preferred embodiment iDSP-QoSM executes the following functions: (1) Monitoring the constant processing load on the server in the system. (2) Distribute the load from the overload server to its peer server. (3) Negotiate service requirements with client applications to facilitate temporary storage of any additional load on the server. (4) To predict the future load on the server based on the specific characteristics of the individual objects served by the server. (5) The algorithm execution period prediction is based on the processor time period, not based on the processing time: In this way, the algorithm execution time prediction is not constrained by the processor's operating frequency. Printed by the Consumers' Cooperative of the Central Standards Bureau of the Ministry of Economic Affairs (Please read the precautions on the back before filling this page)

Texas Instruments TMS320C62XX DSPs have a built-in (on-chip) data memory. With the exception of TMS320C6211 (and its derivatives), the TMS320C62XX DSP does not have a data cache (Data Cache) for external memory (chip) access efficiency. The internal memory system is the highest level in the TMS3 20C62XX DSP data memory hierarchy. Therefore, all algorithms performed on the TMS320C62XX DSP want to use internal memory as its data workspace, because internal memory is the highest level of efficiency for accessing data memory. -14- This paper size applies to the Chinese National Standard (CNS) A4 specification (210X 297 mm) 514832 A7 B7 V. Description of the invention (12) Generally speaking, the developed DSP algorithm assumes that it has the entire DSP processor. So it has all the internal memory of the DSP. This makes it extremely difficult to integrate several different algorithms to be the same (homogeneous) or different (heterogeneous). Algorithm developers need a set of rules for common access methods and the use of system resources such as internal memory. The preferred embodiment provides a method for increasing processor utilization when executing multiple algorithms on a DSP without a data cache by using the data paging architecture of the DSP's internal memory. Using Texas Instruments XDAIS standard can realize the development or conversion of DSP algorithms in accordance with the data paging architecture. This standard requires algorithm developers to define one or more memory areas of all data memory used to support the algorithm. Among these user-defined memory areas, the algorithm developer will choose one or all of them to execute in the internal memory of TMS320C62X DSP. Within the scope of the application's DSP system software, the internal memory is divided into system support and data workspaces (paging). All algorithms in the DSP application share this workspace and own the entire workspace during execution. According to the content switching between the two algorithms, the DSP system software will separately process the transfer between the work area and the external shadow memory of each algorithm. The preferred embodiment provides: Printed by the Consumer Cooperatives of the Central Standards Bureau of the Ministry of Economic Affairs (please read the notes on the back before filling this page) (1) No data cache between two or more DSP algorithms The internal data memory is shared in the local DSP to increase processor utilization. (2) When accessing data memory, multiple algorithms are executed from the same shared internal memory so that each algorithm can enjoy the maximum efficiency in the TMS320C62X DSP environment. -15- This paper size applies to Chinese National Standard (CNS) A4 (210X297 mm) 514832 A7 B7 V. Invention Description (13 Printed by the Consumer Cooperatives of the Central Standards Bureau of the Ministry of Economic Affairs. Any unit of the DMA utility that has internal access to the processor operates on the processor. Mind 83. ⑷ Only implement data efficiency paging architecture on data input reduction. How to provide support for read-only algorithms Asymmetric pagination of data. The data flow in the application may vary from algorithm to algorithm, and the preferred embodiment provides that the data is stored in one or more DSp, and, is executed for each algorithm. Shipped to / from GPp via bus No. 2. DSP ORB in dual processor configuration configuration Figure 1 shows the comparison of dual processor configuration configuration including general purpose processor (GPP) and digital signal processor (DSP). The best embodiment ("iDSPOrb") architecture, where the GPP is used as the client, and Dsp is used as the "server". Please note that iDSPOrb includes Quality of Service (QoS) administrators. 1 shows a client application that calls two DSP algorithm objects' 'a " and ,, B,'. First, iDSPOrb provides proxy (user stub) objects "a" and "b" on GPP. Object binding. For example, "a, and " B" may be the DSPIDL interface extension of the decoder (DEC), as follows: module DEC {interface IDecoder {int process ([in] BUFFER input, [ out] BUFFER output); -16 This paper size is applicable to Chinese National Standard (CNS) A4 specification (2 丨 〇297mm) (Please read the precautions on the back before filling this page) Order 514832 A7 B7 V. Description of the invention ( 14) interface A: IDecoder {} interface B: IDecoder {}} will use the algorithm interface provided by the DSPIDL compiler to build a DSP application (called an iDSP server): DEC-A—Handle DEC-A create (IALG_Params * p); int DEC_A_decode (BUF_Handle in, BUF_Handle out); It will also use the agent interface provided by the DSPIDL compiler to build GPP application programs: DEC A * DEC-A a create (DSPORB a Params * p); int DEC-A-decode (DSPORB-Buffer * in, DSPORB-Buffer * out); iDSP Orb or dynamic invocation interface process. At runtime, "a" can be called from the GPP client application to process the buffer. Then, send this data to the actual object "A" on the DSP side. Use the object link data flow to 'connect the output of' A 'to the input of' B ', so the intermediate data buffer will not be transferred back To GPP., B "Call, B" causes another process step to transfer the data back to GPP. IDSPOrb's dynamic activation process interface supports synchronous and asynchronous activation processes. No need between GPP and a single DSP Split iDSPOrb, iDSPOrb can also be executed in a configuration with multiple DSPs. In this case, the QoS manager (server side) will perform DSP algorithm load balancing among the available DSPs. Other configuration configurations ASIC (-17- this paper is a fixed function (CNS) A4 ^ (2iGx297 ^ jy 514832) 5. Description of the invention (15) A7 B7 DSP printed by the Consumer Cooperative of the Central Standards Bureau of the Ministry of Economic Affairs) or ASIC plus RISC It consists of the algorithm interface provided to the client application program. 2a. DSPIDL compiler iDSPOrb supports DSPIDL (— an IDL (Interface Definition Language) interface definition language, which has the following key Word: module: interface specification set. For example, H263 module can contain Decoder and Encoder interface 〇 interface: interface (interface) specifications. In: indicates input parameters out: indicates output parameters BUFFER: indicates buffer type and J STREAM: Representation of data stream type 丨 J RESULT ·· Representation of function return type and other items are for memory utilization. The general form of real-time DSPIDL file is module modulename {interface algorithm-1 [: alg 1, alg2, ...] {algorithm-1 (PARAMS) // constructor method method_ 1 method_2 method 3 -18- This paper size applies to Chinese National Standard (CNS) A4 specification (210X29? mm) (Please read the precautions on the back before filling this page)

514832 A7 B7 Printed by the Consumer Cooperatives of the Central Standards Bureau of the Ministry of Economic Affairs V. Invention Description (16 where method is a RESULT function ([direction] TYPE, ···)) direction is in, out or [in, out], and TYPE is BUFFER Or STREAM. For example, H263 IDL can generate the algorithm and proxy interface as shown in Figure 2. 2b. Frame and data stream processing The frame and data stream processing have the following differences. Keyword: BUFFER: has one by one Frame-based argument type. In addition, the BUFFER function of the J process order. STREAM: A function that has STREAM as the argument type process order of the frame data stream, usually by Spawn work. The function calls DSPORB_Buffer c〇nnect (DSPORB_Buffer * out, DSPORB_Buffer * in) and DSPORB_Stream_connect (DSPORB_Stream * out, DSP 0 RB__Stre am * in) to provide the connection of the object output to the input ( (Frames or data streams, respectively.) As far as buffers are concerned, the connection operator will cause DSPORB to create a memory buffer on the DSP ', which will store the output of one method's initiation process for another method The input of the dynamic process is used (object link). For example: DSPORB Buffer-connect (yuvframe out, yuvframe one in); -19- This paper size is suitable for financial standards Τ ^ ΰ ^ ΰΐ〇Χ297 石 (Please read the back first (Notes for filling in this page)

Printed by the Consumer Cooperative of the Central Standards Bureau of the Ministry of Economic Affairs 514832 A7 ______B7 _ V. Description of the invention (17) H263—TIDEC—decode (h263frame_in, yuvframe_out); YUV_TI_toRGB (yuvframe in, rgbframe out); In terms of data stream processing, A proxy initiation process such as H2 6 3-TI DEC-dec ode Stream (in-stream, out-stream); usually results in the establishment of work on the DSP side to process the two data streams SIO data stream (the implementation of H263_TIDEC_decodeStream (Work that will spawn this job). It does not provide I / O between the client agent and the server as a connected data stream. 2c. The real-time QoS manager iDSPOrb can provide hard real-time Q〇s, which is through the DSPORB_ System_setTimeConstraint〇 and DSPORB_ System_ setPriority () interface, to configure the set of time constraints required to perform a given job Resources. The GPP / DSP channel I / O driver allows multiple threads to operate in parallel. The QoS manager is part of the iDSPOrb on the DSP. It is responsible for (1) individualizing algorithms according to user needs, (2) updating condition constraints from client applications, and managing resources to meet condition constraints (or return Unable to meet condition constraints), and (3) more features. 2d. IDSPORB registration service iDSPOrb provides a category registration service, so server objects can register their services. For example, you can use iDSPOrb to register server objects that decode MP3 audio. Client objects personalize server objects by providing the name of the requested service. By providing a set of standard Moniker for audio and video services, iDSPOrb registration services can be used for any DSP object service category that is compatible in the media field: -20- This paper size applies to the Chinese National Standard (CNS) M specifications (210X297 Mm) (Please read the notes on the back before filling out this page)

514832 A7 B7 V. Description of the Invention (18) Audio Service Video Service MP3 Audio Decode (MP3 Audio Decode) MPEG1 Video Decode (MPEGl Video Decoding) MP3 Audio Encode (MPl Audio Encoding) MPEG1 Video Encode (MPEGl Video Encoding) MPEG1 L2 Audio Decode (MPEGl L2 MPEG2 Video Decode) Decoding) MPEG1 L2 Audio Encode (MPEGl L2 MPEG2 Video Encode) G. 723 Decode (G. 723 Decoding) G. 723 Encode (G. 723 Encoding) G. 729 Decode (G. 729 decoding) G. 729 Encode (G. 729 encoding) MPEG4 Video Decode (MPEG4 video decoding) MPEG4 Video Encode (MPEG4 video encoding) Η. 263 Decode (H · 263 decoding) Η. 263 Encode (H. 263) (Please read the precautions on the back before filling out this page.) Order the iDSPOrb registration service for employees' cooperatives of the Central Standards Bureau of the Ministry of Economic Affairs to allow iDSPOrb to dynamically individualize server objects during the execution period. When individualizing server objects, iDSPOrb dynamically assigns low-order I / O channels between the microprocessor and the DSP. Client objects can access these low-level channels via the iDSPOrb plug-and-play interface (see DSPORB-Stream interface). The iDSPOrb registration service also provides information, allowing iDSPOrb to find DSPs that provide specific services, and allows QOS administrators to perform load balancing and scheduling (see Instant QoS Manager). For example, using a dynamic triggering process model, calling DSPORB — ALG_create (“MP3 Audio Decode ", NULL) will personalize an instance of the MP3 audio decoder. The iDSPOrb load balancing system, and the user end will be subject to the actual implementation of the decoder. DSP Details 21-This paper size applies Chinese National Standard (CNS) A4 specification (210X29 * 7mm) 514832 A7 B7 V. Description of the invention (19) Protection and what low-level data stream configuration will be configured to transmit data. Users The client can also query the iDSPOrb to enumerate the list of currently registered server X types. You can use the DSPORB—Alg * DSPORB—System_ getServices () function to get the enumerator of the services of the current column. Then, you can Call char * DSPORB_System_next (DSPORB-Alg * enum) to get the name of each registered service. By calling DSPORB_System_reset (DSPORB_Handle * ennm), you can reset the enumeration to the starting point. 2e. Media Architecture Support. IDSPOrb can be used to support media processing acceleration by providing components of a specific media architecture, such as

DirectShow (Windows Media): A filter object that can be implemented to wrap the iDSPOrb codec client object and insert it into the DirectShow framework.

RealMedia Architecture (RealSystem G2): A translatable plug-in that can be implemented to wrap (iDSPOrb) codec client objects and plug into RealSystem G2 architecture. Printed by the Consumer Cooperatives of the Central Standards Bureau of the Ministry of Economic Affairs. You can also use the same method to insert DSPOrb into JMF and QuickTime 〇 Seal the iDSPOrb API in the DSPORB module. The data types and functions of the DSPORB on the client (GPP) side will be described below. 2f. Data Type: DSPORB Alg: Client Agent for DSP Algorithm Objects. DSPORB—Fxn: Function objects to be used in conjunction with the dynamic invocation process. -22- This paper size applies to China National Standard (CNS) A4 (210X 297 mm) Printed by the Consumer Cooperatives of the Central Standards Bureau of the Ministry of Economic Affairs 514832 A7 B7 V. Description of the invention (20) DSPORB_Arg: It must be compatible with the dynamic activation process Function argument used. DSPORB-Buffer and DSPORB-Stream are DSPSubB-Arg's 'Subclass'. DSPORB_Params: Provides parameters of the algorithm that conforms to the parameter structure of the DSP's IALG-Params algorithm. DSPORB_Buffer: buffer object. DSPORB_Stream: Data stream object. 2g. DSPORB Buffer Interface -DSPORB—Buffer * DSPORB_Buffer_ci; eate (int size, int direction); Create a buffer object that can reference the size data size. direction is one of DSPBUFFER INPUT or DSPBUFFER_OUTPUT. The direction of the buffer must conform to the signature of the function initiation process (Signature), otherwise the iDSPOrb runtime error will occur. Or, DSPORB_Buffei: * DSPORBJBuffer_create (DSP ORBAlg *, int, int); the buffer used by the object. -unsigned char * DSPORB—Buffer—getData (); Get the data referenced by the buffer object. If the buffer is connected to another buffer, NULL is returned. -void DSPORB—Buffer_setData (unsigned char * data) Set the buffer data index. If the buffer is connected to another buffer, this operation is incorrect because the memory space used for the data in this buffer is in the DSP memory space. -void DSPORB_Buffer_setSize (int) -23- This paper size applies to Chinese National Standard (CNS) A4 (210X297 mm) (Please read the precautions on the back before filling this page) Order 514832 A7 B7 V. Description of the invention (21) Set the actual data size. -int DSPORB_Buffer_getSize () Get the actual data size. -void DSPORB_Buffer_delete (DSPORB_Buffer * buffer) -int DSPORB_Buffer_connect (DSPORB_Buffer * output, DSPORB_Buffer * input) Connect the input buffer to the output buffer on the DSP. When connected to these buffer objects, the data is saved on the DSP and is not passed back to GPP (a buffer created by iDSPOrb on the DSP to save intermediate results). 2h. DSPORB_Stream interface This stream interface has the following methods. -DSPORB_Stream * DSPORB_Stream_create (int n, int direction); Create a data stream that can hold n buffers. direction is one of DSPSTREAM_INPUT or DSPSTREAM_OUTPUT. Printed by the Consumer Cooperatives of the Central Standards Bureau of the Ministry of Economic Affairs (please read the precautions on the back before filling out this page) -int DSPORB_Stream_issue (DSPORB_Buffer * buf); has an input buffer buf transmitted on the input data stream, or placed in the Fill an empty buffer on the queue of the output data stream. As far as the connected data streams are concerned, this operation is useless because the data streams are directly connected between algorithms. -DSPORB_Buffer * DSPORB_Stream_reclaim (); Obtain the output buffer from the output data stream; or retransmit it on the input data stream-24 Printed by the Consumer Cooperative of the Municipal Bureau 514832 A7 B7 V. Input buffer of invention description (22). As far as the connected data stream, this operation has no effect. 0 -DSPORB_Stream_select (DSPORB_Stream arrayd, int n_streams, int * mask, long millis); block, directly prepare the data stream for I / O. -DSPORB_Stream "dle (DSPORB_Stream * str); idle (Idle) data stream. -DSPORB_Stream_close (DSPORB_Stream * str); closes a stream ° -DSPORB_Stream_connect (DSPORB_Streain * out, DSPORB Stream * in); connect the output data stream to the input data stream. Now, these two data streams will operate in half of the DSP processor space, and GPP cannot access this Two data streams. 2i · DSPORB dynamic triggering process interface The dynamic triggering process interface has the following methods. -Int DSPORB_System_init (); This method must be called first Individual DSPOrb 〇-DSPORB_Alg * DSPORB_Alg_create (const char * name, DSPORB_Params * params); the algorithm instance (reference) to which the symbol 'name' is created. -Void DSPORB_Alg_delete (DSPORB_Handle alg); delete the instance of the algorithm. -DSPORB—Fxn * DSPORB_Alg_getFxn (DSPORB—Alg * alg, const char * fxn—name); returns with the symbol 'fxn—name' Off 25- This paper size Applicable to China National Standard (CNS) Α4 specification (210 × 297 mm) (Please read the precautions on the back before filling this page) 4.

Printed by the Consumer Cooperative of the Central Standards Bureau of the Ministry of Economic Affairs of the 1T. 514832 A7 B7 V. Description of the invention (23) Linked function objects. -int DSPORB_Fxn_setTimeConstraint (DSPORB-Fxn * fxn); Set the time limit for fxn execution. DSPOrb will allocate enough resources to meet this condition constraint, or return 0. -int DSPORB_Fxn_setPriority (DSPORB_Fxn * fxn); Set priority level from 1 to 15. -int DSPORB—Fxn an invoke (DSPORB_Fxn * fxn5 DSPORB—Arg * argsO); call the function on the input and output. This boot process is blocked until all data is available on the unconnected output. For connecting fDSPORB_Buffer_connect 'input and output, it can be transmitted, NULL'. -int DSPORB—Fxn invokeAsync (DSPORB_Fxn * fxn, DSPORB Arg * argsD); Call functions on input and output. This triggering process returns immediately; the application uses 'DSPORB_getData' to retrieve data from the output argument object. -unsigned char * DSPORB_Arg_getData (DSPORB_Arg * output, long timeout); Retrieves data from the output argument object. Block until 弋 imeout ’(nanosecond expiration) in nanoseconds has occurred; or if‘ timeout = -1 *, block indefinitely. -void DSPORB_Arg_setCallback (DSPORB_Arg * output, unsigned char * (* getData) (DSPORB—Arg *)); set a callback function on the output argument; call getData when data is available. -void DSPORB —System—close (); Close DSPOrb. -26- This paper size applies Chinese National Standard (CNS) A4 specification (210X297mm) (Please read the precautions on the back before filling this page) Order 514832 A7 B7 V. Description of the invention (24) 2j. IDSPOrb example first The project example shows how to use iDSPOrb to connect to a TI 263.263 decoder, which uses a dynamic invocation process interface. The second example shows the same program written using agent stubs. / * * testH263-dii.cpp program to test DSPOrb 氺 * Use DSPOrb to read pure H.263 files, parse, decode frames, and * write YUV files.氺 * Usage: testH263 out_file * / #include #include #include " dsporb. H ”#include ffh263. Hff const int MEMSIZE = 4 * 176 * 144 * 3; / * enough for CIF * / static DSPORB_Alg * h263decoder; Printed by the Consumer Cooperative of the Central Standards Bureau of the Ministry of Economic Affairs (please read the precautions on the back before filling this page) static DSPORB_Fxn * h263decoderFxn; static DSPORB_Buffer * h263inputArg; static DSPORB_Buffer * h263outputArg; static DSPORB_Arg h263decoderFxnArgs [2]; int main ( int argc, char ** argv) {/ * frame is encoded H.263; buffer is YUV data * / -27- This paper size applies Chinese National Standard (CNS) A4 (210X297 mm) Central Standard of the Ministry of Economic Affairs Printed by the Bureau's Consumer Cooperative 514832 A7 B7 V. Description of the invention (25) unsigned char * frame = (imsigned char *) malloc (MEMSIZE); unsigned char * buffer = (unsigned char *) malloc (MEMSIZE); DSPORB-System one init〇; h263decoder = DSPORB__Alg create (MH263D_TIDECff, NULL); h263decoderFxn = DSPORB_Fxn_getFxn (h263decoder, Mdecodeff); h263inputArg = DSPORB a Buffer-cre ate (); h263outputArg = DSPORB_Buffer_create〇; h263decoderFxnArgs [0] = (DSPORB_arg *) h263inputArg; h263decoderFxnArgs [1] = (DSPORB_arg *) h263outputArg; / * in is H. 263 file; out is YUV file * / FILE * in = fopen (argv [l] ,,, rb ,,); FILE * out = fopen (argv [2], “wb ''); int n_bytes_in_ £ rame; H263-initReader (in); while ((n_bytes_in_frame = H263_readFrame ( frame, MEMSIZE)) > 0) {DSPORB_Buffer_setSize (h263inputArg, n-bytes in_frame); DSPORB-Buffer_setData (h263inputArg, frame); DSPORBJBuffer_setSize (h263outputArg, MEMSIZE); DSPORB_Buffer_setData ( h263outputArg, buffer); DSPORB_Fxn_invoke (h263decoderFxn, h263decoderFxnArgs); int s = DSPORB_Buffer_getsize (h263outputArg)); printf ("% d->% d \ n", n one bytes one in-frame, s); if (s > 0) fwrite ((constvoid *) buffer, 1, s, out); -28- This paper size applies to China National Standard (CNS) A4 specification (210X 297 mm) (Please read the precautions on the back before filling this page )

V. Description of the invention (26) Printed by the Consumer Cooperatives of the Central Standards Bureau of the Ministry of Economic Affairs 514832 A7 B7 fclose (in); fclose (out); DSPORB_System_close ();} Then the stubs version: / * * testH263-stubs.cpp Test DSPOrb 氺 * Use DSPOrb to read pure H.263 files, parse, decode frames, and * write YUV files.氺 * Usage: testH263 injile out_file * / #include ^ include #include f, dsporb. Hff #include ”h263.h” elude " H263—TIDEC · h ”const int MEMSIZE = 4 * 176 * 144 * 3; / * Enough for CIF * / static H263_TIDEC * h263decoder; static DSPORB_Buffer * h263inputArg; static DSPORB_Buffer * h263outputArg; int main (int arge, char ** argv) {-29- Chinese paper standard (CNS) A4 Specifications (2l0X29 * 7mm) (Please read the precautions on the back before filling in this page)

Printed by the Consumer Cooperatives of the Central Standards Bureau of the Ministry of Economic Affairs 514832 A7 B7 V. Description of the invention (27) / * frame is encoded H.263; buffer is YUV data unsigned char * frame = (unsigned char *) malloc (MEMSIZE); Signed char * buffer = (unsigned char *) malloc (MEMSIZE); DSPORB-init〇; h263decoder = H263_TIDEC_create (NULL); / * in is H. 263 file; out is YUV file * / FILE * in = fopen (argv [l], "rb"); FILE * out = fopen (argv [2], "wb ,,); int nbytes-in-frame; H263_mitReader (in); while ((n-bytes-in-frame: H263_readFrame (frame, MEMSIZE))> 0) {DSPORB_Buffer-SetSize (h263inputArg, n_bytes_in_frame); DSPORB_Buffer_setData (h263inputArg, frame); DSPORB_Buffer_setSize (h263outputArg, MEMSIZE ); DSPORB_Buffer_setData (h263outputArg, buffer); H263_TIDEC_decode (h263inputArg, h263outputArg); int s = DSPORB_Buffer_getSize (h263outputArg)); printf (”% d->% d \ n '', n one byte one in— frame, s); if (s > 0) fwrite ((const void *) buffer, 1, s, out);} fclose (in); fclose (out); DSPORB_close (); -30- This paper size applies to China National Standard (CNS) A4 specifications (210X297 Gongchu 1 (Please read the precautions on the back before filling this page)

514832 A7 B7 Printed by the Consumer Cooperatives of the Central Bureau of Standards of the Ministry of Economic Affairs. 5. Description of the invention (28) 3. Service 口 口 (Quality 〇f Service; QoS) administrator A defined iDSPOrb quality of service manager (iDSPOrb Quality of Service) Manager; iDsP-QSM) is composed of a main processor and a digital signal processor (DSP) pool as a peer server. The umbrella QoS manager that executes all functions needed to maintain a particular quality of service is responsible for managing this dsp server pool. The main processor is often a general purpose processor (Gpp), which is connected to the DSp through hardware such as shared memory or a bus-type interface. The QoS administrator may be part of iDSPOrb, or a separate administrator on the DSP is more common. The system can be driven by hardware and software interrupts. A better implementation is to execute the main user (client) application on the GPP in a share-sharing manner, and to perform specific services on the DSP. Applications that are executed concurrently with the QOS administrator on all processors may be architectures such as the iDsp Media Framework (iDSP media architecture). The iDsp_Q〇s administrator executes three main functions: (1) object classification; (2) object scheduling; and (3) object execution time prediction. These functions are explained below in terms of media-specific examples of use in a GPP / multiple DSP environment. 3a. Object classification In the GPP / multiple DSP environment, objects are transferred to media codecs / filters (algorithms). You can classify media objects based on their data stream type, application type, or algorithm type. The QoS manager defines the known metric information (codec_cycle), filtering cycle (Fiher-Cycle), etc. according to the type of algorithm. -31-National standard against paper rulers (CNS> Α4 · (2ΐ () χ 297 公 楚 (Please read the precautions on the back before filling in this page)

514832 A7 B7 Printed by the Consumer Cooperatives of the Central Standards Bureau of the Ministry of Economic Affairs V. Invention Description (29) 3b. Object Scheduling (Hard Deadline) iDSP-Q〇SM is based on a two-stage scheduling program to read algorithmic objects. The first stage is a high-level scheduling program that determines whether new media asset streams can be scheduled on DSp and sets a hard real-time deadline for the Codec t N ^ cycle. The second phase schedules individual media messages such as q using the hard real-time deadline of the first phase. The first phase is performed during the object transposition time and is usually on the host (GPP). The second stage is performed on a DSP (server) and is performed on a per-frame basis. The first stage of scheduling is when the QoS manager determines on average whether the objects that are already running can support the objects. It is also necessary to consider whether there is sufficient memory to support the object as part of the first stage scheduling. At this time, the object memory buffer, input and wheel for internal use must be statically fixed ’to eliminate the uncertainty of dynamic memory allocation.

The Media platform only executes algorithms that conform to XDAIS. The developer must define the processing time under different conditions. At the time of initialization, the approximate time required to transfer data back and forth between the algorithms will be determined. When setting each == deadline, the QOS administrator will include this time as an important factor. File picking will require each DSP object to supply the following information to the Q ^ s ^ codec cycle (Codec < ycle) and the number of frames ^ umber of Frame (default: number of frames / second). (Um ei: T ... take the number of H (DSP) cycles as the average time of a single encoder cycle (CGdee-eyele). K-different coding solution

Tad uses the number of target server (DSP) cycles as a single early to display the codec-cycle time. (Please read the precautions on the back before filling in this page) E-booking-32- Printed by the Employees' Cooperative of the Central Standards Bureau of the Ministry of Economic Affairs 514832 A7 B7 V. Description of the invention (30) For video codecs, n is usually continuous Within the number of frames (for example, 15 frames). Also, Tace is usually the sum of the maximum total time required plus the average time required for p and B frames. q〇s The administrator keeps track of the Tccd of all media objects. This time (in terms of the DSp period) is based on the current frame speed. For example, for a 30 fps video stream and η = 15, assume Tecd = 125 Mcycles.

Now, the QoS manager can decide whether or not to schedule a new data stream, as follows. Suppose S is the sum of the codec-cycles (Ta ...) of all DAFs currently scheduled. As far as new data Feng is concerned, if (S + Taec) of the new data stream is less than Tced, the data stream can be scheduled, otherwise the data stream cannot be scheduled. For example, suppose there is a 0 b j e c t-A with η = 15, T = 39.5 Mcycles (158 ms), and Tccd = 125 Mcycles (500 ms), and no work is scheduled on the DSP (so S = 〇). The QoS manager is notified to schedule resources for the new data stream requesting Object-A. Because s + 39.5 = 39.5 Mcycles < 125 Mcycles (500 ms), the data flow can be scheduled. When a second data stream requiring Object-A appears, the second data stream is also scheduled because S + 39.5 = 79 Mcycles (316 ms) < 125 Mcycles (500 ms). A third data stream can also be scheduled. However, because the fourth stream requires 158 Mcycles (632 ms), it does not meet the 500 ms hard deadline to schedule the fourth stream. At this point, the QoS administrator negotiates to reduce the frame rate of the data stream. If it fails, it will completely reject the data stream. A modification allows the scheduler to process heterogeneous media objects with different codec-cycle times. Long Tccd-33- This paper size is applicable to Chinese National Standard (CNS) A4 (210X297mm) ''-(Please read the precautions on the back before filling this page) · > Packing. Order 514832 A7 B7 5. The invention description (31) will be allocated to the smallest Teed in proportion. For example, suppose there is an OBject-B with η = 30, Taxc = 40 Mcycles (160 ms), and Tccd = 169 Mcycles (675 ms), and There are two Object-A objects scheduled on the DSP (as defined above) (so S = 79 Mcycles / 316 ms). Because S + 40 * (125/158) = 1 10.45 Mcycles (S + 160 * 500/675 = 435 ms), so you can schedule a new OBject-B data stream. Since (79 + 40 < 125) Mcycles / (316 + 160 < 500) ms can prove correctness, I can actually guarantee all The data streams are all within the minimum 500 ms codec-cycle hard deadline. What happens when a second data stream requiring Object-B is scheduled? 11〇45 + 40 * 125 / 158 = 139 > 125 M cycles / 435 + 160 * (500/675) = 554 ms> 500 ms. Therefore, the scheduler rejected this stream and started negotiation, As mentioned above, iDSP-QoSM will negotiate with the application or its agent to reserve sufficient processing bandwidth for media objects based on codec-cycle. This negotiation will take into account the need of the object Memory, requested QoS level, and MIPS (million instructions per second) of Dsp with other concurrently running Dsp applications. When object selection items change, QOS administrators will perform DSP processing and bandwidth negotiation. Q () The input parameters of the Sf manager negotiation process require the application to define the following items for the object: (l) DSP memory requirements (the number and size of input / output buffers) ⑺ The expected QGS level (usually based on per second) (3) The worst-case execution period of the startup object. ⑷ A hard real-time deadline with a sequence of media frames, called the codec-34-

514832 A7 B7 32 V. Description of the invention (Codec · cycle) (Number of frames and average execution time). Please read the precautions before filling in this book. This page is set in the iDSP-QoS administrator, the second Phase scheduling is based on two perspectives, whose deadline expires first, and who has higher priority. Consider the following example, if Object-Α has a 10ms deadline and Object-D has a 3ms deadline , Even if Object-A has a higher priority, the iDSP-QoS administrator will still schedule to execute Object-D first. Since the approximate execution time of the object is known, when the object must be started, it can be decided "No delay" (No Late〇, so it still meets its deadline. In Figure 3, it is predicted that Object-A will be completed before Object-A's "No Later" starting point. In this case, the higher priority Object_A and There is no deadline conflict between Object-D. Therefore, Object-A will be executed after the lower-priority Object-D. The Employee Consumer Cooperative of the Central Bureau of Standards of the Ministry of Economics will print another scheduled instance (which will (A priority is added to a deadline) It is assumed that the "No Late" prediction of Object-A at a higher priority is before the expected completion time of Object-D. In this case, since Object-A A has higher priority, so Object-A will be executed first, and Object-D can be executed after Object-A as long as Object-D matches the frame placement parameters specified by the object's individualization time; Figure 4 In terms of iDSP QoS used to manage deadlines to achieve the best possible efficiency, as long as the maximum time between the arrival time of the object and the deadline is allowed, the GPP must immediately allow data to be entered into the DSP subsystem. The longer the time between the arrival and the deadline of the data frame, the 1jiDSP-QoSM will also follow in the process of separately scheduling objects and other simultaneous objects -35 This paper size applies Chinese National Standard (CNS) Α4 Specifications (210 × 297 mm) Printed by the Consumer Cooperatives of the Central Standards Bureau of the Ministry of Economic Affairs 514832 A7 ___B7 V. Description of the invention (33) Improve efficiency. 3c. Prediction of the execution period of objects (soft deadline) iDS The core function of PQ〇SM is to predict the processing time required for the next input frame under all scheduled objects. This prediction is very important and specific to the object. The QoS manager uses the statistics of the previous execution period to calculate all Predict the execution time of the next input frame in this way to predict the execution time of the object. The expected execution time of an object is a function (object-specific function) of the previous execution time and the maximum possible actual change (determined for each object). In the case of video objects, for example! The periodicity of p, p, and B v. Boxes are deterministic. So ‘can predict future processing time based on the current frame type and its position within the periodicity of the video frame. Such predictions performed on all concurrently executed algorithms directly contribute to the dynamic reconfiguration of priorities based on the predicted processing time and the upcoming hard deadline. These forecasts are key enablers for managing soft deadlines and processing time instability. iDSP-QOMSM reschedules the objects to be processed immediately based on predictions. Immediate rescheduling occurs within the codec-cycle deadline time (a hard deadline defined by the mean) of an individual object. These individual methods are weighted because the individual frames are weighted based on hard and soft deadlines. In the above example, suppose that when calculating the average 値 of a 500ms workload that partially overlaps with Object-A, all frames in Object-B require the same total hours. However, since the frame of Object-B may take more time during the actual overlap period, or Object-B may not provide the average total hours, this is not the case. Therefore, it is closest to the last issue of its Codec-cycle -36- This paper size is applicable to China National Standard (CNS) A4 (210 X 297 mm) (Please read the precautions on the back before filling (This page)

Order 514832 A7 B7 Printed by the Consumer Cooperatives of the Central Standards Bureau of the Ministry of Economic Affairs 5. The frame of the invention description (34) has a higher priority. If the predicted execution period meets the time requirements defined using I, then QoS management will take one of several possible actions. In a single DSP configuration: (Layer 1) Simple Binary Cutoff: This will automatically discard the frame. The item under consideration should be able to indicate whether discarding the frame has serious consequences. (Layer 2) At the end of the allocation time, clear the objects first, and generally shorten the allocation time of lower priority objects. This may or may not cause the frame to be dropped. (Layer 3) Requires the object to be able to receive QOS commands such as recalling the quality of the output data. In the multiple DSP configuration configuration: (1) At the end of each QoS time limit, a message with load data is transmitted from each DSP to the GPP. (2) Gpp will turn to reassignment only in the absence of the assessed deadlines. Received from the servo Dsp at 〇1) (〇113 layer). After the load is loaded, the new configuration work is performed in the load. However, in order to shorten the work switching time, it is highly desirable to operate all DSPs from a common cluster of external memory space. Everything that executes in the iDSP system must be determined at execution time. DgP objects can be divided into three types: compressed data (encoding), decompressed data (decoding), and data conversion (pre- or post-processing of object data). Objects are presented in the form of blocks to be processed; these blocks are called input data frames. The object processes the input data frame and generates an output data frame. Dispatch -37- This paper size is applicable to Chinese National Standard (CNS) a4 size (210X 297mm) (Please read the notes on the back before filling this page)-Order printed by the Central Consumers Bureau of the Ministry of Economic Affairs 514832 A7 B7 V. Description of the invention (35) When calculating data, the boundaries of the input data frame and the output data frame will be defined in terms of size and amount of processing. Depending on the size of any given input frame, the maximum amount of processing that the DSP (or any other computer in this regard) must perform on that input frame is precisely determined. Before integrating objects into an iDSP system, each object is required to declare the worst-case execution time of a single frame object. The worst-case execution time is used to calculate the execution time of the first input data frame to enable the object to start. Before executing the object, QoS cannot describe the characteristics of the input data frame. Since the encoder and decoder objects are hardly poisoned to perform in the worst case, the first input data frame is very valuable (because they have to predict the worst case). Worst case scheduling is likely to result in a larger actual execution period than the first frame. The only question is whether the actual execution period is greater than the worst case schedule. As mentioned above, the processing time for algorithmic objects will vary depending on the input frame. Initially, iDSP-QoSM will start from the worst case of the first data input frame. After the first frame, the QoS manager uses the predicted characteristics of the algorithm and the measured processing time of the first frame to predict the processing time of the next input frame. For each subsequent frame, the QoS administrator predicts the approximate processing time based on the semantics and the history of the algorithmic objects. For example, the encoder object uses object semantics (for example, I, P, and B frame types) to average the encoding time of input frames similar to the previous one to predict future encoding time requirements. Whenever the execution time of an encoder object is scheduled, the schedule encoder object processes input frames of the same size. Processing time changes are caused by factors such as frame activity levels, frame-to-frame movement, and so on. However, the boundaries of these changes will be defined -38- This paper size applies the Chinese National Standard (CNS) A4 specification (210X 297mm) (Please read the precautions on the back before filling this page)

Printed by the Consumer Cooperatives of the Central Standards Bureau of the Ministry of Economic Affairs 514832 A7 B7 V. Description of Invention (36) Limit. Therefore, the processing time between frames will have a limited maximum rate, and it can be added to the predicted processing time to determine the worst case processing time for the next frame. See Figures 5 to 6. Decoded objects are usually presented as variable-size input frames. The processing time of the input data frame is directly proportional to its size. In order to determine whether the processing time of the next frame will increase, the QoS manager will check the difference in the current and next data input frame sizes. When cooperating with the encoder, similar parameters of the decoder are also saved, that is, the boundaries for processing the difference between two semantically similar frames are defined. The maximum or worst case processing time of the decoder is the maximum possible buffering defined for the object. See Figure 7. Because the transform object and the encoder object must handle input frames of the same size, the transform object performs similarly to the encoder object. Each frame must require the same amount of total processing time, and must be a single pass input frame. Therefore, the processing time of each input frame must be kept constant. Each object receives a relative time from the user application, which is when the object must finish sending frames. An example is an application that specifies that this frame must be processed in the next 7mS. Since there is no common software timing between the GPP and the DSP, the deadlines can only be specified in relative deadlines. It is assumed that the data frame transmission time between GPP and DSP is deterministic. The iDSP system maintains the internal timing, and the data frame cooperates with the internal timing to receive the arrival time stamp, and then calculates the predicted processing time. After calculating the predicted processing time, the QoS Manager schedule data frame is now executed. Before scheduling the objects, the QoS manager compares other objects to be executed at the same time to determine the proper order of execution objects. If there is no other place-39- This paper size applies to Chinese National Standards (CNS> A4 size (210X297 mm) (Please read the precautions on the back before filling this page). Clothing · A7 B7 Staff of the Central Bureau of Standards, Ministry of Economic Affairs Cooperative prints 5. Description of invention (37) The processing input frame will immediately schedule the execution object frame. If there are other materials being executed, the QgS administrator is responsible for determining the execution order. Object priorities, expected deadlines, and hard or soft real-time requirements. See Figure 8. When multiple objects with different execution time priorities are combined into the same, QOS administrators will perform according to the object-specific execution Period calculation to calculate the execution period prediction for each object. Then different tasks (TBD) are scheduled according to the objects in the schedule. The following three transfer conditions may occur: ⑴ Execute all objects to complete a given input Message box, and completed within the deadline set by the application 2 4 Assumptions like the one shown in Figure 9, ... Wang Si, all objects of the + + are before the deadline of each object Completed. If all objects are completed before their respective deadlines, the work requirements of the QoS manager are minimal. (2) The processing load of one or more objects (for example, OBject-B) increases, but no errors are caused Predicted deadline for the next object. May increase the load on one or more objects such as Object-B. Depending on the object, it may be acceptable if subsequent data frames for the same object are processed within the object's deadline limit Missing deadline. For example, the "1" frame in an H263 encoder may require the longest calculation time. The frame that follows the "Γ" frame must be the "P" frame and usually has very small processing Requirements. Therefore, the "1" frame is allowed to occupy the processing period of the subsequent "P" frame. Therefore, if there is enough processing space to process the next frame, the frame / frame that misses a frame is not There will be serious consequences. Because the deadline of Object-B has expired, it must be decided that the overall system is -40- This paper size applies to China National Standards (CNS) M specifications (210 > < 297) ) (Please read the back of the precautions to fill out this page)

1T 514832 A7 B7 Printed by the Consumer Cooperatives of the Central Standards Bureau of the Ministry of Economic Affairs 5. Description of invention (38) Systematic effect. If OBject_B misses the deadline and does not cause the predicted deadline of the next item to be missed, then the harm to the entire system is low. See Figures 10 to 11. (3) The processing load of one or more objects (for example, bje calls) increases, but the result is missed—the predicted deadline of the objects. See Figure 12. In this case, the missed deadline of Objeet.B will cause the most missed prediction of the next object. The deadline ^ even under this material, the harmful effects of the system can be ridiculous or not the lowest. It may take up cycles of subsequent frames, so avoid missing deadlines. iDSP-QSM proposes a set of rules for soft deadline management. This set of rules is used to limit the missed deadline snowball effect caused by a single critical missed deadline. (1) All algorithm objects provide the maximum number of frame drops per second to the QoS manager. (2) Each object will update the number of "missing deadline" execution counts as the moving average after each processing period. (3) When the object exceeds the miss deadline limit, the priority of the object will be changed to the highest priority. Once the number falls outside the limit, the original priority is restored. (4) All subsequent frames with incorrect deadlines after the restriction will be discarded. This will temporarily reduce QOS to the next intermediate level. Then 'report to the client a momentary degradation in QoS (should happen rarely). (5) Frames are discarded in accordance with rules only if the DSP has not activated the object even after the object has passed its deadline. 3d. Throttling control of periodic media translation For a given algorithmic object, iDSP-QoSM [Assuming at any time -41-(CNS) 210X297 public (please read the precautions on the back before filling this page)

1T 514832 A7 B7 V. Description of Invention (39) There is only one request in the prepared queue. In general, media streams have periodic deadlines (for example, 30 frames / second for video streams) that are specified as QoS manager quality of service constraints. The audio and video translation components in the media system can buffer frames to handle changes in arrival times, allowing for slightly earlier arrivals. But these buffers are valid, so the upstream components of the media system must carefully adjust the relative speed of processing frames. DSP-QoSM provides two mechanisms to adjust the processing speed of algorithm objects. (1) The client of the DSP algorithm object controls the speed at which it calls the algorithm object processing function (server). If the request is made within the time period that must be completed, this will cause the QoS administrator to schedule the sub-optimal behavior of the algorithm. For example, consider the previous algorithmic object A, where buffer A1 must be processed during time period T1, and buffer A2 must be processed during time period T2. The figure shows that T1 and T2 are two consecutive cycles, [X] indicates the buffer X that has arrived, and {x} indicates the buffer X that has been processed. See Figure 13a. Printed by the Consumer Cooperatives of the Central Standards Bureau of the Ministry of Economic Affairs (Please read the notes on the back before filling this page) (2) The QOS administrator controls the media data flow adjustment. This mechanism allows the client to cooperate with the input buffer to call the processing function of the algorithm object as soon as possible. The QoS administrator then appends a "startup deadline" to the input buffer. The scheduler will not schedule this buffer until after the “Startup Deadline”. The client blocks until the buffer processing is complete. See Figure 13b. Therefore, in both cases, there is at most one request for each algorithm object in the queue prepared by the QoS manager at any time. 4. Memory Page-42- This paper size is applicable to Chinese National Standard (CNS) A4 specification (210X297 mm) 514832 A7 B7 V. Description of invention (40) In order to be the best on DSP (or any other computer in this regard) To effectively perform multiple algorithms, a set of rules must be established to share system resources fairly between algorithms. These rules specify access to processor peripherals such as DMA, internal memory, and scheduling methods for algorithms. Once a set of rules is received, the system interface applicable to the algorithm can be developed to enable the algorithm to access system resources. The common system interface provides well-defined boundaries for algorithm developers, because algorithm developers can concentrate entirely on algorithm development and have no system support for opening questions, so they can develop algorithms quickly. The Texas Instruments iDSP Media Platform DSP architecture is an example of such an interface. All accesses between the algorithm and the TMS320C62XX DSP occur through this architecture. Printed by the Consumers' Cooperative of the Central Standards Bureau of the Ministry of Economic Affairs (Please read the precautions on the back before filling this page)

The Texas Instruments XDAIS standard requires the establishment of rules that allow one of the rendering algorithms to be plugged into the iDSP media platform, allowing system integrators to quickly combine production quality systems from one or more algorithms. The XDAIS standard requires algorithms to meet a common interface requirement called the Alg interface. The XDAIS standard proposes several rules. The most important is that algorithms cannot directly define memory or directly access hardware peripherals. System services are provided to all algorithms through a single common interface. Therefore, the system integrator only provides the DSP architecture supporting the Alg interface to all algorithms. The Alg interface also provides a means for the algorithm developer to access the system services and the algorithm activation process. Algorithms must precisely define their internal memory requirements. This is a necessary measure of the paging architecture to support multiple algorithms to access the same space in internal memory. An algorithm that complies with XDAIS is required to specify its internal and external memory. -43- This paper size applies to the Chinese National Standard (CNS) A4 specification (210X297 mm) 514832 A7 B7 V. Description of the invention (41) requirements. The internal (on-chip) memory must be divided into two regions. The first section is the system's internal operation (overhead) area. This is the 0S (operating system) data structure used to support the special DSP system configuration. The second area is for the algorithm, but it can only be used when the algorithm is scheduled to be executed. The size of these two memory areas must be fixed. These two memory regions are called the on-chip work area; in other terms, this work area is also called the data overlap or data memory paging area. See Figure 14. To determine the _memory size used by the workspace on the algorithm chip, the system developer takes the total amount of internal data memory space available and subtracts the software needed to support system software such as paging architecture OS support and data support The amount of memory. The system DSP designer should set the 0S configuration configuration such as work, semaphores, etc., to maximize the number of total algorithms that the supporting designer needs to execute simultaneously. This minimizes internal operating time for OS support and increases the algorithm work area. For algorithms executed in this environment, their internal memory requirements must be less than the size of the work area. Otherwise, the system integrator cannot integrate algorithms; the limitation is that each algorithm has only one page. This architecture does not support algorithmic multiple paging. Printed by the Consumer Standards Cooperative of the Central Standards Bureau of the Ministry of Economic Affairs (please read the notes on the back before filling this page) The algorithm workspace is divided into three components: stacked (mandatory), persistent memory, and non-persistent memory. A fourth component is sometimes made to handle the read-only portion of persistent memory, as explained below. See Figure 15. When the algorithm is running, it will only use the on-chip workspace. When the algorithm is scheduled to be executed, the DSP system software will calculate the algorithm's work-44- This paper size applies to the Chinese National Standard (CNS) A4 specification (210X297 mm) 514832 A7 B7 V. Description of the invention (42) Its internal storage location (shadow storage area) is transferred to the work area on the internal wafer. When the algorithm gives up control, the DSP system software will determine the next algorithm to be executed. If it is the same algorithm, it is not necessary to shift the work area. If the next algorithm is a different algorithm, the current workspace will be stored in the shadow storage location of external memory, and the workspace of the next algorithm will be transferred. See Figure 16. The entire workspace of the algorithm is not transferred during content exchange time. Only the used portion of the stack and persistent data memory are migrated. When the algorithm is at the highest level of its call stack, the algorithm's stack is at its highest level (last used). In other words, the algorithm is at its entry point. The optimal algorithmic content exchange occurs when the algorithmic stack is at its highest level, as this means that very little data is transferred from off-chip to the shadow storage area. See Figure 17. Printed by the Consumer Cooperatives of the Central Bureau of Standards of the Ministry of Economic Affairs (please read the notes on the back before filling out this page). The preferred embodiment of the data paging structure requires content exchange to achieve the highest efficiency. The internal operation time of the content exchange process will take the time that the DSP can execute the algorithm. Since it is the best time to exchange content when the algorithm is at its call boundary, the algorithm should be completely emptied beforehand to a minimum. When the algorithm stack is larger than the minimum, it will degrade the entire system. This should be a requirement, but based on very limited circumstances, preemption is acceptable. See FIGS. 18 to 19. A special case of the algorithm workspace is if the algorithm requires read-only persistent memory. This type of memory system is used by lookup tables used by algorithms. Since this memory is never modified, it is read-only memory and does not require a write function. This asymmetric pagination transfer will shorten the algorithm -45- This paper size applies to the Chinese National Standard (CNS) A4 specification (210X 297 mm) 514832 A7 B7 V. Description of the invention (43) Internal operation time of content exchange. Using this data paging structure, a single algorithm can be individualized multiple times. Since the algorithm has defined its internal memory requirements, the DSP system integrator can personalize the same algorithm multiple times. The DSP system software continuously tracks multiple instances and when each instance of the algorithm is scheduled. The number of instances is limited by how much external memory the DSP system has to maintain a shadowed version of the algorithm's instances. The DSP system software must manage each instance in order to correctly map to the algorithm data immediately after scheduling the algorithm. Since most of the DSP algorithm individuals will work, the DSP system software can use the working environment indicators as a means of managing the algorithm execution instances. 5. Data Flow with Links The preferred embodiment of the data flow relies on the entire processing element to provide shared memory space, and directly transfer data between processing elements without the need for GPP intervention. Figure 21 shows such a system. Printed by the Consumer Cooperatives of the Central Bureau of Standards of the Ministry of Economic Affairs (please read the precautions on the back before filling this page). When the processing element 卩 £ & finishes processing the chunk, the generated data will be written to the shared memory In a predefined output buffer in the volume. PEa then informs the next processing element via the appropriate control path to inform PEb that it is in the chain. The notification indicates which shared memory buffer PEb should be used as input. PEb then reads the data from the input buffer for further processing. In this method, data is transferred between all necessary processing elements until all data has been consumed. As mentioned above, a set of buffers are used to communicate data between processing elements and include I / O channels between these processing elements. Any processing element -46- This paper size is applicable to Chinese National Standard (CNS) A4 specification (210X 297 mm) 514832 A7 B7 V. Description of the invention (44) Multiple I / O channels are available between the pieces, so that the system can simultaneously (I.e., parallel) Processing multiple data streams. Fig. 22 An example of parallel processing of multiple data streams -si and s2-. The channel chain is constructed by a series of processing elements connected to the I / O channels. Several channel chains can be defined in special systems. For medium-chain processing elements, each input channel has an associated output channel. The terminal processing element has only input channels or output channels. The input channel of the processing element defines from which buffers data is to be read. The output channel of the processing element defines which buffers the data is to be written to and which processing element is to be notified afterwards. The following lists the types of control messages between the data processing element and the central control processor (CCP). (1) Status messages: data stream processing start, stop, abort, pause, resume, etc. (2) Service quality information: time stamp, system load, resource availability / use, etc. (3) Data flow control messages: start, stop, pause, resume, switch back, etc. (4) System load messages: number of work in progress, number of channels in use, number of channels per processing element, etc. Printed by the Consumer Cooperative of the Central Standards Bureau of the Ministry of Economic Affairs (please read the notes on the back before filling this page). In a preferred embodiment, the configuration file can be read statically through the configuration file that can be read at the system initialization time. Methods define I / O channels and associate them with processing elements. For each type of bit stream to be processed, the configuration profile defines the channel chain (that is, the data path) connected to the appropriate processing element. Aggregating and processing all processing elements in a channel chain results in complete data consumption. -47 · This paper size applies the Chinese National Standard (CNS) A4 specification (210X 297 mm) 514832 A7 B7 V. Description of the invention (45) In the case where there are multiple data paths that can be used for positioning element streams, alternatives can be defined Or backup channel chain. If any of the processing elements of the main channel chain are unavailable, bit streams can be delivered to these alternative or backup channel chains. The bit type and dynamic QoS analysis determined during the execution period choose which channel chain to use to deliver data. During the execution period, all legal channel chains of the system are fixed and cannot be modified. In another preferred embodiment, when a new bit stream arrives at the communication processor, channel chains of different bit streams can be dynamically constructed. The bit stream information obtained during the execution period is transmitted to the CCP via a control message. The CCP determines the required processing elements, and dynamically allocates I / O channels between the processing elements. This approach will allow resources to be obtained in the event of a failure, and will go online during execution, allowing the system to adjust automatically. In a shared memory heterogeneous system, the data flow between processing elements will pass through the external shared memory without the need for CCP intervention. No data will ever appear on the bus, so the speed of data changes is determined by the shared memory access time, not the bus transmission time. Because CCP intervention is also minimized, the entire data flow time eliminates CCP response and processing delays. By minimizing the data transfer time between processing elements, the overall system processing capacity can be enhanced. Printed by the Consumer Cooperatives of the Central Standards Bureau of the Ministry of Economic Affairs (please read the notes on the back before filling out this page) 5a. Examples The typical applications of the data flow techniques discussed here are applicable to media processing systems. Such a system will initialize and control the broadband data stream to facilitate processing such as decoding, encoding, translation, conversion, scaling, and so on. It can handle the media data stream originating from the local disk, or pass -48- This paper size applies the Chinese National Standard (CNS) A4 specification (210X 297 mm) 514832 A7 'B7 V. Description of the invention (46) Communication media such as cable modems, DSL or wireless systems are media data streams from remote machines / servers. Figure 23 shows an example of such a system. The media processing system of FIG. 23 includes five processing elements:

(1) DSL or cable modem I / O front-end DSP

(2) Media processing DSP (3) Video / graphic overlay processor (4) Η · 263 decoder works (5) Color space converter works Η.263 data stream travels along the channel chain of numbered arcs 1 to 3, enter Front-end I / O DSP. Each channel connects two processing elements and consists of a set of I / O buffers used to transfer data between the elements. The control flow shown in the figure is through dotted lines. The global shared memory defines the 定义 .263 data stream that flows from the I / O front-end DSP to the channel 1 I / O buffer. The I / O front-end DSP notifies the destination processing element associated with channel 1, that is, the Η.263 decoder on the media processing DSP operates, notifying that the input buffer is full and ready for reading. Η. The 263 decoder works to read the channel 1 I / O buffer, decodes the data, and writes the generated YUV data to the channel 2 I / O buffer in the local shared memory. Note that channels may be channels between processors or within processors. Data can be transferred between processors via a global shared memory (inter-processor) or a "regional" shared memory (in-processor) for a given processor. In Figure 4, channels 1 and 3 are both Channel between processors, channel 2 is the channel inside the processor. 6. Modification -49-This paper size applies to China National Standard (CNS) A4 specification (210X 297 mm) 514832 A7 B7 V. Description of invention (47) Various methods are available To modify the preferred embodiment, and retain the functions of the patent application. Printed by the Consumer Cooperatives of the Central Standards Bureau of the Ministry of Economic Affairs -50 (Please read the notes on the back before filling this page)

This paper size applies to China National Standard (CNS) A4 (210X 297 mm)

Claims (1)

A B c D 514832 Patent Application No. 090110000 Chinese Patent Application Amendment (September 91) VI. Patent Application Scope 1. A client-server scheduling method, which includes the following steps: (a)-user A first-stage schedule on the server to set an immediate deadline for a task that is coupled to the server of the client; and (b) a second-stage schedule of the child tasks of those tasks on the server This second stage schedule uses the immediate deadline of step (a). 2. The scheduling method of item 1 of the scope of patent application, wherein: (a) the jobs include decoding of a media stream; and (b) the sub-jobs include frame decoding for the media stream. 3. An object request broker method suitable for a master-slave system, the method includes the following steps: (a) hide a first client request back and a second client request call; and ( b) Linking the output of a first server object to the input of a second server object, wherein the first server object and the second server object correspond to the first and second client requests, respectively. 4. The method according to item 3 of the patent application scope, wherein: (a) the link is established by creating a buffer for the intermediate results in the server (the output of the first server object and the second Server objects). 5. —A method for managing server processor memory in a master-slave system, the method includes the following steps: (a) Allocating a first portion of a processor memory to the processor's internal operating time (overhead) ; And this paper size applies Chinese National Standard (CNS) A4 specification (210 X 297 mm) 514832 ^ Application for patent enclosing (b) Allocating a second part of the processor memory to the work area, where only a single job can occupy the second part at a time. 6. The method of claim 5 in the scope of patent application, wherein: (a) the second part of the processor memory includes a stack component, a persistent memory component, and a non-persistent memory component. 7. A data flow method in a heterogeneous system, the heterogeneous system having a bus connected to a control processor and a bus connected to each of a plurality of processing elements, the method comprising the following steps: (a ) Transfer data between the processing elements by using a common memory separate from the bus. -2- This paper size applies to China National Standard (CNS) A4 (210 X 297 mm)