AU2003251399B2

AU2003251399B2 - System for multimedia rendering in a portable device

Info

Publication number: AU2003251399B2
Application number: AU2003251399A
Authority: AU
Inventors: Srinivas Patwari
Original assignee: Qualcomm Inc
Current assignee: Qualcomm Inc
Priority date: 2002-06-04
Filing date: 2003-06-04
Publication date: 2009-07-23
Anticipated expiration: 2023-06-04
Also published as: IL165278A0; HK1075521A1; WO2003103212A3; CN100401281C; KR20050010851A; MXPA04012143A; JP2010288301A; EP1522024A2; BR0311545A; WO2003103212A2; AU2003251399C1; CN1659536A; EP1522024A4; CA2488470A1; NZ536703A; JP4943650B2; JP2005528849A; AU2003251399A1

Description

WO 03/103212 PCT/US03/17690 1 SYSTEM FOR MULTIMEDIA RENDERING IN A PORTABLE DEVICE CROSS REFERENCE TO RELATED APPLICATIONS [0001] This Application claims the benefit of priority from co-pending U.S. Provisional Patent Application Serial No. 60/385,942 entitled "Multimedia Processing" filed on June 4, 2002. This Application also claims the benefit of priority from co-pending U.S. Provisional Application entitled "Multimedia Processing" filed on April 25, 2003. The disclosures of both of the above-identified Provisional Applications are incorporated by reference herein in their entirety for all purposes. BACKGROUND I. FIELD [0002] The present invention relates generally to multimedia processing, and more particularly, to a system for multimedia rendering in a portable device having an embedded controller. II. DESCRIPTION OF THE RELATED ART [0003] Advances in technology have resulted in smaller and more powerful personal computing devices. For example, there currently exist a variety of portable cellular telephones, personal digital assistants (PDAs), and paging devices that are small, lightweight, and can be easily carried by users. Typically, these devices include an embedded controllerwith limited memory resources. For example, the amount of available memory may be limited by the small size of the device. [0004] However, there is an increasing need for these devices to handle larger amounts of data and to execute programs that are more sophisticated. For example, users are demanding remote access to interactive programs, such as gaming programs that require the portable device to provide fast and efficient rendering of multimedia applications and content. For example, users would like to use their portable devices to view and hear multimedia content, such as video clips, animations, movies, and audio, such as music content provided in MP3 format.

2 In order to render multimedia content on a portable device, it is necessary to control the rendering resources of the device. For example, one type of portable device may include a small low resolution display screen or an audio speaker, and another type of portable device may include a larger and higher-resolution display and two-speaker stereo capabilities. Thus, each type of portable device 5 may have different multimedia rendering resources with which to render multimedia content. Therefore, compatibility problems have to be overcome when providing multimedia content for use on a wide variety of portable devices having various configurations and rendering resources. One technique used to overcome compatibility problems requires that the multimedia content be in a format that is specifically compatible with the rendering resources available on the portable 0 device. For example, if a content developer wishes to produce video content for use on a specific portable device, the developer tailors the video content to be exactly compatible with the video rendering resources available on the portable device. Unfortunately, it is expensive and inefficient to produce multimedia content that is tailored for each portable device that the content is to be rendered on. For example, a content developer would 5 need to modify the content for different portable devices, and thereby generate multiple versions of the same content. Also, the content developer would have to constantly update its content as new rendering resources become available. Therefore, what is needed is a system that allows multimedia content to be rendered on a variety of portable devices without having to tailor the content to account for different rendering 3 resources available on each device. The system should operate to allow all types of multimedia content to be rendered and provide an extensible architecture that allows for the rendering of new multimedia formats as they are developed. SUMMARY 5 Aspect I. A method for rendering multimedia content in a portable device having an embedded processor, the multimedia content having an associated multimedia type, the portable device further including one or more rendering resources, and the method including: processing the multimedia content to create a multimedia object based on information obtained in the media data, the multimedia object having an interface including a play method; 0 associating a format handler with the multimedia object, the format handler selected from a plurality of format handlers each respectively associated with a specific multimedia type and supporting at least a play function; and invoking said play method of said multimedia object to render the multimedia content, said play method calling said play function of said format handler to control a selected rendering resource 5 of said device to render the multimedia content from the multimedia object.

3 Aspect 2. The method of aspect 1, further including generating a multimedia data structure based on the multimedia content. Aspect 3. The method of aspect I or 2, wherein the step of controlling includes: 5 implementing a state machine in the format handler to control the selected rendering resource; and rendering the multimedia content using the state machine in the format handler. Aspect 4. A computer-readable medium tangibly storing computer-executable instructions, 0 which when executed by an embedded processor of a portable device, causes said portable device to render multimedia content, the multimedia content having an associated multimedia type, by performing a method, including: processing the multimedia content to create a multimedia object based on information contained in the media data, the multimedia object having an interface including a play method; 5 associating a format handler with the multimedia object, the format handler selected from a plurality of format handlers each respectively associated with a specific multimedia media type and supporting at least a play function; and invoking said play method of said multimedia object to render the multimedia content, said play method calling said play function of said format handler to control a selected rendering resource ) of said device to render the multimedia content from the multimedia object. Aspect 5. The method of aspect 4, further including generating a multimedia data structure based on the multimedia content. 5 Aspect 6. The method of aspect 4 or 5, wherein the step of controlling includes: implementing a state machine in the format handler to control the selected rendering resource; and rendering the multimedia content using the state machine in the formation handler. O Aspect 7. A multimedia system for rendering multimedia content, the multimedia content having an associated multimedia type, in a portable device having an embedded processor, the portable device further comprising one or more rendering resources, and the multimedia system including: means for processing the multimedia content to create a multimedia object based on 5 information contained in the media data, the multimedia object having an interface including a play method; 4 means for associating a format handler with the multimedia object, the format handler selected from a plurality of format handlers each respectively associated with a specific multimedia media type and supporting at least a play function; and means for invoking said play method of said multimedia object to render the multimedia 5 content, said play method calling said play function of said format handler to control a selected rendering resource of said device to render the multimedia content from the multimedia object. Aspect 8. The multimedia system of aspect 7, further including means for generating a multimedia data structure based on the multimedia content. 0 Aspect 9. The multimedia system of aspect 7 or 8, wherein the means for controlling a selected rendering resource includes: means for implementing a state machine in the format handler to control the selected rendering resource; and 5 means for rendering the multimedia content using the state machine in the formation handler. Aspect 10. A method for rendering on a device having a plurality of rendering resources a multimedia presentation, the multimedia presentation having an associated multimedia type, the method, including: ) providing a plurality of format handlers, each format handler respectively associated with a specific multimedia type and supporting at least a play function for rendering the specific multimedia type on said device by controlling at least one rendering resource on said device; creating a media data structure, said media data structure storing said associated multimedia type, a reference to a source of said multimedia presentation, and a size of said multimedia 5 presentation; creating a multimedia object, wherein said multimedia object encapsulates a reference to said media data structure, and an interface of said multimedia object includes a play method for rendering said multimedia presentation on said device; and invoking said play method of said multimedia object; 0 wherein said play method includes calling said play function from a selected format handler of said plurality of format handlers, said selected format handler being associated with a same multimedia type as said associated multimedia type. Aspect 11. The method of aspect 10, wherein said interface of said multimedia object further 5 includes: a registration method for providing notification of asynchronous events associated with said multimedia presentation.

4a Aspect 12. The method of aspect 10 or 11, wherein said multimedia presentation is a sequence of data. Aspect 13. The method of aspect 12, wherein said sequence of data is stored in a file. 5 Aspect 14. The method of aspect 12, wherein said sequence of data is stored in a memory buffer. Aspect 15. The method of any one of aspects 12 to 14, wherein said sequence of data is transmitted via a network stream. 0 Aspect 16. The method of any one of aspects 10 to 15, wherein said multimedia presentation is an application generating a sequence of data. Aspect 17. The method of any one of aspects 10 to 16, wherein said interface of said multimedia 5 object further includes a stop method for stopping said rendering of said multimedia presentation. Aspect 18. The method of aspect 17, wherein said interface of said multimedia object further includes a resume method, for resuming rendering of said multimedia presentation at approximately a location where rendering of said multimedia presentation had been previously stopped. 0 Aspect 19. The method of any one of aspects 10 to 18, wherein said associated multimedia type is a MIME type. Aspect 20. The method of aspect 19, wherein said associated multimedia type includes: ?5 a MIDI type; a MP3 type; a MPEG4 type; and a ADPCM type. 0 Aspect 21. The method of any one of aspects 10 to 20, wherein said plurality of rendering resources includes native software of said device and native hardware of said device. Aspect 22. The method of any one of aspects 10 to 21, wherein said size of said media data structure stores either an actual size of said multimedia presentation, if known, or 0 if said actual size 5 is not known.

4b Aspect 23. A method for recording a multimedia presentation having an associated multimedia type on a device, including: providing a plurality of format handlers, each format handler respectively associated with a specific multimedia type and supporting at least a record function for recording the specific 5 multimedia type by said device by controlling at least one recording resource on said device; creating a media data structure, said media data structure storing said associated multimedia type, a reference to a destination of said multimedia presentation, and a size of said multimedia presentation; creating a multimedia object, wherein said multimedia object encapsulates a reference to said 0 media data structure, and an interface of said multimedia object includes a record method for recording said multimedia presentation by said device; and invoking said record method of said multimedia object; wherein said record method includes calling said record function from a selected format handler of said plurality of format handlers, said selected format handler being associated with a same 5 multimedia type as said associated multimedia type. Aspect 24. The method of aspect 23, wherein said interface of said multimedia object further includes: a registration method for providing notification of asynchronous events associated with said D multimedia presentation. Aspect 25. The method of aspect 23 or 24, wherein said multimedia presentation is a sequence of data. 5 Aspect 26. The method of any one of aspects 23 to 25, wherein said multimedia presentation is an application generating a sequence of data. Aspect 27. The method of any one of aspects 23 to 26, wherein said device is a wireless device and said supplement set of format handlers is provided in a download over a wireless network. 0 Aspect 28. The method of any one of aspects 23 to 27, wherein said interface of said multimedia object further includes a stop method for stopping said recording of said multimedia presentation. Aspect 29. The method of any one of aspects 23 to 27, wherein said interface of said multimedia 5 object further includes a resume method, for resuming recording of said multimedia presentation at approximately a location where rendering of said multimedia presentation had been previously stopped.

4c Other aspects, advantages, and features of the present invention will become apparent after review of the hereinafter set forth Brief Description of the Drawings, Detailed Description of the Invention, and the Claims. 5 BRIEF DESCRIPTION OF THE DRAWINGS The foregoing aspects and the attendant advantages of the embodiments described herein will become more readily apparent by reference to the following detailed description when taken in conjunction with the accompanying drawings wherein: FIG. I illustrates a data network that includes a portable wireless device with an embedded controller 0 suitable for implementing one embodiment of a multimedia system to render multimedia content; WO 03/103212 PCT/US03/17690 5 [0019] FIG. 2 shows a functional block diagram illustrating one embodiment of the portable device of FIG. 1 that includes one embodiment of a multimedia system that operates to render multimedia content; [0020] FIG. 3 shows a detailed block diagram of one embodiment of the multimedia system of FIG. 2; [0021] FIG. 4 shows one embodiment of a state machine managed by a format handler to render multimedia content; and [0022] FIG. 5 shows one embodiment of a method for rendering multimedia content in a portable device. DETAILED DESCRIPTION [0023] The following detailed description describes a multimedia system, including methods and apparatus, for rendering multimedia content in a portable device. In one or more embodiments, the portable device has an embedded controller and limited resources (i.e., limited memory capacity), and the system operates to render multimedia content using the available rendering resources of the device. [0024] In one or more embodiments, the multimedia system interacts with a runtime environment executing on the device that is used to simplify operation of the device, such as by providing generalized calls for device specific resources. One such runtime environment is the Binary Runtime Environment for Wirelessa (BREW

T

m) software platform developed by QUALCOMM, Inc., of San Diego, California. In the following description, it will be assumed that the multimedia system is implemented on a portable device executing a runtime environment, such as the BREW software platform. However, one or more embodiments of the multimedia system are suitable for use with other types of runtime environments to render multimedia content on a portable device. [0025] FIG. 1 illustrates a data network 100 that includes a portable wireless device 102 with an embedded controller suitable for implementing one embodiment of a multimedia system to render multimedia content. For example, the device 102 comprises one embodiment of a multimedia system 104 that will hereinafter be referred to as "IMedia." In the network 100, the wireless device 102 communicates with a network server 106 over a wireless network 108 using wireless communication channels 110.

WO 03/103212 PCT/US03/17690 6 [0026] In one embodiment, the device 102 comprises a cellular telephone that may transmit and/or receive voice information over the wireless network 108. However, one or more embodiments of the described multimedia system are suitable for use with other types of portable devices. For example, other suitable portable devices include, but are not limited to, PDAs, email devices, pagers, tablet computers, mobile telephones or virtually any other type of portable device that includes multimedia-rendering resources. [0027] The device 102 may also receive multimedia applications or content over the wireless network 108. For example, multimedia applications 112 and 114 may be downloaded to the device 102 from the network server 106. Additionally, multimedia content 116 may also be downloaded to the device 102 from the network server 106. It is also possible to download multimedia applications or content to the device 102 from any other network entity coupled to the wireless network 108. [0028] In one embodiment, the device 102 also couples directly to a local system, such as a local workstation 118, via a direct link 120. In one embodiment, the local workstation 118 downloads multimedia applications or content to the device 102 using the direct link 120. For example, the multimedia application 122 is downloaded to the device 102 from the workstation 118 using the link 120. [0029] The IMedia system 104 may be downloaded from the server 106 to the device 102 and operates on the device 102 to render multimedia applications or content. In another embodiment, the IMedia system 104 may be downloaded to the device 102 from the workstation 118 via the link 120, or may be installed in the device during manufacture. [0030] In one embodiment, the IMedia system 104 is provided on a computer readable media, such as a floppy disk, and is loaded onto the system 118 for transmission to the device 102. In another embodiment, the IMedia system 104 may be stored on a computer readable memory device, such as a memory card (not shown), and plugged directly into the device 102, so that the IMedia system 104 may execute on the device 102. Thus, the device 102 may receive the IMedia system 104 in a wireless transmission, a wired transmission, or by retrieving it directly from a memory device. [0031] The network 100 may also include other types of portable devices, such as other types of wireless and non-wireless portable devices, and these devices may communicate with the server 106 or any other network entities utilizing various network types and communication architectures, including wireless and non-wireless networks, 7 private and public networks, switched and non-switched networks, direct links and any combination thereof to download one or more embodiments of the IMedia system 104 to render multimedia content. FIG. 2 shows a functional block diagram illustrating one embodiment of the 5 device 102 that includes one embodiment of the IMedia system 104 that operates to render multimedia content. The device 102 comprises processing logic 202 that is coupled to an internal data bus 204 and a stack memory 216. Also coupled to the internal data bus 204 are instruction memory 206, multimedia application/content memory 208, heap memory 210, user interface 212, display logic 215, audio logic 218, 10 and input/output (1/0) interface 214. During operation of the device 102, the processing logic 202 executes program instructions stored in the instruction memory 206 to generate a runtime environment 220. The runtime environment 220 may be the BREW environment or other suitable runtime environment. To assist with instruction execution, the processing logic 202 15 utilizes the stack memory 216 to store program data or instructions on a temporary basis. For example, the processing logic 202 may store constants, variables, program addresses, pointers, instructions or other information items on the stack memory 216. In another embodiment, the processing logic 202 may store information on a temporary basis in the heap memory 210. The heap memory comprises virtually any type of 20 memory suitable for the storage and retrieval of information by the processing logic 202. In one or more embodiments, the processing logic 202 comprises a CPU, gate array, hardware logic, software, or a combination of hardware and software. Thus, the processing logic 202 generally comprises logic to execute machine-readable 25 instructions. The instruction memory 206 comprises RAM, ROM, FLASH, EEROM, or any other suitable type of memory, or a combination thereof. In one embodiment, the instruction memory 206 is located internal to the device 102, and in another embodiment, the instruction memory 206 comprises a removable memory card or 30 memory device that may be selectively attached to the device 102, and thereby couple to the internal bus 204. Thus, the instruction memory 206 may comprise virtually any type of memory that is capable of storing instructions that may be executed by the processing logic 202.

8 The user interface 212 receives user input, for example, from a keypad, pointing device, touch pad, or other input mechanisms to allow a user to interact with the device 102. The display logic 215 comprises logic to output information to a device display. For example, the device may include an LCD, LED or other type of display resource. 5 The audio logic 218 comprises logic to output audio information to a device speaker(s), remote speaker(s) or audio system, or other type of sound resource. For example, a pair of remote speakers may receive the output of the audio logic 218 to render audio information to a device user. Thus, the display logic 215 and the audio logic 218 may comprise hardware and/or software in any combination to allow the device 102 to 10 render visual or audio information to the device user. The I/O interface 214 operates to transmit and receive information between the device 102 and external devices, systems, and/or networks. For example, in one embodiment, the I/O interface 214 comprises a radio transceiver circuit (not shown) that operates to transmit and receive information over a wireless data network using, for 15 example, communication link 106. For example, the transceiver comprises circuitry that modulates information received from the processing logic 202 and converts the modulated information into high frequency signals suitable for wireless transmission. Similarly, the transceiver also comprises circuitry to convert received high frequency communication signals into signals suitable for demodulation and subsequent 20 processing by the processing logic 202. In another embodiment, the 1/0 interface 214 comprises a transceiver that operates to transmit and receive information over a hardwired communication link, such as a telephone line, to communicate with a remote system on a public data network, such as the Internet. 25 In still another embodiment, the 1/0 interface 214 comprises circuitry that operates to communicate with local devices, such as the local workstation 116 using the link 120. The 1/0 interface 214 may also include circuitry (such as serial or parallel port logic) to communicate with a printer or other local computer or device, such as floppy disk or memory card. Thus, the 1/0 interface 214 may comprise any type of hardware, 30 software, or combination thereof to allow the device 102 to communicate with other local or remotely located devices or systems. During operation of the device 102, execution of program instructions by the processing logic 202 causes the IMedia system 104 to be generated. For example, 9 IMedia instructions that when executed generate the liMedia system 104 may be stored in the instructions memory 206. The IMedia system 104 interacts with the runtime environment 220 to render multimedia applications and content on the device 102. For example, multimedia applications and content may be downloaded to the device 102 via 5 the wireless network 108 and stored in the memory 208. In one embodiment, the IMedia system 104 retrieves the multimedia content stored in the memory 208 via the internal bus 204 and processes the content to render it on the device 102 by interacting with the device's rendering resources (i.e., display logic, audio logic, etc.) via the runtime environment 220. The IMedia system 104 also operates to process and render 10 streaming multimedia content. For example, streaming multimedia content downloaded from the network 108 to the device 102 is rendered on the device 102 using the [Media system 104. It should be noted that the configuration of the device 102 is just one configuration suitable for implementing one embodiment of the IMedia system 104 to 15 render multimedia applications and content. It is also possible to implement the IMedia system 104 using other devices or device configurations within the scope of the present invention. FIG. 3 shows a detailed block diagram of one embodiment of the IMedia system 104 for use in the device 102. The device 102 includes native hardware 302 that 20 comprises processing logic 202 and may optionally include a multimedia processor 304 that is specifically designed to process multimedia information. It will be assumed without further description that the native hardware 302 includes logic to operate the device's rendering resources (i.e., display logic 215, etc.) and that the processing logic 202 and the multimedia processor 304 may operate to control these resources. 25 The device 102 also includes native multimedia software 306 that comprises audio 308, audio2 310 and video 312 modules. These modules represent low-level software modules that are used to directly communicate with the native hardware 302 to control selected rendering resources. For example, the video module 312 communicates with the multimedia processor 304 to control video rendering resources. Although one 30 embodiment is shown, it is also possible to have fewer or more multimedia software modules 306 and these modules may communicate with all types of native hardware 302. Thus, other native hardware 302 and software 306 configurations are possible within the scope of the invention.

WO 03/103212 PCT/US03/17690 10 [0044] During operation, the device 102 executes program instructions to generate the runtime environment 220, which in one embodiment, is the Brew environment. It is then possible to execute the IMedia instructions to generate the IMedia system 104 to render multimedia content in conjunction with the runtime environment. For example, the IMedia instructions may be downloaded to the device via the wireless network 108 and stored in the instruction memory 206. [0045] The IMedia system 104 operates to abstract multimedia content handling and define a framework of multimedia interfaces. In one embodiment, the IMedia system 104 operates to provide the following features. 1 . Provides generic and easy access to device multimedia features and capabilities. 2. Shields complex application specific integrated circuit (ASIC) level multimedia APIs and resource management. 3. Provides building blocks to develop complex multimedia applications. 4. Optionally derives media type based on file extension, file content, or a signature in file. [0046] The IMedia system 104 includes a create media utility 324, media data structure 326, and format handlers 314, 316, 318, and 320. Also shown as part of the IMedia system 104 is an IMedia abstraction 328, which illustrates a basic IMedia object that is created during operation of the IMedia system 104. A detailed description of the compdnents of the IMedia system is provided as follows. IMEDIA ABSTRACTION [0047] The IMedia abstraction 328 is an abstract interface enabling registration and handling of a multimedia content type. The IMedia abstraction 328 maintains a reference to media data in the media data structure 326. MEDIA DATA STRUCTURE [0048] The media data structure 326 operates to encapsulate the media data source or sink. This allows the IMedia system 104 to stream media data from or to various sources or destinations. In one embodiment, the media data structure 326 is formed from data structure logic that comprises the processing logic 202 executing program instructions to generate the media data structure 326 in the memory 210.

11 CREATE MEDIA UTILITY The create media utility 324 performs several functions. First, the create media utility 324 operates to receive multimedia applications or content for rendering. For example, the multimedia applications or content may be stored in the memory 208, 5 and this content includes a function call to the create media utility 324. Second, the create media utility 324 processes the multimedia content to create an IMedia object based on the information contained in the media data 326. After creating the IMedia object, it sets the media data and places the IMedia object in the ready state. In one embodiment, the create media utility 326 is formed from create media logic that 0 comprises the processing logic 202 executing program instructions to create the IMedia object in the memory 210 and perform the functions of the create media utility described herein. In one embodiment, the create media utility 326 includes selection logic that comprises the processing logic 202 executing program instructions to select a format 5 handler to associate with the IMedia object and perform other functions described herein. For example, the selection logic evaluates the data structure 326 associated with the media content to determine which format handler to select. FORMAT HANDLERS 3 In one embodiment, the IMedia system 104 comprises a collection of format handlers (FH). For example, the IMedia system 104 comprises MIDI 314, MP3 316, MPEG4 318, and ADCPM 320 format handlers. These format handlers provide interfaces 322 between IMedia objects and the multimedia rendering hardware and software of the device 102. In one embodiment, the format handlers interpret a !5 common instruction set so that multimedia applications or content may utilize the same instructions to instruct the format handlers to perform similar tasks, such as play, stop, rewind, etc. Any number of format handlers may be used, and new format handlers may be added to render new types of multimedia content. In one embodiment, the format handlers are formed from format handling logic that 0 comprises the processing logic 202 executing program instructions to perform the functions of the format handlers described herein.

WO 03/103212 PCT/US03/17690 12 [0052] In one embodiment, a common instruction set instructs the format handlers to render multimedia content and provides a mechanism for passing parameters and status. A sample of the instructions included in the common instruction set is as follows. 1. IMediaSetMediaParm() ;sets parameters of the FH 2. IMediaGetMediaParm() ;gets parameters from the FH 3. IMediaSetMediaData() ;sets media data 4. IMediaGetMediaData() ;gets media data 5. IMediaGetClassID() ;gets class ID 6. IMediaGetStateo ;gets state 7. IMediaRegisterNotify() ;registers a call back with FH 8. IMediaPlay() ;issues play command to FH 9. IMediaStop() ;issues stop command to FH 10. IMediaSetRecto ;set video parameters [0053] For example, the IMediaSetMediaParm() instruction may be included in the IMedia object to instruct the format handler to set specific parameters, settings, or variables associated with the format handler. For example, parameters associated with a format handler designed to render MPEG video can be set using the IMediaSetMediaParm() instruction. In other embodiments, specific format handlers may also interpret additional instructions (not shown above) that may be included in the IMedia object 328 and that are used to control specific function of the resource being controlled. [0054] In one embodiment, a format handler is software running on the portable device as part of the IMedia system 104. In another embodiment, the format handler may be a combination of hardware and software. Furthermore, format handlers may be added to the IMedia system as they become available so that the IMedia system is extensible and able to render new types of multimedia content as it becomes available. [0055] During operation, format handlers register themselves (their MIME type) with the runtime environment and they can be detected by multimedia applications by querying for the MIME type or enumerating a list of supported (registered) format handlers in the system. Each format handler provides a selection of functions that include, but is not limited to, the following. 1. Providing an application program interface (API) that: WO 03/103212 PCT/US03/17690 13 a. Performs playback and recording of multimedia content, including controlling functions such as seek, pause, resume, etc.; b. Sets and/or Gets audio and video control parameters; c. Handles asynchronous events from the IMedia object. 2. Sending asynchronous media events via registered callback function. 3. Managing device multimedia rendering resources. 4. Managing a state machine that is extensible by derived classes. [0056] FIG. 4 shows one embodiment of a state machine 400 managed by a format handler to render multimedia content, for example, the format handler 318. The initial state of the format handler is IDLE 402. Once the format handler is associated with an IMedia object, the state of the format handler goes to READY 404. From the READY state 404, the format handler may proceed to either PLAY 406 or RECORD 408 states based on commands associated with the multimedia content. From the PLAY 406 and RECORD 408 states, the format handler may go to PLAYPAUSE 410 or RECPAUSE 412 states. From these states, it is also possible to resume the corresponding PLAY 406 and RECORD 408 states. The format handler may also STOP 414 and return to the READY 404 state. [0057] Therefore, using the state machine 400, it is possible for each format handler to control the device's rendering resources to render multimedia applications or content (IMedia object) that has been associated with that formal handler. In another embodiment, it is also possible for the format handler to implement a different state machine than the one shown in FIG. 4. Furthermore, in other embodiments, it is possible to define additional states, which may be used during the rendering process. [0058] FIG. 5 shows one embodiment of a method 500 for rendering multimedia content in a portable device. [0059] At block 502, a multimedia application or content is created using IMedia constructs so that it may be rendered on a portable device. For example, application developers create IMedia objects using the create media utility 324. The number of IMedia objects and the capabilities, such as simultaneous playback or recording, are dependent on the device. All IMedia objects follow the basic state machine shown in FIG. 4. In one embodiment, the following steps can be used to create and use an IMedia object.

WO 03/103212 PCT/US03/17690 14 [0060] 1. Create a media data structure and initialize it based on the media data source/sink. The AEEMediaData is defined as follows. typedef struct { AEECLSID clsData; // Type of media data void * pData; // Context sensitive data (see table) uint32 dwSize; // Context sensitive data (see table) } AEEMediaData; [0061] 2. Initialize the members of the above structure based on the information in the following table. clsStream pData dwSize Comments MMDFILENAME File Name 0 For playback and recording. The file is not opened by IMedia, but the filename is passed sown to the ASIC-level API, which opens the file MMDBUFFER Buffer Ptr Size of Buffer For playback and recording. The buffer must contain the complete media data. MMSSource ISource Total Size of Real-time streaming data or 0 [0062] At block 504, the multimedia content is downloaded to the portable device for rendering. For example, the application or content may be downloaded to the device via the wireless data network 108. The content may be rendered as streaming content, or stored in memory and rendered at a later time. [0063] At block 506, a media data structure is created and initialized based on media source or sink characteristics included in the received content. For example, the media data structure is stored in the media data structure 326. In one embodiment, the media type is derived based on file extension, file content, or a signature included in the file.

15 At block 508, an IMedia object is created using the associated data structure. For example, the create media utility 324 is used to create the IMedia object. The IMedia object is returned in the ready state 404. At block 510, an optional steD may be performed to register a notification 5 function if it is desirable to receive asynchronous events from the IMedia object. At block 512, the IMedia object is associated with a format handler and the multimedia application or content is rendered on the portable device. For example, the format handler may provide any of the functions of the state machine 400 to render to multimedia content. 10 At block 514, after the content has been rendered and the IMedia object is no longer needed, it is released. IMPLEMENTATION EXAMPLE The following is an implementation example to illustrate how in one embodiment an IMedia object is created. The example includes references to the 15 corresponding blocks in the method of FIG. 5 and it is assumed that the runtime environment used by the portable device 102 is the BREW environment. Thus, "CApp" is an instance of a BREW applet global structure. static void AppPlayMedia(CApp * pme) 20 { int nRet; AEEMediaData md; // Create Media Data structure (FIG. 5-block 506) 25 md.clsData = MMD_FILENAME; md.pData = (void *)"media/midifmt1.mid"; md.dwSize = 0; // Create the IMedia object and put in Ready state (FIG. 5-block 508) 30 nRet = AEEMediaUtilCreateMedia(pme->a.pIShell, &md, pme->m-plMedia); // If successful, register App .MediaNotify() as the registered 16 callback for IMedia events (FIG. 5-block 510) if (SUCCESS != nRet) DisplayErrorDlg(pme, IDSERRCREATEMEDIA); else if (SUCCESS != IMEDIARegisterNotify(pme->m-pI Media, 5 AppMediaNotify, pme)) DisplayErrorDlg(pme, IDSERRREGISTERNOTIFY); else if (SUCCESS != IMEDIAPlay(pme->m-plMedia)) // Begin the playback (FIG. 5-block 512) DisplayErrorDlg(pme, IDSERRPLAY); In one or more embodiments included in the present invention, a multimedia system including methods and apparatus operates to render multimedia applications and content on a resource-limited portable device having an embedded controller. Accordingly, while one or more embodiments of the methods and apparatus have 5 been illustrated and described herein, it will be appreciated that various changes can be made to the embodiments without departing from their spirit or essential characteristics. Therefore, the disclosures and descriptions herein are intended to be illustrative, but not limiting, of the scope of the invention, which is set forth in the following claims. 3 Throughout the specification and the claims that follow, unless the context requires otherwise, the words "comprise" and "include" and variations such as "comprising" and "including" will be understood to imply the inclusion of a stated integer or group of integers, but not the exclusion of any other integer or group of integers.

Claims

3. The method of claim I or 2, wherein the step of controlling includes: implementing a state machine in the format handler to control the selected rendering resource; 3 and rendering the multimedia content using the state machine in the format handler.
4. A computer-readable medium tangibly storing computer-executable instructions, which when executed by an embedded processor of a portable device, causes said portable device to render 5 multimedia content, the multimedia content having an associated multimedia type, by performing a method, including: processing the multimedia content to create a multimedia object based on information contained in the media data, the multimedia object having an interface including a play method; associating a format handler with the multimedia object, the format handler selected from a 0 plurality of format handlers each respectively associated with a specific multimedia media type and supporting at least a play function; and invoking said play method of said multimedia object to render the multimedia content, said play method calling said play function of said format handler to control a selected rendering resource of said device to render the multimedia content from the multimedia object. 5 18
5. A multimedia system for rendering multimedia content, the multimedia content having an associated multimedia type, in a portable device having an embedded processor, the portable device further comprising one or more rendering resources, and the multimedia system including: means for processing the multimedia content to create a multimedia object based on 5 information contained in the media data, the multimedia object having an interface including a play method; means for associating a format handler with the multimedia object, the format handler selected from a plurality of format handlers each respectively associated with a specific multimedia media type and supporting at least a play function; and 0 means for invoking said play method of said multimedia object to render the multimedia content, said play method calling said play function of said format handler to control a selected rendering resource of said device to render the multimedia content from the multimedia object.
6. A method for rendering on a device having a plurality of rendering resources a multimedia 5 presentation, the multimedia presentation having an associated multimedia type, the method, including: providing a plurality of format handlers, each format handler respectively associated with a specific multimedia type and supporting at least a play function for rendering the specific multimedia type on said device by controlling at least one rendering resource on said device; 0 creating a media data structure, said media data structure storing said associated multimedia type, a reference to a source of said multimedia presentation, and a size of said multimedia presentation; creating a multimedia object, wherein said multimedia object encapsulates a reference to said media data structure, and an interface of said multimedia object includes a play method for rendering Z5 said multimedia presentation on said device; and invoking said play method of said multimedia object; wherein said play method includes calling said play function from a selected format handler of said plurality of format handlers, said selected format handler being associated with a same multimedia type as said associated multimedia type. 30
7. A method for recording a multimedia presentation having an associated multimedia type on a device, including: providing a plurality of format handlers, each format handler respectively associated with a specific multimedia type and supporting at least a record function for recording the specific 5 multimedia type by said device by controlling at least one recording resource on said device; 19 creating a media data structure, said media data structure storing said associated multimedia type, a reference to a destination of said multimedia presentation, and a size of said multimedia presentation; creating a multimedia object, wherein said multimedia object encapsulates a reference to said 5 media data structure, and an interface of said multimedia object includes a record method for recording said multimedia presentation by said device; and invoking said record method of said multimedia object; wherein said record method includes calling said record function from a selected format handler of said plurality of format handlers, said selected format handler being associated with a same 0 multimedia type as said associated multimedia type.
8. The method of claim 7, wherein said interface of said multimedia object further includes: a registration method for providing notification of asynchronous events associated with said multimedia presentation. 5
9. The method of claim 7 or 8, wherein said multimedia presentation is a sequence of data.
10. The method of any one of claims 7 to 9, wherein said multimedia presentation is an application generating a sequence of data. I1. The method of any one of claims 7 to 10, wherein said device is a wireless device and said supplement set of format handlers is provided in a download over a wireless network.
12. The method of any one of claims 7 to 11, wherein said interface of said multimedia object 5 further includes a stop method for stopping said recording of said multimedia presentation.
13. A method as claimed in claim 1, substantially as herein described with reference to the accompanying drawings. o 14. A computer readable medium as claimed in claim 4, substantially as herein described with reference to the accompanying drawings.
15. A multimedia system as claimed in claim 5, substantially as herein described with reference to the accompanying drawings. 5
16. A method as claimed in claim 6, substantially as herein described with reference to the accompanying drawings. 20
17. A method as claimed in claim 7, substantially as herein described with reference to the accompanying drawings.
18. A method substantially as herein described with reference to any one of the embodiments of 5 the invention illustrated in the accompanying drawings.
19. A computer readable medium substantially as herein described with reference to any one of the embodiments of the invention illustrated in the accompanying drawings. 0 20. A multimedia system substantially as herein described with reference to any one of the embodiments of the invention illustrated in the accompanying drawings.