CN101669328A - Method and apparatus for the adaptation of multimedia content in telecommunications networks - Google Patents

Abstract

An apparatus for adapting media content in one or more networks includes a controller module, one or more network interface modules, and one or more protocol interface modules coupled to the controller module. The apparatus also includes a script interpretation module coupled to the controller module and one or more media adaptation modules coupled to the controller module. The one or more media adaptation modules may support audio, video, or meta data. The apparatus further includes a management interface module coupled to the controller module. Media may be received at the apparatus as streaming media and be provided from the apparatus as a download or a progressive download. Additionally, media may be received at the apparatus as a download or progressive download and be provided from the apparatus as streaming media.

Description

The adaptive method and apparatus of content of multimedia that is used for communication network

Cross reference to related application

The application requires the U.S. Provisional Patent Application No.60/889 that submitted on February 9th, 2007, and 237 priority should be in whole disclosures of first to file all unconditionally by incorporated herein by reference.The application also requires the U.S. Provisional Patent Application No.60/889 that submitted on February 9th, 2007, and 249 priority should be in whole disclosures of first to file all unconditionally by incorporated herein by reference.

Below two conventional U.S. Patent applications (comprising the application) submitted to simultaneously, and whole disclosures of another application are all unconditionally by incorporated herein by reference:

The application (acting on behalf of case number No.021318-006510US) of " the Method and apparatus for the adaptation ofmultimedia content in telecommunications networks (the adaptive method and apparatus that is used for the content of multimedia of communication network) " by name that submitted on February 11st, 2008; And

" Method and apparatus for a multimediavalue added service delivery system (method and apparatus that is used for multimedia value-added service transfer system) " (the acting on behalf of case number No.021318-006610US) by name of submitting on February 11st, 2008.

Background technology

The present invention generally relates to field of telecommunications and relates more specifically to be used for the adaptive method and apparatus of content of multimedia of multiple communication network.Along with popularizing of communication network and equipment, content of multimedia may be with various forms by demand now, when these forms during by precoding, can consume quite a large amount of complexity that are used for memory storing, the bandwidth that is used to exchange and can cause management.

There is the set of the device type of a plurality of video creation source, constantly development and performance and huge content that the user generates, wherein has only the sub-fraction content to be checked.The scope of network enlarges (2.5G, EDGE, 3G, LTE, 4G, WiFi and WiMax) constantly, and in view of (for example such as device screen size and video/audio compression coding and decoding/form, agreement support (download, gradual or streaming), feature support, coding tools or metadata access/demonstration), the factor of the network bandwidth, transmittability and error rate or available memory or CPU and so on, network and equipment should be transmitted the requirement of continuous variation for which content.

The variation of network, equipment and agreement has caused constantly out-of-date to the concern of the minimum common performance of (equipment with better performance/situation is produced quality influence) equipment or network or stronger device dependence and administration overhead and device-specific content.

In existing systems, content be converted in advance usually may be available various forms, and Content Management System finds configuration file at equipment to provide by the content of appropriate formatization.The various characteristics that may need conversion (in general sense), for example encoding and decoding, bit rate, frame sign, Container Type, metamessage and medium are handled, and cause the index of maintained usually target numbers to increase and can be with there being the processing that have delay in a large number before up to content.Because have only welcome clauses and subclauses often to be visited, so the object that great majority are converted will be to be converted very lavishly and to store.

Therefore, need to be used for the improved method and system of the content of multimedia of adaptive various communication networks in this area.

Summary of the invention

Embodiments of the invention provide by the equipment that can not discern specific format or by the network of not supporting described specific format, speed or transmission method or other characteristic and provide the method and system with the visit and the exchange of the stored content of specific format.These functions are provided, and have improved the efficient that user interface and content at content are provided simultaneously.Embodiments of the invention provide under given restrictive condition the suitable distribution to the bandwidth and the quality of medium, service and experience.

According to embodiments of the invention, a kind of device that is used for the media content of adaptive one or more networks is provided.This device comprises controller module, one or more Network Interface Module and the one or more protocol interfaces that are coupled to controller module.This device also comprises the script explanation module that is coupled to controller module, is coupled to one or more medium adaptation module of controller module and the management interface module that is coupled to controller module.

According to embodiments of the invention, provide a kind of adaptive device of content of multimedia that is used on diverse network and host-host protocol, providing from multiple form and quality to multiple other form and quality.This device allow mode with a kind of resource high-efficiency with content of multimedia from multiple format adaptation to multiple other form.This device is stored with a kind of form easily by the adaptive permission content of multimedia to content of multimedia and is accessed from any display device.This device can also be expanded as required.

The present invention has a lot of benefits for conventional art.For example, the adaptive method and system of content of multimedia that provides on multiple network, network condition and host-host protocol from a plurality of forms and quality to a plurality of other forms and quality is provided embodiments of the invention.Depend on embodiment, one or more and other benefit in these benefits can be obtained.These and other benefit will be explained in more detail at whole specification, and describe more specifically in conjunction with the accompanying drawings below.

Description of drawings

Embodiments of the invention equipment that provides and the simplification diagrammatic sketch of serving annexation are provided Figure 1A;

Embodiments of the invention equipment that provides and the simplification diagrammatic sketch of serving annexation are provided Figure 1B;

Fig. 2 shows the simplification diagrammatic sketch of the example that content is conducted interviews according to the use HTTP of the embodiment of the invention;

Fig. 3 shows the simplification diagrammatic sketch of the example that content is conducted interviews according to the use RTSP of the embodiment of the invention;

Fig. 4 shows the high-level overview according to the media path of the embodiment of the invention;

Fig. 5 A is according to the interconnected simplification system assumption diagram of the module of the embodiment of the invention;

Fig. 5 B is according to the interconnected simplification system assumption diagram of the module of the embodiment of the invention;

Fig. 5 C shows the exploded view according to the main system element of the embodiment of the invention;

Fig. 6 shows the simplification diagrammatic sketch according to the Function Decomposition of the system of the embodiment of the invention;

Fig. 7 A-7C is the simplification connection layout that obtains and retrieve from element by content adapter of being used for according to the embodiment of the invention;

Fig. 8 A-8C shows the simplification system assumption diagram according to the position arrangement of the content adaptation control of the embodiment of the invention;

Fig. 9 shows the simplification system assumption diagram according to the decomposition texture of the content adapter of the embodiment of the invention;

Figure 10 shows the simplification system assumption diagram according to the decomposition texture of the content adapter in the HTTP Proxy role of the embodiment of the invention;

Figure 11 shows the simplification diagrammatic sketch of the content adaptation before the caching in HTTP GET method according to the embodiment of the invention;

Figure 12 shows the simplification diagrammatic sketch according to the content adaptation after the caching in HTTP GET method of the embodiment of the invention;

Figure 13 shows the simplification diagrammatic sketch of the content adaptation before the caching in HTTP PUT method according to the embodiment of the invention;

Figure 14 is redirected the simplifying the operation of the URL that relates to content adapter according to the embodiment of the invention;

Figure 15 is the simplification linked system structure according to the content adapter that the HTTP content adaptation is provided of the embodiment of the invention;

Figure 16 shows the sequence chart that is connected to the HTTP source according to the RTSP client of the embodiment of the invention;

Figure 17 shows the sequence chart that is connected to the HTTP source according to the RTSP client of the embodiment of the invention;

Figure 18 shows the sequence chart that is connected to the HTTP source according to the HTTP client of the embodiment of the invention;

Figure 19 shows the sequence chart that is connected to the RTSP source according to the RTSP client of the embodiment of the invention;

Figure 20 shows the sequence chart that is connected to the RTSP source according to the HTTP client of the embodiment of the invention.

Embodiment

Embodiments of the invention provide a lot of benefits.These benefits are provided by the adaptive method and system of content of multimedia that is provided on multiple network and the host-host protocol from a plurality of forms and quality to a plurality of other forms and quality.Described quality can comprise for example frame sign, frame rate, agreement, bit rate coding feature (for example fixed bit rate, variable bit rate), coding tools, bit rate, specific coding, for example error correction coding (FEC) etc.Another benefit provides power and memory-efficient content adaptation method.But another benefit provides the load that can efficiently handle increase and can be expanded to the expanding system that additional capabilities is provided simply.Another benefit provides a kind of permission with multiple form cache contents and reduce the content adaptation method of the load on the content server.Another benefit provides a kind of adaptation method of user interface efficiently.Another benefit provides and allows the mutual communicating devices of a plurality of man-machine interfaces.Another benefit provides and allows in real time and the system of non real-time content adaptation.Another benefit provide a kind of permission fast and accurately user-generated content to the system that uploads of content data base.Another benefit provides a kind of system that expands content data base by the content that allows content database stores to generate by multiple form.Another benefit provides a kind of permission for example waits the content of the different qualities of weighing with encoding and decoding, size, speed from the common source contents extraction system.Another benefit provides a kind of method of creating the multiple version of identical content efficiently at the network or the network condition of variation.Another benefit provides a kind of method of the best edition of definite content that should be provided automatically.Thereby another benefit provides the method that the content of the multiple form of a kind of buffer memory efficiently reduces adaptive requirement.Another benefit provides the method for a kind of short-term and long-term cache contents.Another benefit provides a kind of a plurality of simultaneous devices of arbitrary time that are connected that allow to use different connection protocols.Another benefit provides a kind of method of creating adaptively and revising content adaptation engine.Another benefit provides a kind of system that uses the hardware and software unit to carry out content adaptation efficiently.Another benefit provides a kind of control hardware and software module is come as required, in the method for creating arbitrarily the best content adapter preset time.Another benefit provides a kind of system that uses common interface module and existing network and future network to carry out interface.Another benefit provides a kind of system, and the feature of the service that will provide and service load hint are provided in this system, when obtaining or when recapturing or carry out the code conversion of any necessity when the mixing of obtaining and/or recapturing.Another benefit be adaptive medium to use extendible encoding and decoding, wherein various ranks space or frequency information are transmitted on one or more transmission channels.In this case, receiving equipment can selective reception and is shown various other details of level, if these details are received and/or based on device resource with in good time mode, complete or wrong recoverable mode.Another benefit provides a kind of hardware and software that allows and safeguards the system that the mode with module is performed, thereby allows this system still can operate during safeguarding.Another benefit provides a kind of system that can be used as the redundancy backup of another similar system, bears all or part of load of first system adaptably with application-specific.Another benefit provide a kind of can be by the system with some or all of afore-mentioned characteristics of telemanagement.Another benefit provides a kind of access to content that provides with historical use data and to how optimizing other networking component better so that the system of the suggestion of access to content faster to be provided.Another benefit a kind of providing is provided safeguards and keep system about the ability of the information of usage behavior and statistics, described information the cost of optimization that is used for serving or service or based on before behavior to allow the data mining and the usertracking of the interior perhaps behavior of customization be useful.Another benefit is carried out code conversion by the medium of only client requests being checked with the specific format of being asked, thereby avoids handling, stores, the waste of management and occurrences in human life resource.Another benefit is the adaptability load Sharing method when system is deployed between a plurality of discrete subsystems.

Equipment	Content adapter	Content portal/supplier
			??· 3G mobileLimited H.263, MPEG-4, H.264 do not have equipment to be used to upgrade limited form to support the small screen size PCThe a lot of content source of multiple encoding and decoding (the too many selection) data transfer rate has the needed quality of being changed to of the some kinds of orders of magnitude to pay IPTVThe frame sign (SD or HD) that limited encoding and decoding (MPEG-2, H.264) are fixing	Bit-rate adaptation advertisement insertion of codec code transform frame size conversion and advertisement overlay content watermark stream translation (for example according to be downloaded object) are rule-based/trigger, the conversion of visit (browser/player) type, source address etc.	Expanding covering to 100% 3G and non-3G equipment at the arbitrary content of any apparatus has reduced the requirement that device intelligence is used and expands the content search coverage rate to arbitrary format carry out usertracking via the device label that generates in content adapter

Table 1: when using the content adapter (CA) of embodiment to the benefit of content portal

According to following description in conjunction with the accompanying drawings, will know other purpose of the present invention and advantage, wherein embodiments of the invention are disclosed in the mode of diagram and example.

According to embodiment, a kind of device that is used for adaptive content of multimedia between multiple network and multiple network is provided.Comprise one or more in the following assembly according to the device of embodiment: allow device to carry out the network operating interface module as the part of network, be established to the communication session management module of the connection of other network element, allow user interface to be modified to be fit to the user interface translation module of other network element, the man-machine interface translation module, machinery compartment interface translation module, content adapter is created and control module, the resource control module, the maintenance interface module, one or more memory modules, one or more processing units, bus control unit module and bus, cache module and redundant facility module.

According to embodiment, a kind ofly content of multimedia is provided the one or more steps during this method may further comprise the steps: set up communication session from the method that first communication network is fitted to second telecom network network, the communication session that administrative institute sets up, the place translates interaction data at user interface, translate human-machine interaction data in man-machine interface place, translate the machinery compartment interaction data at the machinery compartment interface, to be adapted for second content of multimedia according to first content of multimedia of first agreement according to second agreement, control is to the visit and the transmission of second content of multimedia, control one or more resources, the working service interface is safeguarded described one or more resource, determine whether redundant process is used and whether definite expansion process is used.

Embodiments of the invention be may reside among a plurality of heterogeneous networks or between or the network element in a plurality of diverse locations in the single network.Figure 1A and Figure 1B show and can be connected or the equipment of serving and the example of service by embodiments of the invention.Equipment operated in network can be packet switching (PS) network or circuit switching (CS) network.Each network utilisation agreement is initiated communication session, user interaction activity is provided and requested content is provided.Content can perhaps be downloaded or progressive download by the data exchange agreement such as HTTP (HTML (Hypertext Markup Language)) or by alternate manner by the stream protocol such as RTSP (real-time streaming protocol) by stream transmission continuously.Content also can be transmitted (promptly not being recorded in advance) in real time from the multiple source such as stream content source (feed), for example via RTSP or such as IETF or 3GPP SIP, H.324, ITU-TH.323 or ITU-T (comprise 3GPP-324M) or have or do not have the proprietary protocol such as Skype Video of independent interconnected work functions that (additional network support can internally or the outside be supported and can adopts at U.S. Patent application No.10/099, the form of the element described in 901, the disclosure of this U.S. Patent application is all incorporated herein by reference).Content also can be extracted from content data base.Content data base can utilize relevant agreement to be connected to content adapter by network, and content data base can be different network element; It can be via direct file access by co and visit.

Embodiment can be established to the connection of the equipment of request content object or network element and can be established to the connection of content data base.The example of this connection is as shown in Fig. 2 and Fig. 3.In Fig. 2, two abilities that are connected element are determined.These abilities can the handling capacity exchange or negotiation, capabilities message determine, perhaps they can be the known features of element, access network or agreement, perhaps are associated with the user via database (for example HSS or HLR) Network Based or otherwise location.These embodiment determine that how the interface of each element is can be by adaptive another element with suitable this ability of use.Under the mutual situation of machinery compartment, device can connect by machinery compartment interface translation module.

Under the situation of man-machine interaction, device of the present invention can utilize machine to arrive people's interface translation module.Machine to people's mutual example as by being connected to the mobile phone access internet web website of 3G (third generation) network.Owing to typical mobile phone has the little and light form that designs at mobility, so check that a lot of web websites are unpractiaca.In this case, device of the present invention can be set up the web website sought and the connection between the mobile device by interface translation module (being that machine is to the people in this case), to realize visit more efficiently and better user experience.This device can also be revised the kind of medium transfer mechanism.In the example of mobile device, typical HTTP download/gradual search modes can be changed and be stream transmission pattern at web.This may be particularly useful for available finite bandwidth and memory space on a lot of mobile devices of compensation.The conversion of download/progressive download as shown in Figure 3.From the unanimity that server provides and streaming media also assists in ensuring that the experience of those users by different network type access services subsequently.Embodiment also by the adaptive conversion content object that is used for to send miscellaneous equipment to less or different abilities, perhaps by adaptive standard with certain capability set that meets the requirements, example embodiment can transmit otherwise will be with DVB such as the DVB-H receiving equipment or based on the incompatible content of the equipment of DVB.

Embodiments of the invention are realized with the decomposing module as shown in Fig. 5 A.Network/terminal module, medium adaptation module, session are set up module, Subscriber Interface Module SIM, control module and content data base and are connected.Alternate embodiment of the present invention realizes with the decomposing module as shown in Fig. 5 B.The network interface of the agreement processing such as SIP (RFC 3261 and the RFC that is associated), RTSP (RFC 2326) and the HTTP (RFC 2616), media filter module, be connected to the administration module of web GUI alternatively, control module and the one or more content server of HTTP, RTSP or SIP that for example uses that is connected to content data base or web application is connected.

Embodiment supports advertisement insertion and medium splicing.Fig. 4 is the high-level overview to the media path among some embodiment, comprises some key points of the process that is used for those embodiment.The ability that the target video advertisement is included in the service of moving and other service is to produce the successfully key factor of service, and therefore the ability of ' splicing ' advertising clip or other Media Stream is provided in an embodiment.

Fig. 4 shows the situation (also can be the situation of advertisement splicing) that a plurality of source of media are used to single client playback session.Buffer memory has cache object so that can be spliced with different modes together after them to be used for resetting and by carrying out the ability that buffer memory provides maximal efficiency in output stage (promptly coding afterwards).Choice point place, the source that is chosen between the source is carried out, and the source choice point can comprise that also the media detection facility switches and can carry out with quick but still quality controlled way with montage in the middle of guaranteeing.

Embodiment has following ability: interrupt the stream that just is being played in single RTSP session, play to substitute montage and in a single day substitute montage after script action/incident and finish, then initial (original) montage restarts.Because embodiment can carry out content adaptation, do not need to have and the identical form of original clip (at aspects such as encoding and decoding, bit rate, frame rate, frame signs) so substitute (being inserted into) montage.

In order to support the advertisement driven service, embodiment can collect user action and incident and can provide feedback with the form of visible message and notice.

One embodiment of the present of invention realize with decomposing module as shown in Figure 6.The scope of some functions is limited in the figure for illustrated purpose.In the figure, the service range in user's space and cyberspace, access to content space, system management space, account management space and the content data base space separates.

In case the connection between requestor and the content source is established, some embodiment just translate the relevant information of interacting activity between the network element.Because different networks may adopt different communication protocol, so the interactive information that is transmitted between end user and content server may need to be translated to from the set of a kind of message, signal and data the set of another kind of message, signal and data.This translating can be performed by the interface translation module.In case system is proposed content requests, device just determines whether content of multimedia needs by adaptive.Whether need adaptive decision to make based on information collected during capabilities exchange and according to content determined possibility information itself and any relevant network characteristic about content by control module (being the session control module).For example, the user may want the 3GP file is uploaded to the content data base that is set to accept avi file.In this case, the control module in the device will determine that content adaptation is necessary.Notice that in some instances, content adaptation is unwanted and data directly transmit.

In case it is adaptive that control module need to be determined, then next will determine to adopt the adaptive of which kind of type.Multi-medium data can be divided into the multiple different rank that is associated with its storage.Medium itself can use multiple technologies to be encoded, some of these technology are by standardization, for example at video H.263, H.264, MPEG-1,2,4 (having all different parts), the GSM-AMR at voice, AMR-WB, G.711, G.723.1, G.726, G.729, SMV, EVRC and at AAC, AAC+, MPEG-1 the 3rd part (MP3) of audio frequency.Other technology such as Flash video encoder, real audio frequency and the real medium of Adobe (Macromedia) is privately owned.

Under situation based on the medium of file, medium are stored in the so-called container, and described container is a kind of file format (benefit of using container is that data can be organized and describe content or the metadata that is associated with content can be saved in the mode of holding certainly in mode more efficiently).Identical content can be stored in two different containers in a different manner, perhaps is stored in the same containers type with different qualities or different metadata.To container storage a kind of substitute be in real time or streaming media () situation for example, the TV type of service, the content that wherein is encoded can be transmitted through procotol (for example RTP/UDP/IP).This means that content adaptation may be performed in some or all levels in container levels, media-level or Network Transmission level.The session control module that is provided in the embodiments of the invention is determined needed adaptation stage.

The present invention among some embodiment also comprises adapter establishment and control module.This module collects that () other information of use level for example, processing, data transmission/bandwidth, memory resource etc., this module uses these information to create needed content adapter efficiently about system resource.Needed content adapter is determined according to the information that the session control module is provided.Content adapter comprises container demultiplexing module, media codes conversion module and container Multiplexing module.The container demultiplexing unit is from container and any medium meta-data extraction media data.The media codes conversion module uses the medium metadata of extracting from container and media data to produce with required/media data of desirable form coding and the metadata that is associated with the data with format.The container Multiplexing module is organized media data and any suitable metadata after code conversion with the needed form of target container.In certain embodiments, Multiplexing Unit may not need.For example, some streams transmit agreement not to be needed to use container to come media data, so just do not need container Multiplexing module (but can need the Network Transmission processor in some cases) in this case.Equally, the situation for receiving stream content may not need demultiplexing module.Adapter is created and control module determines whether to need mentioned a certain module, and then should use which kind of version if desired, because each Container Type requires corresponding demodulation multiplexer and multiplexer type.In case no longer need content adapter, then adapter establishment and control module will free system resources.

Realize more efficient and access to content fast for auxiliary, embodiments of the invention also comprise cache module.This cache module can be time granularity (time-granular) buffer memory (aging) module, so that be stored in the memory more than the information at older content at the information of in the recent period accessed content.Because the information that provided is based on the content that exchanged between the different network elements and interactive information, so its performed processing helps to produce about the content that exchanged and the information of data.For example, accessed content will all be provided recently, and nearer accessed content may have only according to the metadata of its storage and some media datas, and accessed the earliest content (still being considered to be fit to the content of buffer memory) is the process metadata of storage to some extent.Carry out adaptive when needing to be performed once more after helping so more quickly.The example of process metadata can comprise bitrate information, entropy information or allow second to handle in some instances first and second times coded messages of carrying out with less processing and/or higher code efficiency or quality.Cache module comprises memory and buffer memory control module.

In order to realize system maintenance, embodiment also comprises the maintenance interface module, and this module is used to directly to visit and control various system resources and provides management information with the form of alarm, trap and utilance statistics.At last, this device comprises and allows it to implement module as the backup of another similar system or as the redundancy of the expanding system of another similar system.

The function of content adaptation can not be realized in existing gateway well and the code conversion engine mainly uses cascade (tandem) code conversion.It is that the basis provides at the multimedia memory technology of multiple standardsization and nonstandardized technique and the adaptive needed flexibility of efficient multimedia of client, equipment or network distortion that these gateways lack with each session.It is passive that these network elements also are built as usually, because they are made the as a whole efficient that is exchanged or improve in the interactive experience of end user and the network element that the content of seeking is provided and do not bear positive role improving content.In order to address these and other problems, embodiment as described herein provides a kind of and has allowed efficiently and the scheme of content adaptation flexibly, the described adaptive experience that has improved efficient that content transmits (at speed, power, quality, the memory that consumed etc.) and end user to network element that content is provided.

In order to obtain these and other benefit, embodiments of the invention have made up a plurality of modules.For example, Network Interface Module, communication session management module, user interface translation module, man-machine interface translation module, machinery compartment interface translation module, content adapter establishment and control module, resource control module, maintenance interface module, memory module, processing unit, bus control unit module and bus, cache module and redundant facility module are combined in an embodiment.

The use of interface translation module is the new introducing to multimedia code conversion and adaptive space, it also is the same that content adaptation is created with the use of control module, and described content adaptation is created and control module has made up dissimilar a plurality of sub-components to make up needed content adapter.These sub-components include but not limited to container demultiplexing module, media codes conversion module and container Multiplexing module.Comprise that the intelligent buffer module has become another novel feature of embodiments of the invention, wherein said cache module comprises memory and allows to be buffered the buffer memory control module that data constantly are revised and changed when needed, thereby extendible buffer memory of time is provided.

Connect between Network Interface Module permission any other network element in the present invention and diverse network.This module comprises hardware and software.The nextport hardware component NextPort of this module is limited by current standard or popular privately owned unit.In one particular embodiment of the present invention, hardware is the Ethernet card of working with suitable driver software.The component software of Network Interface Module explains that the received communication session with foundation and the device of being invented and other network element is the message of purpose, and prepares message to set up these sessions.

In the exemplary embodiment, component software comprises http communication module and RTSP communication module, and wherein the HTTP module is used to through the Internet to communicate by letter with content data base and the RTSP module is used to set up that stream transmits session and content spread deliver to 3G mobile phone on the packet network.Because this assembly of the present invention is modular, so when being loaded needed software module and having equipped correct hardware, can set up communication session with any other networked devices neatly.Additional embodiments is supported SIP communication module and can streaming and progressive download and with the module of another medium tunnel transfer medium.The RTMP of Adobe (Macromedia) is supported by the RTMP communication module.

The communication session of communication session module management from the starting point to the terminal point.In case be established with the session of another equipment, then the purpose of this session is just determined by this module.In case purpose is determined, communication session just is associated with communication session another equipment with that set up.In one embodiment, the 3G mobile device is connected to this device by Network Interface Module, and content data base also is connected to this device by Network Interface Module.Afterwards, two communication sessions are by associated with each other by communication session.

When stopping connecting for one in the equipment, the session that the communication session module notifies another equipment and shutoff to be terminated, and needn't turn-off another session, because may becoming or may become with the 3rd session, this another session is associated.In example embodiment, 3G equipment user can require and obtain the content from content data base.Two equipment has the independent session of getting up by the communication session module relation.The 3G equipment of being mentioned can stop its session subsequently, and the content data base session can become subsequently with the 3rd session from another 3G mobile device and is associated.

The present invention among some embodiment also comprises the interface translation module.This module can comprise the interface translation module of user interface translation module, man-machine interface translation module, machinery compartment interface translation module or other type.A lot of interfaces in the existing network (especially man-machine interface) are designed at specific application or platform.Which usually, determine at platform (platform refers generally to have the machine of hardware and software, but platform also can be one group of standardized ability) based on the acceptance of platform in market.Current multimedia gateway and adapter be almost only with the emphasis that is adapted for to medium, and do not carry out adaptive to the interface as the front end of content.

The information that the interface translation module can use communication session module and Network Interface Module to be provided is determined carrying out contingent problem when mutual with given interface from connected equipment.In one embodiment of the invention, if the 3G portable equipment is associated with the web content server with HTML (hypertext markup voice) interface, then the end user can not navigate to this interface well, unless make amendment.In certain embodiments, the HTML interface is translated to XHTML (expansion HTML) interface and is carried out alternately to allow the 3G equipment user.This is translated by with the whole interface of taking away and generating automatically at the identical function among the XHTML of interface.This translation module is an intelligent object, because it determines the purpose of the different assemblies on the interface as the people.In this manner, this translation module is novel.

Content adapter establishment and control module included in the embodiments of the invention are responsible for carrying out adaptive in needed level to the content of multimedia that is exchanged arbitrarily.Digital multimedia was compressed to use memory better before storage usually.Information after the compression can further be organized in the predetermined container, and described container provides about the metadata of medium and the information that may help the exchange of these signals.For example, the ISO Container Format limits audio frequency, video and metadata and can how to be organized hereof, and in the metadata some can be how be transmitted server by stream and use to reduce stream and transmit the needed workload of described file (prompting (hinting)).The content adapter control module adopts the data of the form of predetermine one from communication session module and that acquisition of interface translation module, what needs adaptive if necessary to judge, and based on described judgement, the sub-component of determining which type should be combined in come together to carry out efficiently needed adaptive.This step comprises considers resource needed and according to the use of the demand of resource being dispatched efficiently to resource.

In one embodiment, the content that is stored as the Flash video file is checked in the request of 3G mobile device, and this mobile device itself can not playing flash video (a kind of privately owned container that can comprise standardized compression method).Content creating and control module determine that according to capabilities exchange described 3G equipment can receive that (video is compressed with form H.263 as RTSP stream in this example, audio frequency is compressed with the GSM-AMR form) medium, and this content is available in using at the H.263 form of video and the Flash video container at MP3 (MPEG-1, third part) the form compression of audio frequency.In this particular example, thus the content adapter creation module is connected to MP3 to GSM-AMR code conversion module and will be connected to the RTSP/RTP output module from the video of first module with from the audio frequency of second module with Flash video demultiplexing module.Data between the module and metadata move through the qualification memory and how bus resource is managed at the mobile connecting object that distributes of data.Control module in the present embodiment is guaranteed to handle moves and scheduling can be managed for available resources, and up-to-date information is offered system control module can not transship to guarantee system.

In another embodiment, device characteristics detect and be performed (at media formatsization, bandwidth etc.) in systems.Except selecting form and type characteristic, the buffering of equipment and play capability or access technology also can be determined.After buffering/reproducing characteristics was known, these characteristics can be used in the content choice in the integrity service.For example, have only short montage can be provided for some equipment,, perhaps support to flow transmission for a long time but bit rate is too low if for example these equipment do not support stream to transmit.The montage that is used for these constrained devices can be the contents version of emphasizing that allows the weak point of the additional cushion between the montage, and the content of emphasizing that perhaps can be used as independent Downloadable integration or demonstration together is provided, but not single content source.

Content adapter can also be carried out the cross media type conversion, and wherein medium can become another kind of form from a kind of formal transformation.For example, content adapter can be to voice utilization lecture identifier.The text that is identified can be superimposed on the video or be added to the metadata captions, and the text that is identified also can be used by machine translation and with multilingual, and every kind of language can be stored in the metadata.Also can carry out further cross-current for other purpose and handle, thereby lecturer's identification can be used to the mark lecture medium, be shown as captions or more fully a cover can be used for explaining the metadata of these materials subsequently.Face recognition can be used with similar mode.Another example that cross media is handled is such situation, and wherein the web page (also may have plug-in) based on standard HTML is provided for the RTSP Media Stream to present to the equipment (for example SIP visual telephone, TV set-top box or other media information booth (kiosk)) of not supporting all or part of browser capabilities.

Additional embodiments utilizes the homogenization performance of content adapter and some abilities in its cross media disposal ability to strengthen search engine to the extracting (crawling) of content of multimedia with to being labeled the storage of data, so that retrieve efficiently in order to search for the purpose of media play afterwards.

Additional embodiments comprises the mechanism that is used for mean allocation load between different processors or server.When system receives when request, it is known having only the details of the form that target medium should be converted into, and in most of the cases, the attribute of source medium (code/decode type, bit rate, frame rate etc.) is unknown at this one-level place.System estimates real-time processing requirements based on the known parameters relevant with the output medium form, and based on before session or other similar session the processing requirements (cost) of source medium is estimated.This load is used to select suitable processor based on employed load balancing algorithm, and session subsequently continues.In case the source medium are accessed, parameter can be known and estimate and can be generated through the load of correction so, and this estimation is used to update system load balancer (and the estimation after being used to alternatively) subsequently.

The present invention among some embodiment also comprises extendible intelligent buffer module of time.The purpose that this intelligent buffer module is reached provides the fast access in the recent period accessed data, thereby improves overall network and system responses.Because device can be visited the content of various forms, so be not a simple thing to the buffer memory of content.In a particular embodiment, content can be in the 3GP container and medium with accessed in the compressed AVI container of different forms.Always safeguard in buffer memory that the identical content with multiple different-format is not too efficiently.In one embodiment of the invention, be used to determine that from the metadata of contents extraction described content is whether buffer memory.If described content has still had different forms in buffer memory, determine then whether buffer memory preserves process metadata (data that help to accelerate adaptation processing or use resource more efficiently), perhaps whether the medium in the buffer memory itself are more suitable for form adaptive but not that directly regeneration needs from source data (this situation may be present in and handles in the cache file of back, these files comprise to be handled the back metamessage and for example points out, perhaps the element-specific of media object track for example).If buffer memory is not preserved any relevant process metadata, then content adaptation be performed and process metadata be added to buffer memory and with existing relevance.By process metadata and media data and metadata association are got up, system " association " how adaptive better content or utilize before to content make hard-working.The method of this buffered media provides multiple benefit for content provider and content user.Because the visit of particular content entry is reduced, so the amount of the media data of preserving at described content is reduced and replaces with the process metadata that is designed to consume memory still less.

Along with particular content entry is not visited more continually, metadata is further reduced, till all data relevant with described particular content entry are removed from buffer memory.In this manner, buffer memory is that the time is extendible, and this is another novel feature that embodiments of the invention provide.In this particular example, the expiration rule supported of the variety of protocol such as HTTP is observed the correct tracking to guarantee that the source owner uses content.In some cases, additional metadata can be provided for the content owner, and these additional metadata are relevant with the details (for example its type, Network Access Method) of the equipment of received content and form that be provided with content is relevant.

In relating to an embodiment of buffer memory and a plurality of adapter engines, same buffer memory can be shared between a plurality of adapter engines.Buffer memory can utilize the external disk array to be provided, so storage can increase according to the needs of disposing.In relating to other embodiment of buffer memory and script, buffer memory access strategy can utilize script to be customized to meet the needs of deployment.

Cache management among the embodiment drives with such algorithm, and this algorithm is considered popularity, aging degree (welcome montage last month now may be not welcome equally), cost (being used for code conversion, code conversion and storage once more), weight (selectable distributed by business rules).Be used for automatic facility, the buffer memory audit facility of off-line buffer memory and the input (for example input of webAPI) of arriving the cache management interface.

In certain embodiments, the API (API) that is used for inner script logic and external control is provided.When system received client-requested, the PHP script will be based on protocol type (for example HTTP or RTSP) and/or based on client URL selected (can based on the Available scripts different with the regular expression match selection of URL).

Inner script can according to the web server in use the similar mode of PHP to be performed, wherein the request header data are provided as a series of global variables.Other distortion also can exist, and includes but not limited to Python, PERL or Java.Script among embodiment has two major functions.At first, its limits encoding and decoding and the transmission option that the medium be returned to client should use.Secondly, it identifies the position of the not homology media object in the different source storages (for example HTTP or RTSP server or Local or Remote disk, SAN, NFS).The mapping of request can relate to a plurality of source of media that are used for splicing (advertisement of the mode of rolling before or after for example adopting is inserted) or mixed-media (picture-in-picture, advertisement stack or watermark).This has realized from need not be at each equipment separately forward type and the rear-mounted advertisement options of the seamless way of the multiple source of coding and management.

Script facility support among some embodiment allows script and database or other web service to carry out mutual standard feature and storehouse with the wide region of creating senior application.In addition, customization CDR and client's navigation model can be supported.

In certain embodiments, when request was received by the PHP script, a plurality of script global variables were available, for example in global array.These variablees can be in the following variable, a plurality of or whole: CallID (the employed value of system when login and report, perhaps as unique call reference of system), request URI (the complete URI that client is issued, comprise any query string), query string (from the query string of client URI), client address (the IP address of issue client terminal, for example, point mark string format), user agent's (character string of its user agent of indication that some clients send, can be used to device type detects), server host (the IP address of protocol processor) and request head (for the HTTP request, this can be the PHP hash function of the HTTP head in the request).

In certain embodiments, the PHP script is provided with the function of the various aspects that are used to control the medium affairs.For purpose of description, these functions can be divided into session processing capacity, target medium attributive function and medium and execution control function.

The session processing capacity can comprise following these functions, and for example RequestAccept[indicates request on the horizon to be accepted.Usually after request is determined effectively in some way (for example, check that client ip address is from the effective coverage, contrast advance payment payment system is checked or is confirmed that with certain alternate manner request is effective).Common this message can be sent out when session begins and the TargetMedia option is defined in script executing (in some cases, needed media parameter when these are used to determine to accept, for example the SDP to the session in RTSP for example limits)], RequestReject (reason code) [utilizes described reason-code indication session should not be accepted.If this function is called, then script after should withdraw from immediately and client will be disconnected connection], SetProtocolHeader (parameter, value) [allowing parameter proprietary to agreement in connection response (being the response of HTTP GET or RTSP DESCRIBE) (for example at HTTP Content-Type head) to define], RequestAbort () [end client and connect (this connections has used RequestAccept to be accepted before).This function can be called at the place, arbitrfary point in script, and script will withdraw from afterwards.Typical reason may be the failure of the specific source contents clauses and subclauses of visit.] and RequestClose (reason-code) [turn-off delicately that client connects and realize the collection (to record CDR) of conversation statistics, if feasible and be configured to and can catch].

The target medium qualified function can comprise as minor function: SetTarget (parameter, value) [is used to limit the media formats that should export at the special session system.This function was published before session (being utilized RequestAccept) is accepted usually, and can accept the independent variable of two character strings as key value centering.A plurality of SetTarget orders can be issued and be used for specifying the output medium form based on several parameters.These parameters can be compulsory or optionally or only need at some conversation type.Example parameter is audio format, audio frequency Maximum Bit Rate, video format, video Maximum Bit Rate, video frame rate, video width, video height, file format, sdp configuration file, sdp audio configuration file, sdp video profile, sdp total bandwidth, sdp audio bandwidth, sdp video bandwidth].

Medium and execution control function can comprise as minor function: QueueMedia (URL) [rank to send client to URL by indication mechanism.If in media queue, there are not other clauses and subclauses etc. pending, then handle and to begin immediately.Any mistake can not blocked or return to this function.Described URL should (be used PHP htmlspecialchars () function or similar function) and be encoded to remove such as ‘ ﹠amp; ' and so on the URL of any spcial character.This can follow calling EventWait (incident wait) after calling] and EventWait (time) [blocking-up script executing when the incident of waiting for such as the transmission of end of handling and media clip.The blocking-up maximum duration can specify with millisecond, and if the time be set to 0, then can forever block.This function can return the field/keyword PHP hash function of have indication event type/kind (" system event " or " media event "), ' incident ' and event-specific parameter when finishing.The example incident can be meant that the media event/stream of the system event that is shown with or do not have the session of release cause and stop/be terminated/the do not have stream parameter in parameter or release cause and the detailed description hash function stops/do not have parameter or AdapterPlayInfo.]

Other function also can be comprised, LogCdr (cdr) for example, this function will call (as transparent character string) detail records as input and record CDR file according to script.

By being exposed to the request entity parameter that can touch script, can realize that to the addressable operation of the advanced client of code conversion parameter this can realize by inquiry string or other method.This can realize that some aspect of asking determined with protection privacy or because the known connection performance that just can know or be easier to know in client in client only by this locality at client device.This client decision can be encoded in server or the determined content page of client-side or be determined at the device port place in advance.There are a variety of transmissible information, as about medium and adaptive information and the out of Memory such as location-based information of medium.An example can be that client is indicated its position (utilizing for example longitude on inquiry string and latitude) and received the related advertisement of reception text with distinguishing the geographic area.

Employed example makes target medium option and origin url are encoded to the URI that is requested that is received from client by base 64 among some embodiment.URI has such as http://contentprovider/source.wmv﹠amp; Vc=h263﹠amp; The inquiry string coding form that the value of vbr=128 etc. and so on is right.The value that is transmitted in the inquiry string can be used to determine the behavior in the content adapter subsequently.

Medium after the code conversion that is transmitted in this manner can be carried out meticulous adjustment to mate requested parameter, rather than the situation of " best under certain condition " (this situation is only carrying out being only when montage is selected feasible at content encoded in advance), and obviously can be continuously rather than handle each parameter discretely, therefore make be issued in known limitation best in quality.

Be supported to comprise DRM or user or product mark as other parameter that directly passes a parameter.RTSP request URI or the distortion that the modification of SIP head or URI also are in an embodiment of the present invention to be supported.

In certain embodiments, in order to integrate with Content Management, can have such as advance payment and post-pay payment standard P HP data base querying or via SOAP or similar agreement to other port service the request of web service.The form of integrating comprises that the value of using data base key to replace base64URL to encode to identify the special assets in the Content Management System and use device databases to replace being encoded in institute's request URL determines the target codec parameters.

The action that some embodiment support scripting is to add from screen or to remove figure/image/video at watermark, sign/trade mark, advertisement prompting or service message.Image is allowed to have multiple form and has their controlled sizes, position and stack type.Can realize that also image information mixes with the Alpha of following layer video (at watermark) and prevent obscuring of image information and following layer video (at menu and prompting).

Some embodiment give script with user's input transfer, for example use ' eventwait ' function.Incident comprises RTSP PLAY (be used to initiate skill and play (trick play)-F.F. (FF)/rewinding (RW) operation); RTSP OPTIONS order (being used in some user's players application) and SIP INFO/UII incident etc. to realize non-standard function.

In certain embodiments, system can Be Controlled or receives input or be directed from external system.This can use remote process call (RPC) to API or be exposed to being defined in the agreement of those controlled entities and realize.Control can be limited at being identified or authorized users or equipment.Control can change the one or several behaviors of a lot of probable behaviors, and described behavior is for example disposed, update content tabulation, script, advertisement, be allowed to user etc.

Output file after some embodiment buffer memory code conversions is so that other client reuses in subsequent session.This buffer memory is performed in the mode that keeps audio sync.Some embodiment allow (initial data) cache contents before content is encoded once more, insert to allow personalized montage, if the control script allows.Original clip and be inserted into montage and can be buffered.

The function of accessed content when some embodiment are supported in the request before of content basis by code conversion and buffer memory.This has realized sharing the code conversion achievement.

The content that is buffered can be stored with the metadata that is associated: one group can be origin url, frame sign, buffer memory expiration, bit rate and frame rate (bit rate and frame rate may not need when the RAW content is buffered, because can be limited by the target code level) with the stored exemplary meta-data of content.

In certain embodiments, metadata is saved to database, and in other cases, metadata can be encoded into (to exempt the needs of internal appearance database) in the filename that is saved to the object on the disk.When clauses and subclauses are derived from the web server, can be obtained from HTTP buffer memory header fields at the buffer memory of each content item expiration rule.Also can ignore HTTP buffer memory rule.

Log cache can be generated, and shows being buffered request, request time, object size and the delivery time of object (hit and miss).Can also generate more buffer memory statistics in certain embodiments, display object distribution in period, clicking rate, period distribute, preceding 100 buffer memorys are clicked object, minimum/maximum/on average look for time, minimum/maximum/average transfer rate.

Some embodiment are made of four main system components as shown in Fig. 5 C.These assemblies can be placed on the individual server or be disposed on a plurality of servers so that customizable solution to be provided.Fig. 5 C shows decomposition texture.These four elements are managent component, protocol processor, system controller and media filter.

Managent component (may have only a managent component in the whole system) is used to collection of statistical data and incident and can be used to check and revise system configuration.

The request that (one or more) protocol processor can be used for from client provides service.A plurality of protocol processors can be set in the system, but in most of the cases, only need a protocol processor at every kind of agreement that will be supported.

System controller can be the element of construction system core, and is used to carry out the application scenarios at each client-requested.System controller also management of media is handled the load balance between the resource (media filter) and the daily record and the CDR of management system and application specific.

(one or more) media filter can be the needed one or more media filters of any system.These media filters provide rudimentary media conversion and video capability.Load on these elements can be by the system controller uniform distribution.

For small-scale service, all elements can be used the solution of ' box ' on individual server, move.Single IP address is used to all services (RTSP and http protocol processor, Web GUI and exterior content request).Capacity can be restricted to max-session number (this number can require according to the code conversion at medium to be reduced).

For need than individual server the fairly large service of the bigger capacity of the capacity that can provide, commerce services for example, system can be based on a plurality of servers.The server blade that this can be arranged in the frame for example uses IBM blade center hardware.

Some embodiment have the media interface that is used to media data between adapter engine and protocol processor.In certain embodiments, this interface uses SCTP.SCTP has the characteristic that is used for this interface, because it is to have congested control, message framework (allowing discrete RTP grouping to be sent out as message) and the reliable connection-oriented data of multithread transmission (be supported in a plurality of message flows that are re-used on single the connection, the loss of data of a stream can not influence any other stream) to transmit.In some cases, SCTP is better than UDP, because UDP is insecure in essence, and because the content source under a lot of situation is transmitted through reliable connection, may interrupt the media data processed in inside so can not accept content adapter.In addition, UDP itself is not provided for the mechanism of current control.SCTP can also support many hosts (therefore, the flow from a plurality of IP address can be transmitted through public connection).

Some embodiment can realize load balance and supervision.For the performance of management system, a kind of mechanism can be set up, thereby adapter engine can suitably be adjusted its oneself load.Depend on needed load adjustment, this can comprise that single system arrives more complicated system.

In certain embodiments, each adapter engine with the quantity of processor core multiply by megahertz again divided by available processor check figure (some embodiment may processor core as other purposes or because permission former limit its use) as the basic tolerance of performance, provide rough estimate (different chip types may need by standard to selected rank, for example Intel686) to overall performance.Some embodiment are equipped with the maximum load parameter, and the purpose of this parameter is that the maintenance system is no more than known rank (for example 80%) and reduces available total MHz effectively.

When each request follows the affirmation ability of AE to arrive AE from the controller that increases another session (target media type is effectively and is supported), should confirm that resource is available for system.This should be at first by at all cost calculation summations of all ongoing sessions and based target media option and, add the based target media option and session separated the hypothesis " cost " of new session of the estimation (interim) of data code.

In case session is established, just should upgrade cost (being stored in the container file and under the situation that can not read via SDP, described information is source code/decode type, bit rate, frame rate, frame sign etc. for example) based on the available fresh information of any change at video.

Fig. 7 A-7C is used for the simplification connection layout that obtains and retrieve from element by content adapter according to embodiments of the invention.Fig. 7 A represents to deposit and is different from the content that the user generates with the characteristic of retrieval facility and deposits storehouse (UGC) or ordinary circumstance unknown or in the network with different or controlled characteristic (encoding and decoding or un-trusted territory).In this case, CA carry out adaptive/code conversion in the path and in can and supplying with in acceptance.Fig. 7 B shows a kind of visit to UGG and only passes through a CA by the situation of interface, and this example is this situation, and content is stored on a set of equipment with a kind of incompatible form, and CA is as the gateway of these equipment.Fig. 7 C shows the situation of UGC code conversion when obtaining.The example here can be equipment and UGC storage container form or media formats is incompatible and when input by adaptive can be so that they have needed form by other user capture.For example, come the daily record montage to be uploaded to the content ports based on web that can use the Flash video, for all other visit from mobile phone via stream or video phone call.

The offered load factor can also be determined to use the difference among Fig. 7 A-7C in arranging or all arrange.According to business model or based on the possibility of desired access to content, can adopt different strategies.For example, can when obtaining, just be carried out code conversion and be ready to be used for immediately known popular poster according to user id or the definite possible accessed content of similar information, but for other high users of different samples are expected in visit, may will code conversion can not take place.In this manner, resource can be saved.

Fig. 8 A-8C shows the simplification system assumption diagram according to the position arrangement of the content video control of the embodiment of the invention.Fig. 8 A-8C shows in an embodiment, can be present in intrasystem diverse location for different purposes at adaptive control.Fig. 8 A shows the two-way control interface between client and content.It is feasible that this position arrangement generally has situation about obtaining with playback interface for content, and needs client to carry out interface with both.The various particular aspects of uploading and downloading can be in the module of being controlled by this control.Fig. 8 B shows the medium that provide at client and message and by the control of various processing modules to the adaptive execution of described medium.Fig. 8 C show at from content provider's medium and message and medium adaptive/control handled, and this control is afterwards multiplexed before being stored in network of medium, metadata/control messages.

Fig. 9 shows the simplification system assumption diagram according to the decomposition texture of the content adapter of the embodiment of the invention.The figure shows at medium adaptive and control between load balancer.Agreement A interface and agreement B interface can be SIP, RTSP, HTTP or other conversation-based agreement.This element can according to the IMS clause in the similar mode of MRFC be provided.Low layer elements can be counted as adopting the MRFP of same way as.Control interface between the assembly can be MSML/MSMO or can be the application specific interface that more restrictions are arranged.

Figure 10 shows the simplification system assumption diagram according to the decomposition texture of the content adapter of bearing the HTTP Proxy role of embodiment.In this embodiment, the present invention is used by the media services based on web, conversion that wherein need be during uploading (HTTP POST, for example blog or YouTube content are uploaded) be set to public setting from the source format and the encoder of wide region or the conversion that is used for downloading (HTTP GET) from single high-quality source format to the output format of wide region.

Figure 11 shows the simplification diagrammatic sketch according to the content adaptation before the caching in the HTTP GET method of embodiment.Medium were converted before buffer memory.Figure 12 shows the simplification diagrammatic sketch of the content adaptation after the caching in according to the HTTP GET method of the embodiment of the invention.The user makes requests for content, this request can comprise among the URL label with indicate desirable object format or agency can check whether the data relevant with URL have triggered media conversion (requested expansion of content, domain name, IP address range, browser type etc.).The agency will seek correct media store (comprising the URL media label) in content stores, and if exist then it is sent back client.If URL triggers or media label is found, media filter should be enabled (data will be redirected from acting on behalf of that by medium converter) then, (for example size conversion) made then to the request from the data of source (origin url).When data were received, it should be transmitted to medium converter before being cushioned ((block wise) for example, block by block).Single connection can be re-used at a plurality of content requests, to keep superperformance.Analyzed and the buffering to the input data of CA is till source media parameter (encoding and decoding, bit rate, frame sign etc.) can be decoded or be definite.In case the code conversion parameter is known, then medium can be postponed, and to wash the beginning open processed and be transmitted.

Figure 13 is the simplification diagrammatic sketch that illustrates the content adaptation before the caching in according to the HTTP PUT method of the embodiment of the invention.HTTP?PUT/POST。User's (people or equipment) makes POST request to server, and this request can be included in label among the URL object format (for example MSMO) with indicative of desired.The agency will check whether data will come adaptive according to rule (expansion of content of being asked, domain name, IP address range, browser type etc.).Media conversion is then for this stream enables medium converter if desired, and is redirected to CA from source (HTTP client) and agency's data.When data were received, it can be transmitted to medium converter before being cushioned ((blockwise) for example, block by block).The input data analyzed and cushion by medium converter, till source details (encoding and decoding, bit rate, frame sign etc.) is known, in case and the code conversion parameter understood, then medium can from the buffering and the subsequent zone BOB(beginning of block) processed.When medium converter was transmitted, data were normally carried out code conversion and are sent to server in data.

Figure 14 is redirected the simplifying the operation of the URL that is included in content adapter according to the embodiment of the invention.Shown in the behavior of filter be its buffer memory initial data (rather than version of filtered or code conversion) always.Alternative allows system's named cache should still carry out after the filtering before filtering (generally speaking or according to content type/URL).The acting server of standard can not be supported in a plurality of objects of buffer memory under the same URL usually, if thereby they to after code conversion, be stored, then may must revise URL be retained to guarantee a plurality of versions.

Figure 15 is the simplification syndeton according to the content adapter that content adaptation is provided of the embodiment of the invention.The agency can be that HTTP Proxy and remote process call (RPC) client functionality can be provided by XML-RPC.In this architecture, the CA controller provides the system-level control at CA, comprises supply, management and control.It comprises URL analyzer and redirector function, and which kind of object requests is this function be used to control can be handled by CA, and where the data flow of object should be directed to (if they are processed) with controlling at self interface of agency.The URL triggering rule should be able to be defined in order, and should comprise the definition of the target medium form that will transmit at URL object that should be processed.In this architecture, the CA engine is to comprise be useful on becomes file the sub-component of another kind of type from a kind of type conversion senior element.Contain control is gone to or from the interface of CA controller.Originally this assembly is developed to the process/application based on independent server, but in order to transmit desired high density data, and it can be transplanted on the platform based on DSP.Single queue is used to return request and be transmitted to each code conversion object---and these must dynamically be made up by under the situation about understanding in " form " of input data.File Open is used to open file when being transmitted from content server (by the agency) at file.Usually the detail of supposition file type can not be determined based on URL, so these information must find by reading file format (code/decode type, bit rate, frame sign etc.).File format is confirmable in some cases, for example the data of returning from known and believable place.The file packing provides the encapsulation at the needed Voice ﹠ Video assembly of destination file format.Audio frequency Xcode provides the audio frequency media transcoder function, and video Xcode provides the video media transcoder function.Media codes conversion may be provided in tandem transcoder (back-to-back decoding and encoder) or can comprise that many aspects aspect media quality and the processing consumption provide the smart code transducer of better performance.These smart code transducers are in U.S. Patent No. 6,829, and 579, No.7,133,521 and No.7,263,481 and U.S. Patent application No.10/620,329, No.10/693,620, No.10/642,422, No.10/660,468 and No.10/843,844 have further description, and disclosing of these patents and patent application is all unconditionally incorporated herein by reference.

Some embodiment have adaptability and support media processing feature streamline to misdeed, and as U.S. Patent application No.10/762, described in 829, disclosing of this patent application is all unconditionally incorporated herein by reference.Handle the ability (or similarly refresh point) of video intra coded frame, other medium notable feature that perhaps know in advance or that obtain according to passage knowledge can greatly improve the quality when sending the passage of makeing mistakes to.In addition; this system can insert extra intra coded frame; perhaps in audio frame, add redundant, perhaps add the additional materials that comes from a customizable video source or add forward error correction (perhaps by increase redundancy or rise time benchmark the protection of the enhancing that realizes of FEC).

For the service of the very high availability of needs, a plurality of independent identical (or similar) systems can utilize the 4th layer to the 7th layer (L4-L7) load balance switch of standard to be configured in together.These systems can be configured to the configuration of Active-Active or activity-standby.These switches can be outside or be positioned in the exchange trough that is arranged in blade center's frame.The system that utilizes the L4-7 switch can exceed the individual server function adjusts function.The 4th layer switch can distribute the RTSP of acceptance or HTTP to ask on 2 or the more a plurality of individual system.These switches can also be by coming the health degree of surveillance to the client requests of their release cycles; If do not receive correct response, switch will shift out the load Sharing pond to relevant system.Figure 16 shows this process.The HTTP of input and RTSP request are assigned in the available CA system according to load Sharing rule (minimum load, wheel is named scheduling etc.).

Figure 17,18,19 and 20 shows mutual between each assembly of the embodiment of various situations.Client, protocol processor, controller, script, adapter engine and source are illustrated.Figure 17 shows the RTSP client that is connected to the HTTP source.Figure 18 shows the HTTP client that is connected to the HTTP source.

Figure 19 shows the RTSP client that is connected to the RTSP source.In certain embodiments, adapter engine begins RTSP SETUP and PLAY automatically and receives data as early as possible and send it to protocol processor (PH) and cushion.Be established in case connect, RTSP PH just cushions received flow data up to receiving playing request; RTSP PH also should have enough buffered datas by the time to guarantee carrying out real-time playback (considering the transfer rate of content-length and input data).In certain embodiments, protocol processor should keep the SCTP socket to open up to it and be connected (this can take place) by the adapter disconnection when removing, so that a plurality of PLAY request can be scheduled out on public client connects.

Figure 20 shows the HTTP client that is connected to the RTSP source.In certain embodiments, the received flow data of RTSP PH buffering is asked up to receiving PLAY; And wait until always and have enough full buffered data to guarantee to reset in real time (transfer rate of considering content-length and input data).

The present invention also makes up aforementioned modules and system maintaining module and system extension module, allows device of the present invention as the redundancy backup of another similar system or as the expanding element of another system.

In order to carry out the function of being mentioned in this specification, the embodiment of system comprises one or more processing units (for example digital signal processor (DSP), FPGA/ASIC or general processor), one or more memory, one or more operating system (allowing the different hardware resource of visit), has the bus and the resource control unit of bus control unit.

It is in order to make any technical staff of this area can both realize or use the present invention that description for preferred embodiment is provided before.Various modifications to these embodiment all are very tangible to those skilled in the art, and the basic principle that is limited here can be applied to other embodiment with not needing creativity.Thereby the present invention does not wish to be limited to shown embodiment here, but is endowed and principle disclosed herein and novel feature the wideest corresponding to scope.For example, depend on embodiment, above function can be combined or further divide.Some feature also can be added or delete.In addition, the particular order of described feature is unwanted in certain embodiments, but may be very important in other embodiments.Depend on embodiment, these a series of processes can be carried out with computer code and/or hardware.Certainly, those of ordinary skills can identify a lot of other distortion, modification and substitute.

In addition, it should also be understood that example as described herein and embodiment are for illustrative purposes, and under the inspiration of described these contents those skilled in the art can advise various modifications or change and these modifications and change the spirit and scope that all will be included in the application and the scope of claims in.

Claims (15)

1. device that is used for the content of multimedia of adaptive one or more networks, this device comprises:

Controller module;

One or more Network Interface Modules;

Be coupled to one or more protocol interface module of described controller module;

Be coupled to the script explanation module of described controller module;

Be coupled to one or more medium adaptation module of described controller module; And

Be coupled to the management interface module of described controller module.

2. device as claimed in claim 1, a medium adaptation module in wherein said one or more medium adaptation module is configured to medium are carried out code conversion, rate transition, size conversion or filtering.

3. device as claimed in claim 1, a protocol interface module in wherein said one or more protocol interface module are adapted to be the session of handling one or more agreements in HTTP(Hypertext Transport Protocol), Session Initiation Protocol or the real time streaming protocol (rtsp).

4. device as claimed in claim 1, wherein medium are received at described device place as Streaming Media and are provided from described device as download or progressive download, and perhaps medium are received and are provided from described device as Streaming Media at described device place as download or progressive download.

5. device as claimed in claim 1, wherein said one or more protocol interface module are adapted to be to utilize first agreement to receive to be adapted to be from first request of client and second protocol interface module in described one or more protocol interface module and utilize second agreement to send second request to content server.

6. device as claimed in claim 5, wherein said first agreement is selected from the protocol groups that is made of HTTP, RTSP and SIP, and described second agreement is selected from the protocol groups that is made of HTTP, RTSP and SIP.

7. device as claimed in claim 5, wherein said first agreement comprise that the HTTP and second agreement comprise RTSP, and perhaps described first agreement comprises that the RTSP and second agreement comprise HTTP.

8. device as claimed in claim 1, wherein said script explanation module are programmed to notify its decision from the database request data.

9. device as claimed in claim 1, wherein said script explanation module are programmed to determine that according to client-requested and server response medium are adaptive.

10. device as claimed in claim 1, wherein said script explanation module are programmed to determine the target medium attribute according to client-requested.

11. device as claimed in claim 1, wherein said script explanation module is programmed to provide to client according to from first media groups in first source with from second media groups in second source and the adaptive media groups that is transmitted, and wherein the one or more attributes in size, speed and the encoding and decoding are different between the described media groups that is transmitted, described first media groups and described second media groups.

12. device as claimed in claim 1, wherein said one or more medium adaptation module are supported one or more data in audio frequency, video or the metadata.

13. device as claimed in claim 1 also comprises the cache module that is coupled to described one or more medium adaptation module.

14. device as claimed in claim 1 also comprises the load balance module that is coupled to described one or more medium adaptation module.

15. device as claimed in claim 14, wherein said load balance module are configured to estimate load and upgrade described load after determining described one or more session characteristic before determining one or more session characteristic.

Images