JP2011030222A - Optimized content-based and royalty-based encoding and distribution of media data - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method and device for performing content-based and royalty-based encoding. <P>SOLUTION: The method includes steps of: (102) selecting one or more encoding tools to encode media, based on a royalty cost associated with at least one of the one or more encoding tools and a corresponding decoding tool for each of the one or more encoding tools, decoded media quality that each corresponding decoding tool produces, and one or more transmission bandwidth constraints; (103) encoding the media, in accordance with the media content, using the one or more encoding tools; and (104) transmitting encoded data generated by at least one of the one or more encoding tools. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

priority

  [0001] This patent application is a corresponding US Provisional Patent Application No. 60 / 898,324, filed January 29, 2007, entitled "Optimized Content-based and Royalty-Based Encoding and Distribution of Media Data". Is incorporated by reference to the provisional patent application.

  [0002] The present invention relates to the field of media data encoding and distribution, and more particularly, the present invention relates to content-based and loyalty-based encoding of media data.

  [0003] In recent years, media encoding and distribution has undergone significant development. By fusing compression and transport tools that define media standards, media can now be transported over a variety of networks using a wide variety of tools that support a wide variety of end-to-end delivery efficiencies. If you are interested in the most efficient media delivery, you are typically often limited to the latest advanced technology tools with high loyalty / license costs. On the other hand, if a certain degree of inefficiency can be tolerated, it may be possible to reduce the loyalty cost or make the loyalty cost free and achieve distribution.

  [0004] In media standards, the use of such tradeoffs is only very limited due to the profiles defined within the standards. In order to realize such a trade-off, it is also possible to select from various standards. However, the number of alternatives and different standards available in profiles is small and often not very meaningful for any application. It is clear that with the widespread use of general-purpose computing devices, there are many situations that benefit from a series of choices that optimally realize such tradeoffs.

  In conventional solutions, the set of options available to the encoder for media distribution is very limited and coarse. If the encoder uses a given profile for a given standard, media distribution can be achieved with a fixed license cost that covers the use of all tools in that profile. The encoder cannot optimize license costs based on, for example, the content of the media being delivered, the desired media quality at the decoder, and the effective bandwidth of the carrier media. The situation is similar if the encoder only has to choose between different media coding standards.

  The tools included in the media standards are standard ones selected and published during the standardization process. This process occurs in a manner that can only roughly accommodate all desired uses of media encoding. The final choice does not include many good tools because they are not applicable to all possible scenarios, and are effective implementations on all possible hardware platforms. For example, the encoding efficiency is low.

  Currently, some tools selected as standards within the media coding standard are used much more frequently than other standard tools. However, the share of these tools in the revenue pool resulting from standard licensing can be the same or less than other tools.

  In the case of media coding using current technology, it is often necessary to license the standard and pay royalties to the organization that manages the patent's patent pool and / or individual IP owners. When this is done, the media is encoded by obtaining a license from the content owner.

  [0005] Disclosed herein are methods and apparatus for performing content-based and loyalty-based encoding. In one embodiment, the method includes decoding one or more encoding tools for decoding media to at least one of the one or more encoding tools and each of the one or more encoding tools. Using the loyalty cost associated with the tool, the decoding media quality produced by each corresponding decoding tool, and one or more transmission bandwidth constraints, and using one or more encoding tools And encoding the media according to the media content and transmitting encoded data generated by at least one of the one or more encoding tools.

  [0006] The present invention will be more fully understood from the following detailed description and the accompanying drawings of various embodiments of the invention, which limit the invention to the specific embodiments. Rather, it is for explanation and understanding.

を得るための例示的な技術を示す図である。
セグメントの達成可能な面の例を示す図である。 ペア依存関係を示す達成可能な面からの切り取りの例を示す図である。 コンピュータシステムの一実施形態のブロック図である。 コンテンツ配信システムの別の実施形態を示す図である。 2 is a flow diagram of one embodiment of a process for encoding and transporting media. FIG. 6 is a block diagram illustrating examples of different media encodings for different devices and applications. FIG. 4 shows the combined loyalty costs required to generate each encoding. 6 is a flow diagram of another embodiment of a process for encoding and transporting media. It is a figure which shows an example of the chain of the dependency relation of the tool used in the decoding of media. FIG. 3 illustrates an exemplary toolset with equivalent functionality. It is a figure which shows the example which decomposes | disassembles media data into a segment per time. It is a figure which shows the example which generate | occur | produces a certificate in a segment. From encoded media

FIG. 6 illustrates an exemplary technique for obtaining
It is a figure which shows the example of the achievable surface of a segment. It is a figure which shows the example of cutting from the achievable surface which shows a pair dependence. 1 is a block diagram of one embodiment of a computer system. It is a figure which shows another embodiment of a content delivery system.

を解く。 [0020] A method and system for encoding and distributing media data is described. In one embodiment, the techniques described herein take into account the loyalty costs of tools used to encode and / or decode media, along with constraints on the quality of the media to be decoded and transmission bandwidth. And is designed to encode media. In one embodiment, media encoded for a high bandwidth environment with the same decoding media quality can be decoded using a tool with low efficiency but low loyalty costs, whereas low bandwidth Encoding is performed so that the media encoded for the environment can be decoded using a tool with high efficiency but high loyalty costs. Such encoding is achieved by optimizing performance with a function that represents the optimal media encoding point for a tolerance of the three values formed by decoding media quality, effective transmission bandwidth, and loyalty cost. Can be done. Ω determines a set of candidate tools, and Q (ω, ρ), B (ω, ρ), C (ω, ρ) are for a given subset ω of tools with a tool parameter set for a vector of values ρ. Represents quality, bandwidth, and loyalty costs. Then, in one embodiment, for each pair (B ₀ , C ₀ ) targeted for bandwidth and loyalty costs, this is done to obtain the optimal media coding point (Q ₀ , B ₀ , C ₀ ). A function such as

Solve.

により、バッファ制約を組み入れるようにして行うことができる。不等式に基づく制約を許容する他の公知の最適化の公式も使用可能である。同様の方法で、他の符号化／復号制約（例えば、メモリ制約、計算複雑性制約、復号遅延制約など）を、最適化に導入することも可能である。これらの技術は、知的財産権（ＩＰ）の所有者を十分に補償するようにメディア配信が実行されて、復号メディア品質及び伝送帯域幅の制約が緩いユーザが、削減されたＩＰコストからの利益を享受し得ることを、保証する。 [0021] The optimization is, for example, by introducing the variable b that determines the requested buffer size into the above equation together with the corresponding Lagrange multiplier γ, ie

By doing so, buffer constraints can be incorporated. Other known optimization formulas that allow constraints based on inequalities can also be used. In a similar manner, other encoding / decoding constraints (eg, memory constraints, computational complexity constraints, decoding delay constraints, etc.) can be introduced into the optimization. These technologies enable media distribution to be fully compensated for intellectual property rights (IP) owners, allowing users with relaxed decoding media quality and transmission bandwidth constraints to reduce the cost of reduced IP costs. Guarantee that you can enjoy the benefits.

  [0022] In the following description, numerous details are set forth to provide a more thorough explanation of the present invention. However, it will be apparent to one skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form, but not in detail, in order to clarify the present invention.

  [0023] Some portions of the detailed descriptions that follow are presented as algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic notations and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of work to people in other industries. The algorithm is generally considered in this case to be a self-consistent step sequence leading to the desired result. Steps are those requiring physical manipulation of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated. It has proven convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like.

  [0024] It should be noted, however, that not all of these terms and similar terms are associated with the appropriate physical quantities and are merely convenient labels applicable to these quantities. As will be apparent from the following description, unless otherwise specified, throughout this specification, explanations using terms such as “processing”, “calculation”, “calculation”, “decision”, or “display” Manipulate data represented as physical (electronic) quantities in computer system registers and memories, as well as physical quantities in computer system memory or registers or other such information storage, transmission, or display devices It should be appreciated that reference is made to actions or processes of a computer system or similar electronic computing device that convert to other data represented.

  [0025] The present invention also relates to an apparatus for performing the operations herein. This apparatus may be specially constructed for the required purposes, or it may comprise a general purpose computer selectively activated or reconfigured by a computer program stored in the computer. Such computer programs can be any type of disk such as floppy disk, optical disk, CD-ROM, and magneto-optical disk, read only memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic or optical card. Or any type of media suitable for storing electronic instructions, each coupled to a computer system bus, etc., and may be stored on a computer readable storage medium.

  [0026] The algorithms and displays presented herein are not inherently related to any particular computer or other apparatus. Various general purpose systems may be used with the program according to the teachings of the present invention, or it may be convenient to construct a more specialized device to perform the required method steps. The required structure for a variety of these systems will appear from the description below. In addition, the present invention is not described with reference to any particular programming language. It will be appreciated that a variety of programming languages may be used to implement the teachings of the invention as described herein.

  [0027] A machine-readable medium includes any mechanism for storing or transmitting information in a form readable by a machine (eg, a computer). For example, machine readable media may be read only memory (“ROM”), random access memory (“RAM”), magnetic disk storage media, optical storage media, flash memory devices, electrical, optical, acoustic, or other forms Propagation signal (for example, carrier wave, infrared signal, digital signal, etc.).

<Overview>
[0028] A method and system for encoding and distributing media data is described. This method encodes the media according to the media content, taking into account the loyalty cost of the tools used to encode and / or decode the media, as well as the quality of the media to be decoded and the transmission bandwidth constraints. It is designed as follows. Media encoded for high bandwidth environments with the same decoding media quality can be decoded using tools with low efficiency but low loyalty costs, while media encoded for low bandwidth environments However, encoding is performed so that decoding is possible using a tool having high efficiency but high loyalty cost. Such encoding is achieved by optimizing performance with a function that represents the optimal media encoding point for a tolerance of the three values formed by decoding media quality, effective transmission bandwidth, and loyalty cost. It can be carried out. Optimization can be done to incorporate buffers and other encoding / decoding constraints. The system delivers media to fully compensate intellectual property (IP) owners, and users with loose restrictions on decryption media quality and transmission bandwidth will benefit from reduced IP costs. Guarantee that you can enjoy.

  [0029] In one embodiment, the system has a media encoding toolbox and the transport tool can be used with many tools with overlapping functions. The system encodes the media using a subset of available tools as selected by the encoding algorithm. The exact list of tools in the subset is sent to a decoder that can access the same toolbox. The bitstream representing the encoded media is then sent to a decoder that decodes the encoded media using known tools and reconstructs the media with some quality.

  [0030] In one embodiment, each tool used in media encoding has a certain loyalty cost that governs its use. The media itself can also have content licensing costs. In one embodiment, the system encodes the media to minimize the combined loyalty cost given the desired media quality level and the effective bandwidth of the transport media, thereby making an optimal tradeoff. Find out. In one embodiment, the system ensures that content and IP owners are fully compensated.

  [0031] FIG. 1 is a flow diagram of one embodiment of a process for encoding and transporting media. This process is performed by processing logic. The processing logic may comprise hardware (circuitry, dedicated logic, etc.), software (such as is run on a general purpose computer system or a dedicated machine), or a combination of both.

  [0032] Reference is now made to FIG. The process begins with processing logic decomposing the media into multiple segments that are encoded separately (processing block 101). Note that this is not a requirement and the media may be encoded in a single segment.

  [0033] Next, the processing logic includes one or more encoding tools for encoding the media: 1) at least one of the one or more encoding tools and / or one or more encoding tools. Select (processing) based on the loyalty cost associated with each corresponding decoding tool, 2) the decoding media quality produced by each corresponding decoding tool, and / or 3) one or more transmission bandwidth constraints Block 102). In one embodiment, a different encoding tool is selected for each segment. In another embodiment, the segments may be encoded multiple times using different encoding tools so that the media segments can be conveyed with different encodings, depending on the desired quality and / or effective bandwidth.

  [0034] In one embodiment, the selection of the encoding tool maximizes quality for one or both tolerances of effective bandwidth and loyalty cost, and the tolerance of one or both of quality and effective bandwidth. Selecting the optimal tool subset to minimize the cost of loyalty or to minimize the bandwidth cost for one or both tolerances of quality and loyalty costs.

  [0035] In one embodiment, the selection of an encoding tool may include decoding media quality, effective transmission bandwidth, and at least one of the one or more encoding tools and each of the one or more encoding tools. It is performed based on a function that represents the media coding point for a tolerance of three values formed by the loyalty cost associated with the corresponding decoding tool. In one embodiment, the processing logic constructs a three-dimensional graph that represents the function. The graph includes a plane that defines three achievable values of quality, bandwidth, and loyalty cost for each segment. The encoding tool for encoding the segment of media is selected from three achievable values. This will be described in more detail below. In one embodiment, the function incorporates buffers and other encoding / decoding constraints. This is done using a Lagrange multiplier-based framework described above in paragraphs [0019] and [0020], or other constrained optimization framework, as is well known to those skilled in the art. Also good. For example, H.M. See Everett's “Generalized Lagrangian method for solving problems of optimal allocation of resources” (see Opera. Res., Vol. 11, pp. 399-417, 63-417).

  [0036] In one embodiment, one of the selected encoding tools is a tool that has substantially the same decoding media quality and is encoded for a high bandwidth environment, but with low efficiency but low loyalty cost. While the media encoded for the low bandwidth environment can be decoded using a tool with high efficiency but high loyalty cost.

  [0037] Using the selected encoding tool, processing logic encodes the media according to the media content using one or more encoding tools (processing block 102).

  [0038] Once the encoded data is generated, processing logic transmits the encoded data generated by at least one of the one or more encoding tools (processing block 103).

<Media, segments, and toolsets>
[0039] In one embodiment, the media includes audio, video, graphics, and / or auxiliary data. In one embodiment, the video is encoded using a general hybrid DPCM scheme, and the audio is a well-known video and audio compression standard, such as h. As is done in H.264 / MPEG-AVC and MPEG-AAC, media data encoded by a general transform coding framework is transmitted via a packet network, for example, the Internet. Is done.

[0040] In one embodiment, media data is considered on a segment basis. The i-th media segment represents media data within the time interval [T _i−1 , T _i ). Thus, if there are S segments, T _s -T ₀ refers to the duration of the media. In one embodiment, segmentation based on non-time variables or additional variables in addition to time is used (eg, based on time and media quality of layered media encoding applications). In one embodiment, a media encoding and transport algorithm toolset Ω is known for both the encoder and decoder.

  [0041] In one embodiment, each Ω tool has a loyalty cost that determines the license cost of the tool for use in encoding the media. In one embodiment, loyalty costs are flexible. For example, the cost may be constant no matter how many times the tool is used during encoding of specific media data, and may be based on the number of times the tool is used during encoding of specific media data. May depend on other tools used to encode specific media data (for example, when used with royalty-free tools to encode specific media data, Loyalty holding tool loyalty may be free) and may be based on the expected quality of the decrypted media (for example, for devices with low resolution displays as opposed to high resolution displays) If sent, video compression-related tools may have different loyalty costs), equivalent bands required to carry media data It can be based on width or application scenario (for example, if media is sent to a mobile phone as opposed to an HDTV set, it can charge a lower loyalty cost, The tool loyalty can be free if used in an environment), or it can be based on media content license terms (eg, used to encode royalty-free content) If you can, you can make the tool loyalty free). In one embodiment, the loyalty cost and license terms are a combined function of these conditions, so the tool or media data itself may have different loyalty costs in different scenarios.

  [0042] Content owner license terms and loyalty costs may vary for different decrypted media quality levels, available bandwidth utilized, and the application to which the media is sent. FIG. 2 is a block diagram illustrating examples of different media encodings for different devices and applications. Referring to FIG. 2, media 201 has been encoded in K different ways to generate encodings 1-K. Encoding 1 and 2 may be intended to be decoded by a mobile phone, and the other encodings 3 to K are intended to be decoded by PDA, HDTV, etc. May be. Each of the encodings 1-K produces different qualities and has different effective bandwidth requirements. This potentially makes the loyalty cost determined by the content owner different.

  [0043] For complex loyalty clauses, the entire system can also be considered as different encodings of media that generate different loyalty costs (eg, content license costs and tool license costs may be combined). FIG. 3 shows the combined loyalty costs required to generate each encoding of FIG. Referring to FIG. 3, each encoding uses a potentially different encoding toolset with different loyalty costs.

[0044] In one embodiment, the loyalty cost and conditions of each tool are stored in a registry that obtains media encoding conditions to determine a certificate Φ _i for the i th segment. For purposes of the description herein, these conditions are parameterized with a vector Γ _i for segment i. In one embodiment, the vector Γ _i is the target media quality Q _i , effective bandwidth B _i , utilization tool Ω _i ⊂Ω, frame rate, frame resolution, expected decoding delay, expected memory usage, expected decoder computational complexity. And associated conditions such as various other encoding / decoding parameters such as expected encoder / decoder buffer fullness. Other parameters include the number of frames in the segment, the type of application (especially if it affects tool licensing), the type of video (eg news, movies, video conferencing, etc.) YUV4: 2: 0), and the number of bits per pixel.

  [0045] FIG. 4 is a flow diagram of another embodiment of a process for encoding and transporting media. This process is performed by processing logic. The processing logic may comprise hardware (circuitry, dedicated logic, etc.), software (such as is run on a general purpose computer system or a dedicated machine), or a combination of both.

[0046] Reference is now made to FIG. The process begins with processing logic segmenting the media data (processing block 401). For a segment i of media data, the encoder processing logic determines a desired condition for media encoding and a set of tools Ω _i that reduces or potentially minimizes the segment loyalty cost under the desired condition. ⊂Ω is selected (processing block 402). The conditions and toolset are then combined in vector Γ _i .

[0047] Next, the processing logic of the encoder obtains the certificate Φ _i of the i th segment by querying the registry using Γ _i (processing block 403). The registry is described in more detail below. In one embodiment, the media is broken down into one segment for non-real time applications such as media streaming, for example, and broken down into segments of uniform length ΔT seconds for real time applications. ΔT is 1, 2, 3,. . . 10,. . . 60,. . 300 or the like. FIG. 7 shows the media broken down into segments in time units. Referring to FIG. 7, media segment 1 is time T _{0 to} T ₁ , media segment 2 is T _{1 to} T ₂ , and so on. Each media segment has a vector Γ _i that includes an associated condition, which in one embodiment is a target media quality Q _i , an effective bandwidth B _i , and a utilization tool Ω _i ⊂Ω (but Other conditions such as the various encoder / decoder parameters described above for the media segment, for example). Thus, media segment 1 has vector Γ ₁ associated with media segment ₁ , media segment 2 has vector Γ ₂ associated with media segment 2, and so on. Similarly, each vector has a certificate associated with it. That is, the vector gamma _1, the certificate [Phi ₁ associated with vector gamma _1, vector gamma ₂ has a certificate, etc. [Phi ₂ associated with vector gamma _2.

[0048] Referring back to FIG. After receiving the certificate, the processing logic of the encoder encodes the media data (processing block 404) and sends the encoded media along with the certificate Φ _i , the list of tools used, and the encoding parameters to the decoder ( Processing block 404).

と呼ぶΓ_ｉの変形を構築する（処理ブロック４０６）。ベクトル

を用いて、処理ロジックは、符号化されたメディア、及び符号化されたメディアを復号する際に使用されるツールセットの合法性を確認する（処理ブロック４０７）。一実施形態においては、デコーダは、ベクトル

をレジストリに伝送し、レジストリは、メディア及びツールセットが合法であることを確認する。 [0049] At the decoder, the processing logic of the decoder decodes the encoded media (processing block 405). In one embodiment, the decoder optionally determines the toolset and conditions to use and is used herein for purposes of this specification.

Constructing a variant of the called gamma _i and (processing block 406). vector

, Processing logic checks the legality of the encoded media and the toolset used in decoding the encoded media (processing block 407). In one embodiment, the decoder is a vector

To the registry, which confirms that the media and toolset is legal.

  [0050] Once the i th segment has been conveyed, the encoder proceeds with encoding the remaining segments and the process of FIG. 4 is repeated. In one embodiment, the granting of certificates associated with the remaining segments (processing block 403) is contingent on one or more confirmations of the previous segment by the registry (processing block 407).

  [0051] In one embodiment, rather than performing loyalty cost optimization on a segment-by-segment basis, the encoder optimizes the loyalty cost of all segments together.

[0052] In one embodiment, the tool set Ω is determined as the encoding and transport algorithm contributed by the IP owner of the applicable compression and transport tool, and the tool of Ω _i ⊂Ω is in the dependency graph The decoder is notified using well-known formats such as XML and RDL that allow detailed description of the tools in FIG. 5 shows an example of the dependency graph. Referring to FIG. 5, a media decoder 501 receives encoded data and decodes it using toolset 1-T or an algorithm to generate decoded media 502. Tools labeled Tools 1-T are interconnected, and the output of one tool is available as an input to another tool to build the well-known media decoding possibilities.

  [0053] The currently developed MPEG-RVC standard tool specification can be used to package information that allows the decoder to identify which encoding tool was used to generate the encoded data. is there. In one embodiment, equivalently functional tools are used in the dependency graph instead of specific tools or instead of multiple tools. FIG. 6 shows an exemplary set of tools with equivalent functionality. Referring to FIG. 6, a set of tools / algorithms and motion compensation algorithms that can be used in decoding compressed video are shown. The tool is progressively sophisticated (integer pixel accuracy to 1/8 pixel accuracy) as the accuracy of the compensateable motion compensation increases. For more advanced tools, better quality is obtained.

<Example of media encoding and distribution system>
[0054] FIG. 8 is a block diagram of a media encoding and distribution system that includes a registry responsible for certificate generation. Referring to FIG. 8, a media server 801 is communicatively coupled to a registry 802 and one or more user terminals 803. This coupling may be a direct connection or a coupling with one or more intervening elements. For example, in one embodiment, media server 801 is communicatively coupled to registry 802 and one or more user terminals 803 over a network (eg, a packet-based network, the Internet, etc.).

[0055] As described above, in one embodiment, the media server 801 includes a processing unit 813 that performs the media processing operations described herein. Such operations include media decomposition, segmentation, vector generation and processing, and other operations not performed by other functional units of the media server 801. In one embodiment, the media server 801 also includes an encoding unit 820 for encoding a segment of media using one or more encoding tools. In one embodiment, the media server 801 includes a communication interface 811 for communicating with a user. The communication interface 811 may provide the user with different delivery options. Different delivery options may be associated with different decoding media quality and / or different transmission bandwidth constraints, where different delivery options are selected for presentation based on one or more encoding parameters. May be. In one embodiment, in such a case, the tool selection unit 812 of the media server 801 may be based on one selection of delivery options received by the user and / or to one or more encoding parameters. Based on this, one or more encoding tools are selected. For example, the following options may be given in terms of quality and bandwidth.

  [0056] In one embodiment, the interface may dynamically adjust the price of the offered option based on the customer requirements of the option. Customer demand may be measured in the last T seconds, where T may be 1, 2, 51, 600, 1000000, or any time interval.

[0057] Using the processing unit 813, the media server 801 breaks the media into segments and provides a vector Γ _i that includes the conditions applicable to the registry 802. In one embodiment, the vector gamma _i of segment i contains targeted media quality Q _i, the effective bandwidth B _i, a utilization tool Ω _i ⊂Ω. In one embodiment, the registry 802 is a trusted server that stores loyalty costs and provides certificates. Thus, in response to the vector Γ _i containing the relevant conditions from the media server 801 for the i th segment, the registry 802 provides the certificate Φ _i for the i th segment. In one embodiment, in a real-time application, a registry 802 is queried for each segment to obtain a certificate.

  [0058] In one embodiment, the registry certificate covers encoding tool licensing and media licensing from media content owners. In one embodiment, registry 802 may be queried by IP and content owner. In this way, IP and content owners can monitor media licensing and legal use. In this way, a combined certificate that allows licensing of usage tools and transported media to be beneficial to license holders and standard patent pools and / or individual IP owners is a single body. Obtained from. A very wide range of license terms may be allowed. That is, system content owners and technical IP owners may define a continuum of license terms that enable new applications that were still impossible due to prohibitively high licensing costs. In this way, licenses can be easily and dynamically secured and distributed, and IP and content can be used legally and fairly.

  [0059] In one embodiment, a sufficient set of possibilities is defined to deliver media to disparate devices and applications so that IP and content owners are fairly compensated. Furthermore, in one embodiment, tools that are not selected for standard normalization can contribute to niche applications that benefit from these tools. In this way, not only the owners of standard tools within the media coding standard, but also the owners of tools that are not selected for standard standardization can be compensated fairly.

  [0060] The encoder unit 810 of the media server 801 may be responsible for communicating with the registry 802 via the communication interface 811. In one embodiment, for non-real-time applications, the encoder may obtain a certificate prior to all media transmissions so that registry 802 is not queried during media distribution.

を出力する。 [0061] After obtaining the certificate, the media server 801 provides the encoded media and associated certificate to the user terminal 803. In one embodiment, the user terminal 803 includes a decoder for decoding the encoded media. The user terminal 803 also includes a detector for detecting loyalty cost related parameters from the encoded media data. The detector may be part of the decoder. FIG. 9 is a block diagram of one embodiment of a detector that detects parameters associated with determining loyalty costs from encoded media. Referring to FIG. 9, the detector 900 includes two detection units that receive the encoded data 900 and perform two types of detection. The detection unit 910 detects the target media quality Q _i and the effective bandwidth B _i , while the detection unit 911 detects the utilization tool Ω _i ⊂Ω necessary for decoding. Detector 900 is a vector of segment i

Is output.

を報告する。この割合は、例えば、１％、７％、１０％、４３％などの多数の異なる割合であってもよい。ユーザ端末の検出器（又はデコーダ）はまた、最新のｋ個のセグメント（ｋ＝１，２,７,４９など）に関する

を報告するよう、レジストリ又は報告サーバによってポーリングされる。 [0062] In one embodiment, the detector may be a randomly selected segment in a registry (eg, registry 802) or another reporting server so that a certain percentage of the segment is used during reporting.

To report. This percentage may be a number of different percentages, such as 1%, 7%, 10%, 43%, for example. The user terminal detector (or decoder) also relates to the latest k segments (k = 1, 2, 7, 49, etc.).

Polled by the registry or reporting server.

  [0063] In another embodiment, the certificate is obtained by the user terminal 803. In such a case, the user terminal 803 communicates with a registry such as the registry 802 (illustrated with a dotted line).

[0064] FIG. 13 illustrates another embodiment of a content distribution system. Referring to FIG. 13, the system includes at least one service provider server 1301 for serving content to one or more customers in encoded form. The system also includes one or more content provider servers 1306 _i- 1306 _N for providing content to server 1301. The content provider may be part of the server 1301. An encoding service 1302 is included for encoding content. The encoding service 1302 encodes the content using an encoder selected based on the loyalty cost. The encoding service may be part of the server 1301. Registry 1303 is communicatively coupled to server 1301 to maintain a record of a certificate associated with at least a portion of the content. As described above, the certificate is used for confirmation of encoded data. These units may be directly coupled to each other or may be communicatively coupled via a network, such as network 1305.

<Achievable decoding media quality>
[0065] In one embodiment, the encoder selects the encoding tool for the tool set Ω using the achievable surface for each media segment. An encoder or other device builds an achievable surface for each media segment to easily select a tool that provides a particular media quality for a given effective bandwidth and loyalty cost. This aspect is allowed by the Ω tool so that the subset of tools that produce the highest quality for a given effective bandwidth and loyalty cost will determine the intercept of the surface at that bandwidth and loyalty cost. Can be generated by encoding the media segment in any possible way. FIG. 10 illustrates an exemplary aspect defined by three values that can be achieved: [quality, bandwidth, loyalty cost]. Referring to FIG. 10, all three values below the face are considered “attainable”.

  [0066] Also, the pair dependency on the surface as shown in FIG. 10, that is, two of the three parameters quality, effective bandwidth, and loyalty cost fluctuate, and the third parameter is constant. A relationship is obtained that shows the effect of the state that remains. FIG. 11 shows an example of cutting from the plane showing the pair dependency.

  [0067] The graph may be represented in many other ways. For example, in one embodiment, the graph is represented as a table. In other embodiments, the graph is represented using simpler functions and associated parameters. In another embodiment, the program takes input to the function and generates the output of the function. These are only examples and many other embodiments are available.

  [0068] In one embodiment, the encoder uses a parameterized model-based approximation to generate an achievable surface. In one embodiment, these approximation parameters are determined from the media being encoded (e.g., video includes statistics that quantify frame-level standard deviation or other statistics of pixel intensity and motion activity). Can be used and audio can use statistics to quantify the height of the frequency). The model can be formed using a well-known velocity strain or velocity perceptual quality model using a subset of acceptable tools. One such model is described in Yin et al., “Rate-Distortion Models for Video transcoding” (SPIE Conference on Image and Video Communications and Processing, Vol. 5022, pp. 467-488, Jan.

  [0069] In one embodiment, when a plane is generated in a desired range of three values of interest [quality, effective bandwidth, loyalty cost], the encoder Maximize quality with tolerance, minimize loyalty cost with tolerance of [Quality, Effective Bandwidth], or minimize bandwidth cost with tolerance of [Quality, Royalty Cost] Select a subset of tools to limit.

  [0070] Reasons that are not applicable to all possible scenarios, or not suitable for efficient implementation on all possible hardware platforms, reduce coding efficiency, etc. From tools that are not included in the final selection of media standards may be feasible (since the subset of tools used is adaptively determined), efficiency is almost one aspect of the selection process Please note that.

<Example of computer system>
[0071] FIG. 12 is a block diagram of an exemplary computer system that may perform one or more of the operations described herein. With reference to FIG. 12, the computer system 1200 may comprise an exemplary client or server computer system. Computer system 1200 includes a communication mechanism or bus 1211 for communicating information, and a processor 1212 coupled to information processing bus 1211. The processor 1212 includes a microprocessor, but is not limited to a microprocessor such as a Pentium (trademark) processor or a PowerPC (trademark) processor.

  [0072] System 1200 further comprises random access memory (RAM) or other dynamic storage device 1204 (referred to as main memory) coupled to bus 1211 for storing information and instructions executed by processor 1212. ing. Main memory 1204 may also be used to store temporary variables or other intermediate information during execution of instructions by processor 1212.

  [0073] The computer system 1200 also includes a read only memory (ROM) and / or other static storage device 1206 coupled to the bus 1211 for storing static information and instructions for the processor 1212, and magnetic and optical disks. And a data storage device 1207 such as a disk drive corresponding thereto. Data storage device 1207 is coupled to bus 1211 for storing information and instructions.

  [0074] The computer system 1200 may further be coupled to a display device 1221 such as a cathode ray tube (CRT) or a liquid crystal display (LCD) coupled to the bus 1211 for displaying information to a computer user. An alphanumeric input device 1222 that includes alphanumeric and other keys may be coupled to the bus 1211 for communicating information and command selections to the processor 1212. An additional user input device communicates direction information and command selection to the processor 1212 and is coupled to the bus 1211 for controlling cursor movement on the display 1221, a mouse, trackball, trackpad, stylus, or cursor A cursor control 1223 such as a direction key.

  [0075] Another device that may be coupled to the bus 1211 is a hard copy device 1224 that may be used to mark information on media such as paper, film, or similar types of media. Another device that may be coupled to the bus 1211 is a wired / wireless communication function 1225 for communicating with a telephone or handheld palm device.

  [0076] Note that any or all of the components of system 1200 and associated hardware may be used in the present invention. However, it should be understood that other configurations of the computer system may include some or all of these devices.

  [0077] Many variations and modifications of this invention will become apparent to those skilled in the art after reading the foregoing description, but any particular embodiment shown and described by way of example is not limiting. It should be understood that it is not intended to be regarded as a thing. Accordingly, references to details of various embodiments are not intended to limit the scope of the claims that enumerate only the features that are considered essential to the invention.

  501 ... Media decoder, 502 ... Media, 801 ... Media server, 802 ... Registry, 803 ... User terminal, 810 ... Encoder unit, 811 ... Communication interface, 812 ... Tool selection unit, 813 ... Processing unit, 820 ... Encoding unit.

Claims (31)

One or more encoding tools for encoding the media may be a royalties associated with at least one of the one or more encoding tools and a decoding tool corresponding to each of the one or more encoding tools. Selecting based on the cost, the decoding media quality generated by each corresponding decoding tool, and one or more transmission bandwidth constraints;
Encoding the media in accordance with the media content using the one or more encoding tools;
Transmitting encoded data generated by at least one of the one or more encoding tools;
Including methods. Further comprising disassembling the media into a plurality of segments;
Encoding the media includes encoding each of the plurality of segments;
The method of claim 1. Further comprising the step of constructing a three-dimensional graph having surfaces defining three achievable values of quality, bandwidth, and loyalty cost for each segment;
The step of selecting the one or more encoding tools to encode each segment is performed from the achievable three values.
The method of claim 2. Obtaining at least one certificate for each segment;
A list of any encoding tools used to encode the transmitted encoded data, a certificate associated with the encoded data, and a certificate associated with the encoded data, and one or more encodings; Sending to the decoder along with the parameters;
The method of claim 2 further comprising:   The method of claim 1, further comprising decoding the encoded data. Determining a list of the encoding tools and the conditions under which the encoding tools in the list were selected;
Sending the list of encoding tools and the conditions to a registry to verify the legality of the media and the encoding tool list;
The method of claim 5 further comprising:   The method of claim 1, further comprising detecting one or more loyalty cost related parameters from the encoded media data. Selecting one or more encoding tools comprises:
Maximize quality with acceptable bandwidth and / or loyalty costs,
Minimize loyalty costs within one or both tolerances of quality and effective bandwidth, or
To minimize bandwidth costs within one or both tolerances of quality and loyalty costs
The method of claim 1, comprising selecting an optimal tool subset.   At least one of the selected encoding tools is a medium encoded for a high bandwidth environment with substantially similar decoding media quality, using a tool with low efficiency but low loyalty cost Claims wherein the media is encoded such that media encoded for a low bandwidth environment can be decoded using a tool with high efficiency but high loyalty costs while decodable. Item 2. The method according to Item 1.   The step of selecting the one or more encoding tools includes decoding media quality, effective transmission bandwidth, and each of the at least one of the one or more encoding tools and each of the one or more encoding tools. The method of claim 1, wherein the method is performed based on a function that represents a media coding point with a tolerance of three values formed by a loyalty cost associated with a corresponding decoding tool.   The method of claim 10, wherein the function incorporates a buffer constraint.   The method of claim 1, further comprising obtaining a certificate for each segment of the encoded media prior to transmitting the encoded data. Further comprising sending instructions to the user for a plurality of different delivery options;
Each delivery option is associated with at least one of the decoded media quality and transmission bandwidth constraints;
Selecting the one or more encoding tools is based on a user selection of one of the different delivery options;
The method of claim 1.   The method of claim 13, further comprising dynamically adjusting one or more of the different delivery options based on a user request. A toolset selection unit for selecting an encoding tool for encoding a video segment based on encoding parameters;
An encoder coupled to the toolset selection unit, segmenting the video into a plurality of segments and using a tool selected by the toolset selection unit, based on encoding parameters provided by the encoder; The encoder for encoding each of the plurality of segments;
A media server comprising:   16. The media server of claim 15, further comprising an interface for obtaining a certificate for each segment of the media encoded prior to transmitting the encoded data. Further comprising an interface for sending an indication of a plurality of different delivery options to the user;
Each delivery option is associated with at least one of the decoded media quality and transmission bandwidth constraints;
The toolset selection unit selects the one or more encoder tools based on a user selection of one of the different delivery options;
The media server according to claim 15. The media server of claim 17 , wherein the interface adjusts one or more of the different delivery options based on a user request.   The media server of claim 15, wherein the encoder selects an encoding tool for at least one of the plurality of segments based on a best loyalty cost at a given quality and bandwidth.   16. The media server of claim 15, wherein the encoder packages the encoded video segment using MPEG RVC. A system that enables media delivery transactions,
A toolset selection unit that selects an encoding tool for encoding a segment of media based on parameters including quality, bandwidth, and loyalty cost;
An encoder coupled to the toolset selection unit, segmenting media into a plurality of segments, and using the tools selected by the toolset selection unit, based on parameters provided by the encoder The encoder for encoding each of the segments and delivering a bitstream of encoded video data to another location of the system;
A media server comprising:
A registry that is communicatively coupled to the media server and maintains instructions for the tools used by the media server to encode the video content;
A system comprising: The registry generates a certificate for each segment encoded by the encoder;
The certificate is the encoded data corresponding to each segment and confirms the legality of the encoded data transmitted as part of the bitstream by the encoder;
The system of claim 21.   23. The system of claim 22, wherein the certificate specifies an encoding tool used by the encoder to generate the encoded data corresponding to each segment.   The system of claim 21, wherein the registry keeps track of content and loyalty costs, instructions for tools used by the encoder, and quality and bandwidth at which the media was delivered.   The system of claim 21, wherein the registry stores loyalty costs and conditions associated with each of the plurality of tools and determines a certificate for each segment in response to media encoding conditions.   The media coding condition includes at least one of a group consisting of effective bandwidth, target media quality, and one or more parameters associated with either or both of the encoder and decoder at the beginning of the segment. 26. The system of claim 25.   27. The system of claim 26, wherein the toolset selection unit selects from a plurality of different encoding tools that each have a different pricing and produce different quality decoded data at the same bandwidth. An article of manufacture having one or more computer-readable storage media storing instructions, wherein the instructions are executed by the system when executed by the system,
One or more encoding tools for encoding the media are loyalty associated with at least one of the one or more encoding tools and a decoding tool corresponding to each of the one or more encoding tools. Selecting based on cost, decoding media quality generated by each corresponding decoding tool, and one or more transmission bandwidth constraints;
Encoding the media in accordance with the media content using the one or more encoding tools;
Transmitting encoded data generated by at least one of the one or more encoding tools;
An article of manufacture that causes a method comprising: The method comprises
Obtaining at least one certificate for each segment;
A decoder for the transmitted encoded data and a certificate associated with the data, along with a list of optional encoding tools and one or more encoding parameters used to encode the transmitted encoded data; 29. The article of manufacture of claim 28, further comprising: The method comprises
Decoding the encoded data;
Determining the encoding tool list and the conditions under which the encoding tools in the list were selected;
Sending the encoding tool list and conditions to a registry to verify the legality of the media and the encoding tool list;
30. The article of manufacture of claim 28, further comprising: A service provider server for serving content to one or more customers in an encoded format;
One or more content provider servers for providing the content to the server;
An encoding service for encoding content, wherein the content is encoded using an encoder selected based on a loyalty cost; and
A registry communicatively coupled to the server, wherein the registry maintains a record of a certificate associated with at least a portion of the content and used for verification of encoded data of the at least portion of the content. Registry,
A system comprising:

Images