EP2039172A2

EP2039172A2 - Image complexity computation in packet based video broadcast systems

Info

Publication number: EP2039172A2
Application number: EP06787390A
Authority: EP
Inventors: Praveen A. Mohandas
Original assignee: Symmetricom Inc
Current assignee: Microsemi Frequency and Time Corp
Priority date: 2006-07-10
Filing date: 2006-07-12
Publication date: 2009-03-25
Also published as: CA2622548A1; WO2008127217A2; CN101502112A; WO2008127217A3; MX2008014372A; KR20090045882A; EP2039172A4; BRPI0614591A2; JP2009543513A

Abstract

A method to determine real time image complexity in video streaming, IPTV and broadcast applications using a statistical model representing channel bandwidth variation and image complexity that considers scene content changes. Available channel bandwidth is distributed unevenly among multiple video streams in proportion to bandwidth variation and image complexity of the broadcast video stream. The distribution of available channel bandwidth is determined based upon an image complexity factor of each video stream as determined from probability matrices considering bandwidth variations and image complexity.

Description

IMAGE COMPLEXITY COMPUTATION IN PACKET BASED VIDEO BROADCAST SYSTEMS

DESCRIPTION

BACKGROUND OF THE INVENTION

[Para 1 ] The present invention relates generally to broadcast systems. More particularly, the present invention pertains to methods of estimating the complexity of a series of images in compressed video programs that use MPEG compatible encoding. [Para 2] In typical broadcast systems, such as in IPTV (Internet Protocol Television) and direct broadcast satellite (DBS) applications, multiple video programs are encoded in parallel, and the digitally compressed bitstreams are multiplexed onto a single, constant or variable bit rate channel. The available channel bandwidth could be distributed unevenly among programs, in proportion to the information content/complexity of each of the video sources. The monitoring system that computes video quality by measuring impairments could take into account the image complexity factor of the video stream to calculate the different effects of impairments on lesser or more complex images. [Para 3] MPEG encoded variable bit rate (VBR) video traffic is expected to dominate the bandwidth of broadband networks. This could be delivered in streaming, on demand, IPTV or DBS types of environments. Accurate models of VBR or CBR video complexity is necessary to enable monitoring systems for prediction of performance of any proposed network during its operation. FIG. 1 shows components that are involved in delivering video content in a typical IPTV environment. Video source that originates as analog signal is encoded using an encoder and packetized and sent using an IP network. It could be sent as manage subscribers and traffic flows. The content is stored in content servers and delivered on demand upon user request. At various points in the network, measurements can be performed for impairments by service assurance managements systems. [Para 4] MPEG coding standards define three picture types (I, B and P) and encodes pictures with a fixed arrangement. Picture type changes could occur due to scene transitions. In the event of an abrupt transition, the first frame of the new scene is intra- coded (l-frame) in order to avoid severe coding errors. During a gradual scene transition, the distance between two reference frames (I or P) can be changed to improve the picture quality. During most of these gradual transitions, temporal correlation tends to be reduced. This situation demands more frequent placement of predicted reference frames (P-frames) to uphold the required picture quality. When the video sequence contains rapid motions, this may also require frequent P-frames in order to improve picture quality. This increases the bit rate. On the other hand, if the scene does not contain any rapid motions or gradual scene transitions, the inter-frame (l-frame) reference distance can be increased without affecting the picture quality. This is due to the strong correlation between frames. [Para 5] Accordingly, what is needed is a process to analyze the Video Coding Layer (VCL) complexity indication changes and bit rate changes in the video stream by analyzing VCL parameters including, but not limited to slice, macroblocks, quantization, INTER/INTRA coded reference and non-reference macroblock/slice/picture types and arrive at a statistical model to compute image complexity dynamically, so that impairment monitors could use this value to determine their effect on a sequence of complex images.

SUMMARY OF THE INVENTION by statistical analysis of VCL parameters and bandwidth variation in video program stream. This value could be used by monitoring and other applications to estimate video quality in loss states, and make a better estimate on perceived quality by the human visual system. [Para 7] The process for broadcasting multiple video streams on a single channel begins with analyzing complexity indication changes and bit rate changes in a video coding layer of each of the multiple video streams. Next, a statistical model is created to dynamically compute the image complexity of each of the multiple video streams. The effect of the image complexity of each of the multiple video streams on the broadcast is then determined. Available channel bandwidth is distributed among the multiple video streams based upon the determined effect of the image complexity of each of the multiple video streams.

[Para 8] The process further involves estimating video quality in certain loss states. [Para 9] Analyzing the complexity indication changes involves analyzing changes in parameters of discrete sections of the video streams. The discrete sections of the video streams include slice, macroblocks, quantization, inter-coded reference blocks, intra-coded reference blocks, and non-reference macroblock/slice/picture types. [Para 10] Creating the statistical model involves creating a first statistical model of video coding layer complexity indication changes for discrete sections of each video stream. Further, a second statistical model of video coding layer bit rate changes or bandwidth variation is created for the same discrete sections of each video stream. The first and second statistical models from the discrete sections of each video stream are then combined. Image complexity of the discrete sections of each video stream is calculated based upon the combined first and second statistical models. types for picture/slice/macroblock types are counted by determining quantization changes in each video stream. Bandwidth variation is counted by determining the bandwidth of the video coding layer data in each video stream. The counting is accomplished by incrementing a first counter for each quantization change, a second counter for each macroblock, a third counter for each slice, and a fourth counter for each low, average and high bandwidth state transition.

[Para 1 2] A probability for complexity of the video coding layer complexity for discrete sections of each video stream is computed using the first, second, third and fourth counters. Further, a probability for low, average and high bandwidth states for the discrete sections for each video stream is computed using the first, second, third and fourth counters. A first transition probability matrix is constructed for video coding layer complexity transition of the discrete sections of each video stream and a second transition probability matrix is constructed for bandwidth state transition of the discrete sections of each video stream. An image complexity value of the discrete sections of each video stream is computed using limiting state probabilities obtained from each transition probability matrix.

[Para 1 3] The method can be used by collectors to get image complexity value from distributed remote probes; to facilitate computation of impairments in packetized video stream using image complexity as a variable to get more accuracy towards perceived video quality; to provide image complexity at regular intervals for packetized video applications; to provide an estimation on video complexity as perceived by human visual system; to provide Image complexity measurements for typical industry wide video quality assessment models, including and not limited to Peak Signal to Noise Ratio (PSNR), MPQM, MQUANT and Root Mean Square Error (RMSE); to provide offline and real time image complexity , VOD servers (video on demand), broadcast servers and video quality measurement equipments; to provide a statistical model for bandwidth variation that contributes to image complexity; to provide a statistical model for video coding layer complexity that contributes to scene transitions; and to determine the statistical distribution of series of images in a low complexity state and a high complexity state.

[Para 14] Other features and advantages of the present invention will become apparent from the following more detailed description, taken in connection with the accompanying drawings which illustrate, by way of example, the principals of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS

[Para 1 5] The accompanying drawings illustrate the invention. In such drawings:

[Para 16] FIGURE 1 shows an example of an IPTV (IP television) distribution network with potential points where measurements for image complexity could be done;

[Para 1 7] FIGURE 2 shows a typical protocol stack where MPEG frames are encapsulated in

IP (Internet Protocol) and where the values for measurement are extracted at the VCL level;

[Para 1 8] FIGURE 3 shows a statistical model for computing Image Complexity with the final curve fit equation;

[Para 19] FIGURE 4 shows a Markov transition process for a Bandwidth model;

[Para 20] FIGURE 5 shows a Markov transition process for a Video Coding Layer

Complexity model;

[Para 21 ] FIGURE 6 shows the counters and transition matrix relationship for the

Bandwidth model; _{1 1} jf (Uj_j»jH|,!,IMBV ' ^l i r ri V

Coding Layer Complexity model;

[Para 23] FIGURE 8 shows the transition probability matrix for a Bandwidth variation model;

[Para 24] FIGURE 9 shows the transition probability matrix for Video Coding Layer complexity model;

[Para 25] FIGURE 10 shows the probability values and curve fit equation relationship that computes image complexity; and

[Para 26] FIGURE 1 1 shows the flowchart for the bandwidth and Video Coding Layer model computation

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[Para 27] A preferred embodiment of the present invention is illustrated in FIGS. 2-10. An embodiment of the present invention can be utilized in an IPTV delivery system such as that illustrated in FIG. 1 .

[Para 28] The present invention relates to a method of estimating image complexity in a series of images in a video stream supporting MPEG type picture encoding. The method includes creating, during a flow of encoded video stream, a statistical model representing the VCL parameters as quantization, macroblock/slice counts, macroblock sizes 16x16, 1 6x8, 8x8, 4x4, 8x16, picture type variation as inter, intra, I/B/P frame/macroblock types variation that determines the probability of causing scene transitions. During the same flow of encoded video stream, a statistical model representing bandwidth variation that determines the probability of high and low bandwidth states is also created. Image complexity is then determined from the two statistical models created from the same flow estimate perceived video complexity.

[Para 29] The method also includes: determining the quantization changes to count the high quantization transitions, slice/macroblock counts for the monitoring interval, Inter/lntra prediction types for picture/slice/macroblock types (I, B, P) and determining the bandwidth of VCL data to count the bandwidth variation; incrementing a counter for quantization changes, incrementing counters for macroblock and slice types and sizes, and incrementing a counter for bandwidth low, average and high state transitions; computing probability from the counters for state transitions for video coding layer complexity, and computing probability from the counters for state transitions for low, average and high bandwidth states; and computing a transition probability matrix for video coding layer complexity transition and computing a transition probability matrix for bandwidth state transition.

[Para 30] As outlined above, FIG. 1 shows a typical IPTV distribution network 21 that includes Video content acquisition 1 2, IPTV management system 14, IPTV content distribution 16 and IPTV consumer 18. Video Source 20 is usually acquired in analog form and encoded in MPEG 1 /2/4 format by a video encoder 22 and sent to a Video on Demand (VOD) server 24 or a Broadcast server 26. The VOD server 24 encapsulates the content into a program stream for transport to a network core 28. The network core 28 is a relatively higher bandwidth pipe. An IPTV network 21 also consists of a variety of management, provisioning and service assurance elements. Typically it includes the Operation Support System (OSS) 30, Subscriber management system 32 and application servers 34 to create new value added services. Following the management, provisioning and service assurance, the content could be stored in a VOD server 36 or a broadcast server 38 that is accessible by the consumer. It is typically located at an edge 40 of the network 21 . A consumer has , is typically connected to a setop box 46 that decodes the video stream to component output.

[Para 31 ] A protocol stack for a packetized video stream is illustrated in FIG. 2. Media dependent attachment 48 could be Ethernet, Sonet, DS3, cable, or DSL interface. A PHY 50 does the media dependent packet processing. An IP (Internet Protocol) 52 is the network layer part that provides mainly addressing for packet routing in the IPTV network 21 . A UDP/RTP 54 is the transport layer that provides application level addressing for ports. The video stream can be encapsulated in the UDP/RTP or just UDP layer 54. The encoded video can be compressed in MPEG 1 /2/4 and sent as a transport stream or in RTP encapsulation for video 56. There can be an optional Network Abstraction Layer 58 as is the case for H.264/AVC. A video coding layer packet input 60 is decoded and necessary parameters are extracted to get the values for measurement 62 for the image complexity model, as described below.

[Para 32] FiG. 3 provides the high level logic for the statistical models in an embodiment of the present invention. MPEG VCL input 64 is provided to both a VCL complexity (l-frame) model 66 and a bandwidth model 68 to compute the counters needed for the statistical models. A curve fit equation 70 takes the model output parameters and computes image complexity 72.

[Para 33] FIG. 4 illustrates discrete Markov process state transitions for the bandwidth model 68. The bandwidth variations in video sequence are modeled into a three state Markov process to determine the probability of low and high bandwidth state transitions. State one (Sl ) 74, State two (S2) 76 and State three (S3) 78 respectively represent states of the model 68 in low, constant and high bandwidth states. . .« complexity quantization model 66. The quantization transitions retrieved from the macroblock layer is modeled into a two state Markov process. Kl 80 and K2 82 show states of the VCL layer complexity model 66 - quantization high and quantization low states. [Para 35] FIG. 6 illustrates the counters 86 used to compute the transition probabilities 90 of the bandwidth model 68. A VCL bandwidth monitor 84 monitors the bandwidth variations in the VCL stream and updates counters cXY 86, where X represents the initial state and Y represents the resulting state. The initial and resulting states may be low, constant or high bandwidth states designated as 1 , 2 or 3, respectively. For instance, Cl 1 represents the state transition event from a low bandwidth state 74 to a low bandwidth state 74, and C23 represents the state transition event from a constant bandwidth state 76 to a high bandwidth state 78.

[Para 36] State transition probabilities 90 are computed to get a transition matrix 88. The State transition probabilities 90 are represented by pXY where X represents the initial state and Y represents the resulting state. The initial and resulting states may be low, constant or high bandwidth states designated as 1 , 2 or 3, respectively. For instance, pi 2 is the transition probability to go from the low bandwidth state (Sl ) 74 to the constant bandwidth state (S2) 76. From the transition probabilities 90, the transition matrix 88 is formed. From the transition matrix 88, limiting state probabilities are computed without the initial conditions to get BPl 01 92 and BPl 03 94. These values represent the probability to stay in the low bandwidth state and the high bandwidth state, respectively. [Para 37] Counters 98 used to compute the transition probabilities for the VCL layer complexity quantization model are seen in FIG. 7. A VCL slice and macroblock monitor 96 monitors the quantization parameter in the macroblock and updates counters dXY where X represents the initial state and Y represents the resulting state. The initial and resulting , , respectively. For instance, dl 2 represents the state transition event counts from a quantization high received state to a quantization low received state. State transition probabilities are computed to get a transition matrix 100 for the VCL layer quantization model 66. From the transition matrix 100, the probability of a high quantization occurrence in a picture sequence is computed and set in variable IPl 00 1 02. [Para 38] The transition probability matrix 88 for the bandwidth model 68 is illustrated in FIG. 8. States Sl 74, S2 76 and S3 78 represent low, average and high bandwidth states, as outlined above, and each cell in the matrix 88 represents the probability of state transition from one state to another.

[Para 39] FIG. 9 shows a transition probability matrix 100 for the VCL layer quantization model 66. States Kl 104 and K2 106 represent high quantization and low quantization occurrence states, and each cell in the matrix 100 represents the probability of state transition from one state to another.

[Para 40] FIG. 10 illustrates the VCL layer complexity model 66 and bandwidth model 68 out parameters BPl 01 92, BPl 03 94 and IPl 00 1 02 used in the curve fit equation 73 of FIG. 3 to get image complexity (F) 72 which ranges in value from 2 to 3. [Para 41] FIG. 1 1 illustrates a flow chart for the main functional blocks of the inventive process. A bandwidth model initialization 108 is the first step that needs to be performed to run the bandwidth 68 and VCL layer complexity 66 models. Variables to compute an average bandwidth are initialized 1 10. A VCL input is read from the NAL (Network Abstraction Layer)/ transport stream 1 1 2. Average bandwidth for the VCL packets is computed 1 14 and set 1 1 6. During this operation, a bandwidth model 68 and a VCL layer complexity model 66 are run in parallel 1 1 8. The bandwidth model 68 is initialized for transition counters 1 20. The VCL packet size is read from the NAL/transport layer stream . u| transition probability matrix is updated 1 28. The next step is to compute the high and state limiting state probabilities 1 30 using equations (1 ) and (2), as detailed below. The variables BPl Ol and BPl 03 are set 1 32. For every macroblock, the VCL complexity model

66 is run at the same time 1 1 8. The counters are initialized 1 36 and macroblock and slice quantization parameters are read from the NAL/transport stream 1 38 by decoding slice data from the VCL The VCL complexity quantization transition probability matrix is computed 140 and limiting state probabilities are computed 142. The IPl OO variable is then set 144. The final curve fit equation is computed 146 using variables BPl 01 , BPl 03 and IPl OO.

[Para 42] The operation of an embodiment will now be explained in greater detail. A bandwidth model 68 is constructed using the Markov model in FIG. 4. The states Sl 74, S2

76, and S3 78 pertain to the state of a VCL packet rate at any instance in time after processing a certain number of VCL packets or discrete sections. The bandwidth model 68 is initialized after the MPEG video stream creation. At this stage, the bandwidth model 68 determines average bandwidth of the video stream for each discrete section, i.e., at every sampling instance. The procedure to determine average bandwidth is as follows: o Initialize counters Al 00, Al 01 , Al 02, Al 03, Al 04 to zero; o From the MPEG layer read VCL packet size for every NAL / transport layer packet received and set Al 00 for cumulative size received; o Increment Al 03 for every INTRA macroblock/picture type; o Increment Al 04 for every slice type; o Set Al 01 to first VCL received time in milliseconds; o Set Al 02 for every VCL received time in milliseconds; and o At each sampling instance, compute average bandwidth. n v * .

Al OO = Al OO + VCL_size_rcvd from MPEG layer

If (Al Ol = 0) then Al 01 = current time

Al 02 = current time

Cl OO = Al 00 * 8 / (Al 02 - Al Ol ) / 1000 (in kbps)

[Para 44] Average Bandwidth (Cl 00) range will be Cl 00 ± 1 0 kbps.

[Para 45] The model is run only when a minimum pre-defined count of Al 03 is received.

This counter indicates scene transitions and multiple scene transitions are needed to compute the model effectively. The model will be in the bandwidth low state (Sl ) if the current video stream bandwidth is lower than Cl 00 - 10 kbps; for bandwidths higher than

Cl 00 + 10 kbps the model will be in the bandwidth high state (S3). If the bandwidth is within the average bandwidth value, the model is in the bandwidth constant state (S2).

[Para 46] Average bandwidth (Cl 00) is determined continuously for the VCL packets, the bandwidth variation can be modeled using the Discrete transition Markov Process illustrated in FIG. 4. The three states' (Sl , S2 and S3) transitions are calculated by monitoring the video stream bandwidth variation. The transition matrix 88 (FIG. 8) is obtained, where each cell represents the probability of a state transition from a particular state to the next state.

Since the Markov model for this process has no periodic states and its recurrent states form a single chain, the limiting-state probabilities are independent of the initial conditions.

This condition could be applied to obtain Pl (probability to be in Sl state), P2 (probability to be in S2 state) and P3 (probability to be in S3 state). For limiting-state probabilities the following equations hold well:

0 = Σ P/p/j - Pj j = 1 , 2, 3 Equation (1 )

/

1 = Σ P/ Equation (2) j [Para 47] Since there are three variables (Pl , P2, P3) to solve, three simultaneous equations are needed, each of which can be created from the transition matrix 88 (FIG. 8).

The transition matrix 88 is constructed from the MPEG video stream bandwidth variation statistics. The transition matrix 88 is obtained by computing the probability to transition from a particular state to any other possible state, as illustrated in FIG. 8. For instance, the probability of staying in state Sl is represented by pi 1 .

[Para 48] These transition probabilities are entered into equations (1 ) and (2) to obtain three simultaneous equations that can be solved to obtain Pl , P2 and P3, where they represent the following: Pl (probability of the model to stay in low bandwidth state); P2

(probability of the model to stay in average/constant bandwidth state); and P3 (probability of the model to stay in high bandwidth state).

[Para 49] The probability of low and high transitions goes in to the final curve fit equation 70 to obtain an image complexity value 72. The algorithm to obtain Pl , P2 and P3

is described as follows: o Initialize counters cl 1 , cl 2, cl 3, c21 , c22, c23, c31 , c32, and c33 to 0; o state = S2; and o For several VCL packets in MPEG video elementary stream (configurable count)

If (Al 03 > 5000 (configurable count) | | Al 04 > 100 (configurable count)) current_bandwidth = ( vcLsize * 8 ) / (current_time - previous_time) / 1000 If (current_bandwidth > (Cl 00 + 10) ) If (state = S1 ) ++cl 3. else if ( state = S2)

++c23. else

++C33 else if (current_bandwidth < (cl 00 - 1 0)) if (state = S1 ) b / else if (state = S2)

++c21 else

++c31 else if (state = Sl )

++cl 2 else if (state = S2)

++c22 else

++c32 update state to current state

[Para 50] At every sampling instance (e.g., 10 seconds), a transition matrix 88 is computed from the above. The transition probabilities are calculated from the relative frequencies of state transition. pi 1 = cl 1 / (el l + cl 2 + cl 3) pi 2 = cl 2 / (el l + cl 2 + cl 3) pi 3 = cl 3 / (el l + cl 2 + cl 3) p21 = c21 / (c21 + c22 + c23) p22 = c22 / (c21 + c22 + c23) p23 = c23 / (c21 + c22 + c23) p31 = c31 / (c31 + c32 + c33) p32 = c32 / (c31 + c32 + c33) p33 = c33 / (c31 + c32 + c33)

[Para 51 ] From the transition matrix, the probabilities Pl (low rate probability), P2 (constant/average rate probability) and P3 (high rate probability) are computed using three simultaneous equations formed utilizing equations (1 ) and (2).

[Para 52] Putting the transition probabilities into equation (1 ), the following is obtained:

0 = Pl * (pi 1 - 1 ) + P2 * p21 + P3 * p31 Equation (3)

0 = Pl * pi 2 + P2 * (p22 - 1 ) + P3 * p32 Equation (4) i :: s o B / K* y «4.7 T

From equation (2), the following is obtained:

1 = Pl + P2 + P3 Equation (5)

[Para 53] After the equations are solved, the probabilities are assigned into these variables:

BPl Ol = Pl BPl 03 = P3

[Para 54] After the above three equations are solved, Pl and P3 are computed to use in the curve fit equation 70 to get the final image complexity 72. [Para 55] For every VCL input, a VCL layer complexity model 66 needs to be run in parallel. VCL parameters are monitored for scene transitions and picture quality. The INTER/INTRA macroblock types are analyzed to determine scene transitions and quantization parameters inside the macroblock are read to determine picture quality as contributing to image complexity. After the VCL complexity model 66 is run, the curve fit equation 70 for image complexity can be solved to get the final image complexity value 72. [Para 56] Computing VCL complexity probability follows a process similar to the one described above, but the Markov states are limited to two states. FIG. 10 shows the state transition process of the VCL complexity model. The states represent:

Kl (state where quantization high macroblock is received); and

K2 (state where a quantization low macroblock is received). [Para 57] A transition matrix 100 (FIG. 9) that contains transition probabilities is

computed. Each cell represents the state transition, e.g., pi 2 represents the probability of having a quantization low (K2) in quantization high received state (Kl ). [Para 58] The procedure to compute transition probabilities is as follows:

[Para 59] For every VCL input, in an MPEG video elementary stream, o Initialize all counters dl 1 , dl 2, d21 , d22 to zero. Set state = Kl . π αic . initial quantization value from either picture parameter set (as in MPEG4) or from a preconfigured value if it is not available. Set Cl 01 to this value. o Set Cl 02 to zero for macroblock counts o Set Cl 03 to zero for INTRA macroblock types o Set Cl 05 to zero for Slice types o For every macroblock that is processed increment Cl 02 o For every INTRA macroblock type increment Cl 03 o For every Slice type increment Cl 05 o For every macroblock where quantization is available, read the quantization value in

Cl 04

If (state = Kl )

If (Cl 04 > Cl 01 && Cl 05 > 100 (configurable) && Cl 03 > 3000 (configurable))

++dl l ; else

++d! 2; else if (state = K2) If (Cl 04 > Cl 01 && Cl 05 > 100 (configurable) && Cl 03 > 3000 (configurable))

++d21 else

++d22 update state to current state [Para 60] At every sampling instance (e.g., 10 seconds) from the above counters, transition probabilities are computed to get the transition matrix 1 00 above pl l = dl l / (dl l + dl 2) pi 2 = dl 2 / (dl l + d! 2) p21 = d21 / (d21 + d22) p22 = d22 / (d21 + d22) [Para 61 ] From the transition probabilities, the probability of quantization high occurrence (Pl ) and quantization low occurrence (P2) can be computed. The probability of P f ". T / Il Ji ¹S Ii I Ib . ■^■" irf 7 Mh 7 complexity value 72.

[Para 62] Since the limiting state probabilities are independent of initial conditions, the simultaneous equations for the limiting-state probabilities can be solved, as follows:

0 = Σ P/p/y - Pj 7 = 1 , 2 Equation (6)

/

1 = Σ Pj 7 = 1 , 2 Equation (7)

7 [Para 63] Substituting and expanding the transition probabilities in equations (6) and (7) above,

0 = Pl * (pi 1 - 1 ) + P2 * p21 Equation (8)

1 = Pl + P2 Equation (9)

[Para 64] The above two equations are solved to get Pl and P2. Assign, IPl 00 = Pl (probability of VCL layer complexity high occurrence in the macroblocks) to be used in image complexity equation.

[Para 65] BPl 01 , BPl 03 and IPl 00 are used in the curve fit equation 70 (FIG. 10) to get an image complexity 72 that falls in the range of between 2 and 3.

Image Complexity ( T )

r = 2 + In (1 + IPl 00) + In (2 + Bl 03 - Bl 01 ) Equation (1 0) for (r > 3) r= 3

[Para 66] Although several embodiments have been described in detail for purposes of illustration, various modifications may be made to each without departing from the scope and spirit of the invention.

Claims

[Claim 1 ] A process for broadcasting multiple video streams on a single channel, comprising the steps of: analyzing complexity indication changes and bit rate changes in a video coding layer of each of said multiple video streams; creating a statistical model to dynamically compute image complexity of each of said multiple video streams; determining the effect of the image complexity of said multiple video streams on said broadcast; and distributing available channel bandwidth among said multiple video streams based upon the determined effect of the image complexity of each of said multiple video streams.

[Claim 2] The process of claim 1 , wherein the analyzing step includes the step of analyzing video coding layer complexity indication changes in parameters of discrete sections of the video streams.

[Claim 3] The process of claim 2, wherein the discrete sections of the video streams include slice, macroblocks, quantization, inter-coded reference blocks, intra-coded reference blocks, and non-reference macroblock/slice/picture types.

[Claim 4] The process of claim 1 , further comprising the step of estimating video quality in loss states.

[Claim 5] The process of claim 1 , wherein the creating step includes the steps of: creating a first statistical model of video coding layer complexity indication changes for discrete sections of each video stream; creating a second statistical model of video coding layer bit rate changes or bandwidth variation for the same discrete sections of each video stream; P¹ ₁. / iyi ::πι ιι,,n en .•^• id H- . stream; and calculating the image complexity for the discrete sections of each video stream based upon the combined first and second statistical models.

[Claim 6] The process of claim 5, further comprising the steps of: counting high quantization transitions, slice/macroblocks and inter/intra prediction types for picture/slice/macroblock types by determining quantization changes in the discrete sections of each video stream; and counting bandwidth variation by determining bandwidth of video coding layer data in the discrete sections of each video stream.

[Claim 7] The process of claim 6, wherein the counting steps comprise incrementing a first counter for each quantization change, incrementing a second counter for each macroblock, incrementing a third counter for each slice, and incrementing a fourth counter for each low, average and high bandwidth state transition.

[Claim 8] The process of claim 7, further comprising the steps of: computing a probability for video coding layer complexity for the discrete sections of each video stream using the first, second, third and fourth counters; and computing a probability for low, average and high bandwidth states for the discrete sections of each video stream using the first, second, third and fourth counters.

[Claim 9] The process of claim 8, further comprising the steps of: constructing a first transition probability matrix for video coding layer complexity transition of the discrete sections of each video stream; and constructing a second transition probability matrix for bandwidth state transition of the discrete sections of each video stream. complexity of the discrete sections of each video stream using limiting state probabilities obtained from each transition probability matrix.

[Claim 1 1 ] A process for broadcasting multiple video streams on a single channel, comprising the steps of: analyzing complexity indication changes and bit rate changes in a video coding layer in parameters of discrete sections of each of said multiple video streams; creating a statistical model to dynamically compute image complexity of each of said multiple video streams; determining the effect of the image complexity of said multiple video streams on said broadcast; distributing available channel bandwidth among said multiple video streams based upon the determined effect of the image complexity of each of said multiple video streams; and estimating video quality in loss states.

[Claim 1 2] The process of claim 1 1 , wherein the discrete sections of the video streams include slice, macroblocks, quantization, inter-coded reference blocks, intra-coded reference blocks, and non-reference macroblock/slice/picture types.

[Claim 1 3] The process of claim 1 1 , wherein the creating step includes the steps of: creating a first statistical model of video coding layer complexity indication changes for the discrete sections of each video stream; creating a second statistical model of video coding layer bit rate changes or bandwidth variation for the same discrete sections of each video stream; combining the first and second statistical models from the discrete sections of each video stream; and Ir IL I! ^■•■' ιι,,,ιι • if Ui o .•'^■ ιι:« i'¹' 'I" >' ■'' the combined first and second statistical models.

[Claim 1 4] The process of claim 1 3, further comprising the steps of: counting high quantization transitions, slice/macroblocks and inter/intra prediction types for picture/slice/macroblock types by determining quantization changes in the discrete sections of each video stream; and counting bandwidth variation by determining bandwidth of video coding layer data in the discrete sections of each video stream.

[Claim 1 5] The process of claim 14, wherein the counting steps comprise incrementing a first counter for each quantization change, incrementing a second counter for each macroblock, incrementing a third counter for each slice, and incrementing a fourth counter for each low, average and high bandwidth state transition.

[Claim 1 6] The process of claim 1 5, further comprising the steps of: computing a probability for video coding layer complexity for the discrete sections of each video stream using the first, second, third and fourth counters; and computing a probability for low, average and high bandwidth states for the discrete sections of each video stream using the first, second, third and fourth counters.

[Claim 1 7] The process of claim 16, further comprising the steps of: constructing a first transition probability matrix for video coding layer complexity transition of the discrete sections of each video stream; and constructing a second transition probability matrix for bandwidth state transition of the discrete sections of each video stream.

[Claim 1 8] The process of claim 1 7, further comprising the step of computing image complexity of the discrete sections of each video stream using limiting state probabilities obtained from each transition probability matrix. 11? ir: T^" / 11 j s o it » / di 7¹Mi- 77 comprising the steps of: analyzing complexity indication changes and bit rate changes in a video coding layer of each of said multiple video streams; creating a first statistical model of video coding layer complexity indication changes for discrete sections of each video stream; creating a second statistical model of video coding layer bit rate changes or bandwidth variation for the same discrete sections of each video stream; combining the first and second statistical models from the discrete sections of each video stream to dynamically compute image complexity of each of said multiple video streams; determining the effect of the image complexity of said multiple video streams on said broadcast; distributing available channel bandwidth among said multiple video streams based upon the determined effect of the image complexity of each of said multiple video streams; calculating the image complexity for the discrete sections of each video stream based upon the combined first and second statistical models; and estimating video quality in loss states.

[Claim 20] The process of claim 19, further comprising the steps of: counting high quantization transitions, siice/macroblocks and inter/intra prediction types for picture/slice/macroblock types by determining quantization changes in each video stream; and counting bandwidth variation by determining bandwidth of video coding layer data in each video stream.

[Claim 21 ] The process of claim 20, wherein the counting steps comprise incrementing a first counter for each quantization change, incrementing a second counter for each P r- T/' y S0 B /!?ϊ7 ^|l*.77^l for each low, average and high bandwidth state transition.

[Claim 22] The process of claim 21 , further comprising the steps of: computing a probability for video coding layer complexity for the discrete sections of each video stream using the first, second, third and fourth counters; and computing a probability for low, average and high bandwidth states for the discrete sections of each video stream using the first, second, third and fourth counters.

[Claim 23] The process of claim22, further comprising the steps of: constructing a first transition probability matrix for video coding layer complexity transition of the discrete sections of each video stream; and constructing a second transition probability matrix for bandwidth state transition of the discrete sections of each video stream.

[Clai m 24] The process of claim 23, further comprising the step of computing image complexity of the discrete sections of each video stream using limiting state probabilities obtained from each transition probability matrix.

[Claim 25] The process of claim 1 9, wherein the analyzing step includes the step of analyzing video coding layer complexity indication changes in parameters of the discrete sections of the video streams, and wherein the discrete sections of the video streams include slice, macroblocks, quantization, inter-coded reference blocks, intra-coded reference blocks, and non-reference macroblock/slice/picture types.