JP2014090456A - Processing method and device of coded bitstream - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a method for improving bitstream quality evaluation of a multimedia system.SOLUTION: In a method for extracting information from a coded bitstream, extracted information is used for improvement of bitstream quality evaluation or adaptation of a decoding sub-system. From a coded bitstream received from a bitstream source, complexity related information indicating the coded complexity of a bitstream is extracted. On the basis of complexity related information thus extracted, one or more data setting is selected, and then the data setting is used as an input for improving the bitstream quality evaluation in a decoding device or a network node, or for adapting the bitstream decoding sub-system in the decoding device.

Description

  The present specification relates to a method and apparatus for processing an encoded bit stream, taking into account the complexity of the preceding encoding process.

  With the advent of new communication technologies, for example, the coverage and quality of media content such as video that can be delivered to various types of user devices, including both fixed and mobile user devices, for example Multimedia services such as multimedia streaming are becoming increasingly popular.

  In order to ensure good experience quality of such media services, operators of wireless and wired network communication systems typically have tools and tools to locate and avoid problematic locations in networks or services that are already close to the source. Use the measurement method.

  A tool that recognizes important characteristics of media content can use the results of analysis to optimize the network, thereby providing more users with the network resources available to service providers. It is possible to have appropriate quality experience of service.

  Some services require measuring a few network parameters, such as throughput, in order to give a good quality estimate to the end user. In the case of multimedia services such as video streaming, the experience quality measurement task is not a trivial problem because there are several factors that contribute to quality degradation including the media itself.

  Video quality assessment is related to how people feel the video and can be divided into subjective and objective methods. Subjective test methods usually produce the best results, but if a reliable result is achieved, many test personnel must rate each video sequence before calculating the average opinion score (MOS). Subjective testing is expensive and time consuming. Objective models can be used for real-time quality assessment and are often designed to output a score that estimates the subjective MOS.

  Objective models can be divided into three categories: perceptual models, parametric models, and bit stream models.

  Perceptual models often rely on multiple image analysis algorithms that enhance and quantify video artifacts. Perceptual models are often divided into full reference (FR) models, reduced reference (RR) models, and non-reference (NR) models. The perceptual NR model uses only the decoded video as input to the model, while the FR model also uses the associated uncoded reference as processing input. Instead of using a full reference, the RR model uses information about the reference as input via a narrow side channel.

  Regarding the perceptual model, for example, there are a plurality of standards such as Patent Document 1, Patent Document 1 relates to various FRTV multimedia (MM), Patent Document 2 relates to an RRMM model, and Patent Document 3 describes an FRMM model. It is about. However, the performance of the NR perception model is still too low to standardize the model.

  Parametric models known in the prior art commonly use various network parameters and codec information as inputs to estimate the bit stream quality score. Although the performance of parametric models tends to be worse than that of FR perceptual models, parametric models are generally less complex and do not require the computational power normally required by perceptual models. Parametric models can therefore be easily implemented in both network nodes and user equipment, such as set top boxes (STBs) and mobile devices.

  When trying to estimate the quality of the bit stream, the video bit stream model uses the compressed video bit stream as input. The bit stream model is somewhat more complex than the parametric model because more data needs to be analyzed. However, because information about the encoding can be obtained, these types of models have the ability to provide video quality estimates with better accuracy than other models.

  There are many new encoders available on the market today, such as hand breaks, quick time and helix. When analyzing the bit streams from these encoders, it can be seen that even if the same profile and level are used, the encoder feature settings can be different. The possible encoder features affect the computational power required for the encoder, and as a result, the time required to encode a frame may vary depending on the setting. This also affects the amount of computational power required on the decoder side, so the encoder can choose to limit the feature if the feature is known, or on the decoder side, eg There are several types of resource limits, such as computing power limits or memory limits.

  With the use of various encoding feature settings in the encoder, these settings can be viewed as corresponding to different levels of complexity, where the macroblock is divided into smaller partitions and Motion vector resolution in the motion estimation process is a typical feature that can affect computational power.

  This type of feature is typically used to create an opportunity to reduce the remaining encoder loops and improve the quality of the encoded stream.

  U.S. Patent No. 6,057,034 describes a method for analyzing the quality of a video bit stream by using information about quantization and DCT coefficients.

  Using transmission and video stream parameters to calculate the bit stream quality score, or for certain encodings, eg, motion vector information and intra / inter prediction frame / slice ratio, ie actual bits It is also widely known that data such as information related to a stream is used as a basis for the calculation.

  Non-Patent Document 4, which is a conference paper, presents an IPTV bit stream model that depends only on the quantization parameter (QP) when calculating the bit stream quality score.

  A common problem with prior art bit stream quality model solutions is that the complexity of the encoding procedure may vary from case to case, but this depends on which bit stream quality model to choose to obtain the bit stream quality score. It is not reflected in processing. As a result, the bit stream quality model that is optimal for the complexity of a particular encoder will use the same data settings for different scenarios involving encoders that use different complexity.

  As a result, the accuracy of the resulting bit stream quality score may vary from measurement to measurement, even if the same data settings are used as input for the bit stream quality model used.

  Thus, there is a need for a method and apparatus that enables improved bit stream quality assessment.

US Pat. No. 6,011,868

ITU-T recommendation J-144 ITU-T recommendation J-246 ITU-T recommendation J-247 “Proposal of a new QOE assessment application for quality management of IPTV services”, ICIP2008

  It is an object of the present invention to overcome at least some of the disadvantages of the prior art by improving the method and apparatus.

  According to one aspect, a method is provided for improving bit stream quality assessment of a multimedia system, extracted from an encoded bit stream that is typically a video bit stream received from a bit stream source. The bit stream quality score is calculated based on the complexity related information indicating the complexity of the preceding encoding at the bit stream source.

  The complexity related information is used to select one or more data settings, which are used as input to a bit stream quality model that calculates a bit stream quality score.

  By calculating the bit stream quality score based on the information indicating the encoding complexity, the accuracy of the resulting bit stream quality score is due to the possibility of effective adaptation of the bit stream quality model. Improved compared to prior art solutions.

  According to another alternative, the complexity related information to be extracted can be used instead to adapt the decoding subsystem to obtain performance improvements. Such adaptation may include, for example, computing power distribution or memory allocation adaptation within the decoding subsystem.

  By considering the encoding complexity, this information can be valuable for adapting certain sub-modules of the decoding subsystem.

  The proposed method can be used based on the complexity related information to be extracted in combination with information on additional quality.

  Complexity-related information is a parameter related to complexity indicating, for example, one of the macro block partition size, motion vector resolution, skipped macro block, or number of reference images to be used in the encoded bit stream. Can be included.

  The complexity related information to be extracted is used to select an appropriate data setting, either directly or indirectly, alone or in combination with other information, and a complexity level is first selected. In the latter case, bit stream quality score or decoding subsystem adaptation is performed based on the selected complexity level.

  The present invention also relates to each device configured to perform any of the proposed methods.

  Such a device can be implemented, for example, as either a cellular telephone, a set top box, a PC, a laptop computer or a network node.

  The present invention will now be described in detail by way of exemplary embodiments with reference to the accompanying drawings.

2 shows two separate flowcharts of a decoding device receiving an encoded bit stream from a bit stream source and method steps at the bit stream source. Two flow charts for each method step when the encoded bit stream is instead provided from the bit stream source to the network node. 5 is a flowchart of a method for extracting and using complexity related information according to an embodiment. 6 is a flowchart of a method for extracting and using complexity related information according to another embodiment. FIG. 2 shows an exemplary intra macro block that is divided into various intra partitions. FIG. 4 shows an exemplary inter macro block that is divided into various inter partitions. FIG. 3 is a block diagram of an apparatus configured to extract and use complexity related information from which a bit stream quality score is derived, according to an illustrative embodiment. FIG. 4 illustrates a scenario of how parameters can be extracted from an encoded bit stream and used to adapt a decoding subsystem, according to an example embodiment. FIG. 6 illustrates a scenario of how parameters can be extracted from an encoded bit stream and used to adapt a decoding subsystem according to another embodiment. FIG. 3 is a block diagram of an apparatus configured to extract and use coding complexity related information for decoding subsystem adaptation, according to an exemplary embodiment.

  When predicting the bit stream quality score of an encoded multimedia bit stream carrying multimedia content such as video, the use of input data provided to the bit stream quality model is Is necessary for good accuracy that can be achieved. In addition, multimedia decoding devices, particularly mobile video decoding devices, often require several coprocessors or accelerator blocks that can handle complex bit streams. Such coprocessors and / or accelerator blocks are often configured to turn on and off when needed. More particularly, since such a subsystem consumes more or less power, it is not desirable to keep such subsystem operating unless it is absolutely necessary. In addition, the allocation of memory resources for the bit stream decoding process at the decoding device is often based on the signal profile and level used by each codec.

  What is needed is an improved method that allows analysis of bit stream encoding characteristics, and details of how to perform bit stream encoding for a selected complexity level are considered. Having such a mechanism can improve at least some of the above problems, resulting in improved prediction accuracy of the bit stream quality model, or less resource usage in the decoding device. Optimize. Thus, a method is proposed that uses in some way the indication of the complexity of the bit stream encoding, ie the complexity used by the encoder applied during the encoding of the extracted bit stream. The term “indication” is to be interpreted here in the broadest sense, and the information to be extracted contains more or less detailed information and is used by the encoder to encode the bit stream when processing. You can give more or less detailed indications of complexity.

  The degree of complexity at the encoder may depend on one or more parameters extracted from the encoded bit stream and the rules applicable when trying to process the extracted information / parameters.

  The proposed method is typically applicable to encoded video bitstreams, but can also be used for corresponding processing of other types of encoded multimedia bitstreams.

  The proposed method is based on extracting and processing complexity related information at a decoding and rendering device or network node.

  In order to obtain a bit stream quality score with better accuracy, we propose a parameter extraction method based on one or more parameters indicating the complexity of the encoding process performed on the bit stream source.

  According to the proposed method, analysis of the encoded bit stream at a decoding device or network node that requires analysis of the received bit stream typically encodes one or more predefined parameters. Extract from the generalized bit stream. The analysis can be performed by any conventional procedure for analyzing information from a bit stream carrying multimedia content.

  Once extracted, such parameters may be used alone or as parameters related to quality read by any prior art solution to provide a bit stream quality score with improved accuracy. It can be used in combination with other parameters.

  The processing of the encoded bit stream will typically result in specific requirements for various parts of the decoding subsystem of the decoding device, such as coprocessors and / or accelerator blocks. . Since these subsystems consume more or less power, it is usually undesirable to keep such subsystems running unless absolutely necessary. Due to changes in coding complexity, there may be a need to turn on and off various parts of the decoding subsystem as required. Such adaptation or optimization of the decoding subsystem can be achieved through the use of information related to the complexity extracted.

  In other words, the proposed method uses a complexity-related parameter indicating the complexity of the encoder used in the bit stream source as input for selecting the appropriate data settings at the decoding device or network node. To do. Such a data setting can be used as an input to a bit stream quality model that obtains a bit stream quality score with good accuracy or as an input to adapt the decoding subsystem.

  The general principles on which the proposed method is based will be described with reference to the two flowcharts of FIGS. 1a and 1b, respectively, where these flowcharts are encoded from a bit stream source 100 such as a terminal or server. The method steps performed in the decoding device 101 for receiving and processing multimedia content in the form of a compressed bit stream and the steps performed in the corresponding bit stream source 100 are indicated in general terms. is there.

  It should be understood that all steps performed in the bit stream source 100 can be referred to as conventional steps for performing conventional multimedia delivery methods.

  In the case of a video bit stream, such video delivery may be related to a video stream, video telephony service, or any other service that enables compressed video delivery. Such services may relate to any mobile TV such as MBMS or DVB-H, video conferencing, eg digitally broadcast TV such as DVB-T, DBV-C or DVB-S. . A terminal used as a decoding device may be, for example, a mobile device, a video conferencing terminal, a set top box (STB) or any other type of terminal configured to decode compressed video.

  FIG. 1 a shows the method steps at two entities connected via a communication network 102 where multimedia content such as compressed video can be transmitted, but the parameter extraction mechanism proposed here is: There is no network at all, so instead the decoding device encodes the encoded bit stream from a bit stream source integrated in the decoding device or directly connected to the decoding device. It should be understood that the present invention can also be applied when receiving a coded bit stream.

  In the simplified scenario of FIG. 1a, a bit stream source 100, such as a sending terminal or server, for example, in a first step 1: 1, video or any type of multimedia content from the bit stream source. To win. The multimedia content is encoded using any conventional type of encoding technique, as shown in the next step 1: 2, and then the encoded bit stream is transferred to the next step 1: 3. As shown, it is transmitted to the decoding device 100 via the network 102 (provided directly to the decoding device 100 if no communication network is required).

  As shown in another step 1: 4, once received at the receiving terminal 100, the coding complexity is related in the form of one or more predetermined parameters as shown in another step 1: 5. Information is extracted from the video bit stream, typically by analysis of the video bit stream.

  As described above, additional parameters, such as other quality parameters, can be extracted in addition to the complexity parameters.

  In the next step 1: 6, the extracted parameters are processed so that one or more data settings can be selected. The resulting data settings are either in the decoding of the bit stream, as in the alternative path shown as alternative step 1: 7a, or for the calculation of the bit stream quality score in another alternative step 1: 7c. Step 1: As shown in 7b, can be used as input to the bit stream quality model.

  In the next step 1: 8, the bit stream is decoded and the results of the parameter processing can be used in any of the procedures detailed below. However, if the purpose of the parameter extraction procedure is to obtain a bit stream quality score as indicated by steps 1: 7b and 1: 7c, the decoding step of step 1: 8 can be performed in a conventional manner. In final step 1: 9, the encoded multimedia content can be displayed or presented to the end user.

  According to another alternative scenario, the proposed parameter extraction mechanism can be applied, for example, in a test unit of a network node, where the received coded bit stream is analyzed and requires one or more Complexity parameters can be extracted from the bit stream and used, for example, for service or maintenance purposes.

  FIG. 1b shows such an alternative method step, in which no rendering of the received multimedia content takes place. Such an embodiment may be applicable, for example, to network nodes that require a bit stream quality score. Such network nodes typically also include conventional functions that allow the transfer of a bit stream to the network node once it has extracted the required information.

  Details of an alternative method of extracting complexity-related information and using the information read as a result of the extraction procedure are described below.

  FIG. 2a is a flowchart that uses complexity related information to select a data setting for a bit stream quality model, which is used to estimate the quality of a video bit stream with good accuracy. Illustrated in the video bit stream quality model, FIG. 2b shows how the results of the proposed parameter extraction mechanism are used to adapt the decoding subsystem to obtain a more efficient decoding procedure. It is a flowchart which shows.

  These alternative uses of complexity-related information extracted from the encoded video bitstream can be used directly or indirectly to obtain improved video performance for display to the end user of the multimedia system. obtain.

  According to FIG. 2a, a video bit stream is received from a bit stream source in a first step 2: 1.

  In the next step 2: 2, typically one or more parameters indicating the complexity of the encoder are extracted from the video bitstream as a result of the analysis of at least part of the received encoded bitstream. Is done. The parameters to be extracted can be used as input to select one or more data settings according to predefined rules, as shown in another step 2: 3.

  As already mentioned, additional parameters, for example quantization parameters, can also be extracted from the bit stream in step 2: 2, in which case one or more data settings are a combination of the extracted parameters. Selected based on data input including

  According to an alternative embodiment, step 2: 3 can be performed by comparing the data input obtained in extraction step 2: 2 with a preset value of a predefined complexity level, where the parameter value Is interpreted as indicating a low complexity encoding, and thus an evaluation of another parameter value combination that may result in a low complexity level associated with each bit stream is The result may be a lower complexity level associated with the encoding of the stream, and the evaluation of another parameter value combination may result in the selection of a complexity level that represents the higher complexity of the encoding.

  In the final step 2: 4a, when using a certain video quality model, the data settings are used as input to calculate the bit stream quality score.

  As shown in FIG. 2b, steps 2: 1 to 2: 3 are the same as the first step performed in FIG. 2a. However, when using a quality model to calculate the bit stream quality score of a video bit stream, instead of using the information extraction results for accuracy improvement, the results are stored in the decoding subsystem of the decoding device. It can be used to obtain more efficient use of one or more specific categories of limited resources.

  Based on a procedure that adapts or optimizes the complexity related information of the encoding of the video bitstream, typically results in more efficient use of resources during decoding without compromising the decoding result. Can be obtained as

  The complexity-related information is, for example, a reference picture used in the process of encoding various intra or inter macro block partition sizes, skip macro blocks (MB), motion vector resolution or video bit stream. Information indicating various aspects of the encoding can be included, such as the presence and distribution of information about the quantity.

  For example, one complexity parameter to be extracted from a video bitstream by analyzing the bitstream indicates, for example, the percentage of intra or inter partition size used during the video bitstream encoding process. Parameters, other parameters may indicate, for example, the percentage of skipped MBs in the received video bit stream sequence.

  H. H.264 defines three different intra partition sizes, denoted intra 16x16, intra 8x8, and intra 4x4, respectively. The partition used for each MB in the video bit stream may be, for example, the number of bits used for encoding, the difficulty of encoding the current area with the required quality and / or the time and processing power available to the encoder. The encoder can be determined based on various viewpoints. For video sequences with low complexity, the encoder can typically skip large amounts of MB without compromising the visual quality of the video.

  Encoders that are considered to represent encoders with low complexity due to processing constraints can be selected without examining all available intra partitions, but in the course of encoding, For the partitions used, restricting the checks associated with the encoding procedure to some preferred partitions is generally accompanied by lower quality results than in what would be considered a more complex encoding procedure.

  FIG. 3a is an example of a typical example of an intra MB 300a. The intra MB 300a is divided into two intra 8 × 8 partitions 301 and 302 located diagonally and eight intra 4 × 4 partitions 303 to 310 located diagonally. ing. A typical reflection of the most recent example is that sections 303 to 306 and 307 to 310 of MB 300a contain more changes, thus requiring more capacity for encoding than sections 301 and 302.

  H. Inter MB H.264 defines seven different partition sizes: 16x16, 16x8, 8x16, 8x8, 8x4, 4x8 and 4x4. As with intra MBs, the encoder must determine which of these available inter MBs to use in the encoding procedure.

  Also for inter MB checking, checking all possible partitions is not feasible at the cost of a low complexity encoder because the encoder processing is very large.

  FIG. 3b is a typical example of inter MB 300b, which inter 16300 in the process of encoding, one 16x8 partition 311, two 8x4 partitions 312 and 313, one 4x8 partition 314, and two 4x4. It is divided into partitions 315 and 316.

  When the MB is very similar to the corresponding MB of the previous image, or when the number of available bits is considered too low for MB encoding, the encoder chooses to skip the MB can do.

  For motion vectors, H. H.264 defines three resolutions: integer pixel, half pixel and quarter pixel resolution. Integer pixel resolution is the simplest resolution that an encoder / decoder can handle, and to handle half-pixel and quarter-pixel resolution, interpolation between adjacent pixels must be performed on the current reference data. I must. A 6-tap FIR filter is commonly used to perform half pixel interpolation, and for linear pixels, bilinear interpolation is performed on the half pixels.

  The encoder must determine the accuracy to use for each inter block based on, for example, one or more criteria as described above. In order to examine each of the half or quarter pixel resolution motion vectors, the encoder must perform interpolation which consumes processing power, and the various variations of the partition due to large variations between different parts of the image. The consumption is particularly large in the MB having the size.

  For example, by choosing to use only integer pixel resolution at the encoder, the computational complexity may be reduced on both the encoder and decoder side.

  Such determination can also affect the computational power of encoding and decoding since several reference frames can be used in the video bitstream.

  On the encoder side, this can result in an increase in the amount of reference data that must be examined, which will ultimately result in an increase in demand for computing power.

  On the decoder side, reference data needs to be read from different parts of the memory multiple times in the same frame, so the use of multiple reference frames reduces the computational power left to perform decoding. Also in this particular situation, the encoder must decide on the number of reference frames to use. The determination can be made based on various viewpoints such as the number of bits used, the degree of difficulty in encoding with good quality in the current area, and the time and processing capacity available for the encoder. it can.

  Thus, there are multiple ways to obtain an indication of the complexity level at the encoding step. By extracting certain predefined parameters from the encoded bit stream and processing these parameters according to the parameters, typically by conventional data extraction means, after encoding, the bit stream Complexity related information used by the encoder at the source can be obtained. By defining the complexity level used in the process of evaluating the extraction parameters, the measurement of the required complexity level can give a more predictable indication of the complexity.

For example, from the coded bit stream associated with the decoding of the coded bit stream at the decoding device or the coded bit stream associated with processing the coded bit stream at the network node, the following parameters are set: Can be extracted.
P1 = Intra 16 × 16 partition ratio P2 = Intra 8 × 8 partition ratio P3 = Intra 4 × 4 partition ratio P4 = Inter 16 × 16 partition ratio P5 = Inter 16 × 8 partition ratio P6 = Inter 8 × 16 partition ratio P7 = Inter 8 × 8 partition ratio P8 = Inter 4x8 partition ratio P9 = inter 8x4 partition ratio P10 = inter 4x4 partition ratio P11 = number of reference images (nbrRef)
P12 = motion vector resolution expression (MVres)
Here, for example, MVres can be set to 1 when integer pixel resolution is used, and MVres can be set to 2 when half pixel resolution is used.
P13 = Average quantization parameter (AQP)
P14 = Average frame rate (AFR)

The exemplary extraction parameters can be used in a linear quality model, for example, the following quality model referenced as a predictive quality score (PQS) model.
PQS = a ₀ + a ₁ × P ₁ + a ₂ × P ₂ + a ₃ × P ₃ + a ₄ × P ₄ + a ₅ × P ₅ +
a ₆ × P ₆ + a ₇ × P ₇ + a ₈ × P ₈ + a ₉ × P ₉ + a ₁₀ × P ₁₀ +
_{a 11 × P 11 + a 12} × P 12 + a 13 × P 13 + a 14 × P 14 (1)

Alternatively, a non-linear PQS model such as the following model can be used as the extraction parameter.
PQS = a ₀ + a ₁ × P ₁ ^b1 + a ₂ × P ₂ ^b2 + a ₃ × P ₃ ^b3 + (2)

Where a ₀ , a ₁ ... And b ₁ , b ₂ ... Are specific coefficients of the bit stream quality model, for example, the bit stream quality model for subjective test data or It can be preselected by adjusting the PQS model, or it can be selected based on the complexity level. In this example, P13 and P14 are quality parameters used in addition to the parameters P1 to P12 related to complexity. Based on the calculation of the linear or non-linear quality model, a bit stream quality score is derived.

  Instead, applying a predefined complexity level, a predefined combination of extracted parameter sets can typically be defined as, for example, one or more predefined complexity levels that can be defined as thresholds, and Can be compared.

  The presence of only one specific type of partition in the encoding process means that a decoding device or network that requires a bit stream quality score indicates that a high complexity encoding was used at the bit stream source. It can be an instruction to the node.

  For example, if the decoding device recognizes that an intra 4x4 partition was used for encoding at the bit stream source, that is, if P3 is extracted by the decoding device and P3P3 has a non-zero value, this is high An indication that a complexity level, for example, a level 4 complexity should be selected on a scale of 1 to 4, and if P3 is 0, an indication that a complexity less than level 4 should be selected instead It can be. Other rules on how complexity levels depend on one or more parameters can also be pre-defined by multiple possible combinations.

  If the extracted information is to be used for adaptation of the decoding subsystem, it first compares a certain combination of one or more parameters with a predefined complexity level set in the same way as described above. This can be used indirectly as an input parameter. Based on the results, an acceleration block with a decoder can be deactivated to reduce power consumption and / or some memory blocks can be released to save allocated memory space.

  Decoder sub-system adaptation may include preparing the sub-system with complexity related information. For example, if it is determined from the extracted information that half pixel resolution is to be used, the decoder will require interpolation, so when adapting the decoder sub-system, the decoder sub-module configured for this particular task will be It can be operated.

  In the above description, H.C. H.264 has been used for exemplary problems that arise in the encoding / decoding process, the solution proposed here is the H.264 standard. For example, H.264 is not limited thereto. 261, H.H. It should be understood that it can be applied to other similar types of video codecs, such as H.263, MPEG-2 and MPEG-4. The partition size and motion vector resolution used in the encoding process are not the same as other codecs, but the principle is the same.

  It should also be understood that in accordance with any of the above embodiments, types of complexity related information other than the complexity related information described above can also be extracted and processed from a video bit stream or any other multimedia bit stream. .

  Referring to FIG. 4, configured to provide a bit stream quality score by using a bit stream quality model and one or more parameters indicating encoder complexity as input to the quality model. An exemplary apparatus will be described.

  FIG. 4 illustrates an example apparatus 400 that receives an encoded bit stream, such as a video bit stream, from a bit stream source 100, eg, a media quality monitoring device. Referred to as Together with the device 400, the bit stream source 100 forms a multimedia system 401, according to the figure, where direct access is provided between two units. As already described in FIGS. 1a and 1b, the bit stream can be delivered directly from the bit stream source 100 to the device 400, as shown, but any type of conventional communication. It should also be understood that it can be distributed over a network.

  An apparatus 400 or network node that is part of a drawing device and includes a decoding function, referred to as network node / decoding device 402 in the figure, is configured to derive a bit stream quality score representing the received bit stream; This specification includes a unit referred to herein as parameter extraction unit 400a, which is configured to extract information in one or more predefined parameter formats, the extraction information comprising at least in part a bit stream, Indicates the complexity used in the encoding at the source 100, in other words, the extracted information gives an indication of the complexity level used in the encoding, and this indication can be used in subsequent decoding or processing of the encoded bit stream. Can be interpreted. The apparatus 400 also includes another unit, referred to herein as a control unit 400b, which is configured to generate a bit stream quality model from a plurality of pre-defined data settings for the bit stream quality model. Configured to select data settings. A bit stream quality score representing the received bit stream is then calculated based on the selected data settings. This is typically accomplished by the control unit 400b and ensures that the selected data settings are used for the predefined quality model used to calculate the requested bit stream quality score. Such a calculation procedure may be performed in an individual unit, such as the quality model unit 400d of FIG. 4, according to an exemplary configuration.

  According to one embodiment, control unit 400b may be configured to determine one or more appropriate data settings using one or more extraction parameters as a direct input to this determination. Alternatively, the control unit 400b can make such a determination based on a comparison between the extracted information and a predefined complexity level set, which is defined herein as a complexity level set. It can typically be obtained from a database called level database 400c.

  Instead of determining based solely on the complexity parameters, the control unit 400b may extract from the received video bit stream, eg, quantization parameters, frame rate, bit rate and / or one or more other quality parameters. Other information may also be considered, such as In the latter case, the parameter extraction unit 400a may be configured to also extract this additional type of parameter. Alternatively, such parameters may be extracted from another (not shown) parameter extraction unit.

  As a result, the control unit 400b is configured to select one or more data settings as a function of one or more extraction parameters, which data settings are executed by the quality model unit 400d. Used for data entry. Once derived, the resulting bit stream quality score is transferred from the quality model unit 400d, or any other that would require the bit stream quality score of the QoE server or encoded bit stream. Can be read by any type of entity.

  Alternatively, the bit stream quality score is monitored directly at the network node / decoding device, where the bit stream quality score is measured by any type of conventional method.

  In addition to the extraction function described above and any other conventional function that utilizes the read bit stream quality score, the device 400 can receive, for example, a decoding function and / or a video display or video routing, for example. Other conventional functions can also be included, such as functions that allow conventional processing of the encoded bit stream.

  Since this type of optional functionality is not necessary for understanding the proposed extraction mechanism, no further description of such additional functionality will be given for this device.

  As described above, the proposed parameter extraction method, according to an alternative embodiment, is used for the purpose of adaptation of the decoding device to the decoding subsystem, in order to optimize the decoding procedure for the resource usage of the decoding subsystem. Can be done.

  Such an adaptation or optimization procedure can be configured in various alternative ways. According to the first embodiment, the encoded bit stream can be analyzed before the start of decoding. Such a configuration, however, requires buffering of the bit stream, so when applying such a procedure, the optimal display of multimedia content provided by the encoded bit stream may be delayed. Absent.

  These individual procedures are shown as steps 5: 1a and 5: 1b, respectively.

  FIG. 5 shows such an adaptation / optimization mechanism. As shown, the encoded bit stream provided from the bit stream source 100 to the decoding subsystem 500 is first buffered by a buffer 502 and at the same time one or more parameters related to encoding complexity. Are extracted from the encoded bit stream by the adaptation unit 501.

  The extracted parameters are processed by the adaptation unit 501 as shown in another step 5: 2. This processing step directly or indirectly as a function of one or more extraction parameters, i.e. selecting one or more data settings after comparing the extraction information with a predefined complexity level set. Is included. Based on the data settings, the decoding subsystem 500 can be adapted and the use of one or more categories of few resources or other selectable functions of the decoding subsystem can be adapted. Such an adaptation step is shown in step 5: 3. Once adapted, the content from the encoded bit stream can be decoded in the decoding process, which makes the various parts of the decoding subsystem 500 more efficient as shown in step 5: 4 in the decoding process. Use in a typical way.

  According to another alternative embodiment described next with reference to FIG. 6, the decoding subsystem 600 follows the conventional procedure, ie, the received encoded bits according to a procedure that does not initially allow adaptation of the decoding procedure. Start decoding the stream.

  In a first step 6: 1, an encoded bit stream is received from the bit stream source 100. Once received by decoding subsystem 600, the encoded bit stream is decoded using any conventional decoding technique, as indicated by processing step 6: 2a. At the same time, the adaptation unit 601 extracts one or more predefined complexity parameters from the received bit stream, as shown in another step 6: 2b. The adaptation unit 601 processes the extraction parameters according to the corresponding processing step 5: 2 in FIG. This is shown in another processing step 6: 3 in FIG.

  Once processed, the appropriate data settings are used as input to the decryption subsystem 600, after which the decryption subsystem 600 is adapted according to predefined rules. This is shown in step 6: 4. As shown in the final step 6: 5, the decoding procedure is performed according to the adaptation performed in step 6: 4.

  The adaptation described by any of the proposed embodiments above can be performed during a limited time interval or iteratively repeated, thereby continuously adapting the decoding subsystem. It should be understood that

  It should also be understood that the embodiments of FIGS. 5 and 6 are merely schematic and simplified system schematics, omitting additional system entities and steps that are not necessary for understanding the proposed adaptation mechanism. is there.

  For example, the encoded bit stream is typically provided to the decoding subsystem and adaptation unit via a communication network. Typically, the decoding subsystem is also connected to some kind of display means that allows rendering functions, ie the display of the decoded video to the end user.

  With reference to FIGS. 5 and 6, an exemplary apparatus configured to perform decoding subsystem adaptation according to any of the embodiments described above will be described in further detail with reference to FIG. .

  Similar to FIG. 4, FIG. 7 shows an apparatus 700 configured to receive and decode an encoded bit stream from the bit stream source 100, and the bit stream source 100 along with the apparatus 700 Forming a media system 701, where these two entities are interconnected either directly as shown or interconnected via a network.

  It should also be understood that, as with the device of FIG. 4, only the units necessary for understanding each device are shown.

  The apparatus 700 of FIG. 7 that is typically part of the decoding device 702 includes a parameter extraction unit 700a, which is typically the parameter extraction unit 400a of FIG. Corresponding to a unit configured to parse the bit stream, thus allowing complexity related information in the form of one or more predefined parameters to be extracted from the encoded bit stream, or In other words, it is possible to extract information indicating the complexity used in the encoding process of the encoded bit stream. Therefore, by processing this extracted information accordingly, it becomes possible to use the complexity related information used in the preceding encoding process.

  The apparatus 700 comprises a control unit 700b configured to select one or more data settings according to the extraction information obtained from the parameter extraction unit 700a. The parameter extraction unit 700a and the control unit 700b together form an adaptation unit 501, 601 shown in FIGS. 5 and 6, respectively. A data set is selected according to rules that depend on one or more extraction parameters, or a predefined complexity level set that is typically accessible from a database referred to herein as the complexity level database 700c. The data setting can be selected as a result of the comparison.

  If the device 700 is configured to perform the method according to the first embodiment described with reference to FIG. 5, the device 700 is parallel to extracting information from the encoded bit stream. Also provided is a buffer 700e configured to buffer the encoded bit stream.

  If the apparatus 700 is used in the method according to the second embodiment described with reference to FIG. 6, the decoding sub-system 700d is a complex extracted from the video bitstream by the parameter extraction unit 700a. At the same time as the control unit 700b configured to receive and process the degree related information, the encoded bit stream provided from the bit stream source 100 will be received.

  Although the invention has been described with reference to specific exemplary embodiments, the description is intended to be broadly illustrative of the concepts of the invention and should not be construed as limiting the scope of the invention. The scope is defined by the appended claims.

  Specifically, the apparatus proposed in FIG. 4 and the apparatus described with reference to FIG. 7 extract complexity related information from the encoded bit stream and use the information in the described embodiment. Only show a simplified example of how to configure a device or network node, which is a functional entity widely used in this type of device, but is within the scope of this document. It should be understood that parts unnecessary for understanding the extraction mechanism are omitted in each figure for the sake of brevity.

Abbreviation List DVB-H Digital Video Broadcast DVB-T Digital Video Ground Broadcast DVB-C Digital Video Broadcast DVB-S Digital Video Broadcast FR Full Reference MB Macro Block MBMS Multimedia Broadcast Multicast Service MOS Average Opinion Score NR No Reference PEVQ Video Quality Perception Evaluation PSNR Peak Signal-to-Noise Ratio OoE Experience Quality RR Reduced Reference VQEG Video Quality Expert Group

Claims (22)

A method in a decoding device (101,700) for adapting a decoding subsystem (500,600) comprising:
Receiving an encoded bit stream from a bit stream source (100) (1: 4, 2: 1, 6: 1);
Extracting complexity-related information indicating the complexity of encoding at the bit stream source (100) from the encoded bit stream (1: 5, 2: 2, 5: 1b, 6: 2b) )When,
Selecting at least one data setting of the decoding subsystem based on the extracted complexity related information (1: 6, 2: 3, 5: 2, 6: 3);
Adapting the decoding subsystem based on the data settings (1: 7a, 2: 4b, 5: 3, 6: 4);
A method comprising the steps of: The extracting step includes:
Performed by parsing (5: 2) at least a portion of the encoded bit stream;
The method according to claim 1. The at least a portion of the encoded bit stream is analyzed (5: 2) before the decoding subsystem starts decoding the encoded bit stream;
The method according to claim 2. The extracting step (6: 2b)
Performed during decoding of the encoded bit stream;
The method according to claim 1 or 2, characterized in that The step of adapting (1: 7a, 2: 4b, 5: 3, 6: 4) consists of the distribution of computing power or the allocation of memory, coprocessors or accelerator blocks within the decoding subsystem. Adapting at least one,
The method according to any one of claims 1 to 4, characterized in that: A method for improving the bit stream quality assessment of a multimedia system (401) comprising:
Receiving an encoded bit stream from a bit stream source (100) (1: 4, 2: 1);
Extracting from the encoded bit stream complexity related information indicating the complexity of encoding at the bit stream source (100) (1: 5, 2: 2);
Selecting at least one data setting for the bit stream quality model based on the extracted complexity related information (1: 6, 2: 3);
Calculating a bit stream quality score of the encoded bit stream using the bit stream quality model based on the selected data settings (1: 7, 2: 4a);
A method comprising the steps of: The extracting step (1: 5, 2: 2) further includes extracting additional information related to quality from the encoded bit stream;
The selecting step further includes selecting at least one data setting based on a combination of the extracted complexity related information and additional information related to the quality.
A method according to any one of claims 1 to 6, characterized in that The extracting step (1: 5, 2: 2) further comprises at least a part of the coded bit stream macroblock partition size, motion vector resolution, skip macroblock or reference used Reading one or more complexity parameters indicating any of the number of images;
A method according to any one of claims 1 to 7, characterized in that The one or more complexity-related parameters include at least information about the presence, distribution, or relative amount of each information type indicated by each parameter.
The method according to claim 8, wherein: The selecting step (1: 6, 2: 3) further comprises:
Comparing the extracted information to a predefined complexity level set;
Selecting at least one of the data settings based on the comparison;
10. The method according to any one of claims 1 to 9, comprising: The encoded bit stream is a video bit stream;
11. A method according to any one of claims 1 to 10, characterized in that An apparatus (700) for adapting a bit stream decoding subsystem (500, 600, 700d) in a multimedia system (701), comprising:
Parameter extraction configured to extract complexity related information indicating the complexity of encoding at the bit stream source (100) from the encoded bit stream received from the bit stream source (100) A unit (700a);
A control unit (700b) configured to select at least one data setting of the decoding subsystem based on the extracted complexity related information and to adapt the decoding subsystem based on the data setting When,
A device characterized by comprising: The parameter extraction unit (700a)
Configured to extract the complexity related information by analyzing at least a portion of the encoded bitstream;
The apparatus according to claim 12. The apparatus (700) further includes a buffer (502, 700e),
The buffer (502, 700e) is configured to buffer the encoded bit stream in parallel with the extraction of the complexity related information from the encoded bit stream.
14. Apparatus according to claim 12 or 13, characterized in that The control unit (700b) is further configured to start decoding the encoded bit stream in the decoding subsystem (700d) when at least a portion of the encoded bit stream is buffered. Configured to,
The apparatus according to claim 14. An apparatus (400) for improving bit stream quality assessment of a multimedia system comprising:
Parameter extraction configured to extract complexity related information indicating the complexity of encoding at the bit stream source (100) from the encoded bit stream received from the bit stream source (100) A unit (400a);
A control unit (400b) configured to select at least one data setting based on the extracted complexity related information;
A quality model unit (400d) configured to calculate a bit stream quality score of the encoded bit stream based on the data settings;
A device characterized by comprising: The parameter extraction unit (400a, 700a) is further configured to extract additional information related to quality from the encoded bit stream;
The control unit (400b, 700b) is configured to select one or more of the data settings based on both the extracted complexity related information and the extracted additional information related to the quality.
Device (400, 700) according to any one of claims 12 to 16, characterized in that. The control unit (400b, 700b) is configured to compare the extracted information with a predefined complexity level set and to select at least one data setting based on the comparison.
18. The device (400, 700) according to any one of claims 12 to 17, characterized in that A complexity level database (400c, 700c) for storing the predefined complexity level set;
The apparatus (400, 700) according to claim 18, characterized in that: The device (400, 700) is mounted on any of a cellular phone, a set top box, a PC, or a laptop computer.
20. Apparatus (400, 700) according to any one of claims 12 to 19, characterized in that The device (400) is implemented in a network node (402);
20. Apparatus (400) according to any one of claims 16 to 19, characterized in that The encoded bit stream is a video bit stream;
Device (400, 700) according to any one of claims 12 to 21, characterized in that

Images