TW201444342A - Complexity aware video encoding for power aware video streaming - Google Patents

Abstract

Video encoding and decoding may take into account decoding complexities, e.g., complexity levels that may be employed by decoders in user devices, such as wireless transmit/receive units (WTRUs). Complexity aware encoding may take into account decoding complexity that may be related to motion compensation, coding mode, and/or deblocking. An encoder, e.g., a server, for example, may encode a video stream by taking into account decoding complexity. The decoding complexity may include one or more of a motion estimation, a mode decision, or a deblocking option. For example, an encoder may encode video data by receiving an input video signal. The encoder may generate a prediction signal based on the input video signal. The encoder may generate an encoded bitstream as a function of the prediction signal. The prediction signal or the encoded bitstream, or both, may be generated as a function of a decoding complexity.

Description

Power-aware video stream complexity perception video coding

CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Provisional Patent Application No. 61/773,528, filed on March 6, 2013.

With improvements in single-chip systems (SoCs) and wireless networking technologies (eg, 4G and/or WiFi), computing power (eg, CPU frequency and/or multi-core) and/or mobile device bandwidth has been greatly improved . With the rapid increase in the number of mobile users, for example, there has been a tremendous increase in the generation and delivery of mobile video content. Providing high quality mobile video services on resource-constrained and diverse mobile devices can be challenging due to changes in display size, processing power, network conditions, and/or battery capacity. Content production, processing, distribution, and consumption methods used at a server (eg, an encoder) and/or at a user device (eg, a decoder) may not be suitable.

Systems, methods and means are provided to implement video encoding and decoding by considering decoding complexity (e.g., the level of complexity that can be employed by a decoder in a user device such as a WTRU). Complexity-aware coding can take into account decoding complexity, which can be related to motion compensation, coding modes, and/or deblocking. An encoder such as a server can encode the video stream, for example, by considering decoding complexity. The decoding complexity may include one or more of a motion estimation, a mode decision, or a deblocking option. In the exemplary encoding, the encoder can receive the input video signal. The encoder can generate a prediction signal based on the input video signal. The encoder can generate a stream of encoded bits based on the prediction signal. The prediction signal or coded bit stream or both can be generated based on the decoding complexity. For example, motion estimation can be performed based on decoding complexity. This may include selecting a motion vector associated with the minimum motion vector cost of the prediction unit. The motion vector cost may be based on decoding complexity, for example, the motion vector cost may include cost elements associated with decoding complexity. The coding mode may be selected based on the decoding complexity, such as by determining the cost associated with the coding mode based on the decoding complexity. The deblocking process can be selectively enabled or disabled based on the deblocking cost, where the deblocking cost is determined based on the decoding complexity.

A detailed description of the illustrative embodiments will now be described with reference to the various drawings. While the description provides a detailed example of possible embodiments, it should be noted that the details are not to be construed as limiting the scope of the application. Further, the drawings may show a flowchart, which is meant to be exemplary. Other embodiments can be used. The order of the messages can be changed as appropriate. The message can be omitted if not needed, and an additional process can be added. FIG. 1 illustrates an exemplary Hypertext Transfer Protocol (HTTP) based video streaming system 100. Service providers such as Netflix, Amazon, etc. can use the Internet infrastructure to deploy their services on the cloud (OTT, over the top). OTT deployment can reduce deployment costs and/or time. In the exemplary video streaming system illustrated in Figure 1, the captured content can be compressed and cut into small segments. The segmentation period in the streaming system can be, for example, between 2 and 10 seconds. Segments can be stored in an HTTP streaming server and distributed, for example, via a content delivery network (CDN). Information such as bit rate, byte range, and/or segmentation performance of a Uniform Resource Locator (URL) may be assembled, for example, in a Media Presentation Description (MPD) manifest file. At the beginning of the streaming session, the client 102 can request an MPD file. Client 102 may determine the segments it may need, for example, based on its capabilities (e.g., profiling, available bandwidth, etc.). For example, according to the request of the client, the server can send the data to the client. Segments transmitted via HTTP can be cached into the HTTP cache server 104, which can allow these segments to be used by other users. System 100 can provide streaming services on a large scale. As more applications use mobile network platforms, the power of mobile devices may continue to be a factor to consider. The power consumption rate can vary for each application. Video decoding may be an application with high power consumption as it may involve intensive computing and memory access. Video playback applications can display images with sufficient brightness levels, which can be very power hungry. FIG. 3 is a block diagram showing an exemplary video playback system 300. Video playback system 300 can include a receiver 302, a decoder 304, and/or a display 306 (eg, a renderer). 4 shows an exemplary block diagram of a block-based single layer decoder 400 that can receive a stream of video bits, such as that produced by encoder 600, illustrated in the manner illustrated in FIG. 6, and can be reconstructed The video signal that will be displayed. At video decoder 400, the bitstream can be parsed by entropy decoder 402. The residual coefficient may be inverse quantized at 404 and inverse transformed at 406 to obtain a reconstructed residual. For example, using spatial prediction at 408 and/or temporal prediction at 410, the coding mode and prediction information can be used to obtain the prediction signal. The prediction signal and reconstruction residuals can be added together at 412 to obtain reconstructed video. The reconstructed video may be loop filtered at 414 and may be stored in reference picture store 416, to be displayed, and/or to be used to decode future video signals. 6 is a diagram showing an exemplary block-based single layer video encoder 600, which may be used to generate bits for the streaming system 200 as exemplarily shown in FIG. flow. Single layer video encoder 600 may employ, for example, spatial prediction 602 (eg, which may be referred to as intra-picture prediction) and/or temporal prediction 604 (eg, which may be referred to as inter-picture prediction and/or motion compensated prediction) to predict Enter a video signal to get a valid compression. Encoder 600 may have mode decision logic 606 that may select the form of the prediction based, for example, on a combined criteria or condition such as rate and/or distortion considerations. Encoder 600 may transform at 608 and/or quantize the prediction residual (e.g., the difference signal between the input signal and the prediction signal) at 610. Quantized residual and mode information (eg, intra-picture and/or inter-picture prediction) and prediction information (eg, motion vector, reference picture index, intra-picture prediction mode, etc.) may be compressed and compressed at entropy encoder 612 To the output video bit stream. Encoder 600 may generate a reconstructed video signal, for example, by applying inverse quantization at 614 and applying an inverse transform to the quantized residual at 616 to obtain a reconstruction residual and adding a reconstruction residual to the prediction signal at 618. The reconstructed video signal may be loop filtered (e.g., deblocking filtered, sample adaptive offset, and/or adaptive loop filtered) at 620 and may be stored in reference picture store 622. The reconstructed video signal can be used to predict future video signals. High Efficiency Video Coding (HEVC) can be a video compression standard being developed by the ITU-T Video Coding Experts Group (VCEG) and the ISO/IEC Mobile Imaging Experts Group (MPEG). HEVC can use block-based hybrid video coding. Encoders and decoders using HEVC can operate as shown in the examples in Figures 4 and 6. HEVC can allow the use of video blocks (eg, large blocks) and can use quadtree partitioning to signal block coded information. A picture or slice may be partitioned into coding tree blocks (CTBs) that may have the same size (eg, 64x64). The CTB may be partitioned into coding units (CUs) having one or more quadtrees, and the CUs may be partitioned into prediction units (PUs) and/or transform units (TUs) having one or more quadtrees. The CU and its PU can use any number of split modes after inter-picture coding. Figure 7 shows eight example split modes. Temporal prediction (eg, motion compensation) can be applied to reconstruct the inter-picture encoded PU. Depending on the accuracy of the motion vector (eg, up to a quarter of a pixel in HEVC), linear filtering can be applied to obtain pixel values at fractional locations. In HEVC, the interpolation filter can have seven or eight taps for brightness and four taps for chrominance. The deblocking filter in HEVC can be content based. Different deblocking filtering operations may be applied at the TU and/or PU boundaries depending on multiple factors such as coding mode differences, motion differences, reference picture differences, pixel value differences, and the like. For entropy coding, HEVC may employ context adaptive binary arithmetic coding (CABAC) for some or most of the block level syntax elements. Binary examples that may be used in CABAC coding may include context-based coding regular bins and context-free bypass coding bins. In the HEVC decoder module, motion compensation, deblocking, and entropy coding may include power consuming operations. FIG. 2 illustrates an exemplary complexity aware streaming system 200. On the server side, for example, multiple video versions with different decoding complexity can be stored (eg, at similar bit rates). The decoding complexity information can be embedded in the MPD file 202. When the streaming session begins, the MPD file 202 can be obtained by a client (e.g., a user device such as a WTRU) 204. The client 204 can request the video version (eg, the video bit stream identified in the MPD file 202) based on the quality of the available bit rate and the decoding complexity level based on the remaining battery power and/or the remaining video playback time. segment). The complexity-aware streaming system 200 can ensure that the entire length of the video can be played back before the device battery power is completely exhausted. If the bit complexity segment is consumed at the current complexity level to consume excessive power to maintain the entire length of playback using the remaining available power on the device, the client 204 can switch to a bitstream segment with a lower level of complexity. If battery power is sufficient, the client 204 can switch to a bitstream with a higher level of complexity for better quality. The methods, systems, and means disclosed herein may allow an encoder to generate bitstreams having different decoding complexities. For example, a streaming system employing the H.264 coding standard can handle power considerations for mobile platforms without regard to decoding complexity on the server side. An encoder (eg, a HEVC reference software encoder or HM encoder) can improve (eg, optimize) coding efficiency while maintaining a certain video quality. Video quality can be measured with objective quality metrics and/or can be subjectively measured by human observers, such as peak signal-to-interference ratio (PSNR), video quality metric (VQM), structural similarity (SSIM), etc. . The universal encoder can ignore the optimization of the bit stream complexity at the decoder side. Complexity-limited coding for mobile phones can take into account mobile phone decoding capabilities. This restriction allows the phone to decode the bit stream in real time. Some power-aware streaming systems may encode video content, for example, at a variable complexity level, profiling, and/or bit rate. To support a power aware streaming system, the encoder can consider rate distortion performance, decoding complexity, and/or power consumption on the decoder side. The methods, systems, and means disclosed herein can provide encoding that can take into account decoding complexity. Figure 5 illustrates an exemplary profiling result (e.g., based on a task's decoding time and percentage of all decoding time) implemented using, for example, an HEVC decoder implementation (e.g., an optimized HEVC decoder implementation). The results for the various modules included in the decoding may include motion compensation (MC), intra-picture reconstruction (Intra_rec), inter-picture reconstruction (Inter_Rec), loop filtering (eg, deblocking (DB) filtering or LF), entropy Decode sampling adaptive offset (SAO), etc. WVGA (eg, 832 x 480) bitstream decoding can be performed on a tablet; an example profiling result for WVGA bitstream decoding is shown at 502. HD (eg, 720p) bitstream decoding can be performed on a personal computer (PC). An example profiling result for HD bitstream decoding is shown at 504. As shown in Figure 5, motion compensation, in-loop filtering, and entropy decoding can take a lot of processing time and can consume a lot of power. Complexity-aware coding techniques may consider decoding complexity, for example, related to the decoding complexity of motion compensation (MC), coding modes, and/or decoding filtering. The decoder can adjust its decisions during motion estimation, mode decision, and/or intra-loop filtering, for example to account for complexity-related performance. A balance between performance and complexity can be obtained. The motion vector can have fractional pixel precision (eg, up to a quarter of a pixel in HEVC and/or H.264). Motion compensation may use one or more interpolation filters to obtain pixel values at fractional pixel locations. The complexity of motion compensation can depend on the fractional position of the motion vector (MV). Figure 8 shows an exemplary possible fractional position, for example, where the motion vector has a quarter pixel accuracy. As shown in Fig. 8, for example, there may be sixteen cases, which may be set according to the value of the fractional part of the MV. A factor that can affect the complexity of the MC can be the direction in which the interpolation can be applied. The interpolation filter can maintain the same horizontal and/or vertical symmetry position. For example, the interpolation filter can keep the fractional positions 802 and 804 the same, can keep the decimal positions 806 and 808 the same, and the like. The computational complexity can remain the same. The memory access efficiency for vertical interpolation and/or horizontal interpolation can be different. For horizontal interpolation, the decoder can extract multiple reference pixels together because the memory arrangement can be organized in lines. For vertical interpolation, the reference pixels can be placed away from each other, for example, depending on the memory location. This setting can affect the extraction speed and can increase the memory bandwidth. The interpolation filter can be a factor that can affect the complexity of the MC. If the interpolation filter has symmetric coefficients, the computational complexity can be reduced because fewer multiplication operations can be required. Table 1 shows exemplary interpolation filter features for the locations shown in FIG. Table 2 shows exemplary memory sizes for interpolation of the locations in Figure 8. Based on these features, a 4x4 complexity matrix can be defined for the motion estimation on the encoder side. In complexity matrices, vertical interpolation can be more complex than horizontal interpolation, and asymmetric interpolation can be more complex than symmetric interpolation.

In the search window, the motion estimation (ME) may look for a motion vector for each prediction unit, for example associated with a minimum mobile cost. Equation (1) can be used to determine the cost of the motion vector.

The motion vector cost can be used in ME processing. As shown in equation (1), the cost calculation can be calculated without considering the complexity of the decoding considerations. It can be, for example, the cost of the motion vector for the current PU . It may be the distortion between the motion compensated prediction signal and the original signal. It can be measured by metrics such as absolute error sum (SAD), absolute transform error and (SATD), and/or squared error sum (SSE). Can be used to encode motion vectors The number of bits of the value. It can be a lambda (lambda) factor associated with the encoding bit rate. Equation (2) can provide an exemplary motion vector cost calculation in which the encoder can consider decoding complexity.

In addition to the terms in equation (1) that contribute to the motion vector cost, equation (2) may also consider decoding complexity in determining the motion vector cost. In equation (2), Can be expressed as The fractional part of the moving vector. Can be an element in the complexity matrix CM . Size (PU) may be, for example, the size of a prediction unit in a pixel. Equation (2) can use the same or different lambda factor values for the second and third terms in the motion vector cost calculation . For example, the first lambda factor Can be used as a multiplier for the Rate term R _mv and a second lambda factor Can be used as a multiplier for complexity items. Motion estimation can choose a trade-off between prediction efficiency and motion compensation complexity (eg, optimal trade-off). For example, if two motion vectors mv ₁ and mv ₂ have similar prediction errors, the encoder can select motion vectors with lower interpolation complexity (eg, lower computational complexity and/or lower memory extraction complexity) Sex).

As provided in equation (3), It can be expressed as a Hadamard (eg, entrywise) product of the interpolation complexity matrix ( IC ) and the scaling matrix ( S ).

The IC may be a basic complexity matrix that considers the interpolation complexity of, for example, different fractional interpolation locations in Figure 8. The IC can reflect the different complexity at different fractional locations according to Tables 1 and 2. Table 3 shows an exemplary IC matrix.

Table 4 shows exemplary non-flat S matrices such as intermediate complexity levels (Table 4a) and low complexity levels (Table 4b). If S is flat, it can be populated with, for example, a value of 2 at the intermediate complexity level position and a value 16 at the low complexity level position.

HEVC can support inter-picture coding mode (for example, merge mode). In merge mode, the current PU can refer to mobile information from spatial and temporal neighboring PUs. The HEVC encoder may select a candidate from the merge candidate (eg, merge candidates in HEVC may include movements from different spatial and/or temporal neighboring PUs) having a minimum cost estimated using equation (4) ( For example, the best candidate).

The complexity-aware coder may use a merge candidate cost calculation in equation (5), and equation (5) may include an item based on complexity considerations.

In equations (4) and (5), Can be the prediction distortion between the original signal and the motion compensated signal, for example, in the mobile information from the merge candidate In case of being applied. Can be used for encoding The number of bits. And Size(PU) can be, for example, in a pixel The complexity and the size of the prediction unit.

The mode decision process may select an encoding mode based on the rate distortion cost estimated using equation (6).

It may be distortion between the reconstructed signal using the encoding mode mode and the original signal. It can be the lambda factor used in the mode decision. If the CU is encoded in the encoding mode mode , then It may be the number of bits used to encode the information of the current coding unit CU. The complexity-aware encoder can use the mode cost, for example, provided by equation (7).

item Can be related to coding mode complexity. For example, you can use equation (8) to measure .

It can be a factor that can be used to balance complexity and compression efficiency. Larger values may correspond to lower decoding complexity. For example, in equation (8), because Mobile compensation complexity Add to CU to calculate The inter-picture coding mode complexity can be considered. Encoding mode complexity can be considered for intra-picture coding mode. Different intra-picture prediction modes can be considered to have different complexity. For example, intra-picture blocks encoded using DC prediction mode may be considered less complex than intra-picture blocks that use mode (angle) prediction mode coding that may use interpolation of neighboring pixel values. When calculating The complexity difference between the inter-picture coding mode and the intra-picture coding mode can be considered and reflected.

Video coding standards (eg, HEVC and H.264) can support in-loop deblocking filtering, which can effectively suppress block effects. Deblocking can consume the decoding time and the percentage of device power (eg, as shown in the example in Figure 5). The complexity aware encoder may decide whether to enable and/or disable the deblocking process, for example based on the deblocking cost that may include decoding complexity considerations. The cost of no deblocking and/or deblocking can be estimated, for example, as provided by equations (9) and (10), respectively.

P can be the original picture. It can be a picture that was reconstructed with deblocking. It may be a picture that was reconstructed without using deblocking. It can be the distortion between the pictures P and R. It can be a deblocking complexity that can be predefined based on different levels of decoding complexity. It can be a lambda factor for balancing complexity and reconstruction quality. Can influence The factor of the value may include, for example, whether the picture is used as a reference picture, an encoding bit rate, and the like. If the picture is a reference picture, then The value can be small. The distortion term in equation (10) can be more important than the complexity term. The predictive node may include, for example, a hierarchical coding structure 900 as shown in FIG. The bit rates for each of the layers 902, 904, 906, 908 in the hierarchy can be assigned differently. If the picture in the lower layer is referenced by the picture in the higher layer, the lower quantization parameter can be used for the lower layer. The bit rate of the lower layer picture can be higher. For lower level screens Value can be used for higher level screens The value is small. If the cost of deblocking calculated by equation (10) is less than the cost of no deblocking calculated via equation (9), the encoder can enable deblocking. Otherwise the encoder can disable deblocking.

FIG. 10A is a diagram of an example communication system 1000 in which one or more disclosed embodiments may be implemented. Communication system 1000 can be a multiple access system that provides content, such as voice, data, video, messaging, broadcast, etc., to multiple wireless users. Communication system 1000 can enable multiple wireless users to access such content via sharing system resources including wireless bandwidth. For example, communication system 1000 can employ one or more channel access methods, such as code division multiple access (CDMA), time division multiple access (TDMA), frequency division multiple access (FDMA), orthogonal FDMA (OFDMA). Single carrier FDMA (SC-FDMA) and the like. As shown in FIG. 10A, communication system 1000 can include wireless transmit/receive units (WTRUs) 1002a, 1002b, 1002c, and/or 1002d (these are commonly or collectively referred to as WTRUs 1002), radio access networks (RAN). 1003/1004/1005, core network 1006/1007/1009, Public Switched Telephone Network (PSTN) 1008, Internet 1010, and other networks 1012, but it should be understood that the disclosed embodiments contemplate any number of WTRU, base station, network, and/or network element. Each of the WTRUs 1002a, 1002b, 1002c, 1002d may be any type of device configured to operate and/or communicate in a wireless environment. For example, the WTRUs 1002a, 1002b, 1002c, 1002d may be configured to transmit and/or receive wireless signals, and may include wireless transmit/receive units (WTRUs), mobile stations, fixed or mobile subscriber units, pagers, mobile phones, individuals Digital assistants (PDAs), smart phones, laptops, portable Internet devices, personal computers, wireless sensors, consumer electronics devices, and more. Communication system 1000 can include base station 1014a and base station 1014b. Each of the base stations 1014a, 1014b may be any type of device configured to facilitate access to one or more communication networks via wireless interfacing with at least one of the WTRUs 1002a, 1002b, 1002c, 1002d, the network The road can be the core network 1006/1007/1009, the Internet 1010, and/or the network 1012. By way of example, base stations 1014a, 1014b may be base transceiver stations (BTS), Node Bs, eNodeBs, home Node Bs, home eNodeBs, website controllers, access points (APs), wireless routers, and the like. While each base station 1014a, 1014b is described as a single component, it should be understood that base stations 1014a, 1014b can include any number of interconnected base stations and/or network elements. The base station 1014a may be part of the RAN 1003/1004/1005, which may also include other base stations and/or network elements (not shown), such as a base station controller (BSC), radio network. Controller (RNC), relay node, and so on. Base station 1014a and/or base station 1014b may be configured to transmit and/or receive wireless signals within a particular geographic area known as a cell (not shown). The cell can be further divided into cell sectors. For example, a cell associated with base station 1014a can be divided into three sectors. Thus, in one embodiment, base station 1014a may include three transceivers, for example, each transceiver corresponding to one sector of a cell. In an embodiment, base station 1014a may use multiple input multiple output (MIMO) technology whereby multiple transceivers may be used for each sector of a cell. The base stations 1014a, 1014b may communicate with one or more WTRUs 1002a, 1002b, 1002c, 1002d via an empty intermediation plane 1015/1016/1017, which may be any suitable wireless communication link ( For example, radio frequency (RF), microwave, infrared (IR), ultraviolet (UV), visible light, etc.). The null intermediate plane 1015/1016/1017 can be established using any suitable radio access technology (RAT). More specifically, as noted above, communication system 1000 can be a multiple access system and can utilize one or more channel access schemes such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, and the like. For example, base station 1014a and WTRUs 1002a, 1002b, 1002c in RAN 1003/1004/1005 may implement a radio technology such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA) and the technology may be used Broadband CDMA (WCDMA) is used to establish an empty intermediate plane 1015/1016/1017. WCDMA may include communication protocols such as High Speed Packet Access (HSPA) and/or Evolved HSPA (HSPA+). HSPA may include High Speed Downlink Packet Access (HSDPA) and/or High Speed Uplink Packet Access (HSUPA). In an embodiment, base station 1014a and WTRUs 1002a, 1002b, 1002c may implement a radio technology such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which may use Long Term Evolution (LTE) and/or LTE-Advanced (LTE-A) to establish an empty mediator 1015/1016/1017. In an embodiment, base station 1014a and WTRUs 1002a, 1002b, 1002c may implement IEEE 802.16 (eg, Worldwide Interoperability for Microwave Access (WiMAX)), CDMA2000, CDMA2000 1X, CDMA2000 EV-DO, Provisional Standard 2000 (IS-2000) ), Temporary Standard 95 (IS-95), Provisional Standard 856 (IS-856), Global System for Mobile Communications (GSM), Radio Access Technology for GSM Enhanced Data Rate Evolution (EDGE), GSM EDGE (GERAN). As an example, base station 1014b in FIG. 10A may be a wireless router, home node B, home eNodeB or access point, and may use any suitable RAT to facilitate, for example, a business location, a home, a vehicle, a campus, etc. Wireless connections in local areas such as. In an embodiment, base station 1014b and WTRUs 1002c, 1002d may establish a wireless local area network (WLAN) by implementing a radio technology such as IEEE 802.11. In an embodiment, base station 1014b and WTRUs 1002c, 1002d may establish a wireless personal area network (WPAN) by implementing a radio technology such as IEEE 802.15. In an embodiment, base station 1014b and WTRUs 1002c, 1002d may use a cellular based RAT (eg, WCDMA, CDMA2000, GSM, LTE, LTE-A, etc.) to establish picocells or femtocells. As shown in FIG. 10A, the base station 1014b can be directly connected to the Internet 1010. Thus, base station 1014b may not need to access Internet 1010 via core network 1006/1007/1009. The RAN 1003/1004/1005 can communicate with a core network 1006/1007/1009, which can be configured to provide voice, data, to one or more WTRUs 1002a, 1002b, 1002c, 1002d, Any type of network that applies and/or via Voice over Internet Protocol (VoIP) services. For example, the core network 1006/1007/1009 can provide call control, billing services, mobile location based services, prepaid calling, internet connectivity, video distribution, etc., and/or high level security for user authentication. Features. Although not shown in FIG. 10A, it should be appreciated that the RAN 1003/1004/1005 and/or the core network 1006/1007/1009 may use the same RAT or a different RAT as the RAN 1003/1004/1005, either directly or indirectly. The other RANs communicate. For example, in addition to being connected to the RAN 1003/1004/1005, which may use E-UTRA radio technology, the core network 1006/1007/1009 may also be in communication with a RAN (not shown) that uses GSM radio technology. The core network 1006/1007/1009 may also serve as a gateway for the WTRUs 1002a, 1002b, 1002c, 1002d to access the PSTN 1008, the Internet 1010, and/or other networks 1012. The PSTN 1008 may include a circuit switched telephone network that provides Plain Old Telephone Service (POTS). The Internet 1010 may include a globally interconnected computer network and device system using a public communication protocol, which may be TCP in the Transmission Control Protocol (TCP)/Internet Protocol (IP) Internet Protocol suite, use Data Packet Protocol (UDP) and IP. Network 1012 may include a wired or wireless communication network that is owned and/or operated by other service providers. For example, network 1012 can include a core network connected to one or more RANs that can use the same RAT or a different RAT as RAN 1003/1004/1005. Some or all of the WTRUs 1002a, 1002b, 1002c, 1002d in the communication system 1000 may include multi-mode capabilities, for example, the WTRUs 1002a, 1002b, 1002c, 1002d may include multiple transceivers that communicate with different wireless networks over different wireless links. For example, the WTRU 1002c shown in FIG. 10A can be configured to communicate with a base station 1014a that can use a cellular-based radio technology, and with a base station 1014b that can use an IEEE 802 radio technology. FIG. 10B is a system diagram illustrating the WTRU 1002. As shown in FIG. 10B, the WTRU 1002 may include a processor 1018, a transceiver 1020, a transmit/receive element 1022, a speaker/microphone 1024, a keyboard 1026, a display/touchpad 1028, a non-removable memory 1030, and a removable type. Memory 1032, power supply 1034, global positioning system (GPS) chipset 1036, and other wireless microphone peripherals 1038. It should be appreciated that the WTRU 1002 may also include any sub-combination of the aforementioned elements while remaining consistent with the embodiments. Moreover, embodiments contemplate that base stations 1014a and 1014b, and/or base stations 1014a and 1014b may represent nodes that may include some or all of the elements depicted in FIG. 10B and described herein, such as but not limited to transceiver stations (BTS), Node B, Website Controller, Access Point (AP), Home Node B, Evolved Home Node B (eNode B), Home Evolved Node B (HeNB), Home Evolved Node B Gate, And proxy nodes, etc. The processor 1018 can be a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors associated with the DSP core, a controller, a microcontroller Dedicated integrated circuit (ASIC), field programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), state machine, and so on. The processor 1018 can perform signal coding, data processing, power control, input/output processing, and/or any other functionality that enables the WTRU 1002 to operate in a wireless environment. The processor 1018 can be coupled to a transceiver 1020 that can be coupled to the transmit/receive element 1022. Although FIG. 10B depicts processor 1018 and transceiver 1020 as separate components, it should be appreciated that processor 1018 and transceiver 1020 can be integrated into an electronic package or wafer. The transmit/receive element 1022 can be configured to transmit or receive signals to or from a base station (e.g., base station 1014a) via the null intermediate plane 1015/1016/1017. For example, in one embodiment, the transmit/receive element 1022 can be an antenna configured to transmit and/or receive RF signals. In an embodiment, as an example, the transmit/receive element 1022 can be a transmitter/detector configured to transmit and/or receive IR, UV, or visible light signals. In an embodiment, the transmit/receive element 1022 can be configured to transmit and receive both RF and optical signals. It should be appreciated that the transmit/receive element 1022 can be configured to transmit and/or receive any combination of wireless signals. Moreover, although transmission/reception element 1022 is depicted as a single element in FIG. 10B, WTRU 1002 may include any number of transmission/reception elements 1022. More specifically, the WTRU 1002 may use MIMO technology. Thus, in one embodiment, the WTRU 1002 may include two or more transmit/receive elements 1022 (e.g., multiple antennas) that transmit and receive wireless signals via the null intermediaries 1015/1016/1017. The transceiver 1020 can be configured to modulate a signal to be transmitted by the transmission/reception element 1022 and to demodulate a signal received by the transmission/reception element 1022. As noted above, the WTRU 1002 may have multi-mode capabilities. Thus, transceiver 1020 can include multiple transceivers that allow WTRU 1002 to communicate via multiple RATs, such as UTRA and IEEE 802.11. The processor 1018 of the WTRU 1002 can be coupled to a speaker/microphone 1024, a keyboard 1026, and/or a display/touchpad 1028 (eg, a liquid crystal display (LCD) display unit or an organic light emitting diode (OLED) display unit), and can Receive user input from these components. The processor 1018 can also output user profiles to the speaker/microphone 1024, the keyboard 1026, and/or the display/trackpad 1028. In addition, the processor 1018 can access information from any suitable memory (eg, the non-removable memory 1030 and/or the removable memory 1032) and store the data in the memory. The non-removable memory 1030 can include random access memory (RAM), read only memory (ROM), hard disk, or any other type of memory storage device. The removable memory 1032 can include a Subscriber Identity Module (SIM) card, a memory stick, a secure digital (SD) memory card, and the like. In other embodiments, the processor 1018 can access information from, and store data in, memory that is not physically located on the WTRU 1002, where the memory can be located, for example, on a server or a home computer. (not shown) on. The processor 1018 can receive power from the power source 1034 and can be configured to allocate and/or control power for other elements in the WTRU 1002. Power supply 1034 can be any suitable device that powers WTRU 1002. For example, the power source 134 may include one or more dry cells (such as nickel-cadmium (Ni-Cd), nickel-zinc (Ni-Zn), nickel-hydrogen (NiMH), lithium-ion (Li-ion), etc.), solar cells. , fuel cells, etc. The processor 1018 can also be coupled to a GPS chipset 1036 that can be configured to provide location information (eg, longitude and latitude) related to the current location of the WTRU 1002. Additionally or alternatively to the information from the GPS chipset 1036, the WTRU 1002 may receive location information from base stations (e.g., base stations 1014a, 1014b) via the null mediator 1015/1016/1017, and/or based on two or more The timing of the signals received by nearby base stations determines their position. It should be appreciated that the WTRU 1002 can obtain location information using any suitable location determination method while remaining consistent with the implementation. Processor 1018 may also be coupled to other peripheral devices 1038, which may include one or more software and/or hardware modules that provide additional features, functionality, and/or wired or wireless connections. For example, peripheral device 1038 can include an accelerometer, an electronic compass, a satellite transceiver, a digital camera (for photos and video), a universal serial bus (USB) port, a vibrating device, a television transceiver, hands-free headset, Bluetooth R Modules, FM radio units, digital music players, media players, video game console modules, Internet browsers, and more. Figure 10C is a system diagram of RAN 1003 and core network 1006, in accordance with one embodiment. As described above, the RAN 1003 can communicate with the WTRUs 1002a, 1002b, 1002c via the null plane 1015 using UTRA radio technology. The RAN 1003 can also communicate with the core network 1006. As shown in FIG. 10C, the RAN 1003 can include Node Bs 1040a, 1040b, 1040c, each of which can include one or more transceivers that communicate with the WTRUs 1002a, 1002b, 1002c via the null plane 1015. Each of Node Bs 1040a, 1040b, 1040c can be associated with a particular cell (not shown) within RAN 1003. The RAN 1003 may also include RNCs 1042a, 1042b. It should be appreciated that the RAN 1003 may include any number of Node Bs and RNCs while remaining consistent with the implementation. As shown in FIG. 10C, Node Bs 1040a, 1040b can communicate with RNC 1042a. Additionally, Node B 1040c can communicate with RNC 1042b. Node Bs 1040a, 1040b, 1040c can communicate with respective RNCs 1042a, 1042b via an Iub interface. The RNCs 1042a, 1042b can communicate with each other via the Iur interface. Each RNC 1042a, 1042b can be configured to control a respective Node B 1040a, 1040b, 1040c connected thereto. In addition, each RNC 1042a, 1042b can be configured to perform or support other functions, such as outer loop power control, load control, admission control, packet scheduling, handover control, macro diversity, security functions, data encryption, and the like. The core network 1006 shown in FIG. 10C may include a media gateway (MGW) 1044, a mobile switching center (MSC) 1046, a serving GPRS support node (SGSN) 1048, and/or a gateway GPRS support node (GGSN) 1050. While each of the foregoing elements is described as being part of core network 1006, it should be understood that other entities other than the core network operator may also own and/or operate any of these elements. The RNC 1042a in the RAN 1003 can be connected to the MSC 1046 in the core network 1006 via the IuCS interface. The MSC 1046 can be connected to the MGW 1044. MSC 1046 and MGW 1044 may provide WTRUs 1002a, 1002b, 1002c with access to a circuit switched network, such as PSTN 1008, to facilitate communication between WTRUs 1002a, 1002b, 1002c and conventional landline communication devices. The RNC 1042a in the RAN 1003 can also be connected to the SGSN 1048 in the core network 1006 via an IuPS interface. The SGSN 1048 can be connected to the GGSN 1050. The SGSN 1048 and GGSN 1050 may provide WTRUs 1002a, 1002b, 1002c with access to a packet switched network, such as the Internet 1010, to facilitate communication between the WTRUs 1002a, 1002b, 1002c and IP-enabled devices. As noted above, core network 1006 can also be coupled to network 1012, which can include other wired or wireless networks that are owned and/or operated by other service providers. Figure 10D is a system diagram of RAN 1004 and core network 1006, in accordance with one embodiment. As described above, the RAN 1004 can communicate with the WTRUs 1002a, 1002b, 1002c via the null plane 1016 using the E-UTRA radio technology. In addition, the RAN 1004 can also communicate with the core network 1007. The RAN 1004 may include eNodeBs 1060a, 1060b, 1060c, but it should be appreciated that the RAN 1004 may include any number of eNodeBs while remaining consistent with the implementation. Each eNodeB 1060a, 1060b, 1060c may include one or more transceivers to communicate with the WTRUs 1002a, 1002b, 1002c via the null plane 1016. In one embodiment, eNodeBs 1060a, 1060b, 1060c may implement MIMO technology. Thus, for example, eNodeB 1060a may use multiple antennas to transmit wireless signals to, and receive wireless signals from, WTRU 1002a. Each eNodeB 1060a, 1060b, 1060c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, user scheduling in the uplink and/or downlink, etc. Wait. As shown in FIG. 10D, the eNodeBs 1060a, 1060b, 1060c can communicate with each other via the X2 interface. The core network 1007 shown in FIG. 10D may include a mobility management gateway (MME) 1062, a service gateway 1064, and a packet data network (PDN) gateway 1066. While each of the above elements is described as being part of the core network 1007, it should be understood that other entities than the core network operator may also own and/or operate any of these elements. The MME 1062 may be interfaced via the S1. It is connected to each of the eNodeBs 1060a, 1060b, 1060c in the RAN 1004 and can act as a control node. For example, the MME 1062 may be responsible for authenticating the users of the WTRUs 1002a, 1002b, 1002c, bearer activation/deactivation, selecting a particular service gateway during the initial connection of the WTRUs 1002a, 1002b, 1002c, and the like. The MME 1062 may also provide control plane functionality to perform handover between the RAN 1004 and other RANs (not shown) that employ other radio technologies such as GSM or WCDMA. The service gateway 1064 can be connected to each of the eNodeBs 1060a, 1060b, 1060c in the RAN 1004 via an S1 interface. The service gateway 1064 can typically route and forward user data packets to/from the WTRUs 1002a, 1002b, 1002c. The service gateway 1064 can also perform other functions, such as anchoring the user plane during handover between eNodeBs, triggering paging, managing and storing the WTRUs 1002a, 1002b, when the downlink information is available to the WTRUs 1002a, 1002b, 1002c, 1002c context and so on. The service gateway 1064 can also be coupled to the PDN gateway 1066 to provide WTRUs 1002a, 1002b, 1002c with access to a packet switched network, such as the Internet 1010, to facilitate WTRUs 1002a, 1002b, 1002c, and IP assignments. Can communicate between devices. The core network 1007 can facilitate communication with other networks. For example, core network 1007 may provide WTRUs 1002a, 1002b, 1002c with access to a circuit switched network, such as PSTN 1008, to facilitate communication between WTRUs 1002a, 1002b, 1002c and conventional landline communication devices. As an example, core network 1007 can include or be in communication with an IP gateway (eg, an IP Multimedia Subsystem (IMS) server), where the IP gateway acts as an interface between core network 1007 and PSTN 1008. In addition, core network 1007 can provide WTRUs 1002a, 1002b, 1002c with access to network 1012, which can include other wired or wireless networks owned and/or operated by other service providers. Figure 10E is a system diagram of RAN 1005 and core network 1009, in accordance with one embodiment. The RAN 1005 may be an Access Service Network (ASN) that communicates with the WTRUs 1002a, 1002b, 1002c via the null plane 1017 using IEEE 802.16 radio technology. As discussed further below, the communication links between the different functional entities of the WTRUs 1002a, 1002b, 1002c, RAN 1005, and core network 1009 may be defined as reference points. As shown in FIG. 10E, the RAN 1005 may include base stations 1080a, 1080b, 1080c and ASN gateway 1082, but it should be appreciated that the RAN 1005 may include any number of base stations and ASN gateways while remaining consistent with the implementation. . Each of the base stations 1080a, 1080b, 1080c may be associated with a particular cell (not shown) in the RAN 1005, and each base station may include one or more transceivers to communicate with the WTRUs 1002a, 1002b via the null plane 1017. And 1002c communicate. In one embodiment, base stations 1080a, 1080b, 1080c may implement MIMO technology. Thus, for example, base station 1080a can use multiple antennas to transmit wireless signals to, and receive wireless signals from, WTRU 1002a. Base stations 1080a, 1080b, 1080c can also provide mobility management functions such as handover triggering, tunnel establishment, radio resource management, traffic classification, quality of service (QoS) policy enforcement, and the like. The ASN gateway 1082 can act as a traffic aggregation point and can be responsible for paging, user profile cache, routing to the core network 1009, and the like. The null intermediation plane 1017 between the WTRUs 1002a, 1002b, 1002c and the RAN 1005 may be defined as an Rl reference point that implements the IEEE 802.16 specification. In addition, each of the WTRUs 1002a, 1002b, 1002c can establish a logical interface (not shown) with the core network 1009. The logical interface between the WTRUs 1002a, 1002b, 1002c and the core network 1009 can be defined as an R2 reference point that can be used for authentication, authorization, IP host configuration management, and/or mobility management. The communication link between each of the base stations 1080a, 1080b, 1080c can be defined as an R8 reference point that includes protocols for facilitating WTRU handover and data transfer between base stations. The communication link between the base stations 1080a, 1080b, 1080c and the ASN gateway 1082 can be defined as an R6 reference point. The R6 reference point can include an agreement to facilitate mobility management based on mobility events associated with each of the WTRUs 1002a, 1002b, 1002c. As shown in FIG. 10E, the RAN 1005 can be connected to the core network 1009. The communication link between the RAN 1005 and the core network 1009 can be defined as an R3 reference point, which, by way of example, includes protocols for facilitating data transfer and mobility management capabilities. Core network 1009 may include a Mobile IP Home Agent (MIP-HA) 1084, an Authentication, Authorization, Accounting (AAA) server 1086, and a gateway 1088. While each of the foregoing elements is described as being part of the core network 1009, it should be understood that entities other than the core network operator may also own and/or operate any of these elements. The MIP-HA may be responsible for IP address management and may enable the WTRUs 1002a, 1002b, 1002c to roam between different ASNs and/or different core networks. The MIP-HA 1084 may provide WTRUs 1002a, 1002b, 1002c with access to a packet switched network, such as the Internet 1010, to facilitate communication between the WTRUs 1002a, 1002b, 1002c and IP-enabled devices. The AAA server 1086 can be responsible for user authentication and supporting user services. Gateway 1088 can facilitate interworking with other networks. For example, gateway 1088 can provide WTRUs 1002a, 1002b, 1002c with access to a circuit-switched network, such as PSTN 1008, to facilitate communication between WTRUs 1002a, 1002b, 1002c and conventional landline communication devices. In addition, gateway 1088 can provide WTRUs 1002a, 1002b, 1002c with access to network 1012, where the network can include other wired or wireless networks that are owned and/or operated by other service providers. Although not shown in Figure 10E, it should be understood that the RAN 1005 can be connected to other ASNs and the core network 1009 can be connected to other core networks. The communication link between the RAN 1005 and other ASNs may be defined as an R4 reference point, which may include a protocol for coordinating the movement of the WTRUs 1002a, 1002b, 1002c between the RAN 1005 and other ASNs. The communication link between core network 1009 and other core networks may be defined as an R5 reference point, which may include protocols for facilitating interworking between the home core network and the visited core network. The processes and means disclosed herein can be applied in any combination, can be applied to other wireless technologies, and used for other services. The WTRU may relate to an identification code of a physical device, or an identification code of a user such as a user-related identification code (e.g., MSISDN, SIR PRL, etc.). The WTRU may relate to an application based identification code, such as a username that each application may use. Although features and elements of a particular combination are described above, those of ordinary skill in the art will appreciate that each feature can be used alone or in any combination with other features and elements. Moreover, the methods described herein can be implemented in a computer program, software or firmware incorporated into a computer readable medium and executed by a computer or processor. Examples of computer readable media include electrical signals (transmitted via a wired or wireless connection) and computer readable storage media. Examples of computer readable storage media include, but are not limited to, read only memory (ROM), random access memory (RAM), scratchpad, cache memory, semiconductor storage devices, such as internal hard drives and removable magnetics. Magnetic media such as films, magneto-optical media, and optical media such as CD-ROM discs and digital versatile discs (DVDs). A processor associated with the software can be used to implement a radio frequency transceiver for use in a WTRU, UE, terminal, base station, RNC, or any host computer.

100. . . Video streaming system

102, 204. . . user terminal

104. . . Cache server

200. . . Streaming system

202. . . MPD file

300. . . Video playback system

302. . . receiver

304. . . decoder

306. . . monitor

400. . . Single layer decoder

402. . . Entropy decoder

416, 622. . . Reference screen storage

600. . . Single layer coder

602. . . Spatial prediction

604. . . Time prediction

606. . . Mode decision logic

612. . . Entropy encoder

802, 804, 806, 808. . . Decimal position

900. . . Hierarchical coding structure

902, 904, 906, 908. . . Floor

1000. . . Communication Systems

1002, 1002a, 1002b, 1002c, 1002d. . . WTRU

1003, 1004, 1005. . . RAN

1006, 1007, 1009. . . Core network

1008. . . PSTN

1010. . . Internet

1012. . . Other network

1014a, 1014b, 1080a, 1080b, 1080c. . . Base station

1015, 1016, 1017. . . Empty intermediary

1018. . . processor

1020. . . transceiver

1022. . . Transmission/reception component

1024. . . Speaker/microphone

1026. . . keyboard

1028. . . Display/trackpad

1030. . . Non-removable memory

1032. . . Removable memory

1034. . . power supply

1036. . . GPS chipset

1038. . . Peripheral device

1040a, 1040b, 1040c. . . Node B

1042a, 1042b. . . RNC

1044. . . MGW

1046. . . MSC

1048. . . SGSN

1050. . . GGSN

1062. . . MME

1064. . . Service gateway

1066. . . PDN gateway

1084. . . MIP-AA

1086. . . AAA server

1088. . . Gateway

AAA. . . Proof, authorization, accounting

ASN. . . Access service network

DB. . . Deblocking

GGSN. . . Gateway GPRS support node

GPS. . . Global Positioning System

HEVC. . . Efficient video coding

HTTP. . . Hypertext transfer protocol

Intra_rec. . . Intra-image reconstruction

Inter_Rec. . . Inter-picture reconstruction

IP. . . Internet protocol

Iub, IuCS, IuPS, iur, S1, X2. . . interface

MC. . . Motion compensation

MGW. . . Media gateway

MIP-HA. . . Mobile IP local agent

MME. . . Mobility management gateway

MPD. . . Media presentation description

MSC. . . Action exchange center

PDN. . . Packet data network

PSTN. . . Public switched telephone network

R1, R3, R6, R8. . . Reference point

RAN. . . Radio access network

SAO. . . Entropy decoding sampling adaptive offset

SGSN. . . Service GPRS support node

WTRU. . . Wireless transmission/reception unit

Figure 1 shows an exemplary Hypertext Transfer Protocol (HTTP) based video streaming system. Figure 2 illustrates an exemplary complexity aware streaming architecture. Figure 3 illustrates various modules of an exemplary video playback system. Figure 4 shows a block diagram of an exemplary video decoder. Figure 5 shows an exemplary profiling result for efficient video coding (HEVC) decoding of, for example, a WVGA sequence on a tablet and a high profiling (HD) sequence on a personal computer (PC). Figure 6 shows a block diagram of an exemplary video transcoder. Figure 7 shows an exemplary prediction unit (PU) mode in HEVC. Figure 8 shows an exemplary pixel location for luminance shift compensation (MC) in HEVC. Figure 9 shows an exemplary hierarchical coding structure in HEVC. FIG. 10A is a system diagram of an example communication system in which one or more of the disclosed embodiments may be implemented. Figure 10B is a system diagram of an example wireless transmit/receive unit (WTRU) that can be used within the communication system shown in Figure 10A. Figure 10C is a system diagram of an example radio access network and an example core network that can be used within the communication system shown in Figure 10A. Figure 10D is a system diagram of an example radio access network and an example core network that can be used within the communication system shown in Figure 10A. Figure 10E is a system diagram of an example radio access network and an example core network that can be used within the communication system shown in Figure 10A.

802, 804, 806, 808. . . Decimal position

Claims (20)

A method for encoding a video data, the method comprising: receiving an input video signal; generating a prediction signal based on the input video signal; and generating an encoded bit stream according to the prediction signal, wherein the prediction signal and the encoded bit stream At least one of them is generated according to a decoding complexity. The method of claim 1, wherein generating the prediction signal comprises performing a motion estimation based on the decoding complexity. The method of claim 2, wherein performing the motion estimation comprises selecting a motion vector associated with a minimum motion vector cost of a prediction unit. The method of claim 3, further comprising determining a mobile vector cost based on an interpolation complexity. The method of claim 4, wherein determining the motion vector cost comprises considering whether at least one of a horizontal interpolation, a vertical interpolation, an asymmetric interpolation, and a symmetric interpolation is performed. The method of claim 1, wherein generating the prediction signal comprises selecting an encoding mode based on the decoding complexity. The method of claim 6, wherein selecting the encoding mode comprises determining a cost associated with the encoding mode based on the decoding complexity. The method of claim 7, wherein determining the cost associated with the encoding mode comprises determining an encoding mode complexity based on an interpolation complexity. The method of claim 1, wherein generating the encoded bit stream comprises selectively enabling or disabling a deblocking process based on a deblocking cost, the deblocking cost being determined based on the decoding complexity. The method of claim 9, wherein the deblocking cost is predefined for a level of decoding complexity. An apparatus configured to encode a material, the apparatus comprising: a memory; and an encoder, wherein the apparatus is at least partially configured to: receive an input video signal; generate a prediction signal based on the input video signal; The predictive signal produces a coded bitstream, wherein at least one of the predictive signal and the encoded bitstream is generated in accordance with a decoding complexity. The apparatus of claim 11, wherein the encoder is configured to generate the prediction signal by at least performing a motion estimation based on the decoding complexity. The apparatus of claim 12, wherein the encoder is configured to perform the motion estimation by at least selecting a motion vector associated with a minimum motion vector cost of a prediction unit. The apparatus of claim 13, wherein the encoder is further configured to determine a motion vector cost based on an interpolation complexity. The apparatus of claim 14, wherein the encoder is configured to at least consider by considering at least one of a horizontal interpolation, a vertical interpolation, an asymmetric interpolation, and a symmetric interpolation. Execute to determine the cost of the mobile vector. The apparatus of claim 11, wherein the encoder is configured to generate the prediction signal by at least selecting an encoding mode based on the decoding complexity. The apparatus of claim 16, wherein the encoder is configured to select the encoding mode by at least determining a cost associated with an encoding mode based on the decoding complexity. The apparatus of claim 17, wherein the encoder is configured to determine the cost associated with the encoding mode by at least determining an encoding mode complexity based on an interpolation complexity. The apparatus of claim 11, wherein the encoder is configured to generate an encoded bit stream at least by selectively enabling or disabling a deblocking process according to a deblocking cost, the deblocking cost It is determined based on the decoding complexity. The apparatus of claim 19, wherein the deblocking cost is predefined for a decoding complexity level.