WO2014052602A1 - Method and apparatus of edge guided processing for video coding - Google Patents

Description

METHOD AND APPARATUS OF EDGE GUIDED PROCESSING FOR VIDEO

CODING

CROSS REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of U.S. Provisional Patent Application

61/707,477, entitled "Method and Apparatus of Edge Guided Processing of Video Coding," which was filed on September 28, 2012 and is hereby incorporated by reference herein.

BACKGROUND

[0002] With the growth of smart phones and tablets both in resolution and computation capability, there has been an increase in video applications, such as video chat, mobile video recording and sharing, and video streaming. To improve the quality of experience while using the video applications, video coding systems have been used to compress digital video signals. Compressing the digital video signals reduces the storage need and/or transmission bandwidth of such signals. However, when compressing digital video signals, images within the videos that are rich in texture and/or edges may suffer from a loss of detail due to the compression process.

SUMMARY OF THE INVENTION

[0003] Disclosed herein are methods and apparatus for edge guided processing that may be used for video coding. Methods and apparatus herein may be provided to process regions in a video or a reconstructed video that may be rich in texture and/or edges. A. pixel that may be part of a texture may be an edge pixel. This may be done, for example, to improve the prediction accuracy when these regions may be used as a reference. Regions rich in texture and/or edges may suffer from a loss of detaii due to a quantization process, which may remove and/or reduce high frequency information. When these regions may be used as reference in spatial, temporal, or inter layer prediction, the resulting prediction residual may have higher energy, and may be more difficult to compress. The methods and apparaius disclosed herein may recover high frequency information that may be lost during quantization, may recover high frequency

- i - information in regions where such information loss may be likely to have occurred during quantization, and may keep computational complexit '- of edge guided processing low. The methods and apparatus disclosed herein may be applicable to various types of prediction used in video coding, such as spatial, temporal, and inter-layer prediction, or the like.

[0004] Methods and apparatus may be provided for edge guided processing that may be used for video coding. The apparatus may include a processor that may be configured to perform a number of actions. A pixel may be detected that may be part of an edge or a texture that may be within an area of an image. An edge may be a texture. The pixel may be classified in an edge pixel class using a characteristic of the pixel. A filter for the edge pixel class may be determined using the characteristic of the pixel. This may be done, for example, to pre v ent a loss of detail in the area of the image during video coding. The filter may be applied on the pixel.

[0005] The Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, not is it intended to be used to limit the scope of the claimed subject matier. Furthermore, the claimed subject matter is not limited to any limitations that solve any or all disadvantages noted in any part of this disclosure.

[0006] A more detailed understanding may be had from the following description, given by way of example in conjunction with the accompanying drawings.

[0007] FIG, 1 is a diagram illustrating an example of a block-based hybrid video encoder.

[0008] FIG, 2 is a diagram illustrating an example of a block-based video decoder.

[0009] FIG, 3 is a diagram illustrating an example of a scalable video encoder.

[0010] FIG, 4 is a diagram illustrating an example of a 2-layer scalable video decoder.

[0011] FIG, 5 is a diagram illustrating an example of edge guided processing that may occur at an encoder.

[0012] FIG, 6 is a diagram illustrating an example of an edge classification that may be based on edge directions.

[0013] FIG, 7 is a diagram illustrating an example of a 1-d filter.

[0014] FIG. 8 is a diagram illustrating an example of edge guided processing that may occur at a decoder. [0015] FIG. 9 is a diagram illustrating an example of an error map between a picture and an upsampled picture.

[0016] FIG. 1 OA is a system diagram of an example communications system in which one or more disclosed embodiments may be implemented.

[0017] FIG. 10B is a system diagram of an example wireless transmit/receive unit

(WTRXJ) that may be used within the communications system illustrated in FIG. 10A.

[0018] FIG. 10C is a system diagram of an example radio access network and an example core network that may be used within the communications system illustrated in FIG. 10A.

[0019] FIG. 10D is a system diagram of another example radio access network and another example core network (hat may be used within the communications system illustrated in FIG. I OA.

[0020] FIG. 10E is a system diagram of another example radio access network and another example core network that may be used within the communications system illustrated in FIG. 10A.

DETAILED DESCRIPTION

[0021] A detailed description of illustrative embodiments will now be described with reference to the various Figures. Although this description provides a detailed example of possible implementations, it should be noted that the details are intended to be exemplary and in no way limit the scope of the application.

[0022] Disclosed herein are methods and apparatus for edge guided processing that may be used for video coding. Methods and apparatus herein may process regions in video, such as a reconstructed video, that may be rich in texture and/or edges. A pixel that may be part of a texture may be an edge pixel. This may be done, for example, to improve the prediction accuracy when these regions may be used as reference.

[0023] As disclosed herein, edge guided processing may be applied to regions of a reconstructed video. For example, regions rich in texture and/or edges may suffer from loss of detail due to a quantization process, which may remove and/or reduce high frequency

information. When these regions may be used as reference in a prediction, such as a spatial, temporal, or inter layer prediction, the resulting prediction residual may have higher energy and may be more difficult to compress.

[0024] Methods and apparatus disclosed herein may recover high frequency information that may be lost during quantization, may recover high frequency information in regions where such information loss may be likely to have occurred during quantization, and may keep computational complexity of edge guided processing low. Methods and apparatus disclosed herein may be applicable to various types of prediction used in video coding, such as spatial, temporal, inter-layer prediction, or the like. The methods and apparatus disclosed herein may be applicable for single layer video coding where spatial and temporal prediction may be applied, scalable video coding where inter-layer prediction may be applied using a reconstructed video signal from one or more lower layers, or the like.

[0025] A method and an apparatus may be used for edge guided processing for video encoding. The apparatus may include a processor that may be configured to perform a number of actions. An edge pixel may be detected within an image. The edge pixel may be refined. The edge pixel may be classified in an edge pixel class. A filter for the edge pixel class may be trained using a video, which may be an original video. The trained filter may be applied to the edge pixel

[0026] A method and an apparatus may be used for edge guided processing for video encoding. The apparatus may include a processor that may be configured to perform a number of actions. An edge pixel may be detected within an image. The edge pixel may be refined. The edge pixel may be classified in an edge pixel class. A filter for the edge pixel class may be received. The trained filter may be applied to the edge pixel.

[0027] A method or an apparatus may be used to refine an edge pixel and train a filter.

The apparatus may include a processor that may be configured to perform a number of actions. An edge pixel may be refined based on the value of an edge gradient, for example, by discarding the edge pixel when its gradient value is less than a threshold. A density of edges may be determined within an area. The density of edges may be discarded when the density of edges may be lower than a density threshold. A filter may be trained.

[0028] A method or an apparatus may be used to classify an edge pixel and train a filter.

The apparatus may include a processor that may be configured to perform a number of actions. An edge pixel may be classified into one or more categories according to an edge direction and a characteristic of the edge pixel. The characteristic of the edge pixel may be a second derivative in a normal direction of the edge pixel. A filter for a category of the one or more categories may^¬ be trained. The category of the one or more categorizes may be a combined classification category. A mapping may be generating between the filter and the combined classification category. The mapping may be transmitted.

[0029] Methods and apparatus may be provided for edge guided processing that may be used for video coding. The apparatus may include a processor that may be configured to perform a number of actions. A pixel may be detected that may be part of an edge or a texture that may be within an area of an image. The pixel may be classified in an edge pixel class using a characteristic of the pixel. A filter for the edge pixel class may be determined using the characteristic of the pixel. This may be done, for example, to prevent a loss of detail in the area of the image during video coding. The filter may be applied on the pixel.

[0030] Methods and apparatus may be provided for edge guided processing that may be used for video coding. The apparatus may include a processor that may be configured to perform a number of actions, A pixel may be detected that may be part of an edge or a texture that may be within an area of an image. An edge strength may be determined in an edge direction for the pixel. It may be determined that the pixel belongs to an edge when the edge strength for the pixel is above a threshold.

[0031] The pixel may be classified in an edge pixel class using a characteristic of the pixel. For example, an edge direction may be determined for the pixel. A normal value may be determined for the pixel. The normal value may indicate the value of a second order derivative that may be perpendicular to the edge direction. The pixel may be classified in an edge pixel class using the normal value and the edge direction. As another example, a gradient value G_x indicating an edge strength in a first edge direction for the pixel may be determined. A gradient value Gy indicating an edge strength in a second edge direction for the pixel may be determined.

An edge direction may be determined where the edge direction may be equal to tan^{"" 1} \ ~^ \ . As

" - ^■ Gx ' another example, a second order derivative for the pixel may be determined using sd = Θ + ~, where sd is the second order derivative. An edge direction may be determined. The pixel may be classified in the edge pixel class using the second order derivative and the edge direction.

[0032] A filter for the edge pixel class may be determined using the characteristic of the pixel. This may be done, for example, to prevent a loss of detail in the area of the image during video coding. A parameter may be determined for the filter. For example, a parameter may be determined that may optimize the filter for the edge pixel class. As another example, a parameter for the filter may be determined by performing a regression analysis to minimize creating an error in an area of the image when the filter is applied on a pixel. As another example, a parameter may be determined for the filter by generating a second image using the filter, comparing the second image to the image, determining an error for the pixel class, and determining a parameter for the filter that minimizes the error for the pixel class. As another example, a parameter may be determined for the filter to enhance high frequency information that may be lost during video coding. The parameter may be sent to a decoder. The filter may be applied on the pixel. [0033] Due to the increased demand for smart phones and tablet, video applications, such as video chat, mobile video recording and sharing, video streaming, or the like may use video transmission in a heterogeneous environment. These scenarios may be referred to as a 3-screen scenario or an N-screen scenario, A 3-screen scenario or an N-screen scenario may consider a number of consumer devices such as a personal computer, a smart phone, a table, a television, or the like. In an N-screen scenario, video consumption on devices with widely varying capabilities in terms of computing power, memory/storage size, display resolution, display frame rate, or the like may be accommodated. Network and transmission channels that may be used in an N- screen scenario may have widely varying characteristics in terms of packet loss rate, available channel bandwidth, burst error rate, or the like and may be accommodated.

[0034] Video data may be transmitted over a combination of wired networks and wireless networks. Scalable video coding may improve the qualify of experience for video applications ranning on devices with different capabilities over a heterogeneous network. Scalable video coding may encode the signal at a high representation such as a temporal resolution, spatial resolution, quality, or the like. Although the scalable video coding may be encoded at a high representation, the scalable video coding may be decode from a subset of the video stream that may depend on a rate and representation used by an application running on a device. For example, a video may be encoded at a high representation, but an application miming on a smart for may decode the video at a low representation. This may be done, for example, save bandwidth and storage. Scalable video coding may be used with MPEG -2 Video, H.263, MPEG4 Visual, 1 1.2(4. HEVC, or the like.

[0035] Video coding systems may be used to compress digital video signals to reduce the storage and/or transmission bandwidth of such signals. There may be various types of video coding systems, such as block-based, wavelet-based, object-based systems, block-based hybrid video coding systems, or the like. Examples of bl ock-based video coding systems may include international video coding standards such as the MPEG 1/2/4 part 2, H.264/MPEG-4 part 10 AVC and VC-i standards.

[0036] FIG. 1 is a diagram illustrating an example of a block-based hybrid video encoder. Input video signal 302 may be processed block by block. A video block unit for input video signal 302 may be 16x16 pixels. A video block unit may be referred to as a macroblock or MB. In High Efficiency Video Coding (HEVC), extended block sizes may be used to compress high resolution video signals, such as video signal that may be 1080p or larger. The extended block sizes may be referred to as a coding unit or CU. In HEVC, a CU may be up to 64x64 pixels. A CU may be partitioned into prediction units (PU) and a prediction implementation may be applied for a prediction unit. For example, for an input video block (e.g., MB or CU), spatial prediction 360 and/or temporal prediction 362 may be performed.

[0037] At spatial prediction 360, a video block may be predicted from coded neighboring blocks in the same video picture/slice. Spatial prediction may be referred to as intra-prediction. Spatial prediction may reduce spatial redundancy that may be inherent in the video signal. At temporal prediction 362, pixels from already coded video pictures may be used to predict a video block. Temporal prediction may be referred to as inter-prediction or motion compensated prediction. Temporal prediction may reduce temporal redundancy inherent in the video signal. A temporal prediction signal for a video block may include one or more motion vectors. A temporal prediction signal for a video block may include one or more reference picture indexes if multiple reference pictures may be used. A. motion vector and/or a reference picture index maybe used to identify reference pictures in reference picture store 364.

[0038] After spatial and/or temporal prediction, mode decision 380 may choose a prediction mode, for example, based on a rate-distortion optimization implementation. At 31 6, the prediction block may be subtracted from the video block. The prediction residual may be transformed at transform 304 and may be quantized at quantization 306. The quantized residual coefficients may be inverse quantized at inverse quantization 310 and may be inverse transformed at inverse transform 312. This may be done, for example, to form a reconstructed residual that may be added io the prediction block at 326 to form a reconstructed video block. Filtering, such as deblocking filter, adaptive loop filtering, or the like may be applied on the reconstructed video block at loop filter 366. This may be done, for example, before the reconstructed block may be stored in reference picture store 364 and may be used to code future video blocks. To form bitstream 320, which may be a output video bitstream, a coding mode (inter or intra), prediction mode information, motion information, and/or quantized residual coefficients may be sent to the entropy coding unit 308 and may be compressed and packed by entropy coding unit 308 to form bitstream 320.

[0039] As shown in FIG. 1 , during encoding, prediction residual of the base layer blocks may be transformed at 304 and may be quantized at 306. Quantization may incur information loss. Such information loss may result in a loss of details in the reconstructed video in areas that may be rich in edges and/or textures. This may occur when the quantization parameter (QP) may be set high. For example, the level of quantization may be set high. When such a reconstructed video may be used as reference to predict future video signal, the resulting prediction residual mode, which may be the output of summer 316, may have high energy and may become more difficult to compress. To improve prediction accuracy for these areas that may be susceptible to information loss due to quantization, additional processing of the reconstructed video around the edge/texture rich areas may be performed.

[0040] Edge guided processing may be applied to an area of a reconstructed video. This may be done, for example, to improve prediction quality when the reconstructed video may be used as reference to predict future video signal. As described herein, edge guided processing techniques may be used for single layer video coding where spatial and temporal prediction maybe applied, and may be used for scalable video coding where inter layer prediction may be applied using reconstructed video signal from one or more lower layers.

[0041] FIG. 2 is a diagram illustrating an example of a block-based video decoder. The block-based video decoder of FIG. 2 may correspond to the block-based encoder of FIG. 1. Bitstream 202, which may be a video bitstream, may be unpacked and may be entropy decoded at entropy decoding 208. Coding mode and prediction information may be sent to the spatial prediction 260 (e.g., if intra coded) or temporal prediction 2.62 (e.g., if inter coded) to form a prediction block. Residual transform coefficients may be sent to inverse quantization 2.10 and inverse transform 212 to reconstruct a residual block. The prediction block and the residual block may be added together at 226 to form a reconstructed block. The reconstructed block may go through in-foop filtering, which may occur at loop filter 266, before it may be stored in reference picture store 264. Reconstructed video 220 in the reference picture store may be sent to drive a display device and/or used to predict future video blocks.

[0042] FIG. 3 is a diagram illustrating an example of a scalable video encoder. In FIG. 3, a two-layer scalable coding system with a base layer and an enhancement layer may be illustrated. The spatial resolutions between the two layers may be different. For example, spatial scalability may be applied. An enhancement layer video input may be downsampled at downsample 400 to create a base layer video input.

[0043] Base layer encoder 412 may encode a base layer video input, block by block and may generate a base layer bitstream. Base layer encoder 402 may create a base layer bitstream that may be sent to MUX 408. Base layer encoder 412 may be an HEVC encoder. Base layer encoder 412 may be the encoder described with respect to FIG. 1.

[0044] Referring again to FIG. 3, enhancement layer encoder 402 may encode the enhancement layer video input block by block and generate an enhancement layer bitstream. Enhancement layer encoder 402 may create an enhancement layer bitstream that may be sent to MUX 408. Enhancement layer encoder 401 may be an HEVC encoder. Enhancement layer encoder 402. may be the encoder described with respect to FIG. 1. [0045] Referring again to FIG. 3, to improve the coding efficiency of the scalable system, when the enhancement layer video may be encoded at enhancement layer encoder 402, signal correlation from the base layer reconstructed video may be used to improve its prediction accuracy. For example, as shown in FIG. 3, base layer reconstructed video may be processed at inter-layer prediction processing 406. One or more of the processed base lay er pictures may be inserted into enhancement layer Decoded Picture Buffer (DPB) 404 and may be used to predict the enhancement layer video input. The base layer and the enhancement layer videos may be the same video source represented in different spatial resolutions. They may correspond to via the downsampling process that may occur at downsample 400.

[0046] Inter-layer prediction (ILP) processing may be carried out by inter- layer processing and management 406. For example, an upsampling operation may be used to align the spatial resolution of the base layer reconstruction with that of the enhancement layer video. ILP processing and management may receive base layer pictures fro BL DPB 410, base layer video information from base layer encoder 412, and/or enhancement layer video information from enhancement layer encoder 402. ILP processing and management 406 may produce a base layer bitstream, an enhancement layer bitstream produced by the base and enhancement layer encoders.

[0047] Inter-layer prediction information may be produced by ILP processing and management 406. For example, the ILP information may include the type of inter-layer processing that may be applied, the parameters that may be used in the processing, the upsampling filters that may be used, one or more processed base layer pictures that may be inserted into enhancement layer DPB 404, or the like. MUX 402. may multiplex the base layer bitstream, the enhancement layer bitstream, and the ILP information together to form a bitstream. The bitstream may be S-HEVC.

[0048] During encoding, prediction residual of the base layer blocks may be transformed and may be quantized. Quantization may incur information loss. Such information loss may- result in a loss of details in the reconstructed video in areas that may be rich in edges and/or textures. This may occur when the quantization parameter (QP) may be set high (i.e. the level of quantization may be set high). When such a reconstructed video may be used as reference to predict future video signal, the resulting prediction residual mode may have high energy and may become more difficult to compress. To improve prediction accuracy for these areas that may be susceptible to information loss due to quantization, additional processing of the reconstructed video around the edge/texture rich areas may be performed. [0049] Processing a reconstructed video signal in edge and texture rich areas may also be used to improve coding efficiency of a scalable video coding system. In FIG. 3, the base layer and the enhancement layer videos may be the same video source and may be represented in different spatial resolutions; for example, they may coiTespond via the downsampling process. ILP processing may be carried out by the inter-layer processing and management unit in FIG. 3, which may perform the upsampling operation that may be used to align the spatial resolution of the base layer reconstruction with that of the enhancement layer video. Upsampling filters may have low pass filter characteristics, and may introduce further blurring to the upsampled base layer pictures (this blurring may be in addition to the loss of detail that may be introduced due to quantization during coding of base layer video). Improving the signal quality around edge rich areas in the inter-layer reference pictures may improve inter layer prediction in a scalable video coding system.

[0050] Edge guided processing may be applied to an area of a reconstructed video. This may be done, for example, to improve prediction quality when the reconstructed video may be used as reference to predict future video signal. As described herein, edge guided processing techniques may be used for single layer video coding where spatial and temporal prediction may be applied, and may be used for scalable video coding where inter layer prediction may be applied using reconstiucted video signal from one or more lower layers.

[0051] FIG. 4 is a diagram illustrating an example of a 2-layer scalable video decoder.

The 2-layer scalable video decoder of FIG. 4 may correspond to the scalable encoder in FIG. 3. The decoder may perform operations that may be in re v erse of the encoder. DEMUX 502 may de-multiplex a scalable bitstream into a base layer bitstream, an enhancement layer bitstream, and ILP information. Base layer decoder 506 may decode the base layer bitstream. Base layer decoder 506 may produce a base layer reconstruction.

[0052] Enhancement layer decoder 504 may decode the enhancement layer bitstream, for example, with a combination of temporal reference pictures and inter-layer reference pictures (e.g., the processed base layer pictures) to reconstruct an enhancement layer video. As used herein, the terms "inter layer reference picture" and "processed base layer pictures" may be used interchangeably.

[0053] ILP processing and management 508 may receive the ILP information. ILP processing and management 508 may process the base layer reconstruction. This may be done, for example, in accordance with received ILP information. ILP processing and management 508may selectively insert one or more of the processed base layer pictures from base layer DPB into enhancement layer DPB 512. This may be done, for example, in accordance with the received 1LP information.

[0054] Edge guided processing may be applied to an area of a reconstructed video. This may be done, for example, to improve prediction quality when the reconstructed video may be used as reference to predict future video signal. As described herein, edge guided processing techniques may be used for single layer video coding where spatial and temporal prediction may^¬ be applied, and may be used for scalable video coding where inter layer prediction may be applied using reconstructed video signal from one or more lower layers.

[0055] FIG. 5 is a diagram illustrating edge guided processing that may occur at an encoder. As shown in FIG. 5 at 602, edge detection may be performed. A reconstructed video may be received. Edge detection may be applied to the reconstructed video to detect the locations of the pixels thai may represent an edge and/or a textured area. This may be done, for example, to recover a loss of high frequency information in a video reconstruction. An edge detection algorithm, such as the Canny edge detection algorithm, or the like, may be used. The edge detection algorithm may output an edge map, such as a binary map, that may indicate which pixels may be edge pixels. For an edge pixel *¾?/, an edge detection algorithm may output a pair of gradient values G_x(pel) and G_v(pel) that may indicate the edge strengths in both directions, and may output an angle 6 (pel) = tan ^{" !} j— - j tb at may indicate the direction of the edee.

Gx(pe!)

[0056] At 604, edge refinement may be performed. Edge refinement may refine a group of edge pixels. Edge refinement may be used to control the number of edge pixels that may be processed, which may correspond to computational complexity. An edge pixel may be refined based on the value of an edge gradient. For example, an edge pixel may be discarded when it is less than a threshold.

[0057] Gradient values, such as gradient values G_x(pel) and G_y(pel), may be used to remove the weak edges. This may be done, for example, by comparing the gradient values against a threshold. For example, the pixel pel may be removed from a group of edge pixels when the following condition may be true:

\Gx(pei) \ ÷ \ Gy(pel) \ < threshold (1 )

[0058] Parameters that may be used for edge refinement may be sent to a decoder. For example, gradient values such as GJpel) and G_y(pel), a threshold, or the like may be sent to a decoder. This may be done, for example, to allow the decoder to refine edge pixels during decoding. At 612, the parameters that may be used for edge refinement may be sent to a decoder by encoding the parameters in a bitstream, such as a video bitstream. [0059] At 606, edge classification may be performed. Classification may be applied to edge pixels, such as edge pixels that may be output by edge refinement. Different classification methods may be applied to pictures/areas of with different characteristics. Classification may be applied to divide edge pixels into subsets of edge pixels that may share similar characteristics.

[0060] Edge classification may classify edge pixels with similar edge directions. For example, an edge pixel may be classified with other edge pixels that have similar values of θ (ρεΓ), where Θ is a direction angle of an edge that the pixel pel, may belong to.

[0061] FIG, 6 is a diagram illustrating edge classification that may be based on edge directions. The eight directions shown in FIG. 6 may be used to classify the edges into eight direction groups. For example, an edge may be classified in edge classification 620, 625, 630, 635, 640, 645, 650, or 655. To classify the edge, the edge may be compared to the edge classifications and the edge classification that more closely matched the edge may be selected. For example, an edge that may closely match edge classification 625 may be classified in edge classification 625.

[0062] Referring again to FIG. 5, an edge pixel may be grouped or classified with edge pixels that may have similar second order derivatives along a normal direction, such as perpendicular to edge direction. For example, a second order derivative along a normal direction, such as (O (pei) + n/2), may be calculated for an edge pixel. If the edge pixel has a negaiive second order derivative, then the pixel may be classified into a first group. This may occur, for example, when a pixel intensify value may increase and then decrease in the normal direction. If the edge has positive second order derivative, then the edge pixel may be classified into a second group. This may occur, for example, when a pixel intensity value may decrease and then increase along the normal direction.

[0063] Edge classification may use a number of edge classification methods in combination. For example, edge direction and a second order derivative in the normal direction may be combined to classify edge pixels into a number of groups, such as 16 groups. This may allow edge pixels to be grouped with other edge pixels that may share the same characterisiics. Pixels that may be grouped together may be processed in a similar manner.

[0064] At 608, a filter may be trained for an edge pixel class, A bitstream, which may be the original video bitstream may be received. A filter may be determined for an edge class. A filter may be applied to an edge class to generate output that may be compared to the original video bitstream. In comparing the output to the original video bit stream, a margin of error may^¬ be determined and parameters for the filter may be determined to minimize the margin of error. The parameters may optimize the filter so reduce errors. The filter parameters may be sent to a decoder. For example, the filter parameters may be encoded in a bitstream, such as a video bitstream, that may be sent to the decoder.

[0065] A filter may be determined for an edge class by determining a filter type that may- best match the characteristics of the edge class. For example, it may be determined than edge class may have a loss of details. A linear filter may be selected and may be applied on the edge pixels of the edge class to enhance lost details.

[0066] To train the filter coefficients or parameters of a filter, such as a linear filter, an error analysis may be performed. For example, the least square method or the Gaussian-Newton regression method may be applied to minimize the 1 norm of error between an output of the filter and a bitstream, which may be the original video bitstream.

[0067] Filters, such as high pass filters may be selected and/or trained, as these filters may enhance the high frequency information lost during quantization. To improve filter stability and to reduce computational complexity, the linear filters may have one parameter, which may be an additive filter offset. The additive filter offset may be similar to the sample adaptive offset (SAO), which may be supported in HEVC. As described herein, filters may reduce the signaling cost of a filter coefficient.

[0068] At 610, a trained filter may be applied on edge pixels. The trained filter may be applied on edge pixels, for example, to generate an improved video bitstream.

[0069] When filter parameter, such as a linear filter parameters, may have been obtained for each class C, each of the edge pixels, which may be denoted as pel, pel E C, may be fi ltered using the following equation:

val'ipel) ~ f^c · val(N(pel)) + off^' (2) where /^{' L} may be the trained filter for class C, N (pel) may be the neighboring pixels of pel involved in the filtering operation, f ^■ val(N(pel)) may be the filtering operation between the filter f^c and the neighboring pixels N(pel), off^c may be the additive filter offset for class C, and val(-) and val '(') may be the pixel values before and after edge guided filtering. If a filter with an additive filter offset may be used, then the equation (2) may be simplified to the following equation:

val'ipel) = vol ⁽pel) + off^c (3)

[0070] Filter parameters may be sent to a decoder at 612. As shown in FIG. 5, filter parameters may be derived at the encoder side and may be signaled as part of a bitstream to the decoder. This may be done, for example, so that the same filters may be applied to keep the encoder and decoder in sync. Signaling of the filter parameters may incur overhead, which may depend on a number of factors, such as the number of edge pixel classes, the filter tap lengths for a filter in a class, or the like. Disclosed herein are methods and apparatus that may be used to reduce the signaling overhead.

[0071] As different filters may be applied to different classes, the number of classes may affect the amount of signaling overhead. For example, edge pixels may have been classified into many classes, which may increase the signaling cost for a filter coefficient, and may lead to potentially unsta ble trained filters as the number of training pixels in a cla ss decreases.

[0072] Edge classification may be based on the edge direction and may be based on a second order derivati ve in the normal direction. The number of classes may affect the amount of signaling. For example, then there may be 16 filters that may be derived and signaled when there may be eight edge directions and two signs (for example, positive and negative) of a second order derivative.

[0073] Other classification methods may be applied. For example, a reduced number of edge directions may be used. As another example, the number of filters may depend on the second order deri vative along the normal direction, but may not depend on the edge direction. This may reduce the number of filters to be signaled to two. Edge directions may be computed, but may be used to determine the normal directions of the edges such that second order directive along the normal direction may be calculated.

[0074] The following pseudo code may be used to signal the collection of filters and may be used to assign the filters an edge pixel class. A collection of filters, which may be a total of num of filters, may be signaled. For an edge class, a filter idx (filter idx may take a value betwee 0 and num_of_fi.iters- 1 , inclusively) may be signaled to indicate which filter may be assigned to it.

Signal num of filters

for ( m = 0; m < num of filters;

m++ )

i

Signal the m-th filter

i

Signal num_of_ciasses

for ( n = 0; n < num of classes:

n++ )

{

Signal the n-th filter idx

1

/

[0075] Filter characteristics such as filter tap length and the precision of its coefficients may affect the signaling overhead. As described herein, within a set of num_of_filters, one or more of the filters may be simplified to include an additive offset. One coefficient per filter may be signaled. One or more of the filters may have shorter tap length. For example, for an edge with edge direction Θ(ρε1), a one-dimensional filter in the normal direction of θ(ρεί.) 4- π/2 may be applied. An example is shown in FIG. 7.

[0076] FIG, 7 is a diagram illustrating an example of a 1-d filter. As shown in FIG. 7, an edge pixel pel, at 702, may represent horizontal edge 704, such as 6(pel) = 0. To reduce filter size, a 1-d filter that may use neighboring pixels, such as neighboring pixel 706, in the vertical direction may be applied.

[0077] Referring again to FIG. 5, an encoder may decide the number of classes and may decide what filters may be used for the classes based on characteristics of the input video signal. For example, the encoder may make such decisions based on the rate distortion cost, and may choose a set of classification and filtering parameters with a low rate distortion cost,

[0078] Additional parameter signaling may be provided. The number of edge pixels found after the edge detection and edge refinement may be related to computational complexity of the edge guided processing method. For example, if more pixels may be detected as edge pixels, then filtering operations may be applied on more pixels, which may increase the computational complexity. Parameters that may be used in edge refinement, such as the threshold in equation (1), may be used to control the number of edge pixels. Different threshold values may be applied to different edge classes. As shown at 612 , these thresholds may be signaled as part of the bitstream, or these thresholds may be fixed at the encoder and the decoder and may not be signal. Syntax elements that may be used to signal parameters may be signaled as part of the slice header or may be placed in a separate AL unit such as the Adaptation Parameter Set (APS).

[0079] FIG. 8 is a diagram illustrating edge guided processing that may occur at a decoder. As shown in FIG. 8, the decoder may use edge detection to find the location of an edge pixel. The decoder may use edge refmement and may use edge classification to classify an edge pixel into a group. The decoder may use edge detection, edge refinement, and edge classification in a similar manner as described herein with respect to an encoder. The decoder may imply respective filters to the edge pixels according to the classes that they belong to, for example, for each pixel pel, pel £ C, the decoder may apply linear filtering as in equation (2), or may apply filtering with an additive offset as in equation (3).

[0080] At 802, edge detection may be performed. A reconstructed video may be received. Edge detection may be applied to the reconstructed video to detect the locations of the pixels that may represent an edge and/or a textured area. This may be done, for example, to recover a loss of high frequency information in a video reconstruction. An edge detection algorithm, such as the Canny edge detection algorithm, or the like, may be used. The edge detection algorithm may output an edge map, such as a binary map, that may indicate which pixels may be edge pixels. For an edge pixel pel, an edge detection algorithm may output a pair of gradieni values G (pel) and G_y(pel) that may indicate the edge strengths in boih directions, and may output an angle Θ (ρβί) = tan^-1 j ^S^l | that mav indicate the direction of the edge.

Gx(pei)

[0081] At 804, edge refinement may be performed. Edge refinement may refine a group of edge pixels. Edge refinement may be used to control the number of edge pixels that may be processed, which may correspond to computational complexity. An edge pixel may be refined based on the value of an edge gradient. For example, an edge pixel may be discarded when it is less than a threshold.

[0082] Gradient values, such as gradieni values G_x(pel) and G_y(pel), may be used to remove the weak edges. This may be done, for example, by comparing the gradient values against a threshold. An equation, such as equation (1) may be used.

[0083] Parameters that may be used for edge refinement may be received from an encoder. For example, gradient values such as G_x(pel) and G_Y(pel), a threshold, or the like may be received from an encoder. This may be done, for example, to allow the decoder to refine edge pixels during decoding. The parameters that may be used for edge refinement may be received in a bitstream, such as a video bitstream,

[0084] At 806, edge classification may be performed. Classification may be applied to edge pixels, such as edge pixels that may be output by edge refinement. Different classification methods may be applied to pictures/areas of with different characteristics. Classification may be applied to divide edge pixels into subsets of edge pixels that may share similar characteristics.

[0085] Edge classification may classify edge pixels with similar edge directions. For example, an edge pixel may be classified with other edge pixels that have similar values of * (pel), where θ is a direction angle of an edge that the pixeLpeZ, may belong to,

[0086] An edge pixel may be grouped or classified with edge pixels that may have similar second order derivatives along a normal direction, such as perpendicular to edge direction. For example, a second order derivative along a normal direction, such as (0 (pei) + /2 ), may be calculated for an edge pixel. If the edge pixel has a negative second order derivative, then the pixel may be classified into a first group. This may occur, for example, when a pixel intensity value may increase and then decrease in the normal direction. If the edge has positive second order derivative, then the edge pixel may be classified into a second group. This may occur, for example, when a pixel intensity value may decrease and then increase along the normal direction. [0087] Edge classification may use a number of edge classification methods in combination. For example, edge direction and a second order derivative in the normal direction may be combined to classif edge pixels into a number of groups, such as 16 groups. This may allow edge pixels to be grouped with other edge pixels that may share the same characteristics. Pixels that may be grouped together may be processed in a similar manner.

[0088] At 808, a trained filter may be applied on edge pixels. A filter may be determined for an edge class by determining a filter type that may best match the characteristics of the edge class. For example, it may be determined than edge class may have a loss of details. A linear filter may be selected and may be applied on the edge pixels of the edge class to enhance lost details.

[0089] Filter parameters may be used to train a filter for a filter class. For example, the parameters may optimize the filter so reduce errors. The filter parameters may be received from an encoder. For example, the filter parameters may be encoded in a bitstream, such as a video bitstream, that may be received from the encoder. The trained filter may be applied on edge pixels, for example, to generate an improved video bitstream.

[0090] FIG, 9 is a diagram illustrating an error map between an upsampled picture and an original picture. For example, FIG. 9 may show how errors between the upsampled picture and the original picture may be distributed. Pictures 910 and 915 may be original pictures. Images 920 and 925 may be error maps that may correspond between the original and the upsampled pictures. For example, image 920 may be an error map that may correspond between picture 910 and an upsampled picture. As another example, image 925 may be an error map that may correspond between picture 915 and an upsampled picture. In an error map, such as image 920 and 925, the pixel errors may be found at 928, 930, 940, and 942. Edge locations may be found at 926, 944, and 946, Locations w^rhere edges may be collocated with errors may be found at 932, 934, 936, and A large percentage of edge and error locations may collocate. The edge pixels may have more information in frequency domain and it may be difficult to restore them using an upsampling method.

[0091] Edge guided processing may be applied to an area of a reconstructed video. This may be done, for example, to improve prediction quality when the reconstructed video may be used as reference to predict future video signal. As described herein, edge guided processing technique may be used for single layer video coding where spatial and temporal prediction may be applied, and may be used for scalable video coding where inter layer prediction may be applied using reconstructed video signal from one or more lower layers. [0092] Edge guided processing may be applied in single layer and scalable systems. As described herein, the embodiments may be applied to single layer video coding systems, such as in FIG. 1 and FIG. 2, and may be applied to scalable video coding systems, such as in FIG. 3 and FIG. 4. In a single layer coding system, this operation may be applied as part of the loop filter operation, which may be block 366 in FIG. l and may be block 266 in FIG. 2. In this case, the original video that may be used to train the filters in FIG. 5 may be the video input 302 in FIG. 1. When applied to a scalable video coding system, the operations may be applied as part of the ILP processing and management unit in FTG. 3 and FIG. 4. In this case, the original video that may be used to train the filters in FIG. 5 may be the enhancement layer video input in FIG. 3. In a scalable system, other processing, such as upsampling of the base layer reconstruction, may be performed and may be followed by edge-guided processing to further improve the inter layer reference pictures.

[0093] FIG. 1 OA is a diagram of an example communications system 100 in which one or more disclosed embodiments may be implemented. The communications system 100 may be a multiple access system that provides content, such as voice, data, video, messaging, broadcast, etc., to multiple wireless users. The communications system 100 may enable multiple wireless users to access such content through the sharing of system resources, including wireless bandwidth. For example, the communications systems 100 may employ one or more channel access methods, such as code division multiple access (CDMA), time division multiple access (T^'DMA), frequency division multiple access (FDMA), orthogonal FDMA (OFDMA), single- carrier FDMA (SC-FDMA), and the like.

[0094] As shown in FIG. 100 A, the communications system 100 may include wireless transmit/receive units (WTRUs) 102a, 102b, 102c, and/or 102d (which generally or collectively may be referred to as WTRU 102), a radio access network (RAN) 103/104/105, a core network 106/107/109, a public switched telephone network (PSTN) 108, the Internet 1 10, and other networks 1 12, though it will be appreciated that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and/or network elements. Each of the WTRUs 102a, 102b, 102c, 102d may be any type of device configured to operate and/or communicate in a wireless environment. By way of example, the WTRUs 102a, 102 b, 102c, 102d may be configured to transmit and/or receive wireless signals and may include user equipment (UE), a mobile station, a fixed or mobile subscriber unit, a pager, a cellular telephone, a personal digital assistant (PDA), a smartphone, a laptop, a netbook, a personal computer, a wireless sensor, consumer electronics, and the like. [0095] The communications systems 100 may also include a base station 1 14a and a base station 1 14b. Each of the base siaiions 1 14a, 114b may be any type of device configured to wirelessiy interface with at least one of the WTRUs 102a, 102b, 102c, 102d to facilitate access to one or more communication networks, such as the core network 106/107/109, the Internet 1 10, and/or the networks 112. By way of example, the base stations 1 14a, 1 14b may be a base transceiver station (BTS), a Node-B, an eNode B, a Home Node B, a Home eNode B, a site controller, an access point (AP), a wireless router, and the like. While the base stations 1 14a, 1 14b are each depicted as a single element, it will be appreciated that the base stations 1 14a, 1 14b may include any number of interconnected base stations and/or network elements.

[0096] The base station 114a may be part of the RAN 103/104/105, which may also include other base stations and/or network elements (not shown), such as a base station controller (BSC), a radio network controller (RNC), relay nodes, etc. The base station 1 14a and/or the base station 1 14b may be configured to transmit and/or receive wireless signals within a particular geographic region, which may be referred to as a cell (not shown). The ceil may further be divided into cell sectors. For example, the cell associated with the base station 1 14a may be divided into three sectors. Thus, in one embodiment, the base station 1 14a may include three transceivers, i.e., one for each sector of the cell. In another embodiment, the base station 1 14a may employ multiple-input multiple output (MIMO) technology and, therefore, may utilize multiple transceivers for each sector of the cell.

[0097] The base stations 114a, 1 14b may communicate with one or more of the WTRUs

102a, 102b, 102c, 102d over an air interface 1 15/1 16/1 17, which may be any suitable wireless communication Jink (e.g., radio frequency (RF), microwave, infrared (IR), ultraviolet (UV), visible light, etc.). The air interface 1 15/1 16/1 17 may be established using any suitable radio access technology (RAT).

[0098] More specifically, as noted above, the communications system 100 may be a multiple access system and may employ one or more channel access schemes, such as CDMA, TDMA, FDMA, OFDMA, 8C-FDMA, and the like. For example, the base station 1 14a in the RAN 103/104/105 and the WTRUs 102a, 102b, 102c may implement a radio technology such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which may establish the air interface 1 15/1 16/1 17 using wideband CDMA (WCDMA).

WCDMA may include communication protocols such as High-Speed Packet Access (HSPA) and/or Evolved HSPA (HSPA+). HSPA may include High-Speed Downlink Packet Access (HSDPA) and/or High-Speed Uplink Packet Access (FISUPA). [0099] In another embodiment, the base station 1 14a and the WTRUs 102a, 102b, 102c may implement a radio technology such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which may establish the air interface 1 15/1 16/117 using Long Term Evolution (LTE) and/or LTE- Advanced (LTE-A).

[0100] In other embodiments, the base station 1 14a and the WTRUs 102a, 102b, 102c may implement radio technologies such as IEEE 802.16 (i.e.. Worldwide Interoperability for Microwave Access (WiMAX)), CDMA2000, CDMA2000 IX, CDMA2000 EV-DO, Interim Standard 2000 (18-2000), Interim Standard 95 (IS-95), Interim Standard 856 (IS -856), Global System for Mobile communications (GSM), Enhanced Data rates for GSM Evolution (EDGE), GSM EDGE (GERAN), and the like.

[0101] The base station 1 14b in FIG. 100A may be a wireless router, Home Node B,

Home eNode B, or access point, for example, and may utilize any suitable RAT for facilitating wireless connectivity in a localized area, such as a place of business, a home, a vehicle, a campus, and the like. In one embodiment, the base station 1 14b and the WTRUs 102c, 102d may implement a radio technology such as IEEE 802.1 1 to establish a wireless local area network (WLAN). In another embodiment, the base station 1 14b and the WTRUs 102c, 102d may implement a radio technology such as IEEE 802.15 to establish a wireless personal area network (WPAN). In yet another embodiment, the base station 1 14b and the WTRUs 102c, I02d may utilize a cellular-based RAT (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, etc) to establish a picocell or femtocell. As shown in FIG. I OA, the base station 1 14b may have a direct connection to the Internet 1 10. Thus, the base station 1 14b may not be required to access the Internet 1 10 via the core network 106/107/109.

[0102] The RAN 103/104/105 may be in communication with the core network

106/107/109, which may be any type of network configured to provide voice, data, applications, and/or voice over internet protocol (VoIP) services to one or more of the WTRUs 102a, 102b, 102c, 102d. For example, the core network 106/107/109 may provide call control, billing services, mobile location-based services, pre-paid calling, Internet connectivity, video distribution, etc., and/or perform high-level security functions, such as user authentication. Although not shown in FIG. 1 OA, it will be appreciated that the RAN 103/104/105 and/or the core network 106/107/109 may be in direct or indirect communication with other RANs that employ the same RAT as the RAN 103/104/105 or a different RAT. For example, in addition to being connected to the RAN 103/104/105, which may be utilizing an E-UTRA radio technology, the core network 106/107/109 may also be in communication with another RAN (not shown) employing a GSM radio technology. [0103] The core network 106/107/109 may also serve as a gateway for the WTRUs 102a,

102b, 102c, l()2d to access the PSTN 108, the Internet 1 10, and/or other networks 1 12. The PSTN 108 may include circuit-switched telephone networks that provide plain old telephone service (POTS). The Interact 1 10 may include a global system of interconnected computer networks and devices that use common communication protocols, such as the transmission control protocol (TCP), user datagram protocol (UDP) and the internet protocol (IP) in the TCP/IP internet protocol suite. The networks 1 12. may include wired or wireless

communications networks owned and/or operated by other service providers. For example, the networks 1 12 may include another core network connected to one or more RANs, which may employ the same RAT as the RAN 103/104/105 or a different RAT.

[0104] Some or all of the WTRUs 102a, 102b, 102c, 102d in the communications system

100 may include multi-mode capabilities, i.e., the WTRUs 102a, 102b, 102c, 102d may include multiple transceivers for communicating with different wireless networks over different wireless links. For example, the WTRU 102c shown in FIG. 10A may be configured to communicate with the base station 1 14a, which may employ a cellular-based radio technology, and with the base station 1 14b, which may employ an IEEE 802 radio technology.

[0105] FIG. 10B is a system diagram of an example WTRU 102. As shown in FIG. 10B, the WTRU 102 may include a processor 1 18, a transceiver 120, a transmit/receive element 122, a speaker/microphone 124, a keypad 126, a display/touclipad 128, non-removable memory 130, removable memory 132, a power source 134, a global positioning system (GPS) chipset 136, and other peripherals 138. It will be appreciated that the WTRU 102 may include any subcombination of the foregoing elements while remaining consistent with an embodiment. Also, embodiments contemplate that the base stations 1 14a and 1 14b, and/or the nodes that base stations 1 14a and 1 14b may represent, such as but not limited to transceiver station (BTS), a Node-B, a site controller, an access point (AP), a home node-B, an evolved home node-B (eNodeB), a home evolved node-B (HeNB), a home evolved node-B gateway, and proxy nodes, among others, may include some or all of the elements depicted in FIG. 10B and described herein.

[0106] The processor 11 8 may be a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of

microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, Application Specific Integrated Circuits (ASICs), Field Programmable Gate Array (FPGAs) circuits, any other type of integrated circuit (IC), a state machine, and the like. The processor 1 18 may perform signal coding, data processing, power control, input/output processing, and/or any other functionality that enables the WTRU 102 to operate in a wireless environment. The processor 1 18 may be coupled to the transceiver 120, which may be coupled to the transmit/receive element 122. While FIG. 10B depicts the processor 1 18 and the transceiver 120 as separate components, it will be appreciated that the processor 1 18 and the transceiver 120 may be integrated together in an electronic package or chip.

[0107] The transmit/receive element 122 may be configured to transmit signals to, or receive signals from, a base station (e.g., the base station 1 14a) over the air interface

1 15/1 16/1 17. For example, in one embodiment, the transmit/receive element 122 may be an antenna configured to transmit and/or receive RF signals. In another embodiment, the transmit/receive element 122 may be an emitter/detector configured to transmit and/or receive IR, UV, or visible light signals, for example. In yet another embodiment, the transmit receive element 122 may be configured to transmit and receive both RF and light signals. It will be appreciated that the transmit/receive element 122 may be configured to transmit and/or receive any combination of wireless signals.

[0108] In addition, although the transmit/receive element 122. is depicted in FIG. 10B as a single element, the WTRU 102. may include any number of transmit/receive elements 122. More specifically, the WTRU 102 may employ MIMO technology. Thus, in one embodiment, the WTRU 102 may include two or more transmit/receive elements 122 (e.g., multiple antennas) for transmitting and receiving wireless signals over the air interface 1 15/1 16/117.

[0109] The transceiver 120 may be configured to modulate the signals that are to be transmitted by the transmit/receive element 122 and to demodulate the signals that are received by the transmit/receive element 122. As noted above, the WTRU 102. may have multi-mode capabilities. Thus, the transceiver 120 may include multiple transceivers for enabling the WTRU 102 to communicate via multiple RATs, such as UTRA and IEEE 802.1 1 , for example.

[0110] The processor 1 18 of the WTRU 102 may be coupled to, and may receive user input data from, the speaker/microphone 124, the keypad 126, and'Or the displav/touchpad 128 (e.g., a liquid crystal displa (LCD) display unit or organic light-emitting diode (OLED) display unit). The processor 1 18 may also output user data to the speaker/microphone 124, the keypad 126, and/or the displ y/touchpad 128. In addition, the processor 1 1 8 may access information from, and store data in, any type of suitable memory, such as the non-removable memory 130 and/or t e removable memory 132. The non-removable memory 130 may include random- access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. The removable memory 132 may include a subscriber identity module (SIM) card, a memory stick, a secure digital (SD) memory card, and the like. In other embodiments. the processor 18 may access information from, and store data in, memory that is not physically located on the WTRU 102, such as on a server or a home computer (not shown).

[0111] The processor 118 may receive power from the power source 134, and may be configured to distribute and/or control the power to the other components in the W RU 102. The power source 134 may be any suitable device for powering the WTRU 102. For example, the power source 134 may include one or more dry cell batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, and the like.

[0112] The processor 11 8 may also be coupled to the GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) regarding the current locaiion of the WTRU 102. In addition to, or in lieu of, the information from the GPS chipset 136, the WTRU 102. may receive locaiion information over the air interface 1 15/1 16/1 17 from a base station (e.g., base stations 1 14a, 1 14b) and/or determine its location based on the timing of the signals being received from two or more nearby base stations. It will be appreciated that the WTRU 102 may acquire locaiion information by way of any suitable location- determination method while remaining consistent with an embodiment.

[0113] The processor 1 18 may further be coupled to other peripherals 138, which may include one or more software and/or hardware modules that provide additional features, functionality and/or wired or wireless connectivity. For example, the peripherals 138 may include an accelerometer, an e-compass, a satellite transceiver, a digital camera (for photographs or video), a universal serial bus (USB) port, a vibration device, a television transceiver, a hands free headset, a Bluetooth® module, a frequency modulated (FM) radio unit, a digital music player, a media player, a video game player module, an Internet browser, and the like.

[0114] FIG. IOC is a system diagram of the RAN 103 and the core network 106 according to an embodiment. As noted above, the RAN 103 may employ a UTRA radio technology to communicate with the WTRUs 102a, 102b, 102c over the air interface 1 15. The RAN 103 may also be in communication with the core network 106, As shown in FIG. IOC, the RAN 103 may include Node-Bs 140a, 140b, 140c, which may each include one or more transceivers for communicating with the WTRUs 102a, 102b, 102c over the air interface 1 15. The Node-Bs 140a, 140b, 140c may each be associated with a particular cell (not shown) within the RAN 103. The RAN 103 may also include RNCs 142a, 142b. It will be appreciated that the RAN 103 may include any number of Node-Bs and RNCs while remaining consistent with an embodiment.

- 2 Ί - [0115] As shown m FIG. I OC, the Node-Bs 140a, 140b may be in communication with the RNC 142a. Additionally, the Node-B 140c may be in communication with the RNC 142b. The Node-Bs 140a, 140b, 140c may communicate with the respective RNCs 142a, 142b via an Iub interface. The RNCs 142a, 142b may be in communication with one another via an lur interface. Each of the RNCs 142a, 142b may be configured to control the respective Node-Bs 140a, 140b, 140c to which it is connected. In addition, each of the RNCs 142a, 142b may be configured to carry out or support other functionality, such as outer loop power control, load control, admission control, packet scheduling, handover control, macrodiversity, security functions, data encryption, and the like.

[0116] The core network 106 shown in FIG. IOC may include a media gateway (MGW)

144, a mobile switching center (MSG) 146, a serving GPRS support node (SGSN) 148, and/or a gateway GPRS suppori node (GGSN) 150. While each of the foregoing elements are depicted as part of the core network 106, it will be appreciated that any one of these elements may be owned and/or operated by an entity other than the core network operator.

[0117] The RNC 1 2a in the RAN 103 may be connected to the MSG 146 in the core network 106 via an luCS interface. The MSC 146 may be connected to the MGW 144. The MSG 146 and the MGW 144 may provide the WTRUs 102a, 102b, 102c with access to circuit- switched networks, such as the PSTN 108, to facilitate communications between the WTRUs 102a, 102b, 102c and traditional land-line communications devices.

[0118] The RNC 142a in the RAN 103 may also be connected to the SGSN 148 in the core network 106 via an luPS interface. The SGS 148 may be connected to the GGSN 150. The SGSN 148 and the GGSN 150 may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 1 10, to facilitate communications between and the WTRUs 102a, 102b, 102c and IP-enabled devices.

[0119] As noted above, the core network 106 may also be connected to the networks 1 12, which may include other wired or wireless networks that are owned and'Or operated by other service providers.

[0120] FIG, 10D is a system diagram of the RAN 104 and the core network 107 according to an embodiment. As noted above, the RAN 104 may employ an E-UTRA radio technology to communicate with the WTRUs i02a, 102b, 102c over the air interface 1 16. The RAN 104 may also be in communication with the core network 107,

[0121] The RAN 104 may include eNode-Bs 160a, 160b, 160c, though it will be appreciated that the RAN 104 may include any number of eNode-Bs while remaining consistent with an embodiment. The eNode-Bs 160a, 160b, 160c may each include one or more transceivers for communicating with the WTRUs 102a, 102b, 102c over the air interface 1 16, in one embodiment, the eNode-Bs 160a, 160b, 160c may implement MIMO technology. Thus, the eNode-B 160a, for example, may use multiple antennas to transmit wireless signals to, and recei ve wireless signals from, the WTRU 102a.

[0122] Each of the eNode-Bs 160a, 160b, 160c may be associated with a particular cell

(not shown) and may be configured to handle radio resource management decisions, handover decisions, scheduling of users in the uplink and/or downlink, and the like. As shown in FIG. 10D, the eNode-Bs 160a, 160b, 160c may communicate with one another over an X2 interface.

[0123] The core network 107 shown in FIG. 10D may include a mobility management gateway (MME) 162, a serving gateway 164, and a packet data network (PDN) gateway 166. While each of the foregoing elements are depicted as part of the core network 107, it will be appreciated that any one of these elements may be owned and/or operated by an entity other than the core network operator.

[0124] The MME 162 may be connected to each of the eNode- Bs 160a, 160b, 160c in the

RAN 104 via an SI interface and may serve as a control node. For example, the MME 162. maybe responsible for authenticating users of the WTRUs 102a, 102b, 102c, bearer

activation/deactivation, selecting a particular serving gateway during an initial attach of the WTRUs 102a, 102b, 102c, and t e like. The MME 162. may also provide a control plane function for switching between the RAN 104 and other RANs (not shown) that employ other radio technologies, such as GSM or WCDMA.

[0125] The serving gateway 164 may be connected to each of the eNode-Bs 160a, 160b,

160c in the RAN 104 via the S I interface. The serving gateway 164 may generally route and forward user data packets to/from the WTRUs 102a, 102b, 102c. The serving gateway 164 may also perform other functions, such as anchoring user planes during inter-eNode B handovers, triggering paging when downlink data is available for the WTRUs 102a, 102b, 102c, managing and storing con tex ts of the WTRUs 102 a, i 02b, 102c, and the like.

[0126] The serving gateway 164 may also be connected to the PDN gateway 166, which may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 1 10, to facilitate communications between the WTRUs 102a, 102b, 102c and IP -enabled devices.

[0127] The core network 107 may facilitate communications with other networks. For example, the core network 107 may provide the WTRUs 102a, 102b, 102c with access to circuit- switched networks, such as the PSTN 108, to facilitate communications between the WTRUs 102a, 102b, 102c and traditional land-line communications devices. For example, the core network 107 may include, or may communicate with, an IP gateway (e.g., an IP multimedia subsystem (IMS) server) that serves as an interface between the core network 107 and the PSTN 108. In addition, the core network 107 may provide the WTRUs 102a, 102b, 102c with access to the networks 1 12, which may include other wired or wireless networks that are owned and/or operated by other service providers.

[0128] FIG. 10E is a system diagram of the RAN 105 and the core network 109 according to an embodiment. The RAN 105 may be an access service network (ASN) that employs IEEE 802.16 radio technology to communicate with the WTRUs 102a, 102b, 102c over the air interface 1 17. As will be further discussed below, the communication links between the different functional entities of ihe WTRU s 102a, 102b, 102c, the RAN 105, and the core network 109 may be defined as reference points.

[0129] As shown in FIG. 10E, the R AN 105 may include base stations 180a, 180b, 180c, and an ASN gateway 182, though it will be appreciated that the RAN 105 may include any number of base stations and ASN gateways while remaining consistent with an embodiment. The base stations 180a, 180b, 180c may each be associated with a particular cell (not shown) in the RAN 105 and may each include one or more transceivers for communicating with the WTRUs 102a, 102b, 102c over the air interface 1 17. in one embodiment, the base stations 180a, 180b, 180c may implement MIMO technology. Thus, the base station 180a, for example, may use multiple antennas to transmit wireless signals to, and receive wireless signals from, ihe WTRU 102a. The base stations 180a, 180b, 180c may also provide mobility management functions, such as handoff triggering, tunnel establishment, radio resource management, traffic classification, quality of service (QoS) policy enforcement, and the like. The ASN gateway 182 may serve as a traffic aggregation point and may be responsible for paging, caching of subscriber profiles, routing to the core network 109, and the like.

[0130] The air interface 1 17 between the WTRUs 102a, 102b, 102c and the RAN 105 may be defined as an Rl reference point that implements the IEEE 802.16 specification. In addition, each of the WTRUs 102a, 102b, 102c may establish a logical interface (not shown) with the core network 109. The logical interface between the WTRUs 102a, 102b, 102c and the core network 109 may be defined as an R2 reference point, which may be used for

authentication, authorization, IP host configuration management, and/or mobility management.

[0131] The communication link between each of the base stations 180a, 180b, 180c may be defined as an R8 reference point that includes protocols for facilitating WTRXJ handovers and the transfer of data between base stations. The communication link between the base stations 180a, 180b, 180c and the ASN gateway 182 may be defined as an R6 reference point. The R6 reference point may include protocols for facilitating mobility management based on mobility events associated with each of the WTRUs 102a, 102b, 102c.

[0132] As shown in FIG, 10E, the RAN 105 may be connected to the core network 109.

The communication link between the RAN 105 and the core network 109 may defined as an R3 reference point that includes protocols for facilitating data transfer and mobility management capabilities, for example. The core network 109 may include a mobile IP home agent (MIP-HA) 184, an authentication, authorization, accounting (AAA) server 186, and a gateway 188. While each of the foregoing elements are depicted as part of the core network 109, it will be appreciated that any one of these elements may be owned and/or operated by an entity other than the core network operator.

[0133] The MIP-HA may be responsible for IP address management, and may enable the

WTRUs 102a, 102b, 102c to roam between different ASNs and/or different core networks. The MIP-HA 184 may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 1 10, to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices. The AAA server 186 may be responsible for user authentication and for supporting user services. The gateway 188 may facilitate interworking with other networks. For example, the gateway 188 may provide the WTRUs 102a, 102b, 102c with access to circuit-switched networks, such as the PSTN 108, to facilitate communications between the WTRUs 102a, 102b, 102c and traditional land-line communications devices. In addition, the gateway 188 may provide the WTRUs 102a, 102b, 102c with access to the networks 1 12, which may include other wired or wireless networks that are owned and/or operated by other service providers.

[0134] Although not shown in FIG, 10E, it will be appreciated that the RAN 105 may be connected to other ASNs and the core network 109 may be connected to other core networks. The communication link between the RA 105 the other ASNs may be defined as an R4 reference point, which may include protocols for coordinating the mobility of the WTRUs 102a, 102b, 102c between the RAN 105 and the other ASNs. The communication link between the core network 109 and the other core networks may be defined as an R5 reference, which may include protocols for facilitating interworking between home core networks and visited core networks.

[0135] Although features and elements are described above in particular combinations, one of ordinary skill in the art will appreciate that each feature or element can be used alone or in any combination with the other features and elements. In addition, the methods described herein may be implemented in a computer program, software, or firmware incorporated in a computer- readable medium for execution by a computer or processor. Examples of computer-readable media include electronic signals (transmitted over wired or wireless connections) and computer- readable storage media. Examples of computer-readable storage media include, but are not limited to, a read only memor (ROM), a random access memor (RAM), a register, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks, and digital versatile disks (DVDs). A processor in association with software may be used to implement a radio frequency transceiver for use in a WTRU, UE, terminal, base station, RNC, or any host computer.

Claims

CLAIMS What is claimed:

1. An apparatus comprising:

a processor configured to:

detect a pixel within an area of an image, the pixel being part of an edge;

classify the pixel in an edge pixel class using a characteristic of the pixel;

determine a filter for the edge pixel class using the characteristic of the pixel, wherem the filter prevents a detail loss to the area of the image while video coding; and apply the filter on the pixel.

2. The apparatus of claim 1, wherein the processor is further configured to:

determine an edge strength in an edge direction for the pixel; and

determine that the pixel does not belong to a weak edge using the edge strength when the edge strength is above a threshold.

3. The apparatus of claim 1, wherem the processor is configured to classify the pixel in the edge pixel class using the characteristic for the pixel by:

determining an edge direction for the pixel;

determining a normal value for the pixel indicating the value of a second order derivaiive that is perpendicular to the edge direction; and

classifying the pixel in an edge pixel class using the normal value and the edge direction.

4. The apparatus of claim 1, wherein the processor is configured to classify the pixel in the edge pixel class using the characteristic for the pixel by: determining a gradient value G_x indicating an edge strength i a fsrst edge direction for the pixel;

determining a gradient value Gy indicating an edge strength in a second edge direction for the pixel; and

determining an edge direction according to the equation:

Gy

Θ = tan^"1 j— j. where Θ is the edge direction.

¹ Gx ^&

5. The apparatus of claim 4, wherem the processor is further configured to classify the pixel in an edge pixel class using the characteristics of the pixel by:

determining a second order derivative for the pixel according to the equation:

sd = θ + where sd is the second order derivative: and classifying the pixel in the edge pixel class using the second order derivative and the edge direction.

6. The apparatus of claim 1, wherem the processor is further configured to determine a parameter for the filter to optimize the filter for the edge pixel class.

7. The apparatus of claim 6, wherein the processor is further configured to send a parameter for the filter to a decoder.

8. The apparatus of claim 1, wherem the processor is further configured to determine a parameter for the filter to by performing a regression analysis to minimize creating an error in the area of the image when the filter is applied on the pixel.

9. The apparatus of claim 8, wherem the regression analysis is a least mean of square errors analysis.

10. The apparatus of claim 1, wherein the image if a first image and the processor is further configured to:

generate a second image using the filter for the edge pixel class;

determine an error for the edge pixel class by comparing the second image to the first image; and

determine a parameter for the filter to minimize the error for the edge pixel class.

1 1. The apparatus of claim 1, wherein the processor is further configured to determine a parameter for the filter to enhance high frequency infonnation that can be lost during video coding.

12. The apparatus of claim 1, wherein the processor is further configured to receive a parameter for the filter to optimize the filter for the edge pixel class.

13. The apparatus of claim 1, wherein the processor is configured to apply the filter on the pixel by:

generating a filtered pixel according to the equation:

val^! = f^c - val(N) + off^c

where val' (pel)h the filtered pixel, f - vai(N) is a filtering operating for a neighboring pixel, and off^c is an offset for the edge pixel class.

14. The apparatus of claim 6, wherein the processor is configured to apply the filter on the pixel by:

optimizing the filter by applying the parameter to the filter; and

generating a filtered pixel using the optimized filter.

15. A method for edge guided processing for video coding, the method comprising: detecting a pixel in an area of an image, the pixel being part of an edge;

classifying the pixel in an edge pixel class using a characteristic of the pixel;

determining a filter for the edge pixel class using the characteristic of the pixel, wherein the filter prevents a detail loss in the area of the image while video coding; and

applying the filter on the pixel,

16. The method of claim 15, further comprising:

determining an edge strength in the edge direction for the pixel; and

determining that the pixel does not belong to a weak edge using the edge strength when the edge strength is above a threshold.

17. The method of claim 15, wherein classifying the pixel in the edge pixel class using ihe characteristic for the pixel comprises:

determining an edge direction for the pixel;

determining a normal value for the pixel indicating the value of a second order derivative that is perpendicular to the edge direction; and

classifying the pixel in an edge pixel class using the normal value and the edge direction.

1 8. The method of claim 15, wherein classifying the pixel in an edge pixel class using the characteristic of the pixel:

determining a gradient value G, indicating an edge strength in a first edge direction for the pixel;

determining a gradient value Gy indicating an edge strength in a second edge direction for the pixel; and determining an edge direction according to the equation:

8 = tan^~A j ~ |, where Θ is the edge direction.

19. The method of claim i 8, wherein classifying the pixel in an edge pixel class using the characteristic of the pixel further comprises:

determining a second order derivative for the pixel according to the equation:

sd = Θ + -, where sd is the second order derivative; and

2 ' '

classifying the pixel in the edge pixel class using the second order derivative and the edge direction.

20. The method of claim 15, further comprising determining a parameter for the filter to optimize the filter for the edge pixel class.

21. The method of claim 20, further comprising sending the parameter for the filter to a decoder.

22. The method of claim 15, further comprising determining a parameter for the filter to by performing a regression analysis to minimize creating an error in the area of the image when the filter is applied on the pixel.

23. The method of claim 22, wherein the regression analysis is a least mean of square errors analysis.

24. The method of claim 15, wherein the image is a first image and the method further comprises: generating a second image using the filter for the edge pixel class;

determining an error for the edge pixel class by comparing the second image to the first image; and

determining a parameter for the filter to minimize the error for the edge pixel class,

25. The method of claim 15, further comprising determining a parameter for the filter to enhance high frequency information that can be lost during video coding.

26. The method of claim 15, further comprising receiving a parameter for the filter to optimize the filter for the edge pixel class.

27. The method of claim 15, wherein applying the filter on the pixel comprises: generating a filtered pixel according to the equation:

vaV = f^c■ val(N + off^c

where val'(pel~)is the filtered pixel, f^c■ val(N) is a filtering operating for a neighboring pixel, and off⁰ is an offset for the edge pixel class.

28. The method of claim 20, wherein applying the filter on the pixel comprises: optimizing the filter by applying the parameter to the filter: and

generating a filtered pixel using the optimized filter.