Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N7/00—Television systems
  • H04N7/01—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level
  • H04N7/0117—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level involving conversion of the spatial resolution of the incoming video signal
  • H04N7/012—Conversion between an interlaced and a progressive signal
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N7/00—Television systems
  • H04N7/01—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level
  • H04N7/0135—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level involving interpolation processes
  • H04N7/0142—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level involving interpolation processes the interpolation being edge adaptive

Definitions

• the embodiments of the invention relate generally to the field of video signal processing and, more specifically, relate to an angular-directed spatial deinterlacer.
• An interlaced video frame contains two fields.
• the first field consists of the odd lines in the interlaced video frame and the second field consists of the even lines in the interlaced video frame.
• the lines of the two fields are displayed in an alternating fashion. All odd lines of the interlaced video frame are displayed, and then all even lines of the interlaced video frame are displayed.
• interlaced display format is still quite common in video products.
• a deinterlacing method is applied to adapt the interlaced format.
• Current generations of video processors deinterlace video by using a variety of techniques.
• One field of prior art techniques includes intra- field processing (or spatial-only domain processing) such as bob.
• Another field of prior art techniques includes inter-field processing (or spatial and temporal domain processing) such as weave, motion adaptive, and motion compensation.
• the bob and weave techniques are typically applied due to their inherently simple complexity, but at the cost of picture quality. For instance, the bob technique determines an average of the pixels directly above and below (e.g., at 90 degree angles) a pixel being estimated for deinterlacing. This is a simple technique but can lead to reduced picture quality.
• the motion adaptive and motion compensation techniques achieve a better picture quality, but with a more expensive operation. Spatial-only deinterlacers are applied in environments that cannot access temporal reference pictures and require minimal complexity.
• Figure 1 illustrates a block diagram of one embodiment of an angular-directed spatial deinterlacer
• Figure 2 is a block diagram of one embodiment of a target pixel block and its multiple angle candidates
• Figure 3 is a conceptual diagram of one embodiment of determining cost measure for an angle candidate.
• Figure 4 is a flow diagram of one embodiment of a method for determining a local minimum angle measure
• Figure 5 is a block diagram of one embodiment of angle filtering
• Figure 6 illustrates a block diagram of one embodiment of a computer system.
• Embodiments of the invention present an angular-directed spatial deinterlacer to deinterlace interlaced pictures in an efficient and quality manner in a spatial-only domain.
• the angular-direct spatial deinterlace of embodiments of the invention provides layered techniques to select a best angle for interpolation. For instance, the angle selection is based on the local minimum angle measure in order to utilize the resemblance typically found between neighboring pixels in a frame, as well as on a global minimum angle measure to produce consistent prediction.
• a line-based measure is presented to better predict the best angle that is resilient to pixel perturbation and to handle thin line content.
• the spatial-only deinterlacer of embodiments of the invention may work as a standalone deinterlacer solution. However, one skilled in the art may appreciate that embodiments of the deinterlacer may be combined with a temporal measure (e.g., frames other than the target frame) to form a motion adaptive or motion compensated deinterlacer solution.
• the spatial-only deinterlacer of embodiments of the invention includes: (1) a mechanism to deinterlace the interlaced picture by the process of spatial-only operations of angular detection and angular filtering; (2) a mechanism to provide the spatial angular prediction by utilizing the layered technique to search the most suitable angle with the local minimum measure of the layered angle candidates; (3) a mechanism to provide the spatial angular prediction among multiple angle candidates by utilizing the measures from the local minimum of the layered angle candidates and the global minimum of all angle candidates to achieve better filtering; (4) a mechanism to provide the spatial angular prediction among multiple angle candidates by utilizing the variable size of neighboring lines to measure the angle cost; (5) a mechanism to provide the spatial angular prediction by the technique of edge detection to refine the process of angle prediction; and (6) a mechanism to provide the deinterlace filtering by the angles from the local minimum of layered angle candidates and from the global minimum of all angle candidates.
• Figure 1 is a block diagram depicting one embodiment of components for an angle-directed spatial deinterlacer.
• a video input 110 is presented to a deinterlacer 120 of graphics processor in a computer system.
• the video input is sent to a multiple angle cost measure component 121 and a horizontal angle measure component 122.
• a pixel block to be interpolated for the deinterlacing is sent as an input to the deinterlacer.
• the multiple angle cost measure component 121 applies techniques to measure multiple angle costs for the pixel block, while the horizontal angle measure component 122 performs horizontal angle measures on the pixel block.
• the results from the multiple angle cost measure component 121 are sent to both of the global minimum angle measure component 123 and the local minimum angle measure component 124.
• the results from the horizontal angle measure component 122 are sent solely to the local minimum angle measure component 124.
• the two best angle candidates for the particular pixel block are determined based on the global minimum of all angle candidates as determined by the global minimum angle measure component 123, and based on a proposed local minimum of the layered angle candidates determined by the local minimum angle measure component 124.
• the local minimum of the layered angle candidates is, at least in part, determined according to the horizontal angle measure from the horizontal angle measure component 122.
• angle filtering component 125 angular filtering by the angle filtering component 125 is applied to the two best angle candidates resulting from the global minimum angle measure component 123 and the local minimum angle measure component 124.
• the angle filtering results in a value to use for pixel interpolation for deinterlacing.
• the operations of the components 121-125 of deinterlacer 120 are further described in more detail below.
• Figure 2 depicts a location of the target pixel block to be de -interlaced and the associated multiple angle candidates (11 angles) pointing to the neighboring pixels of the opposite field.
• the eleven angles are denoted as 90 degree (90), positive 45 degree (P45), negative 45 degree (N45), positive 27 degree (P27), negative 27 degree (N27), positive 18 degree (P 18), negative 18 degree (Nl 8), positive 14 degree (P14), negative 14 degree (N14), positive 11 degree (Pl 1), and negative 11 degree (Nl 1).
• embodiments of the invention may be expanded beyond the PI l and Ni l degrees.
• embodiments of the invention are not necessarily limited to the specific degree candidates listed above and may include more or less of the same or different angles than those mentioned above.
• Embodiments of the invention utilize a line-based measure to determine angle cost measures for the target pixel block.
• This line-based measure reduces the complexity of deinterlacing. As a result, picture quality is reasonably maintained in deinterlacing.
• the angle cost measure utilized in embodiments of the invention is based on the sum of absolute difference (SAD) of multiple pairs of neighboring lines per each candidate angle.
• Figure 3 shows an example of angle cost measure for the N45 angle by utilizing three pairs of neighboring lines.
• the multiple angle cost measure is also repeated on each of the other candidate angles (other than N45) being measured from the target pixel block.
• the SAD between a first line pair 310, a second line pair 320, and a third line pair 330 is individually calculated. These individual SADs are added together for the total SAD for an angle candidate.
• SADl 310 is the pair of lines above the target pixel block at the N45 angle.
• SAD2 320 is the pair of lines directly above and below the target pixel block at the N45 angle.
• SAD3 330 is the pair of lines directly below the target pixel block at the N45 angle. It should be noted that more or less line pairs may be utilized in calculating the angle cost measure using the SAD, based on the available resources of the system.
• the line size of at least one pair is different from the other pair(s). For instance, SADl 310 and SAD3 330 utilize a 5 pixel line length, while SAD2 320 utilizes a 6 pixel line length. If all line pairs are the same length, then the cost measure would be the SAD between the entire block of pixels (including all of the line pairs). The variability of line pair length and the variability of line pair locations allows for the cost measure to be based on the SAD of each line pair(s), rather than the SAD of an entire pixel block. In addition to calculating the multiple angle cost measures, embodiments of the invention also determine a horizontal angle measurement for the target pixel block. To determine the horizontal angle measure, the well-known first-order Sobel formula is applied to estimate the edge information: -1 -2 - 1
• AM(x) I n9(x) * E_h
• the results of the horizontal angle measure are later utilized as part of the local minimum angle measure determination.
• the global minimum angle measure, G min angle is the angle candidate of the lowest angle cost measure among all of the angle candidates. For instance, with reference to Figures 2 and 3, the global minimum angle measure would be the angle candidate from Figure 2 with the lowest cost measure as determined by the SAD calculation depicted in Figure 3.
• Figure 4 depicts the hierarchical angle selection based on the local minimum of layered angle candidates. As shown, five layers 410, 420, 430, 440, 450 are illustrated. At block 405, the initial default angle is set to 90 degrees. Then, in layer 1 410, the best angle among 90, P45, and N45 is identified. Then, the cost measures associated with each potential candidate angle are subsequently compared at each of the different hierarchical layers. For instance, if the best angle is selected to be P45 or N45, the process continues to the associated branch of the next layer, layer 2 420, to test against the cost measures for P27 or N27 degrees. As illustrated, the process is divided at layer 2 420 between the negative and positive angles.
• threshold values DO, Dl, D, and Th are utilized. These values may be modified adaptively according to the value of horizontal angle measure previously determined. When the horizontal angle measure is bigger than a threshold, there is a horizontal angle, and then the default 90 degree is invalid. In such a case, the process in Layer 1 410 is skipped and instead begins at Layer 2 420 to pick up a lower value between P45 and N45. Then, the process continues normally to the search in the next layers until the local best local minimum angle is found.
• embodiments of the invention present two options of filters.
• the following equation is applied to determine the value used for pixel interpolation:
• angle filtering is based solely on the local minimum angle measure and the LJ ⁇ lter may be applied. Utilizing only the local minimum angle measure may achieve sufficient picture quality for most video signals and is also economical in terms of implementation complexity.
• Figure 6 is a block diagram of one embodiment of a computer system 600.
• computer system 600 includes the components of Figure 1 and performs their associated functions.
• graphics interface card 650 with its graphics processing unit (GPU) and graphics deinterlacer 657 may include the components of Figure 1 and perform the functions described with respect to Figures 2 through 5.
• Computer system 600 includes a central processing unit (CPU) 602 coupled to interconnect 605.
• CPU 602 is a processor in the Pentium® family of processors Pentium® IV processors available from Intel Corporation of Santa Clara, California. Alternatively, other CPUs may be used. For instance, CPU 602 may be implemented multiple processors, or multiple processor cores.
• a chipset 607 is also coupled to interconnect 605.
• Chipset 607 may include a memory control hub (MCH) 610.
• MCH 610 may include a memory controller 612 that is coupled to a main system memory 615.
• Main system memory 615 stores data and sequences of instructions that are executed by CPU 602 or any other device included in system 600.
• main system memory 615 includes one or more DIMMs incorporating dynamic random access memory (DRAM) devices; however, main system memory 615 may be implemented using other memory types. Additional devices may also be coupled to interconnect 605, such as multiple CPUs and/or multiple system memories.
• MCH 610 may be coupled to an input/output control hub (ICH) 640 via a hub interface.
• ICH 640 provides an interface to input/output (I/O) devices within computer system 600.
• ICH 640 may support standard I/O operations on I/O interconnects such as peripheral component interconnect (PCI), accelerated graphics port (AGP), universal serial interconnect (USB), low pin count (LPC) interconnect, or any other kind of I/O interconnect (not shown).
• PCI peripheral component interconnect
• AGP accelerated graphics port
• USB universal serial interconnect
• LPC low pin count interconnect
• MCH 610 is coupled to a graphics interface card 650.
• Graphics interface card 650 includes a GPU 655 and a graphics deinterlacer 657.
• system 600 may vary from implementation to implementation depending upon numerous factors, such as price constraints, performance requirements, technological improvements, and/or other circumstances.
• the embodiments described herein may be performed under the control of a programmed processor, such as CPU 602 or GPU 655, in alternative embodiments, the embodiments may be fully or partially implemented by any programmable or hard coded logic, such as field programmable gate arrays (FPGAs), transistor transistor logic (TTL) logic, or application specific integrated circuits (ASICs). Additionally, the embodiments of the invention may be performed by any combination of programmed general-purpose computer components and/or custom hardware components. Therefore, nothing disclosed herein should be construed as limiting the various embodiments of the invention to a particular embodiment wherein the recited embodiments may be performed by a specific combination of hardware components.
• FPGAs field programmable gate arrays
• TTL transistor transistor logic
• ASICs application specific integrated circuits
• the various embodiments may be performed by a combination of hardware and software.
• Various embodiments of the invention may be provided as a computer program product, which may include a machine-readable medium having stored thereon instructions, which may be used to program a computer (or other electronic devices) to perform a process according to various embodiments of the invention.
• the machine- readable medium may include, but is not limited to, floppy diskette, optical disk, compact disk-read-only memory (CD-ROM), magneto-optical disk, read-only memory (ROM) random access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic or optical card, flash memory, or another type of media/machine-readable medium suitable for storing electronic instructions.
• various embodiments of the invention may also be downloaded as a computer program product, wherein the program may be transferred from a remote computer to a requesting computer by way of data signals embodied in a carrier wave or other propagation medium via a communication link (e.g., a modem or network connection).
• a communication link e.g., a modem or network connection

Landscapes

• Engineering & Computer Science (AREA)
• Computer Graphics (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Television Systems (AREA)
• Image Processing (AREA)
• Image Analysis (AREA)

Priority Applications (3)

Applications Claiming Priority (2)

Publications (1)

Family

ID=39583345

Family Applications (1)

Country Status (6)

Families Citing this family (3)

Citations (3)

Family Cites Families (14)

• 2006
  • 2006-12-27 US US11/646,763 patent/US8295357B2/en not_active Expired - Fee Related
• 2007
  • 2007-11-28 TW TW096145180A patent/TWI400688B/zh not_active IP Right Cessation
  • 2007-12-27 JP JP2009544274A patent/JP5061199B2/ja not_active Expired - Fee Related
  • 2007-12-27 EP EP07869998A patent/EP2098070A4/en not_active Withdrawn
  • 2007-12-27 CN CN2007800484898A patent/CN101573973B/zh not_active Expired - Fee Related
  • 2007-12-27 WO PCT/US2007/088987 patent/WO2008083245A1/en not_active Ceased

Patent Citations (3)

Also Published As

Similar Documents

Legal Events