US20110037771A1 - Display Processing Line Buffers Incorporating Pipeline Overlap - Google Patents
Display Processing Line Buffers Incorporating Pipeline Overlap Download PDFInfo
- Publication number
- US20110037771A1 US20110037771A1 US12/909,248 US90924810A US2011037771A1 US 20110037771 A1 US20110037771 A1 US 20110037771A1 US 90924810 A US90924810 A US 90924810A US 2011037771 A1 US2011037771 A1 US 2011037771A1
- Authority
- US
- United States
- Prior art keywords
- pixel
- image
- line buffers
- pipeline
- antenna
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 239000000872 buffer Substances 0.000 title claims abstract description 37
- 238000000034 method Methods 0.000 claims abstract description 31
- 230000008569 process Effects 0.000 claims abstract description 25
- 230000015654 memory Effects 0.000 claims description 34
- 230000003068 static effect Effects 0.000 claims description 2
- 238000004513 sizing Methods 0.000 claims 1
- 238000004891 communication Methods 0.000 description 6
- 230000037361 pathway Effects 0.000 description 6
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000002372 labelling Methods 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 241000255925 Diptera Species 0.000 description 1
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical compound [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000000135 prohibitive effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 229910052710 silicon Inorganic materials 0.000 description 1
- 239000010703 silicon Substances 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/60—Memory management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
- G09G5/36—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
- G09G5/39—Control of the bit-mapped memory
- G09G5/393—Arrangements for updating the contents of the bit-mapped memory
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/14—Picture signal circuitry for video frequency region
- H04N5/21—Circuitry for suppressing or minimising disturbance, e.g. moiré or halo
Definitions
- Image processing for display is typically done in a raster fashion where pixels are processed serially starting from the left most pixel in the upper most row in a frame and proceeding down through the frame row by row such that all pixels in any particular row are processed together. While processing a given pixel in one row of the frame a pixel processing algorithm or “filter kernel” may need to access surrounding vertical pixels. However, because pixels are usually serially scanned in a raster fashion through a typical processing pipeline the surrounding vertical pixels may not be immediately available.
- a common solution to accessing surrounding vertical pixels is to use line buffers to store pixels of the rows surrounding the pixel being processed so that those vertical pixels are readily available to the filter being implemented.
- the common solution is to employ four line buffers to store the pixels of the four surrounding rows.
- line buffers are costly to implement. This is particularly true in full high-definition (HD) image processing where each line buffer is typically wide enough to store at least 1,920 pixel values.
- HD high-definition
- FIG. 1 illustrates portions of an image processing system in accordance with some implementations of the invention
- FIG. 2 illustrates an example image processing scheme
- FIG. 3 is a flow chart illustrating a process in accordance with some implementations of the invention.
- FIG. 4 illustrates an example image processing pipeline scheme
- FIG. 5 illustrates an example image processing pixel labeling scheme
- FIG. 6 illustrates an example image processing scheme incorporating pipeline overlaps in accordance with some implementations of the invention.
- FIG. 7 illustrates a system in accordance with some implementations of the invention.
- FIG. 1 is a simplified block diagram of portions of an image processing system 100 in accordance with some implementations of the claimed invention.
- System 100 may include a display processing engine (DPE) 102 , a display controller 104 , a memory controller 106 , and memory 108 .
- DPE 102 also includes a set of line buffers 112 .
- DPE 102 may be any combination of logic in hardware, software and/or firmware form capable of supporting processing of pixel data where, in accordance with some implementations of the invention, DPE 102 may include processing logic at least capable of assessing an overall pipeline horizontal overlap associated with a set of filter kernels where that set of filter kernels may be considered to comprise a filter pipeline to be implemented by DPE 102 as will be explained in further detail below.
- DPE 102 may, in accordance with some implementations of the invention, be capable of providing line buffers 112 of sufficient width to support processing of pixel data wherein varying degrees of overlap between horizontally adjacent regions may be provided as will be explained in further detail below.
- DPE 102 may include one or more processing cores (now shown), each of which may be capable of undertaking, in a parallel and/or simultaneous fashion, processes in accordance with implementations of the invention. The uses and functionality of DPE 102 will be explained in greater detail below.
- Display controller 104 may comprise any processing and/or control logic (i.e., in the form of hardware, software, and/or firmware) capable of converting processed pixel data supplied by DPE 102 into a format suitable for driving a display (i.e., display-specific data).
- DPE 102 may provide processed pixel data to controller 104 in a specific color data format, for example in a compressed red-green-blue (RGB) format, and controller 104 may modify such data by generating, for example, corresponding liquid crystal display (LCD) drive data levels etc.
- FIG. 1 liquid crystal display
- DPE 102 and controller 104 as distinct components, the invention is not limited in this regard, and those of skill in the art will recognize that, for example, some if not all of the functions of display controller 104 may be performed by DPE 102 or, further, that the functionality of DPE 102 and/or controller 104 may be provided by a display processor and/or a host processor such as a microprocessor.
- Memory controller 106 may comprise any memory access logic (i.e., in the form of hardware, software, and/or firmware) capable of providing DPE 102 with pixel data (i.e., pixel values) to be processed or filtered by DPE 102 in response to a set of the filter kernels. Controller 106 may also provide DPE 102 with access to memory 108 for the purpose of storing pixel data corresponding to pixel values processed by DPE 102 . Memory controller 106 may also provide display controller 104 with access to pixel data (processed or otherwise) stored or held in memory 108 . The invention is, however, not limited to particular implementations of memory controller 106 and thus this item will not be described in greater detail herein.
- Memory 108 may comprise any memory device or mechanism suitable for storing and/or holding pixel data (e.g., in the form of luminance and color difference signals (i.e., “YUV” data) or red-green-blue (RGB) color pixel data or values, or other possible formats). While memory 108 may comprise any volatile or non-volatile memory technology such as Random Access Memory (RAM) memory or volatile memory technology (e.g., Flash memory), the invention is in no way limited by the type of memory employed for use as memory 108 .
- RAM Random Access Memory
- Flash memory volatile memory technology
- FIG. 2 illustrates a simplified example of a scheme 200 for image processing.
- FIG. 2 is offered solely to aid discussion of processes in accordance with some implementations of the invention, and thus no quantities or items shown in FIG. 2 should be construed to limit the invention in any way.
- FIG. 2 shows an image frame 202 corresponding, for example, to a high-definition (HD) 1920 ⁇ 1080 display format (i.e., where frame 202 includes 1,920 vertical columns and 1,080 horizontal rows of pixel values) where frame 202 has been divided into four horizontally adjacent regions 204 - 207 each of which includes 1,080 rows and 480 columns of pixel values and each of which is defined with respect to adjacent regions by boundaries 208 . While four regions 204 - 207 are shown in FIG. 2 , the invention itself is not limited by the number of regions employed or the width (e.g., measured in columns of pixels per region) of those regions employed.
- HD high-definition
- pixels of any given region 204 - 207 may be processed separately from the pixels of any other regions 204 - 207 .
- pixels in region 205 may be processed separately from pixels in region 206 .
- pixels of any of regions 204 - 206 may be processed separately in a sequential and/or serial manner (i.e., by one processing core) or in a parallel fashion (i.e., by a plurality of processing cores).
- an image processing scheme in accordance with some implementations of the invention may be distinguished from a raster scan image processing scheme in that, for a given row of frame 202 , a raster scan image processing scheme may process all pixels of that row in a sequential and/or serial manner.
- FIG. 3 is a flow chart illustrating a process 300 in accordance with some implementations of the invention. While, for ease of explanation, process 300 may be described with regard to system 100 of FIG. 1 and scheme 200 of FIG. 2 , the claimed invention is not limited in this regard and other processes or schemes supported by appropriate devices in accordance with the claimed invention are possible.
- Process 300 may begin with the determination of a pipeline overlap value associated with an image processing pipeline [act 302 ].
- DPE 102 may undertake act 302 in a preprocessing stage. That is, act 302 may be undertaken when DPE 102 is configured to implement an image processing pipeline where that pipeline employs one or more filter kernels to process pixel data.
- image processing undertaken by devices such as DPE 102 may employ one or more filter kernels where those kernels comprise, in effect, numerical matrices that act, for example, to convolve a given pixel value with the weighted values of surrounding pixels.
- devices such as DPE 102 may, in response to a filter algorithm, employ internal processing logic to process pixel values to implement the corresponding numerical kernel.
- FIG. 4 illustrates an example image processing pipeline 400 employing a series of filter kernels.
- FIG. 4 is offered to aid discussion of process 300 , in particular act 302 , and no quantities or items shown therein should be construed to limit the invention in any way.
- Pipeline 400 includes a series of three algorithms or filter kernels: a first algorithm 402 (corresponding to a 5 ⁇ 5 two-dimensional (2D) filter kernel), a second algorithm 404 (corresponding to a 9-tap horizontal, or one-dimensional (1D), filter), and a third algorithm 406 (corresponding to a 9 ⁇ 9 2D filter).
- N ⁇ N 2D filter kernel e.g., 5 ⁇ 5 2D, 9 ⁇ 9 2D, etc.
- N-tap horizontal filter imply although these terms will be placed in further context below.
- Algorithms 402 - 406 are example algorithms provided in FIG. 4 solely for explanatory purposes and should not be construed to limit the invention to particular image processing algorithms and/or particular image processing pipeline schemes.
- an image processing scheme employing algorithm 402 may require the use of a line buffer 408 having at least four (4) lines to store the pixels values for four (4) rows of pixel values of the rows of pixel values vertically adjacent and/or neighboring (two rows above and two rows below) to the pixel being processed by algorithm 402 .
- employing algorithm 406 may require the use of a line buffer 410 having at least eight (8) lines to store the pixels values for eight (8) rows of pixel values.
- algorithm 404 the 9-tap horizontal algorithm, does not require the use of line buffers storing adjacent rows of pixel values because this algorithm convolves the pixel being processed with values from neighboring and/or adjacent pixels in the same row.
- the width (i.e., the number of pixel values stored in each line or row) of the line buffers 408 / 410 may correspond to a width that is less than the full horizontal resolution of the image frame to be processed by pipeline 400 .
- FIG. 5 illustrates an example scheme 500 for labeling pixels.
- FIG. 5 is offered to aid discussion of process 300 , in particular act 302 , and no quantities or items shown therein should be construed to limit the invention in any way.
- FIG. 5 includes representative portions of two adjacent image regions 504 and 505 separated by a region boundary 506 .
- a pixel 508 within region 504 and lying adjacent to the region boundary (e.g., where pixel 508 is the 240th pixel in one row of a 240 pixel-wide region) is to be processed by DPE 102 employing pipeline 400 ( FIG. 4 ).
- application of algorithm 402 convolves the value of pixel 508 with the weighted values of the 5 ⁇ 5 matrix 510 of pixels surrounding pixel 508
- application of algorithm 404 convolves the value of pixel 508 with the weighted values of the four neighboring pixels 512 located within the same row on either side of pixel 508
- application of algorithm 406 convolves the value of pixel 508 with the weighted values of the 9 ⁇ 9 matrix 514 of pixels surrounding pixel 508 as shown.
- FIG. 5 in the context of pipeline 400 ( FIG. 4 ), it may be recognized that application of pipeline 400 to pixel 508 of region 504 should result in the convolution of some pixels in region 505 with pixel 508 's value if the value of pixel 508 is to be accurately processed by pipeline 400 .
- algorithm 402 (5 ⁇ 5 2D filter) should convolve the values of the adjacent two pixels 516 (e.g., the 241st and 242nd pixels) to the right of pixel 508 .
- algorithm 404 (9-tap horizontal filter) should convolve the values of the adjacent four pixels 517 (e.g., the 241st through the 244th pixels) to the right of pixel 508 . However, when doing so, unless algorithm 402 has convolved the two further adjacent pixels 518 (e.g., the 245th and 246th pixels) to the right of those four pixels 517 , then algorithm 404 may not, for example, use a correct value for the 244th pixel when it convolutes it with the 240th pixel's value (i.e., pixel 508 's value).
- algorithm 406 (9 ⁇ 9 2D filter) should convolve the values of the adjacent four pixels 517 (e.g., the 241st through the 244th pixels) to the right of pixel 508 . However, when doing so, unless algorithm 404 has convolved the four further adjacent pixels 519 (e.g., the 245th through the 248th pixels) to the right of those four pixels 517 and unless algorithm 402 has also convolved the two further adjacent pixels 520 (e.g., the 249th and 250th pixels) to the right of the four pixels 519 , then algorithm 406 may not, for example, use a correct value for the 244th pixel when it convolutes it with the 240th pixel's value.
- algorithm 406 may not, for example, use a correct value for the 244th pixel when it convolutes it with the 240th pixel's value.
- a horizontal overlap may be defined in accordance with the invention where that horizontal overlap includes sufficient pixels taken from regions adjacent to the region being processed.
- an overlap 522 may be defined in accordance with some implementations of the invention where that overlap includes the ten pixels in region 505 that are adjacent to that right-most pixel.
- image processing of region 504 may produce mathematically similar results to those produced by a standard raster scan process.
- a pipeline overlap factor may be defined in accordance with some implementations of the invention where that factor may be expressed as
- act 302 may involve having DPE 102 determine a pipeline overlap factor in accordance with equation 1 where DPE 102 has been provided with the individual horizontal overlap factors h_overlap[i] of the algorithms of the pixel processing pipeline to be used in process 300 .
- the pipeline overlap factor may be provided to DPE 102 by another device such as, for example, a host processor.
- Process 300 may then continue with the provision of one or more line buffers having widths sufficient to accommodate a region width plus twice the pipeline overlap factor [act 304 ].
- FIG. 6 illustrates an example image 600 showing, in accordance with some implementations of the invention, horizontal overlaps associated with image regions where those overlaps have widths corresponding to the pipeline overlap factor determined in act 302 .
- FIG. 6 is offered to aid discussion of process 300 , in particular act 304 , and no quantities or items shown therein should be construed to limit the invention in any way.
- example image 600 has been divided into three horizontally adjacent regions including a left-most region 602 that may, in accordance with some implementations of the invention, have an overlap 604 incorporating a number of columns of an adjacent region 606 where the width of overlap 604 corresponds to the pipeline overlap factor as determined in act 302 for a particular image processing pipeline.
- the next region 606 may, in accordance with some implementations of the invention, have two overlaps, a first overlap 608 incorporating a number of columns of region 604 to the left of region 606 and a second overlap 610 incorporating a number of columns of a region 612 to the right of region 606 .
- the widths of both overlap 608 and 610 may also correspond to the pipeline overlap factor as determined in act 302 .
- the third and right-most region 612 may have an overlap 614 incorporating a number of columns of region 606 to the left of region 612 .
- the width of overlap 614 may also correspond to the pipeline overlap factor as determined in act 302 .
- each of overlaps 604 , 608 , 610 and 614 may have a width equivalent to a pipeline overlap factor of ten (10).
- each of overlaps 604 , 608 , 610 and 614 may have a width sufficient to include ten columns of the respective adjacent regions 606 , 602 , 612 , and 606 .
- each region “j” may have respective horizontal and vertical dimensions region_w(j) and image_h.
- each region may be associated with one or two overlaps of width equal to pipeline_overlap.
- the total width of the pixel data to be processed for each of regions 602 and 612 may be expressed as:
- line buffers may be employed, in accordance with some implementations of the invention, that have a width consistent with equation (2). In other words, using the example above, those line buffers might have a width of 260 pixels as compared to the region width of 240 pixels.
- DPE 102 may, having access to or having determined a pipeline overlap factor (e.g., pipeline_overlap) in act 302 , use that overlap factor and the predetermined region width (e.g., region_w) to provide one or more line buffers where those line buffers have a width corresponding to equation (2) above.
- a pipeline overlap factor e.g., pipeline_overlap
- region_w region width
- act 304 would involve providing ten (10) line buffers each of which has a width corresponding to 260 pixels.
- Process 300 may continue with the receipt of pixel data for processing [act 306 ].
- act 306 may be undertaken by having DPE 102 use memory controller 106 to access pixel data stored in memory 108 .
- act 306 may involve DPE 102 using an internal Direct Memory Access (DMA) engine (not shown) to access pixel data in memory 108 via controller 106 where, in accordance with some implementations of the invention, the width of the pixel data obtained from memory 108 matches the widths of the line buffers provided in act 304 .
- DMA Direct Memory Access
- Process 300 may then continue with the processing of the pixel data through the image processing pipeline [act 308 ].
- DPE 102 may place the pixel data received in act 306 in the line buffers provided in act 304 and may then undertake act 308 by applying a filter pipeline to that pixel data.
- DPE 102 may process the pixel data received in act 306 using, in succession, algorithms 402 , 404 and 406 (i.e., pipeline 400 ) to filter the pixel data.
- both acts 302 and 304 may be undertaken once such as, for example, during a preprocessing stage, and then multiple simultaneous occurrences of acts 306 and 308 may be undertaken by two or more processing cores and/or two or more DPEs 102 where, for example, one occurrence of acts 306 / 308 could be associated with processing pixels in one region (e.g., region 602 ) while another occurrence of acts 306 / 308 could be associated with processing pixels in another region (e.g., region 606 ), etc. Further, at least some of the acts in this figure may be implemented as instructions, or groups of instructions, implemented in a machine-readable medium.
- FIG. 7 illustrates an example system 700 in accordance with some implementations of the invention.
- System 700 may include a host processor 702 , a graphics processor 704 , memories 706 and 708 (e.g., dynamic random access memory (DRAM), static random access memory (SRAM), non-volatile memory, etc.), a bus or communications pathway(s) 710 , input/output (I/O) interfaces 712 (e.g., universal synchronous bus (USB) interfaces, parallel ports, serial ports, telephone ports, and/or other I/O interfaces), network interfaces 714 (e.g., wired and/or wireless local area network (LAN) and/or wide area network (WAN) and/or personal area network (PAN), and/or other wired and/or wireless network interfaces), a display processor and/or controller 716 , and a display 718 .
- I/O input/output
- USB universal synchronous bus
- LAN local area network
- WAN wide area network
- PAN personal area
- System 700 may also include an antenna 715 (e.g., dipole antenna, narrowband Meander Line Antenna (MLA), wideband MLA, inverted “F” antenna, planar inverted “F” antenna, Goubau antenna, Patch antenna, etc.) coupled to network interfaces 714 .
- antenna 715 e.g., dipole antenna, narrowband Meander Line Antenna (MLA), wideband MLA, inverted “F” antenna, planar inverted “F” antenna, Goubau antenna, Patch antenna, etc.
- MLA narrowband Meander Line Antenna
- F narrowband Meander Line Antenna
- F planar inverted “F” antenna
- Goubau antenna Goubau antenna
- Patch antenna etc.
- system 700 may be any system suitable for processing pixel data for display.
- System 700 may assume a variety of physical implementations.
- system 700 may be implemented in a personal computer (PC), a networked PC, a media PC, a server computing system, a handheld computing platform (e.g., a personal digital assistant (PDA)), a gaming system (portable or otherwise), a set-top box (STB), a cellular telephone handset, etc.
- PC personal computer
- PDA personal digital assistant
- STB set-top box
- IC system-on-a-chip
- host processor 702 along with components 704 , 706 , 712 , and 714 may be implemented as multiple ICs contained within, for example, a single PC while display processor 716 may be implemented in a separate device such as display 718 coupled to host processor 702 and components 704 , 706 , 712 , and 714 through communications pathway 710 .
- all items in system 700 other than display 718 may be implemented in a single platform or device, such as, to name a few examples, a media PC or a STB, coupled to display 718 .
- Host processor 702 may comprise a special purpose or a general purpose processor including any control and/or processing logic (i.e., in the form of hardware, software and/or firmware) capable of providing display processor 716 with pixel data and/or instructions. For example, host processor 702 may place an image frame (or a portion thereof such as several rows of an image region) of pixel data in memory 708 from which display processor 716 may access that pixel data. In one implementation, host processor 702 may be capable of performing any of a number of tasks that support display processing line buffers incorporating pipeline overlap.
- These tasks may include, for example, although the invention is not limited in this regard, providing pixel processing algorithms, such as algorithms comprising a filter pipeline (e.g., pipeline 400 ), to display processor 716 , downloading microcode (via antenna 715 and interfaces 714 ) to processor 716 , initializing and/or configuring registers within processor 716 , interrupt servicing, providing a bus interface for uploading and/or downloading pixel and/or image data, etc.
- some or all of these functions may be performed by display processor 716 .
- FIG. 7 shows host processor 702 and display processor 716 as distinct components, the invention is not limited in any way in this regard and those of skill in the art will recognize that processors 702 and 716 possibly in addition to other components of system 700 may be implemented within a single IC.
- Bus or communications pathway(s) 710 may comprise any mechanism for conveying information (e.g., pixel data, instructions, etc.) between or amongst any of the elements of system 700 .
- communications pathway(s) 710 may comprise a multipurpose bus capable of conveying, for example, pixel data between memory 706 or memory 108 and display processor 716 .
- pathway(s) 710 may comprise a wireless communications pathway.
- graphics processor 704 may include a memory controller (not shown) that provides functionality enabling display processor 716 to provide pixel data to or to obtain pixel data from memories 706 / 708 .
- Display processor 716 may comprise any control and/or processing logic (i.e., in the form of hardware, software, and/or firmware) capable of implementing display processing line buffers incorporating pipeline overlap in accordance with some implementations of the invention, and of supplying processed pixel data to display 718 . Further, processor 716 may include one or more instances of DPE 102 or may provide the functionality of DPE 102 as described above. In addition, processor 716 may further process processed pixel data, such as that generated by DPE 102 implementing process 300 , to, for example, convert color pixel values to corresponding display drive data levels etc.
- display processor 716 shows display processor 716 as a distinct component, the invention is not limited in this regard, and those of skill in the art will recognize that, for example, some if not all of the functionality of display processor 716 may be performed by graphics processor 704 and/or host processor 702 .
- display 718 may be any device capable of displaying pixel data provided by display processor 716 and processed by DPE 102 .
- display 718 may be a liquid crystal display (LCD) or a Plasma Display Panel (PDP).
- LCD liquid crystal display
- PDP Plasma Display Panel
- the invention is not, however, limited to any particular type of display panel employed in display 718 .
- a display processor and/or a display processing engine utilizing line buffers sized to include pipeline overlaps may permit complex, high quality pixel processing algorithms (e.g., motion compensated deinterlacing, mosquito noise reduction, etc.) to be employed in a processing scheme operating on an image divided into horizontally adjacent regions where that scheme retains the mathematical accuracy of those algorithms while at the same time minimizing costs in terms of the silicon area devoted to the display processing pipeline.
- complex, high quality pixel processing algorithms e.g., motion compensated deinterlacing, mosquito noise reduction, etc.
- FIG. 1 and the accompanying text may show and describe a single display processing engine 102 coupled to memory controller 106 , those skilled in the art will recognize that display processing systems in accordance with the invention may employ multiple display processing engines, each operating in accordance with the invention, coupled to one or more memory controllers. Clearly, many other implementations may be employed to provide display processing line buffers incorporating pipeline overlap.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Image Processing (AREA)
- Image Generation (AREA)
Abstract
Apparatus, systems and methods for display processing line buffers incorporating pipeline overlap are disclosed. For example, an apparatus is disclosed including processing logic to use pixel processing algorithms to process a pixel value of a first portion of an image, and line buffers coupled to the processing logic. The line buffers to hold at least some pixel values of other portions of the image adjacent to the first portion. Where the pixel values of the other portions of the image held by the line buffers correspond to pixel values of the adjacent portions of the image that are to be convolved by the pixel processing algorithms with the pixel value of the first portion. Other implementations are also disclosed.
Description
- This application is a divisional of U.S. patent application Ser. No. 11/510,045, filed on Aug. 25, 2006.
- Image processing for display is typically done in a raster fashion where pixels are processed serially starting from the left most pixel in the upper most row in a frame and proceeding down through the frame row by row such that all pixels in any particular row are processed together. While processing a given pixel in one row of the frame a pixel processing algorithm or “filter kernel” may need to access surrounding vertical pixels. However, because pixels are usually serially scanned in a raster fashion through a typical processing pipeline the surrounding vertical pixels may not be immediately available.
- A common solution to accessing surrounding vertical pixels is to use line buffers to store pixels of the rows surrounding the pixel being processed so that those vertical pixels are readily available to the filter being implemented. Thus, for example, to use a five-by-five (5×5) two-dimensional (2D) filter kernel the common solution is to employ four line buffers to store the pixels of the four surrounding rows. For a 9×9 2D filter eight line buffers need to be employed. However, line buffers are costly to implement. This is particularly true in full high-definition (HD) image processing where each line buffer is typically wide enough to store at least 1,920 pixel values. For a typical image processing pipeline employing multiple filter kernels the expense associated with using full width line buffers can become prohibitive.
- The accompanying drawings, incorporated in and constituting a part of this specification, illustrate one or more implementations consistent with the principles of the invention and, together with the description of the invention, explain such implementations. The drawings are not necessarily to scale, the emphasis instead being placed upon illustrating the principles of the invention. In the drawings,
-
FIG. 1 illustrates portions of an image processing system in accordance with some implementations of the invention; -
FIG. 2 illustrates an example image processing scheme; -
FIG. 3 is a flow chart illustrating a process in accordance with some implementations of the invention; -
FIG. 4 illustrates an example image processing pipeline scheme; -
FIG. 5 illustrates an example image processing pixel labeling scheme; -
FIG. 6 illustrates an example image processing scheme incorporating pipeline overlaps in accordance with some implementations of the invention; and -
FIG. 7 illustrates a system in accordance with some implementations of the invention. - The following description refers to the accompanying drawings. Among the various drawings the same reference numbers may be used to identify the same or similar elements. While the following description provides a thorough understanding of the various aspects of the claimed invention by setting forth specific details such as particular structures, architectures, interfaces, techniques, etc., such details are provided for purposes of explanation and should riot be viewed as limiting. Moreover, those of skill in the art will, in light of the present disclosure, appreciate that various aspects of the invention claimed may be practiced in other examples or implementations that depart from these specific details. At certain junctures in the following disclosure descriptions of well known devices, circuits, and methods have been omitted to avoid clouding the description of the present invention with unnecessary detail.
-
FIG. 1 is a simplified block diagram of portions of animage processing system 100 in accordance with some implementations of the claimed invention.System 100 may include a display processing engine (DPE) 102, adisplay controller 104, amemory controller 106, andmemory 108. DPE 102 also includes a set ofline buffers 112. DPE 102 may be any combination of logic in hardware, software and/or firmware form capable of supporting processing of pixel data where, in accordance with some implementations of the invention, DPE 102 may include processing logic at least capable of assessing an overall pipeline horizontal overlap associated with a set of filter kernels where that set of filter kernels may be considered to comprise a filter pipeline to be implemented by DPE 102 as will be explained in further detail below. Toward that end, DPE 102 may, in accordance with some implementations of the invention, be capable of providingline buffers 112 of sufficient width to support processing of pixel data wherein varying degrees of overlap between horizontally adjacent regions may be provided as will be explained in further detail below. In addition, DPE 102 may include one or more processing cores (now shown), each of which may be capable of undertaking, in a parallel and/or simultaneous fashion, processes in accordance with implementations of the invention. The uses and functionality ofDPE 102 will be explained in greater detail below. -
Display controller 104 may comprise any processing and/or control logic (i.e., in the form of hardware, software, and/or firmware) capable of converting processed pixel data supplied by DPE 102 into a format suitable for driving a display (i.e., display-specific data). For example, while the invention is not limited in this regard, DPE 102 may provide processed pixel data to controller 104 in a specific color data format, for example in a compressed red-green-blue (RGB) format, andcontroller 104 may modify such data by generating, for example, corresponding liquid crystal display (LCD) drive data levels etc. AlthoughFIG. 1 shows DPE 102 andcontroller 104 as distinct components, the invention is not limited in this regard, and those of skill in the art will recognize that, for example, some if not all of the functions ofdisplay controller 104 may be performed byDPE 102 or, further, that the functionality ofDPE 102 and/orcontroller 104 may be provided by a display processor and/or a host processor such as a microprocessor. -
Memory controller 106 may comprise any memory access logic (i.e., in the form of hardware, software, and/or firmware) capable of providingDPE 102 with pixel data (i.e., pixel values) to be processed or filtered byDPE 102 in response to a set of the filter kernels.Controller 106 may also provideDPE 102 with access tomemory 108 for the purpose of storing pixel data corresponding to pixel values processed byDPE 102.Memory controller 106 may also providedisplay controller 104 with access to pixel data (processed or otherwise) stored or held inmemory 108. The invention is, however, not limited to particular implementations ofmemory controller 106 and thus this item will not be described in greater detail herein. -
Memory 108 may comprise any memory device or mechanism suitable for storing and/or holding pixel data (e.g., in the form of luminance and color difference signals (i.e., “YUV” data) or red-green-blue (RGB) color pixel data or values, or other possible formats). Whilememory 108 may comprise any volatile or non-volatile memory technology such as Random Access Memory (RAM) memory or volatile memory technology (e.g., Flash memory), the invention is in no way limited by the type of memory employed for use asmemory 108. -
FIG. 2 illustrates a simplified example of ascheme 200 for image processing.FIG. 2 is offered solely to aid discussion of processes in accordance with some implementations of the invention, and thus no quantities or items shown inFIG. 2 should be construed to limit the invention in any way.FIG. 2 shows animage frame 202 corresponding, for example, to a high-definition (HD) 1920×1080 display format (i.e., whereframe 202 includes 1,920 vertical columns and 1,080 horizontal rows of pixel values) whereframe 202 has been divided into four horizontally adjacent regions 204-207 each of which includes 1,080 rows and 480 columns of pixel values and each of which is defined with respect to adjacent regions byboundaries 208. While four regions 204-207 are shown inFIG. 2 , the invention itself is not limited by the number of regions employed or the width (e.g., measured in columns of pixels per region) of those regions employed. - In accordance with some implementations of the invention for an image processing scheme applied to a given row of pixels of
frame 202 the pixels of any given region 204-207 may be processed separately from the pixels of any other regions 204-207. In other words, for example, pixels inregion 205 may be processed separately from pixels inregion 206. Further, in accordance with some implementations of the invention, pixels of any of regions 204-206 may be processed separately in a sequential and/or serial manner (i.e., by one processing core) or in a parallel fashion (i.e., by a plurality of processing cores). Moreover, an image processing scheme in accordance with some implementations of the invention may be distinguished from a raster scan image processing scheme in that, for a given row offrame 202, a raster scan image processing scheme may process all pixels of that row in a sequential and/or serial manner. -
FIG. 3 is a flow chart illustrating aprocess 300 in accordance with some implementations of the invention. While, for ease of explanation,process 300 may be described with regard tosystem 100 ofFIG. 1 andscheme 200 ofFIG. 2 , the claimed invention is not limited in this regard and other processes or schemes supported by appropriate devices in accordance with the claimed invention are possible. -
Process 300 may begin with the determination of a pipeline overlap value associated with an image processing pipeline [act 302]. In accordance with some implementations of the invention DPE 102 may undertakeact 302 in a preprocessing stage. That is,act 302 may be undertaken when DPE 102 is configured to implement an image processing pipeline where that pipeline employs one or more filter kernels to process pixel data. Those skilled in the art will recognize that image processing undertaken by devices such asDPE 102 may employ one or more filter kernels where those kernels comprise, in effect, numerical matrices that act, for example, to convolve a given pixel value with the weighted values of surrounding pixels. Further, as those skilled in the art will also recognize, devices such asDPE 102 may, in response to a filter algorithm, employ internal processing logic to process pixel values to implement the corresponding numerical kernel. -
FIG. 4 illustrates an exampleimage processing pipeline 400 employing a series of filter kernels.FIG. 4 is offered to aid discussion ofprocess 300, inparticular act 302, and no quantities or items shown therein should be construed to limit the invention in any way.Pipeline 400 includes a series of three algorithms or filter kernels: a first algorithm 402 (corresponding to a 5×5 two-dimensional (2D) filter kernel), a second algorithm 404 (corresponding to a 9-tap horizontal, or one-dimensional (1D), filter), and a third algorithm 406 (corresponding to a 9×9 2D filter). Those skilled in the art will recognize what the terms N×N 2D filter kernel (e.g., 5×5 2D, 9×9 2D, etc.) and N-tap horizontal filter imply although these terms will be placed in further context below. Algorithms 402-406 are example algorithms provided inFIG. 4 solely for explanatory purposes and should not be construed to limit the invention to particular image processing algorithms and/or particular image processing pipeline schemes. - Those skilled in the art will further recognize that an image processing
scheme employing algorithm 402 may require the use of aline buffer 408 having at least four (4) lines to store the pixels values for four (4) rows of pixel values of the rows of pixel values vertically adjacent and/or neighboring (two rows above and two rows below) to the pixel being processed byalgorithm 402. Similarly, employingalgorithm 406 may require the use of aline buffer 410 having at least eight (8) lines to store the pixels values for eight (8) rows of pixel values. Further, those skilled in the art will recognize thatalgorithm 404, the 9-tap horizontal algorithm, does not require the use of line buffers storing adjacent rows of pixel values because this algorithm convolves the pixel being processed with values from neighboring and/or adjacent pixels in the same row. Moreover, in accordance with some implementations of the invention, ifpipeline 400 is employed to undertake image processing of a frame of pixel data where that frame is divided into horizontally adjacent image regions then the width (i.e., the number of pixel values stored in each line or row) of the line buffers 408/410 may correspond to a width that is less than the full horizontal resolution of the image frame to be processed bypipeline 400. -
FIG. 5 illustrates anexample scheme 500 for labeling pixels.FIG. 5 is offered to aid discussion ofprocess 300, inparticular act 302, and no quantities or items shown therein should be construed to limit the invention in any way.FIG. 5 includes representative portions of twoadjacent image regions region boundary 506. Consider the case where apixel 508, withinregion 504 and lying adjacent to the region boundary (e.g., wherepixel 508 is the 240th pixel in one row of a 240 pixel-wide region) is to be processed byDPE 102 employing pipeline 400 (FIG. 4 ). Forpixel 508, application ofalgorithm 402 convolves the value ofpixel 508 with the weighted values of the 5×5matrix 510 ofpixels surrounding pixel 508, application ofalgorithm 404 convolves the value ofpixel 508 with the weighted values of the four neighboringpixels 512 located within the same row on either side ofpixel 508, and application ofalgorithm 406 convolves the value ofpixel 508 with the weighted values of the 9×9matrix 514 ofpixels surrounding pixel 508 as shown. - In considering
FIG. 5 in the context of pipeline 400 (FIG. 4 ), it may be recognized that application ofpipeline 400 topixel 508 ofregion 504 should result in the convolution of some pixels inregion 505 withpixel 508's value if the value ofpixel 508 is to be accurately processed bypipeline 400. In particular, considering only therow containing pixel 508 where, for the sake of argument,pixel 508 is the 240th and last pixel in region 504 (example region width of 240 pixels) then algorithm 402 (5×5 2D filter) should convolve the values of the adjacent two pixels 516 (e.g., the 241st and 242nd pixels) to the right ofpixel 508. Similarly, algorithm 404 (9-tap horizontal filter) should convolve the values of the adjacent four pixels 517 (e.g., the 241st through the 244th pixels) to the right ofpixel 508. However, when doing so, unlessalgorithm 402 has convolved the two further adjacent pixels 518 (e.g., the 245th and 246th pixels) to the right of those fourpixels 517, thenalgorithm 404 may not, for example, use a correct value for the 244th pixel when it convolutes it with the 240th pixel's value (i.e.,pixel 508's value). - Likewise, algorithm 406 (9×9 2D filter) should convolve the values of the adjacent four pixels 517 (e.g., the 241st through the 244th pixels) to the right of
pixel 508. However, when doing so, unlessalgorithm 404 has convolved the four further adjacent pixels 519 (e.g., the 245th through the 248th pixels) to the right of those fourpixels 517 and unlessalgorithm 402 has also convolved the two further adjacent pixels 520 (e.g., the 249th and 250th pixels) to the right of the fourpixels 519, thenalgorithm 406 may not, for example, use a correct value for the 244th pixel when it convolutes it with the 240th pixel's value. Thus, in order to ensure that the result of applyingpipeline 400 to pixels in a region of an image frame divided into horizontally adjacent regions is mathematically similar to the results that would be obtained in a raster scan pixel processing scheme a horizontal overlap may be defined in accordance with the invention where that horizontal overlap includes sufficient pixels taken from regions adjacent to the region being processed. - For example, as shown in
FIG. 5 , to obtain a mathematically correct application ofpipeline 400 to aright-most pixel 508 inregion 504 anoverlap 522 may be defined in accordance with some implementations of the invention where that overlap includes the ten pixels inregion 505 that are adjacent to that right-most pixel. Thus, in accordance with some implementations of the invention, ifoverlap 522 is incorporated into the processing ofregion 504 then image processing ofregion 504 may produce mathematically similar results to those produced by a standard raster scan process. - Generally speaking, a pipeline overlap factor may be defined in accordance with some implementations of the invention where that factor may be expressed as
-
- where each algorithm “i” in a pixel processing pipeline has an associated horizontal overlap factor h_overlap[i]. Thus, for example,
algorithm 402 inpipeline 400 would have a horizontal overlap factor of two (2), and both ofalgorithms pipeline 400 would have horizontal overlap factors of four (4). Hence, the value of pipeline_overlap (equation 1) forpipeline 400 would be ten (10). In accordance with some implementations of the invention, act 302 may involve havingDPE 102 determine a pipeline overlap factor in accordance withequation 1 whereDPE 102 has been provided with the individual horizontal overlap factors h_overlap[i] of the algorithms of the pixel processing pipeline to be used inprocess 300. Alternatively, the pipeline overlap factor may be provided toDPE 102 by another device such as, for example, a host processor. -
Process 300 may then continue with the provision of one or more line buffers having widths sufficient to accommodate a region width plus twice the pipeline overlap factor [act 304].FIG. 6 illustrates anexample image 600 showing, in accordance with some implementations of the invention, horizontal overlaps associated with image regions where those overlaps have widths corresponding to the pipeline overlap factor determined inact 302.FIG. 6 is offered to aid discussion ofprocess 300, inparticular act 304, and no quantities or items shown therein should be construed to limit the invention in any way. - As
FIG. 6 shows,example image 600 has been divided into three horizontally adjacent regions including aleft-most region 602 that may, in accordance with some implementations of the invention, have anoverlap 604 incorporating a number of columns of anadjacent region 606 where the width ofoverlap 604 corresponds to the pipeline overlap factor as determined inact 302 for a particular image processing pipeline. Because most image processing algorithms such as filter kernels are symmetric, thenext region 606 may, in accordance with some implementations of the invention, have two overlaps, afirst overlap 608 incorporating a number of columns ofregion 604 to the left ofregion 606 and asecond overlap 610 incorporating a number of columns of aregion 612 to the right ofregion 606. The widths of both overlap 608 and 610 may also correspond to the pipeline overlap factor as determined inact 302. - Finally, the third and
right-most region 612 may have anoverlap 614 incorporating a number of columns ofregion 606 to the left ofregion 612. Again, the width ofoverlap 614 may also correspond to the pipeline overlap factor as determined inact 302. Thus, if, for example,pipeline 400 is applied separately to pixels inregions overlaps overlaps adjacent regions - If, as shown in
FIG. 6 , the image to be processed has respective horizontal and vertical dimensions image_w and image_h then each region “j” may have respective horizontal and vertical dimensions region_w(j) and image_h. In addition, as discussed above, each region may be associated with one or two overlaps of width equal to pipeline_overlap. Thus, in accordance with some implementations of the invention, the total width of the pixel data to be processed for each ofregions -
region_w(j)+pipeline_overlap (2) - while the total width of the pixel data to be processed for
region 606 may be expressed as: -
region_w(j)+2*pipeline_overlap (3) - Thus, assuming that all three regions have the same region width of, for example, 240 pixels and a pipeline_overlap value of 10 pixels, the width of pixel data to be processed for
regions region 606 would be 260 pixels. Thus, in order for the application of an image processing pipeline to each region ofimage 600 to have mathematically similar results to what would be attained if theentire image 600 was processed in a raster scan manner, line buffers may be employed, in accordance with some implementations of the invention, that have a width consistent with equation (2). In other words, using the example above, those line buffers might have a width of 260 pixels as compared to the region width of 240 pixels. - It should be noted however, that the invention is not limited to a particular number of regions employed to process an image frame nor is the invention limited to all regions having the same region width. Moreover, the terms image region, image section, image strip, image stride, image swath, and/or image portion may be used interchangeably without departing from the scope and spirit of the invention.
- Returning to
FIG. 3 , in undertakingact 304,DPE 102 may, having access to or having determined a pipeline overlap factor (e.g., pipeline_overlap) inact 302, use that overlap factor and the predetermined region width (e.g., region_w) to provide one or more line buffers where those line buffers have a width corresponding to equation (2) above. For example, referring to pipeline 400 (FIG. 4 ) and image 600 (FIG. 6 ), act 304 would involve providing ten (10) line buffers each of which has a width corresponding to 260 pixels. -
Process 300 may continue with the receipt of pixel data for processing [act 306]. In some implementations of theinvention act 306 may be undertaken by havingDPE 102use memory controller 106 to access pixel data stored inmemory 108. Thus, for example, act 306 may involveDPE 102 using an internal Direct Memory Access (DMA) engine (not shown) to access pixel data inmemory 108 viacontroller 106 where, in accordance with some implementations of the invention, the width of the pixel data obtained frommemory 108 matches the widths of the line buffers provided inact 304. Continuing the example from above, if DPE is to process pixel 508 (FIG. 5 ) and the line buffers provided inact 304 have a width of 260 pixels then act 306 may involveDPE 102 obtaining the pixel data corresponding to the nine rows of region 606 (width=240 pixels) centered about therow containing pixel 508 and the pixel data for theoverlap regions regions -
Process 300 may then continue with the processing of the pixel data through the image processing pipeline [act 308]. In accordance with some implementations of the invention,DPE 102 may place the pixel data received inact 306 in the line buffers provided inact 304 and may then undertakeact 308 by applying a filter pipeline to that pixel data. Continuing the example from above,DPE 102 may process the pixel data received inact 306 using, in succession,algorithms - The acts shown in
FIG. 3 need not be implemented in the order shown; nor do all of the acts necessarily need to be performed. Also, those acts that are not dependent on other acts may be performed in parallel with the other acts. Further, bothacts acts acts 306/308 could be associated with processing pixels in one region (e.g., region 602) while another occurrence ofacts 306/308 could be associated with processing pixels in another region (e.g., region 606), etc. Further, at least some of the acts in this figure may be implemented as instructions, or groups of instructions, implemented in a machine-readable medium. -
FIG. 7 illustrates anexample system 700 in accordance with some implementations of the invention.System 700 may include ahost processor 702, agraphics processor 704,memories 706 and 708 (e.g., dynamic random access memory (DRAM), static random access memory (SRAM), non-volatile memory, etc.), a bus or communications pathway(s) 710, input/output (I/O) interfaces 712 (e.g., universal synchronous bus (USB) interfaces, parallel ports, serial ports, telephone ports, and/or other I/O interfaces), network interfaces 714 (e.g., wired and/or wireless local area network (LAN) and/or wide area network (WAN) and/or personal area network (PAN), and/or other wired and/or wireless network interfaces), a display processor and/orcontroller 716, and adisplay 718.System 700 may also include an antenna 715 (e.g., dipole antenna, narrowband Meander Line Antenna (MLA), wideband MLA, inverted “F” antenna, planar inverted “F” antenna, Goubau antenna, Patch antenna, etc.) coupled to network interfaces 714. In general,system 700 may be any system suitable for processing pixel data for display. -
System 700 may assume a variety of physical implementations. For example,system 700 may be implemented in a personal computer (PC), a networked PC, a media PC, a server computing system, a handheld computing platform (e.g., a personal digital assistant (PDA)), a gaming system (portable or otherwise), a set-top box (STB), a cellular telephone handset, etc. Moreover, while some components ofsystem 700 may be implemented within a single device, such as a system-on-a-chip (SOC) integrated circuit (IC), components ofsystem 700 may also be distributed across multiple ICs or devices. For example,host processor 702 along withcomponents display processor 716 may be implemented in a separate device such asdisplay 718 coupled tohost processor 702 andcomponents communications pathway 710. Alternatively, all items insystem 700 other thandisplay 718 may be implemented in a single platform or device, such as, to name a few examples, a media PC or a STB, coupled todisplay 718. -
Host processor 702 may comprise a special purpose or a general purpose processor including any control and/or processing logic (i.e., in the form of hardware, software and/or firmware) capable of providingdisplay processor 716 with pixel data and/or instructions. For example,host processor 702 may place an image frame (or a portion thereof such as several rows of an image region) of pixel data inmemory 708 from whichdisplay processor 716 may access that pixel data. In one implementation,host processor 702 may be capable of performing any of a number of tasks that support display processing line buffers incorporating pipeline overlap. These tasks may include, for example, although the invention is not limited in this regard, providing pixel processing algorithms, such as algorithms comprising a filter pipeline (e.g., pipeline 400), to displayprocessor 716, downloading microcode (viaantenna 715 and interfaces 714) toprocessor 716, initializing and/or configuring registers withinprocessor 716, interrupt servicing, providing a bus interface for uploading and/or downloading pixel and/or image data, etc. In alternative implementations, some or all of these functions may be performed bydisplay processor 716. Further, whileFIG. 7 showshost processor 702 anddisplay processor 716 as distinct components, the invention is not limited in any way in this regard and those of skill in the art will recognize thatprocessors system 700 may be implemented within a single IC. - Bus or communications pathway(s) 710 may comprise any mechanism for conveying information (e.g., pixel data, instructions, etc.) between or amongst any of the elements of
system 700. For example, although the invention is not limited in this regard, communications pathway(s) 710 may comprise a multipurpose bus capable of conveying, for example, pixel data betweenmemory 706 ormemory 108 anddisplay processor 716. Alternatively, pathway(s) 710 may comprise a wireless communications pathway. In addition,graphics processor 704 may include a memory controller (not shown) that provides functionality enablingdisplay processor 716 to provide pixel data to or to obtain pixel data frommemories 706/708. -
Display processor 716 may comprise any control and/or processing logic (i.e., in the form of hardware, software, and/or firmware) capable of implementing display processing line buffers incorporating pipeline overlap in accordance with some implementations of the invention, and of supplying processed pixel data to display 718. Further,processor 716 may include one or more instances ofDPE 102 or may provide the functionality ofDPE 102 as described above. In addition,processor 716 may further process processed pixel data, such as that generated byDPE 102 implementingprocess 300, to, for example, convert color pixel values to corresponding display drive data levels etc. AlthoughFIG. 7 showsdisplay processor 716 as a distinct component, the invention is not limited in this regard, and those of skill in the art will recognize that, for example, some if not all of the functionality ofdisplay processor 716 may be performed bygraphics processor 704 and/orhost processor 702. - Finally,
display 718 may be any device capable of displaying pixel data provided bydisplay processor 716 and processed byDPE 102. Thus, to name a few more prominent examples,display 718 may be a liquid crystal display (LCD) or a Plasma Display Panel (PDP). The invention is not, however, limited to any particular type of display panel employed indisplay 718. - Thus, in accordance with some implementations of the invention, a display processor and/or a display processing engine utilizing line buffers sized to include pipeline overlaps may permit complex, high quality pixel processing algorithms (e.g., motion compensated deinterlacing, mosquito noise reduction, etc.) to be employed in a processing scheme operating on an image divided into horizontally adjacent regions where that scheme retains the mathematical accuracy of those algorithms while at the same time minimizing costs in terms of the silicon area devoted to the display processing pipeline.
- While the foregoing description of one or more instantiations consistent with the claimed invention provides illustration and description of the invention it is not intended to be exhaustive or to limit the scope of the invention to the particular implementations disclosed. Clearly, modifications and variations are possible in light of the above teachings or may be acquired from practice of various implementations of the invention. For example, while
FIG. 1 and the accompanying text may show and describe a singledisplay processing engine 102 coupled tomemory controller 106, those skilled in the art will recognize that display processing systems in accordance with the invention may employ multiple display processing engines, each operating in accordance with the invention, coupled to one or more memory controllers. Clearly, many other implementations may be employed to provide display processing line buffers incorporating pipeline overlap. - No device, element, act, data type, instruction etc. set forth in the description of the present application should be construed as critical or essential to the invention unless explicitly described as such. Also, as used herein, the article “a” is intended to include one or more items. Moreover, when terms or phrases such as “coupled” or “responsive” or “in communication with” are used herein or in the claims that follow, these terms are meant to be interpreted broadly. For example, the phrase “coupled to” may refer to being communicatively, electrically and/or operatively coupled as appropriate for the context in which the phrase is used. Further, those skilled in the art will realize that the terms pixel, pixel value, pixel data, display data and/or image data may be used interchangeably. Variations and modifications may be made to the above-described implementation(s) of the claimed invention without departing substantially from the spirit and principles of the invention. All such modifications and variations are intended to be included herein within the scope of this disclosure and protected by the following claims.
Claims (10)
1. A method comprising:
dividing an image to be processed into a plurality of adjacent vertical strips; and
providing line buffers sized based on the size of said vertical strips rather than the size of said image.
2. The method of claim 1 including sizing said line buffers to be slightly larger than the size of the strips in order to accommodate pixels adjacent the strip that need to be accommodated.
3. The method of claim 1 including determining a pipeline overlap factor for said strips.
4. An apparatus comprising:
processing logic to divide an image to be processed into a plurality of adjacent vertical strips; and
line buffers sized based on the size of said vertical strips rather than the size of said image.
5. The apparatus of claim 4 , said line buffers sized to be slightly larger than the size of the strips in order to accommodate pixels adjacent the strip that need to be accommodated.
6. The apparatus of claim 4 , said processing logic to determine a pipeline overlap factor for said strips.
7. A system comprising:
processing logic to use pixel processing algorithms to process a pixel value of a first portion of an image;
memory coupled to the processing logic, the memory to store at least the pixel value of the first portion; and
line buffers coupled to the processing logic, the line buffers to hold at least some pixel values of other portions of the image adjacent to the first portion, wherein the pixel values of the other portions of the image held by the line buffers correspond to pixel values of the adjacent portions of the image that are to be convolved by the pixel processing algorithms with the pixel value of the first portion.
8. The system of claim 7 , wherein the memory comprises one of dynamic random access memory (DRAM), static random access memory (SRAM), or non-volatile memory.
9. The system of claim 7 , further comprising an antenna to receive the pixel value of the first portion.
10. The system of claim 9 , wherein the antenna comprises one of a dipole antenna, a narrowband Meander Line Antenna (MLA), a wideband MLA, an inverted “F” antenna, a planar inverted “F” antenna, a Goubau antenna, or a Patch antenna.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US12/909,248 US20110037771A1 (en) | 2006-08-25 | 2010-10-21 | Display Processing Line Buffers Incorporating Pipeline Overlap |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/510,045 US7834873B2 (en) | 2006-08-25 | 2006-08-25 | Display processing line buffers incorporating pipeline overlap |
US12/909,248 US20110037771A1 (en) | 2006-08-25 | 2010-10-21 | Display Processing Line Buffers Incorporating Pipeline Overlap |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/510,045 Division US7834873B2 (en) | 2006-08-25 | 2006-08-25 | Display processing line buffers incorporating pipeline overlap |
Publications (1)
Publication Number | Publication Date |
---|---|
US20110037771A1 true US20110037771A1 (en) | 2011-02-17 |
Family
ID=39107132
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/510,045 Expired - Fee Related US7834873B2 (en) | 2006-08-25 | 2006-08-25 | Display processing line buffers incorporating pipeline overlap |
US12/909,248 Abandoned US20110037771A1 (en) | 2006-08-25 | 2010-10-21 | Display Processing Line Buffers Incorporating Pipeline Overlap |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/510,045 Expired - Fee Related US7834873B2 (en) | 2006-08-25 | 2006-08-25 | Display processing line buffers incorporating pipeline overlap |
Country Status (6)
Country | Link |
---|---|
US (2) | US7834873B2 (en) |
EP (1) | EP2054794A4 (en) |
JP (1) | JP2009545085A (en) |
CN (1) | CN101495952B (en) |
TW (2) | TWI381364B (en) |
WO (1) | WO2008024668A1 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11042962B2 (en) * | 2016-04-18 | 2021-06-22 | Avago Technologies International Sales Pte. Limited | Hardware optimisation for generating 360° images |
Families Citing this family (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8754895B2 (en) * | 2008-09-09 | 2014-06-17 | Sony Corporation | Pipelined image processing engine |
TW201106679A (en) * | 2009-08-05 | 2011-02-16 | Silicon Motion Inc | Image processing device and deinterlacing method |
CN102215324B (en) * | 2010-04-08 | 2013-07-31 | 安凯(广州)微电子技术有限公司 | Filtering circuit for performing filtering operation on video image and filtering method thereof |
JP5835942B2 (en) * | 2010-06-25 | 2015-12-24 | キヤノン株式会社 | Image processing apparatus, control method thereof, and program |
JP5703769B2 (en) * | 2011-01-19 | 2015-04-22 | ソニー株式会社 | Image conversion apparatus, image conversion method, program, and electronic apparatus |
US8761454B2 (en) * | 2011-05-10 | 2014-06-24 | Hewlett-Packard Development Company, L.P. | Detecting streaks in printed images |
KR101344858B1 (en) | 2012-03-23 | 2013-12-24 | 경북대학교 산학협력단 | Device and Method for Processing Image, Computer Readable Recording Medium |
TWI520598B (en) * | 2012-05-23 | 2016-02-01 | 晨星半導體股份有限公司 | Image processing apparatus and image processing method |
CN103474049B (en) * | 2012-06-08 | 2017-05-17 | 晨星软件研发(深圳)有限公司 | Image processing apparatus and image processing method |
GB2533565A (en) * | 2014-12-18 | 2016-06-29 | Stmicroelectronics (Research & Development) Ltd | A method, apparatus and system |
US9749548B2 (en) | 2015-01-22 | 2017-08-29 | Google Inc. | Virtual linebuffers for image signal processors |
US9756268B2 (en) | 2015-04-23 | 2017-09-05 | Google Inc. | Line buffer unit for image processor |
US10204396B2 (en) * | 2016-02-26 | 2019-02-12 | Google Llc | Compiler managed memory for image processor |
US10367639B2 (en) * | 2016-12-29 | 2019-07-30 | Intel Corporation | Graphics processor with encrypted kernels |
US10255655B1 (en) | 2017-06-16 | 2019-04-09 | Apple Inc. | Serial pixel processing with storage for overlapping texel data |
US11431941B2 (en) * | 2018-06-12 | 2022-08-30 | Carl Zeiss Ag | Method, apparatus, and system for processing digital images |
US10681266B2 (en) * | 2018-06-12 | 2020-06-09 | Carl Zeiss Ag | Method, apparatus, and system for processing digital images |
CN110047031A (en) * | 2019-03-26 | 2019-07-23 | 深兰科技(上海)有限公司 | A kind of method and apparatus of pixel fragment splicing |
Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6327000B1 (en) * | 1999-04-02 | 2001-12-04 | Teralogic, Inc. | Efficient image scaling for scan rate conversion |
US6400782B2 (en) * | 1998-09-03 | 2002-06-04 | Conexant Systems, Inc. | Method of frequency domain filtering employing a real to analytic transform |
US6457075B1 (en) * | 1999-05-17 | 2002-09-24 | Koninkijke Philips Electronics N.V. | Synchronous memory system with automatic burst mode switching as a function of the selected bus master |
US6625629B1 (en) * | 2000-05-23 | 2003-09-23 | Microsoft Corporation | System and method for signal processing using an improved convolution technique |
US20040130553A1 (en) * | 2002-12-26 | 2004-07-08 | Canon Kabushiki Kaisha | Image processing apparatus and image processing method |
US20040160452A1 (en) * | 2003-02-18 | 2004-08-19 | Samsung Electronics Co., Ltd. | Method and apparatus for processing pixels based on segments |
US6999105B2 (en) * | 2003-12-04 | 2006-02-14 | International Business Machines Corporation | Image scaling employing horizontal partitioning |
US7251271B1 (en) * | 1999-09-07 | 2007-07-31 | Telefonaktiebolaget Lm Ericsson (Publ) | Digital filter design |
US7376908B2 (en) * | 2003-03-24 | 2008-05-20 | Microsoft Corporation | On-screen display image rendered with MPEG hardware |
US7701472B2 (en) * | 2004-06-08 | 2010-04-20 | Seiko Epson Corporation | Display controller, electronic device, and method of supplying image data |
Family Cites Families (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5469223A (en) * | 1993-10-13 | 1995-11-21 | Auravision Corporation | Shared line buffer architecture for a video processing circuit |
JP3510997B2 (en) * | 1999-04-05 | 2004-03-29 | セイコーエプソン株式会社 | Image processing device and copy system |
JP2000351242A (en) * | 1999-04-05 | 2000-12-19 | Seiko Epson Corp | Image forming apparatus |
FR2797979B1 (en) * | 1999-08-24 | 2002-05-24 | St Microelectronics Sa | ANTI-FLICKER FILTERING METHOD AND SYSTEM |
US6642962B1 (en) * | 1999-09-01 | 2003-11-04 | Neomagic Corp. | Merged pipeline for color interpolation and edge enhancement of digital images |
US7184059B1 (en) * | 2000-08-23 | 2007-02-27 | Nintendo Co., Ltd. | Graphics system with copy out conversions between embedded frame buffer and main memory |
KR100423503B1 (en) * | 2001-09-14 | 2004-03-18 | 삼성전자주식회사 | Apparatus for disposing digital image and a method using the same |
US6765622B2 (en) * | 2001-10-26 | 2004-07-20 | Koninklijke Philips Electronics N.V. | Line-buffer reuse in vertical pixel-processing arrangement |
JP3862621B2 (en) * | 2002-06-28 | 2006-12-27 | キヤノン株式会社 | Image processing apparatus, image processing method, and program thereof |
US7266255B1 (en) * | 2003-09-26 | 2007-09-04 | Sun Microsystems, Inc. | Distributed multi-sample convolution |
JP2005260875A (en) * | 2004-03-15 | 2005-09-22 | Yokowo Co Ltd | Surface mounted patch antenna and its mounting method |
JP2005311745A (en) * | 2004-04-22 | 2005-11-04 | Olympus Corp | Image processor |
US7515766B2 (en) * | 2004-09-22 | 2009-04-07 | Intel Corporation | Apparatus and method for edge handling in image processing |
US20060092320A1 (en) * | 2004-10-29 | 2006-05-04 | Nickerson Brian R | Transferring a video frame from memory into an on-chip buffer for video processing |
JP4219887B2 (en) * | 2004-12-28 | 2009-02-04 | 富士通マイクロエレクトロニクス株式会社 | Image processing apparatus and image processing method |
TWI260547B (en) * | 2005-03-07 | 2006-08-21 | Li Shing Comm Science And Tech | Readable tag having dual reading function |
-
2006
- 2006-08-25 US US11/510,045 patent/US7834873B2/en not_active Expired - Fee Related
-
2007
- 2007-08-16 EP EP20070841000 patent/EP2054794A4/en not_active Ceased
- 2007-08-16 WO PCT/US2007/076089 patent/WO2008024668A1/en active Application Filing
- 2007-08-16 JP JP2009522036A patent/JP2009545085A/en active Pending
- 2007-08-16 CN CN2007800278655A patent/CN101495952B/en not_active Expired - Fee Related
- 2007-08-20 TW TW096130710A patent/TWI381364B/en not_active IP Right Cessation
- 2007-08-20 TW TW101134121A patent/TWI494916B/en not_active IP Right Cessation
-
2010
- 2010-10-21 US US12/909,248 patent/US20110037771A1/en not_active Abandoned
Patent Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6400782B2 (en) * | 1998-09-03 | 2002-06-04 | Conexant Systems, Inc. | Method of frequency domain filtering employing a real to analytic transform |
US6327000B1 (en) * | 1999-04-02 | 2001-12-04 | Teralogic, Inc. | Efficient image scaling for scan rate conversion |
US6457075B1 (en) * | 1999-05-17 | 2002-09-24 | Koninkijke Philips Electronics N.V. | Synchronous memory system with automatic burst mode switching as a function of the selected bus master |
US7251271B1 (en) * | 1999-09-07 | 2007-07-31 | Telefonaktiebolaget Lm Ericsson (Publ) | Digital filter design |
US6625629B1 (en) * | 2000-05-23 | 2003-09-23 | Microsoft Corporation | System and method for signal processing using an improved convolution technique |
US20040130553A1 (en) * | 2002-12-26 | 2004-07-08 | Canon Kabushiki Kaisha | Image processing apparatus and image processing method |
US20040160452A1 (en) * | 2003-02-18 | 2004-08-19 | Samsung Electronics Co., Ltd. | Method and apparatus for processing pixels based on segments |
US7376908B2 (en) * | 2003-03-24 | 2008-05-20 | Microsoft Corporation | On-screen display image rendered with MPEG hardware |
US6999105B2 (en) * | 2003-12-04 | 2006-02-14 | International Business Machines Corporation | Image scaling employing horizontal partitioning |
US7701472B2 (en) * | 2004-06-08 | 2010-04-20 | Seiko Epson Corporation | Display controller, electronic device, and method of supplying image data |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11042962B2 (en) * | 2016-04-18 | 2021-06-22 | Avago Technologies International Sales Pte. Limited | Hardware optimisation for generating 360° images |
Also Published As
Publication number | Publication date |
---|---|
JP2009545085A (en) | 2009-12-17 |
EP2054794A1 (en) | 2009-05-06 |
TW201308309A (en) | 2013-02-16 |
TWI381364B (en) | 2013-01-01 |
TWI494916B (en) | 2015-08-01 |
EP2054794A4 (en) | 2011-09-07 |
CN101495952A (en) | 2009-07-29 |
US20080049037A1 (en) | 2008-02-28 |
CN101495952B (en) | 2012-04-18 |
TW200820213A (en) | 2008-05-01 |
WO2008024668A1 (en) | 2008-02-28 |
US7834873B2 (en) | 2010-11-16 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7834873B2 (en) | Display processing line buffers incorporating pipeline overlap | |
WO2015180371A1 (en) | Display device brightness compensation method, brightness compensation apparatus, and display device | |
US7492371B2 (en) | Hardware animation of a bouncing image | |
US20080012874A1 (en) | Dynamic selection of high-performance pixel shader code based on check of restrictions | |
WO2015180336A1 (en) | Display device brightness compensation method, brightness compensation apparatus, and display device | |
US10347220B1 (en) | Data compression and decompression method for DeMura table | |
US10936943B2 (en) | Providing flexible matrix processors for performing neural network convolution in matrix-processor-based devices | |
US7742636B2 (en) | Method and apparatus for scaling down a bayer domain image | |
US20220084159A1 (en) | Image signal processor, method of operating the image signal processor, and application processor including the image signal processor | |
US10482374B1 (en) | Ensemble learning based image classification systems | |
US11710213B2 (en) | Application processor including reconfigurable scaler and devices including the processor | |
JP5703769B2 (en) | Image conversion apparatus, image conversion method, program, and electronic apparatus | |
US10559244B2 (en) | Electronic apparatus, display driver and method for generating display data of display panel | |
US10290132B2 (en) | Graphics processing | |
US10402946B2 (en) | System and method for performing orthogonal rotation and mirroring operation in a device | |
US20070040844A1 (en) | Reducing memory bandwidth to texture samplers via re-interpolation of texture coordinates | |
US7936359B2 (en) | Reconfigurable floating point filter | |
US8982134B2 (en) | Area-based dependency chain analysis of shaders and command stream | |
US8860739B2 (en) | Method and device for processing digital images | |
US10255659B2 (en) | Method of and apparatus for scaling data arrays | |
WO2020167360A1 (en) | Image processor complex transfer functions | |
JP2013195963A (en) | Image processing device, integrated circuit apparatus, and image display system | |
JP3533197B2 (en) | Image processing device | |
US20180090110A1 (en) | Apparatus and method for video frame rotation | |
US20070286518A1 (en) | Image processing apparatus and image processing method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |