US20160249049A1 - Performance of the JPEG Estimated Spectrum Adaptive Postfilter (JPEG-ESAP) for Low Bit Rates - Google Patents
Performance of the JPEG Estimated Spectrum Adaptive Postfilter (JPEG-ESAP) for Low Bit Rates Download PDFInfo
- Publication number
- US20160249049A1 US20160249049A1 US14/810,959 US201514810959A US2016249049A1 US 20160249049 A1 US20160249049 A1 US 20160249049A1 US 201514810959 A US201514810959 A US 201514810959A US 2016249049 A1 US2016249049 A1 US 2016249049A1
- Authority
- US
- United States
- Prior art keywords
- esap
- jpeg
- ppm
- images
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000003044 adaptive effect Effects 0.000 title claims description 8
- 238000001228 spectrum Methods 0.000 title claims description 6
- 238000000034 method Methods 0.000 claims description 27
- 238000004590 computer program Methods 0.000 claims description 18
- 230000000903 blocking effect Effects 0.000 claims description 12
- 230000006835 compression Effects 0.000 claims description 11
- 238000007906 compression Methods 0.000 claims description 11
- 238000013139 quantization Methods 0.000 claims description 9
- 239000003086 colorant Substances 0.000 claims description 3
- 238000004422 calculation algorithm Methods 0.000 abstract description 13
- 238000001914 filtration Methods 0.000 abstract description 7
- 238000012545 processing Methods 0.000 description 16
- 230000008901 benefit Effects 0.000 description 11
- 230000008569 process Effects 0.000 description 8
- 230000000007 visual effect Effects 0.000 description 8
- 235000002566 Capsicum Nutrition 0.000 description 6
- 241000282537 Mandrillus sphinx Species 0.000 description 6
- 241000758706 Piperaceae Species 0.000 description 6
- 238000004891 communication Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 230000006872 improvement Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 238000012805 post-processing Methods 0.000 description 3
- 238000003491 array Methods 0.000 description 2
- 230000000052 comparative effect Effects 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000002457 bidirectional effect Effects 0.000 description 1
- 230000001364 causal effect Effects 0.000 description 1
- 238000005056 compaction Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 235000019800 disodium phosphate Nutrition 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000012804 iterative process Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000007723 transport mechanism Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/117—Filters, e.g. for pre-processing or post-processing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/124—Quantisation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/136—Incoming video signal characteristics or properties
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/177—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a group of pictures [GOP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/625—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using discrete cosine transform [DCT]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
- H04N19/86—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving reduction of coding artifacts, e.g. of blockiness
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
- H04N19/91—Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
Definitions
- the present invention generally relates to image enhancement, and more particularly, to frequency-based, pixel-adaptive filtering for low bit rates for JPEG format color images.
- JPEG a lossy image compression standard based on the discrete cosine transform (DCT) was introduced in 1992.
- JPEG is firmly entrenched in digital cameras (e.g., NikonTM), smart phones (e.g., Android RAZRTM), and web browsers (e.g., Internet ExplorerTM, FirefoxTM, etc.). Additionally, JPEG is the basis of today's MPEG-2 high definition television (HDTV) digital satellite news gathering (DSNG) broadcast industry and H.262 video compression.
- HDTV high definition television
- DSNG digital satellite news gathering
- JPEG-2000 (i.e., JP2), also a lossy image compression standard, but based on the discrete wavelet transform (DWT), was published in 1999.
- JPEG-2000 has been a JPEG-competing standard since its inception. In 2005, the U.S. Library of Congress embraced JPEG-2000 for permanent electronic storage of digitized images.
- JPEG-2000 is also supported by other smart phone brands (e.g., Apple iPhoneTM)
- the estimated spectrum adaptive postfilter reuses the DCT coefficients of a decoded grayscale JPEG image to postfilter and enhance the image.
- ESAP pixel-adaptive filters minimize the JPEG 8 ⁇ 8 blocking distortion without requiring any additional bit stream overhead.
- a key improvement is that ESAP not only adaptively filters out the JPEG blocking distortion, but also does not introduce the ringing artifacts inherent to JPEG-2000.
- ESAP complies with the Independent JPEG Group (IJG) SoftwareTM, Release 8c, found at http://www.ijg.org.
- Baseline JPEG uses Huffman entropy coding, whereas JPEG-2000 and ESAP-AC both use arithmetic entropy coding (AC).
- JPEG-2000 Most of the quality improvement of JPEG-2000 over baseline JPEG images is likely due to the arithmetic entropy coding utilized by JPEG-2000 versus the Huffman coding initially used by JPEG. Similarly, when JPEG uses AC instead of Hoffman coding, as in the IJG SoftwareTM command example:
- the ESAP-AC image visual and peak signal-to-noise ratio (PSNR) quality either matches or surpasses the quality of JPEG-2000.
- PSNR peak signal-to-noise ratio
- ESAP in its most recent version, operates in the RGB color image space.
- Certain embodiments of the present invention may provide solutions to the problems and needs in the art that have not yet been fully identified, appreciated, or solved by conventional image processing algorithms.
- some embodiments of the present invention pertain to using JPEG-ESAP for frequency-based, pixel-adaptive filtering for low bit rates (i.e., less than 0.25 bits/pixel) for JPEG format color images.
- a computer-implemented method includes creating JPEG quantization coefficients, by a computing system, for each respective color plane of a plurality of color planes.
- the computer-implemented method also includes JPEG decoding each color plane, by the computing system, to generate YCbCr DCT coefficients for ESAP postfiltering.
- the computer-implemented method further includes ESAP postfiltering each of the color planes, by the computing system, and outputting a JPEG image file and an ESAP postfiltered image file, by the computing system.
- a computer program is embodied on a non-transitory computer-readable medium.
- the program configured to cause at least one processor to convert an input image to a portable pix map (PPM) format and decompose the PPM image into color planes in portable graymap (PGM) format.
- the computer program is also configured to cause the at least one processor to create JPEG quantization coefficients for each respective color plane, JPEG decode each color plane to generate YCbCr DCT for ESAP postfiltering and ESAP postfilter each color plane.
- the computer program is further configured to cause the at least one processor to combine PGM JPEG planes for each color, combine PGM ESAP planes for each color into PPM, and output a JPEG image file and an ESAP postfiltered image file.
- an apparatus in yet another embodiment, includes memory storing computer program instructions and at least one processor configured to execute the computer program instructions stored in the memory.
- the at least one processor is configured to receive a video comprising a series of images. For each of the received images, the at least one processor is also configured to create JPEG quantization coefficients for each respective color plane of a plurality of color planes, JPEG decode each color plane to generate YCbCr DCT coefficients for ESAP postfiltering, ESAP postfilter each of the color planes, and output a JPEG image file and an ESAP postfiltered image file.
- the at least one processor is further configured to output a sequence of PPM images and re-code the sequence of PPM images into an enhanced blocking distortion-minimized MPEG-2 or MPEG-4 video sequence.
- FIG. 1 is a block diagram illustrating the grayscale ESAP decoder algorithm.
- FIG. 2 illustrates a series of Lena images showing output of the 2-D, 1:8 ESAP frequency up sampler for the blue plane, according to an embodiment of the present invention.
- FIG. 3 illustrates the Lena, peppers, airplane, Barbara, and mandrill images referred to in Table 2 that were processed according to an embodiment of the present invention.
- FIG. 4 is a flowchart illustrating a process for improved JPEG-ESAP processing of color images, according to an embodiment of the present invention.
- FIG. 5 is a block diagram of a computing system configured to perform improved JPEG-ESAP processing of color images, according to an embodiment of the present invention.
- Some embodiments of the present invention pertain to using JPEG-ESAP for frequency-based, pixel-adaptive filtering for low bit rates (i.e., less than 0.25 bits/pixel) for JPEG format color images.
- RGB for example, an image is decomposed into three color bands—red, green, and blue.
- other bands may be used in certain embodiments, for instance, RGBY.
- the JPEG-ESAP algorithm is then applied for each band (e.g., once for red, once for green, and once for blue in RGB) and the output of each application of the algorithm is rebuilt as a color image by combining the ESAP filtered RGB planes (for RGB schemes) in grayscale portable graymap (PGM) format into color portable pix map (PPM) format.
- PGM grayscale portable graymap
- PPM color portable pix map
- ESAP estimates two-dimensional (2-D) pixel-adaptive bandwidths directly from the dequantized DCT coefficients without requiring any additional overhead.
- the ESAP decoder algorithm 100 that is applied to each color plane is shown in FIG. 1 .
- FIG. 1 For more detail on a grayscale implementation, see Linares I., Mersereau R. M., and Smith M. J. T., “ JPEG Estimated Spectrum Adaptive Postfilter Using Image - Adaptive Q - Tables and Canny Edge Detectors ,” ISCAS '96 International Symposium on Circuits and Systems, IEEE, Atlanta, Ga., pp. 722-725, Vol. 2, May 12, 1996, as well as U.S. Pat. No. 6,760,487.
- Each DCT block 2-D bandwidth is vertically and horizontally separable and determined from Table 1 below.
- FIG. 2 illustrates a series of images 200 showing output of the 2-D, 1:8 ESAP frequency up sampler for the blue plane, according to an embodiment of the present invention.
- Original image 210 is in color, but is shown in grayscale here.
- the 2-D interpolated vertical frequency (IVF) image 220 and the interpolated horizontal frequency (IHF) image 230 which are grayscale, show the interpolated DCT bandwidths w[m] scaled for 8-BPP display.
- the IVF and IHF then drive a separable, pixel-adaptive, non-causal 2-D h m [n] Hamming window low pass filter that is applied in a raster manner to each pixel and its surrounding 17 ⁇ 17 area of support.
- This filtering minimizes the DCT blocking while performing negligible filtering in the high frequency areas of the image consisting of edges or highly textured areas.
- This process takes advantage of the ability of the human visual system to mask the JPEG quantization noise present in the high frequency areas.
- JPEG, ESAP-AC, and JPEG-2000 based on the processed color test images include, but are not limited to, the following: (1) the ESAP-AC PSNR is always higher than the DWT-based JPEG-2000 images compressed at similar bit rates, for typical test images; (2) ESAP-AC performs equally to or better than JPEG-2000 images; (3) ESAP visual quality is better than JPEG-2000 for high frequency content images; (4) ESAP visual quality is comparable to JPEG-2000 for low frequency content images, and the arithmetic coding entropy option for JPEG should be used rather than the baseline Huffman coding option; (5) JPEG AC provides a better quality image than JPEG Huffman coding at comparable bit rates; (6) ESAP-AC should be turned on for bit rates below 0.5 BPP; (7) for bit rates higher than 0.5 BPP, JPEG images should only choose to use JPEG-AC without ESAP post-processing; (8) ESAP may
- An MPEG-2 Group of Pictures is a sequence of JPEG-like, DCT coded intraframes (I), bidirectional (B) frames, and forward predicted (P) frames. All types of frames may be coded with JPEG-AC, and then ESAP may be applied to filter the decoded frames.
- ESAP can also remove blocking artifacts in MPEG-2 GOP frames.
- frame sequences extracted from MPEG-2 video using ffmpeg software, then compressed at half or a third of their original bit rate using ffmpeg, and in parallel, also post-processed with ESAP-AC before reassembling into an ESAP-AC ffmpeg video sequence show significant video improvement, with imperceptible or minimized DCT blocking over their respective 2 ⁇ and 3 ⁇ compressed MPEG-2 sequences.
- ffmpeg video experiments it was verified that the ESAP-AC MPEG-2 video sequences compare very well in visual quality against their respective MPEG-4 sequences coded at the same 2 ⁇ and 3 ⁇ video compression ratios.
- ESAP-AC could make 50% of the digital bandwidth reduction available to approximately double the number of digital MPEG-2 HDTV terrestrial channels over the same bandwidth without changing the broadcasting infrastructure; (2) ESAP-AC could also allow the extra 50% bandwidth to seamlessly integrate, multiplex, and provide UHDTV MPEG-4 content alongside HDTV MPEG-2 over the same digital pipe for dual, HDTV-UHDTV compatible receiver sets; (3) decoupled coexistence of both HDTV standards should provide many more choices for commercial video providers and consumers; and (4) Earth orbiting and planetary spacecraft video would certainly benefit from lower bit rates as well that could improve the SNR of the received signal.
- the ESAP algorithm may be applied to RGB color images.
- the ESAP-AC PSNR is always higher than default settings JPEG-2000 images compressed at similar bit rates.
- ESAP-AC post-processing yields better PSNRs than default settings compressed JPEG-2000 images due to ESAP's excellent ability to remove DCT blocking artifacts without introducing the DWT ringing inherent to JPEG-2000.
- An embodiment of the ESAP algorithm may be implemented in the IJG SoftwareTM discussed above using the cjpeg -arithmetic option and the ImageMagickTM convert utility, which can be found at http://www.imagemagick.org.
- the pseudocode is shown in the esap_script below:
- esap_script image.fmt QF esap_img.ppm // The input image format “fmt” is any of: // PPM, PGM, GIF, JPG, TIFF, BMP, etc.
- the JPEG quality factor “QF” // is a [0..100] integer.
- the output is the filtered esap_img.ppm image. // 1. Convert input image to a portable pix map (PPM) format, // and then decompose into RGB planes in portable graymap (PGM) format. convert image.fmt image.ppm ppm2rgb image.ppm image.r.pgm image.g.pgm image.b.pgm // 2.
- the PSNR of an RGB image is defined as:
- PSNR RGB 10*log 10 (255 2 /(MSE R +MSE G +MSE B )/3) (1)
- MSE mean-square-error
- FIG. 4 is a flowchart 400 illustrating a process for improved JPEG-ESAP processing of color images, according to an embodiment of the present invention.
- the process begins with converting an input image to a PPM format at 405 and then decomposing the ppm image into color planes in PGM format at 410 .
- JPEG quantization coefficients are created for each respective color plane at 415 .
- the colors of the color planes may be RGB, RGBY, or any other suitable color scheme.
- Each color plane is JPEG decoded at 420 and ESAP postfiltered at 425 .
- the decoder generates the YCbCr DCT coefficients necessary to perform ESAP postfiltering.
- the PGM JPEG planes for each color are combined at 430 and the PGM ESAP planes for each color are combined into PPM at 435 .
- Grayscale MSEs are computed for each color plane at 440 and the results are output to files at 445 .
- the purpose of this output is to compute a single color PSNR metric for each decoded and each postfiltered color image.
- the PSNRs are computed for the JPEG and ESAP images at 450 and the PPM CF X:1 ratios and BPPs are computed at 455 .
- the results of the computations are then output to file at 460 .
- the purpose of this output is to gather and record PSNR performance metrics as a function of the compression ratio (or equivalently, as a function of the BPP rate) for each processed image.
- the final output of the color JPEG-ESAP processing is a JPEG image file and an ESAP postfiltered image file, which are created at 465 .
- Baseline Huffman entropy coding or alternatively, arithmetic entropy coding may be used for the creation of both images.
- performance metrics may be computed and appended to a file for each processed image.
- video such as MPEG-2
- the process may be repeated for multiple frames of the video or all frames of the video, as shown by the dashed line.
- the final output of MPEG-2 ESAP video processing is a sequence of PPM images that can be re-coded into an enhanced blocking distortion-minimized MPEG-2 or MPEG-4 video sequence at 470 .
- FIG. 5 is a block diagram of a computing system 500 configured to perform improved JPEG-ESAP processing of color images, according to an embodiment of the present invention.
- Computing system 500 includes a bus 505 or other communication mechanism for communicating information, and processor(s) 510 coupled to bus 505 for processing information.
- Processor(s) 510 may be any type of general or specific purpose processor, including a central processing unit (“CPU”) or application specific integrated circuit (“ASIC”).
- Processor(s) 510 may also have multiple processing cores, and at least some of the cores may be configured to perform specific functions. Multi-parallel processing may be used in some embodiments.
- Computing system 500 further includes a memory 515 for storing information and instructions to be executed by processor(s) 510 .
- Memory 515 can be comprised of any combination of random access memory (RAM), read only memory (ROM), flash memory, cache, static storage such as a magnetic or optical disk, or any other types of non-transitory computer-readable media or combinations thereof.
- computing system 500 includes a communication device 520 , such as a transceiver and antenna, to wirelessly provide access to a communications network.
- Non-transitory computer-readable media may be any available media that can be accessed by processor(s) 510 and may include both volatile and non-volatile media, removable and non-removable media, and communication media.
- Communication media may include computer-readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.
- Processor(s) 510 are further coupled via bus 505 to a display 525 , such as a Liquid Crystal Display (LCD), for displaying information to a user.
- a display 525 such as a Liquid Crystal Display (LCD)
- LCD Liquid Crystal Display
- a keyboard 530 and a cursor control device 535 are further coupled to bus 505 to enable a user to interface with computing system.
- a physical keyboard and mouse may not be present, and the user may interact with the device solely through display 525 and/or a touchpad (not shown). Any type and combination of input devices may be used as a matter of design choice.
- Memory 515 stores software modules that provide functionality when executed by processor(s) 510 .
- the modules include an operating system 540 for computing system 500 .
- the modules further include an improved JPEG-ESAP module 545 that is configured to perform JPEG-ESAP processing of color images.
- Computing system 500 may include one or more additional functional modules 550 that include additional functionality.
- a “system” could be embodied as an embedded computing system, a personal computer, a server, a console, a personal digital assistant (PDA), a cell phone, a tablet computing device, a cable TV set top box, an HDTV or UHDTV set, a digital video disk (DVD) player, a Blu-ray player, or any other suitable computing device, or combination of devices.
- PDA personal digital assistant
- Presenting the above-described functions as being performed by a “system” is not intended to limit the scope of the present invention in any way, but is intended to provide one example of many embodiments of the present invention. Indeed, methods, systems and apparatuses disclosed herein may be implemented in localized and distributed forms consistent with computing technology, including cloud computing systems.
- modules may be implemented as a hardware circuit comprising custom very large scale integration (“VLSI”) circuits or gate arrays, off-the-shelf semiconductors such as logic chips, transistors, or other discrete components.
- VLSI very large scale integration
- a module may also be implemented in programmable hardware devices such as field programmable gate arrays, programmable array logic, programmable logic devices, graphics processing units, or the like.
- a module may also be at least partially implemented in software for execution by various types of processors.
- An identified unit of executable code may, for instance, comprise one or more physical or logical blocks of computer instructions that may, for instance, be organized as an object, procedure, or function. Nevertheless, the executables of an identified module need not be physically located together, but may comprise disparate instructions stored in different locations which, when joined logically together, comprise the module and achieve the stated purpose for the module.
- modules may be stored on a computer-readable medium, which may be, for instance, a hard disk drive, flash device, RAM, tape, or any other such medium used to store data.
- a module of executable code could be a single instruction, or many instructions, and may even be distributed over several different code segments, among different programs, and across several memory devices.
- operational data may be identified and illustrated herein within modules, and may be embodied in any suitable form and organized within any suitable type of data structure. The operational data may be collected as a single data set, or may be distributed over different locations including over different storage devices, and may exist, at least partially, merely as electronic signals on a system or network.
- the process steps performed in FIG. 4 may be performed by a computer program, encoding instructions for the nonlinear adaptive processor to perform at least the process described in FIG. 4 , in accordance with embodiments of the present invention.
- the computer program may be embodied on a non-transitory computer-readable medium.
- the computer-readable medium may be, but is not limited to, a hard disk drive, a flash device, a random access memory, a tape, or any other such medium used to store data.
- the computer program may include encoded instructions for controlling the nonlinear adaptive processor to implement the process described in FIG. 4 , which may also be stored on the computer-readable medium.
- the computer program can be implemented in hardware, software, or a hybrid implementation.
- the computer program can be composed of modules that are in operative communication with one another, and which are designed to pass information or instructions to display.
- the computer program can be configured to operate on a general purpose computer, or an ASIC.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- Discrete Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
Description
- The invention described herein was made by employees of the United States Government and may be manufactured and used by or for the Government for Government purposes without the payment of any royalties thereon or therefore.
- The present invention generally relates to image enhancement, and more particularly, to frequency-based, pixel-adaptive filtering for low bit rates for JPEG format color images.
- JPEG, a lossy image compression standard based on the discrete cosine transform (DCT), was introduced in 1992. JPEG is firmly entrenched in digital cameras (e.g., Nikon™), smart phones (e.g., Android RAZR™), and web browsers (e.g., Internet Explorer™, Firefox™, etc.). Additionally, JPEG is the basis of today's MPEG-2 high definition television (HDTV) digital satellite news gathering (DSNG) broadcast industry and H.262 video compression.
- JPEG-2000 (i.e., JP2), also a lossy image compression standard, but based on the discrete wavelet transform (DWT), was published in 1999. JPEG-2000 has been a JPEG-competing standard since its inception. In 2005, the U.S. Library of Congress embraced JPEG-2000 for permanent electronic storage of digitized images. JPEG-2000 is also supported by other smart phone brands (e.g., Apple iPhone™)
- The estimated spectrum adaptive postfilter (ESAP) reuses the DCT coefficients of a decoded grayscale JPEG image to postfilter and enhance the image. ESAP pixel-adaptive filters minimize the
JPEG 8×8 blocking distortion without requiring any additional bit stream overhead. A key improvement is that ESAP not only adaptively filters out the JPEG blocking distortion, but also does not introduce the ringing artifacts inherent to JPEG-2000. ESAP complies with the Independent JPEG Group (IJG) Software™, Release 8c, found at http://www.ijg.org. Baseline JPEG uses Huffman entropy coding, whereas JPEG-2000 and ESAP-AC both use arithmetic entropy coding (AC). - Most of the quality improvement of JPEG-2000 over baseline JPEG images is likely due to the arithmetic entropy coding utilized by JPEG-2000 versus the Huffman coding initially used by JPEG. Similarly, when JPEG uses AC instead of Hoffman coding, as in the IJG Software™ command example:
-
- cjpeg -quality 5 -arithmetic img.ppm img.jpg,
- a remarkable visual improvement of the JPEG image is observed. When the image is further filtered by the pixel-adaptive ESAP process, the ESAP-AC image visual and peak signal-to-noise ratio (PSNR) quality either matches or surpasses the quality of JPEG-2000. The excellent DCT energy compaction properties are thus exulted with usage of AC.
- A previous version of the ESAP algorithm was applied to grayscale images only. ESAP, in its most recent version, operates in the RGB color image space.
- Certain embodiments of the present invention may provide solutions to the problems and needs in the art that have not yet been fully identified, appreciated, or solved by conventional image processing algorithms. For example, some embodiments of the present invention pertain to using JPEG-ESAP for frequency-based, pixel-adaptive filtering for low bit rates (i.e., less than 0.25 bits/pixel) for JPEG format color images.
- In an embodiment, a computer-implemented method includes creating JPEG quantization coefficients, by a computing system, for each respective color plane of a plurality of color planes. The computer-implemented method also includes JPEG decoding each color plane, by the computing system, to generate YCbCr DCT coefficients for ESAP postfiltering. The computer-implemented method further includes ESAP postfiltering each of the color planes, by the computing system, and outputting a JPEG image file and an ESAP postfiltered image file, by the computing system.
- In another embodiment, a computer program is embodied on a non-transitory computer-readable medium. The program configured to cause at least one processor to convert an input image to a portable pix map (PPM) format and decompose the PPM image into color planes in portable graymap (PGM) format. The computer program is also configured to cause the at least one processor to create JPEG quantization coefficients for each respective color plane, JPEG decode each color plane to generate YCbCr DCT for ESAP postfiltering and ESAP postfilter each color plane. The computer program is further configured to cause the at least one processor to combine PGM JPEG planes for each color, combine PGM ESAP planes for each color into PPM, and output a JPEG image file and an ESAP postfiltered image file.
- In yet another embodiment, an apparatus includes memory storing computer program instructions and at least one processor configured to execute the computer program instructions stored in the memory. The at least one processor is configured to receive a video comprising a series of images. For each of the received images, the at least one processor is also configured to create JPEG quantization coefficients for each respective color plane of a plurality of color planes, JPEG decode each color plane to generate YCbCr DCT coefficients for ESAP postfiltering, ESAP postfilter each of the color planes, and output a JPEG image file and an ESAP postfiltered image file. The at least one processor is further configured to output a sequence of PPM images and re-code the sequence of PPM images into an enhanced blocking distortion-minimized MPEG-2 or MPEG-4 video sequence.
- In order that the advantages of certain embodiments of the invention will be readily understood, a more particular description of the invention briefly described above will be rendered by reference to specific embodiments that are illustrated in the appended drawings. While it should be understood that these drawings depict only typical embodiments of the invention and are not therefore to be considered to be limiting of its scope, the invention will be described and explained with additional specificity and detail through the use of the accompanying drawings, in which:
-
FIG. 1 is a block diagram illustrating the grayscale ESAP decoder algorithm. -
FIG. 2 illustrates a series of Lena images showing output of the 2-D, 1:8 ESAP frequency up sampler for the blue plane, according to an embodiment of the present invention. -
FIG. 3 illustrates the Lena, peppers, airplane, Barbara, and mandrill images referred to in Table 2 that were processed according to an embodiment of the present invention. -
FIG. 4 is a flowchart illustrating a process for improved JPEG-ESAP processing of color images, according to an embodiment of the present invention. -
FIG. 5 is a block diagram of a computing system configured to perform improved JPEG-ESAP processing of color images, according to an embodiment of the present invention. - Some embodiments of the present invention pertain to using JPEG-ESAP for frequency-based, pixel-adaptive filtering for low bit rates (i.e., less than 0.25 bits/pixel) for JPEG format color images. For RGB, for example, an image is decomposed into three color bands—red, green, and blue. However, other bands may be used in certain embodiments, for instance, RGBY. The JPEG-ESAP algorithm is then applied for each band (e.g., once for red, once for green, and once for blue in RGB) and the output of each application of the algorithm is rebuilt as a color image by combining the ESAP filtered RGB planes (for RGB schemes) in grayscale portable graymap (PGM) format into color portable pix map (PPM) format.
- ESAP estimates two-dimensional (2-D) pixel-adaptive bandwidths directly from the dequantized DCT coefficients without requiring any additional overhead. The
ESAP decoder algorithm 100 that is applied to each color plane is shown inFIG. 1 . For more detail on a grayscale implementation, see Linares I., Mersereau R. M., and Smith M. J. T., “JPEG Estimated Spectrum Adaptive Postfilter Using Image-Adaptive Q-Tables and Canny Edge Detectors,” ISCAS '96 International Symposium on Circuits and Systems, IEEE, Atlanta, Ga., pp. 722-725, Vol. 2, May 12, 1996, as well as U.S. Pat. No. 6,760,487. - A simple, one-pass, non-iterative process is used for w [m] bandwidth estimation and for ESAP hm[n] filtering. Each DCT block 2-D bandwidth is vertically and horizontally separable and determined from Table 1 below.
-
Highest DCT Coefficient Normalized 1-D Bandwidth c0 0.125π c1 0.250π c2 0.375π c3 0.500π c4 0.625π c5 0.750π c6 0.875π c7 1.000π -
FIG. 2 illustrates a series ofimages 200 showing output of the 2-D, 1:8 ESAP frequency up sampler for the blue plane, according to an embodiment of the present invention.Original image 210 is in color, but is shown in grayscale here. The 2-D interpolated vertical frequency (IVF)image 220 and the interpolated horizontal frequency (IHF)image 230, which are grayscale, show the interpolated DCT bandwidths w[m] scaled for 8-BPP display. - The IVF and IHF then drive a separable, pixel-adaptive, non-causal 2-D hm[n] Hamming window low pass filter that is applied in a raster manner to each pixel and its surrounding 17×17 area of support. This filtering minimizes the DCT blocking while performing negligible filtering in the high frequency areas of the image consisting of edges or highly textured areas. This process takes advantage of the ability of the human visual system to mask the JPEG quantization noise present in the high frequency areas.
- Some experimental observations regarding JPEG, ESAP-AC, and JPEG-2000 based on the processed color test images include, but are not limited to, the following: (1) the ESAP-AC PSNR is always higher than the DWT-based JPEG-2000 images compressed at similar bit rates, for typical test images; (2) ESAP-AC performs equally to or better than JPEG-2000 images; (3) ESAP visual quality is better than JPEG-2000 for high frequency content images; (4) ESAP visual quality is comparable to JPEG-2000 for low frequency content images, and the arithmetic coding entropy option for JPEG should be used rather than the baseline Huffman coding option; (5) JPEG AC provides a better quality image than JPEG Huffman coding at comparable bit rates; (6) ESAP-AC should be turned on for bit rates below 0.5 BPP; (7) for bit rates higher than 0.5 BPP, JPEG images should only choose to use JPEG-AC without ESAP post-processing; (8) ESAP may be implemented in fast electronics, such as FPGAs, ASICs, or DSPs, which could reduce the processing time by several orders of magnitude; (9) ESAP is an optional post-processing step at the receiver or decoder fully capable of removing the JPEG/MPEG-2 blocking distortion; and (10) ESAP does not increase the coded stream bit rate.
- Some embodiments are also potentially applicable to MPEG-2 video. An MPEG-2 Group of Pictures (GOP) is a sequence of JPEG-like, DCT coded intraframes (I), bidirectional (B) frames, and forward predicted (P) frames. All types of frames may be coded with JPEG-AC, and then ESAP may be applied to filter the decoded frames. A typical MPEG-2 CCIR-601 GOP sequence with an I-frame distance of 15 and a P-frame distance of 3, such as IBBPBBPBBPBBPBB, has an I-frame bit rate of about 1.2 BPP, a P-frame bit rate of 0.6 BPP, and a B-frame bit rate of 0.23 BPP. The equivalent average bit rate for this GOP sequence is 0.38 BPP.
- By extension to each frame, applying JPEG-AC and ESAP to MPEG-2 frames in a GOP sequence could reduce the MPEG-2 bit rate by a factor of two without detrimentally affecting the video quality, and while retaining an equivalent video quality at an approximate average bit rate of 0.19 BPP. Comparative objective PSNR among JPEG, ESAP, and JPEG-2000 versions of a series of
images 300 at 0.20, 0.25, 0.50, and 1.00 BPP is shown in Table 2 below with respect to theLena 310,peppers 320,airplane 330,Barbara 340, andmandrill 350 images. -
COMPARATIVE OBJECTIVE PSNR AMONG JPEG, ESAP, AND JPEG-2000 IMAGES Δ Base- Δ PSNR PSNR line ESAP- ESAP- Δ PSNR ESAP JPEG AC AC over JP2 JP2 over over PSNR PSNR JPEG PSNR JPEG JP2 Image @ 0.20 BPP Lena 26.15 30.88 4.72 29.41 3.26 1.47 Peppers 24.34 29.62 5.28 27.74 3.41 1.88 Airplane 24.08 30.45 6.38 28.70 4.63 1.75 Barbara 22.19 25.59 3.40 25.41 3.22 0.18 Mandrill 19.76 21.55 1.79 20.88 1.13 0.66 Image @ 0.25 BPP Lena 28.55 31.65 3.10 30.11 1.56 1.54 Peppers 27.59 30.48 2.89 28.44 0.85 2.04 Airplane 27.69 31.55 3.87 29.55 1.86 2.01 Barbara 23.91 26.47 2.56 26.36 2.46 0.10 Mandrill 20.52 22.00 1.48 21.33 0.81 0.67 Image @ 0.50 BPP Lena 32.84 33.87 1.03 32.39 −0.55 1.58 Peppers 31.76 32.43 0.67 30.70 −1.06 1.74 Airplane 33.05 34.77 1.72 32.69 −0.36 2.08 Barbara 27.94 30.04 2.10 29.46 1.53 0.57 Mandrill 22.96 23.90 0.94 22.90 −0.06 1.00 Image @ 1.00 BPP Lena 35.68 36.29 0.61 34.40 −1.28 1.89 Peppers 33.93 34.23 0.30 32.56 −1.37 1.67 Airplane 37.18 38.14 0.96 36.00 −1.18 2.14 Barbara 32.86 34.36 1.50 32.94 0.08 1.42 Mandrill 25.44 26.33 0.89 24.92 −0.52 1.41 - A visual and PSNR comparison of baseline JPEG images coded at 1.0 BPP (Table 2, fourth table) versus the quality of the same images coded at half the bit rate (0.5 BPP in the third table) using ESAP-AC (column 3), demonstrates the two times compression potential of ESAP-AC versus baseline JPEG at one times compression. Similarly, comparing column 3 (ESAP-AC) versus column 5 (JPEG-2000 or JP2) images compressed at the same bit rate shows the improved visual quality of ESAP-AC over baseline coded JP2 images. The same comparisons can be made for the 0.5 versus 0.25 BPP rates as well. All of the images listed in Table 2 can also be viewed in full detail at the NASA ITPO website http://techtransfer.gsfc.nasa.gov/ft_tech_esap.shtm. The website details the PSNR, BPP, and CR statistics for each image as well.
- ESAP can also remove blocking artifacts in MPEG-2 GOP frames. As a related example of MPEG-2 laboratory video processing, frame sequences extracted from MPEG-2 video using ffmpeg software, then compressed at half or a third of their original bit rate using ffmpeg, and in parallel, also post-processed with ESAP-AC before reassembling into an ESAP-AC ffmpeg video sequence, show significant video improvement, with imperceptible or minimized DCT blocking over their respective 2× and 3× compressed MPEG-2 sequences. In these ffmpeg video experiments, it was verified that the ESAP-AC MPEG-2 video sequences compare very well in visual quality against their respective MPEG-4 sequences coded at the same 2× and 3× video compression ratios.
- This breakthrough bit rate reduction creates exciting new opportunities for digital video content service providers, webcasting, and satellite video. Some such opportunities include, but are not limited to, the following: (1) ESAP-AC could make 50% of the digital bandwidth reduction available to approximately double the number of digital MPEG-2 HDTV terrestrial channels over the same bandwidth without changing the broadcasting infrastructure; (2) ESAP-AC could also allow the extra 50% bandwidth to seamlessly integrate, multiplex, and provide UHDTV MPEG-4 content alongside HDTV MPEG-2 over the same digital pipe for dual, HDTV-UHDTV compatible receiver sets; (3) decoupled coexistence of both HDTV standards should provide many more choices for commercial video providers and consumers; and (4) Earth orbiting and planetary spacecraft video would certainly benefit from lower bit rates as well that could improve the SNR of the received signal.
- As discussed above, the ESAP algorithm may be applied to RGB color images. The ESAP-AC PSNR is always higher than default settings JPEG-2000 images compressed at similar bit rates. ESAP-AC post-processing yields better PSNRs than default settings compressed JPEG-2000 images due to ESAP's excellent ability to remove DCT blocking artifacts without introducing the DWT ringing inherent to JPEG-2000.
- ESAP Algorithm for RGB Jpeg Image Processing
- An embodiment of the ESAP algorithm may be implemented in the IJG Software™ discussed above using the cjpeg -arithmetic option and the ImageMagick™ convert utility, which can be found at http://www.imagemagick.org. The pseudocode is shown in the esap_script below:
-
esap_script image.fmt QF esap_img.ppm // The input image format “fmt” is any of: // PPM, PGM, GIF, JPG, TIFF, BMP, etc. The JPEG quality factor “QF” // is a [0..100] integer. The output is the filtered esap_img.ppm image. // 1. Convert input image to a portable pix map (PPM) format, // and then decompose into RGB planes in portable graymap (PGM) format. convert image.fmt image.ppm ppm2rgb image.ppm image.r.pgm image.g.pgm image.b.pgm // 2. Create JPEG RGB Quantization Coefficients cjpeg -quality QF -arith image.r.pgm image.r.jpg cjpeg -quality QF -arith image.g.pgm image.g.jpg cjpeg -quality QF -arith image.b.pgm image.b.jpg // 3. JPEG decode each R, G, and B planes. Then ESAP-filter each plane. // The modified djpeg decoder generates the YCbCr DCT coefficients // necessary to perform ESAP postfiltering. djpeg image.r.jpg image.r.djpg.pgm esap image.r.djpg.pgm DCTcoeffs.r esap_image.r.pgm djpeg image.g.jpg image.g.djpg.pgm esap image.g.djpg.pgm DCTcoeffs.g esap_image.g.pgm djpeg image.b.jpg image.b.djpg.pgm esap image.b.djpg.pgm DCTcoeffs.b esap_image.b.pgm // 4. Combine RGB PGM JPEG planes into ppm rgb2ppm image.r.djpg.pgm image.g.djpg.pgm image.b.djpg.pgm\ image.djpg.ppm // 5. Combine RGB PGM ESAP planes into ppm rgb2ppm esap_image.r.pgm esap_ image.g.pgm esap_image.b.pgm\ esap_image.ppm // 6. Compute grayscale Mean Square Errors (MSEs) for each color plane // and output results to file mse image.r.pgm image.r.djpg.pgm >> file.txt mse image.g.pgm image.g.djpg.pgm >> file.txt mse image.b.pgm image.b.djpg.pgm >> file.txt mse image.r.pgm esap_image.r.pgm >> file.txt mse image.g.pgm esap_image.g.pgm >> file.txt mse image.b.pgm esap_image.b.pgm >> file.txt // 7. Compute the peak signal-to-noise ratios (PSNRs) as a function of the // above MSEs for the JPEG and ESAP images. Also compute PPM // compression factor (CF) X:1 ratios and bit rates (BPPs). compute_psnr_cf >> file.txt - The PSNR of an RGB image is defined as:
-
PSNRRGB=10*log10(2552/(MSER+MSEG+MSEB)/3) (1) - For each color plane c={R, G, or B}, the mean-square-error (MSE) is:
-
-
FIG. 4 is aflowchart 400 illustrating a process for improved JPEG-ESAP processing of color images, according to an embodiment of the present invention. The process begins with converting an input image to a PPM format at 405 and then decomposing the ppm image into color planes in PGM format at 410. Next, JPEG quantization coefficients are created for each respective color plane at 415. In some embodiments, the colors of the color planes may be RGB, RGBY, or any other suitable color scheme. - Each color plane is JPEG decoded at 420 and ESAP postfiltered at 425. The decoder generates the YCbCr DCT coefficients necessary to perform ESAP postfiltering. The PGM JPEG planes for each color are combined at 430 and the PGM ESAP planes for each color are combined into PPM at 435.
- Grayscale MSEs are computed for each color plane at 440 and the results are output to files at 445. The purpose of this output is to compute a single color PSNR metric for each decoded and each postfiltered color image. The PSNRs are computed for the JPEG and ESAP images at 450 and the PPM CF X:1 ratios and BPPs are computed at 455. The results of the computations are then output to file at 460. The purpose of this output is to gather and record PSNR performance metrics as a function of the compression ratio (or equivalently, as a function of the BPP rate) for each processed image.
- The final output of the color JPEG-ESAP processing is a JPEG image file and an ESAP postfiltered image file, which are created at 465. Baseline Huffman entropy coding, or alternatively, arithmetic entropy coding may be used for the creation of both images. Also performance metrics may be computed and appended to a file for each processed image. For video, such as MPEG-2, the process may be repeated for multiple frames of the video or all frames of the video, as shown by the dashed line. The final output of MPEG-2 ESAP video processing is a sequence of PPM images that can be re-coded into an enhanced blocking distortion-minimized MPEG-2 or MPEG-4 video sequence at 470.
-
FIG. 5 is a block diagram of acomputing system 500 configured to perform improved JPEG-ESAP processing of color images, according to an embodiment of the present invention.Computing system 500 includes a bus 505 or other communication mechanism for communicating information, and processor(s) 510 coupled to bus 505 for processing information. Processor(s) 510 may be any type of general or specific purpose processor, including a central processing unit (“CPU”) or application specific integrated circuit (“ASIC”). Processor(s) 510 may also have multiple processing cores, and at least some of the cores may be configured to perform specific functions. Multi-parallel processing may be used in some embodiments.Computing system 500 further includes amemory 515 for storing information and instructions to be executed by processor(s) 510.Memory 515 can be comprised of any combination of random access memory (RAM), read only memory (ROM), flash memory, cache, static storage such as a magnetic or optical disk, or any other types of non-transitory computer-readable media or combinations thereof. Additionally,computing system 500 includes acommunication device 520, such as a transceiver and antenna, to wirelessly provide access to a communications network. - Non-transitory computer-readable media may be any available media that can be accessed by processor(s) 510 and may include both volatile and non-volatile media, removable and non-removable media, and communication media. Communication media may include computer-readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.
- Processor(s) 510 are further coupled via bus 505 to a
display 525, such as a Liquid Crystal Display (LCD), for displaying information to a user. Akeyboard 530 and acursor control device 535, such as a computer mouse, are further coupled to bus 505 to enable a user to interface with computing system. However, in certain embodiments such as those for mobile computing implementations, a physical keyboard and mouse may not be present, and the user may interact with the device solely throughdisplay 525 and/or a touchpad (not shown). Any type and combination of input devices may be used as a matter of design choice. -
Memory 515 stores software modules that provide functionality when executed by processor(s) 510. The modules include anoperating system 540 forcomputing system 500. The modules further include an improved JPEG-ESAP module 545 that is configured to perform JPEG-ESAP processing of color images.Computing system 500 may include one or more additionalfunctional modules 550 that include additional functionality. - One skilled in the art will appreciate that a “system” could be embodied as an embedded computing system, a personal computer, a server, a console, a personal digital assistant (PDA), a cell phone, a tablet computing device, a cable TV set top box, an HDTV or UHDTV set, a digital video disk (DVD) player, a Blu-ray player, or any other suitable computing device, or combination of devices. Presenting the above-described functions as being performed by a “system” is not intended to limit the scope of the present invention in any way, but is intended to provide one example of many embodiments of the present invention. Indeed, methods, systems and apparatuses disclosed herein may be implemented in localized and distributed forms consistent with computing technology, including cloud computing systems.
- It should be noted that some of the system features described in this specification have been presented as modules, in order to more particularly emphasize their implementation independence. For example, a module may be implemented as a hardware circuit comprising custom very large scale integration (“VLSI”) circuits or gate arrays, off-the-shelf semiconductors such as logic chips, transistors, or other discrete components. A module may also be implemented in programmable hardware devices such as field programmable gate arrays, programmable array logic, programmable logic devices, graphics processing units, or the like.
- A module may also be at least partially implemented in software for execution by various types of processors. An identified unit of executable code may, for instance, comprise one or more physical or logical blocks of computer instructions that may, for instance, be organized as an object, procedure, or function. Nevertheless, the executables of an identified module need not be physically located together, but may comprise disparate instructions stored in different locations which, when joined logically together, comprise the module and achieve the stated purpose for the module. Further, modules may be stored on a computer-readable medium, which may be, for instance, a hard disk drive, flash device, RAM, tape, or any other such medium used to store data.
- Indeed, a module of executable code could be a single instruction, or many instructions, and may even be distributed over several different code segments, among different programs, and across several memory devices. Similarly, operational data may be identified and illustrated herein within modules, and may be embodied in any suitable form and organized within any suitable type of data structure. The operational data may be collected as a single data set, or may be distributed over different locations including over different storage devices, and may exist, at least partially, merely as electronic signals on a system or network.
- The process steps performed in
FIG. 4 may be performed by a computer program, encoding instructions for the nonlinear adaptive processor to perform at least the process described inFIG. 4 , in accordance with embodiments of the present invention. The computer program may be embodied on a non-transitory computer-readable medium. The computer-readable medium may be, but is not limited to, a hard disk drive, a flash device, a random access memory, a tape, or any other such medium used to store data. The computer program may include encoded instructions for controlling the nonlinear adaptive processor to implement the process described inFIG. 4 , which may also be stored on the computer-readable medium. - The computer program can be implemented in hardware, software, or a hybrid implementation. The computer program can be composed of modules that are in operative communication with one another, and which are designed to pass information or instructions to display. The computer program can be configured to operate on a general purpose computer, or an ASIC.
- It will be readily understood that the components of various embodiments of the present invention, as generally described and illustrated in the figures herein, may be arranged and designed in a wide variety of different configurations. Thus, the detailed description of the embodiments of the present invention, as represented in the attached figures, is not intended to limit the scope of the invention as claimed, but is merely representative of selected embodiments of the invention.
- The features, structures, or characteristics of the invention described throughout this specification may be combined in any suitable manner in one or more embodiments. For example, reference throughout this specification to “certain embodiments,” “some embodiments,” or similar language means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, appearances of the phrases “in certain embodiments,” “in some embodiment,” “in other embodiments,” or similar language throughout this specification do not necessarily all refer to the same group of embodiments and the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
- It should be noted that reference throughout this specification to features, advantages, or similar language does not imply that all of the features and advantages that may be realized with the present invention should be or are in any single embodiment of the invention. Rather, language referring to the features and advantages is understood to mean that a specific feature, advantage, or characteristic described in connection with an embodiment is included in at least one embodiment of the present invention. Thus, discussion of the features and advantages, and similar language, throughout this specification may, but do not necessarily, refer to the same embodiment.
- Furthermore, the described features, advantages, and characteristics of the invention may be combined in any suitable manner in one or more embodiments. One skilled in the relevant art will recognize that the invention can be practiced without one or more of the specific features or advantages of a particular embodiment. In other instances, additional features and advantages may be recognized in certain embodiments that may not be present in all embodiments of the invention.
- One having ordinary skill in the art will readily understand that the invention as discussed above may be practiced with steps in a different order, and/or with hardware elements in configurations which are different than those which are disclosed. Therefore, although the invention has been described based upon these preferred embodiments, it would be apparent to those of skill in the art that certain modifications, variations, and alternative constructions would be apparent, while remaining within the spirit and scope of the invention. In order to determine the metes and bounds of the invention, therefore, reference should be made to the appended claims.
Claims (20)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/810,959 US9414072B1 (en) | 2015-02-20 | 2015-07-28 | Performance of the JPEG estimated spectrum adaptive postfilter (JPEG-ESAP) for low bit rates |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201562118770P | 2015-02-20 | 2015-02-20 | |
US14/810,959 US9414072B1 (en) | 2015-02-20 | 2015-07-28 | Performance of the JPEG estimated spectrum adaptive postfilter (JPEG-ESAP) for low bit rates |
Publications (2)
Publication Number | Publication Date |
---|---|
US9414072B1 US9414072B1 (en) | 2016-08-09 |
US20160249049A1 true US20160249049A1 (en) | 2016-08-25 |
Family
ID=56556088
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/810,959 Active US9414072B1 (en) | 2015-02-20 | 2015-07-28 | Performance of the JPEG estimated spectrum adaptive postfilter (JPEG-ESAP) for low bit rates |
Country Status (1)
Country | Link |
---|---|
US (1) | US9414072B1 (en) |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111654674B (en) * | 2020-06-04 | 2021-10-22 | 上海鸢安智能科技有限公司 | Method, software system and terminal for processing multiple video sources in parallel based on single ffmpeg process |
CN112449200B (en) * | 2020-11-12 | 2023-01-31 | 北京环境特性研究所 | Image compression method and device based on wavelet transformation |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6760487B1 (en) * | 1999-04-22 | 2004-07-06 | The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration | Estimated spectrum adaptive postfilter and the iterative prepost filtering algirighms |
-
2015
- 2015-07-28 US US14/810,959 patent/US9414072B1/en active Active
Also Published As
Publication number | Publication date |
---|---|
US9414072B1 (en) | 2016-08-09 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20200195937A1 (en) | Image processing device and method | |
US9930329B2 (en) | Video encoding and decoding based on image refinement | |
JP4847890B2 (en) | Encoding method converter | |
US9679365B2 (en) | Processing high dynamic range images | |
US9681132B2 (en) | Methods and apparatus for adaptive loop filtering in video encoders and decoders | |
US8831111B2 (en) | Decoding with embedded denoising | |
US8638852B2 (en) | Video coding of filter coefficients based on horizontal and vertical symmetry | |
US9911179B2 (en) | Image decontouring in high dynamic range video processing | |
US20090252425A1 (en) | Scalable picture encoding | |
US20090080517A1 (en) | Method and Related Device for Reducing Blocking Artifacts in Video Streams | |
US20220385949A1 (en) | Block-based compressive auto-encoder | |
US20140056535A1 (en) | Image processing device and method | |
US20140192266A1 (en) | Method and apparatus of reducing compression noise in digital video streams | |
US9414072B1 (en) | Performance of the JPEG estimated spectrum adaptive postfilter (JPEG-ESAP) for low bit rates | |
US20160050442A1 (en) | In-loop filtering in video coding | |
US9275434B2 (en) | Phase control multi-tap downscale filter | |
US8442338B2 (en) | Visually optimized quantization | |
Kim et al. | A lossless color image compression method based on a new reversible color transform | |
US9432614B2 (en) | Integrated downscale in video core | |
KR100803132B1 (en) | Method and apparatus for reduction MPEG noise using wavelet transform | |
US20090046783A1 (en) | Method and Related Device for Decoding Video Streams |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: UNITED STATES OF AMERICA AS REPRESENTED BY THE ADM Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:LINARES, IRVING, DR;REEL/FRAME:036205/0517 Effective date: 20150720 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 4 |
|
FEPP | Fee payment procedure |
Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |