US20140152694A1 - Merging Multiple Exposures to Generate a High Dynamic Range Image - Google Patents

Merging Multiple Exposures to Generate a High Dynamic Range Image Download PDF

Info

Publication number
US20140152694A1
US20140152694A1 US14098230 US201314098230A US2014152694A1 US 20140152694 A1 US20140152694 A1 US 20140152694A1 US 14098230 US14098230 US 14098230 US 201314098230 A US201314098230 A US 201314098230A US 2014152694 A1 US2014152694 A1 US 2014152694A1
Authority
US
Grant status
Application
Patent type
Prior art keywords
image
pixel
exposure
long
hdr
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
US14098230
Inventor
Rajesh Narasimha
Aziz Umit Batur
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Texas Instruments Inc
Original Assignee
Texas Instruments Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date

Links

Images

Classifications

    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/36Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
    • G09G5/37Details of the operation on graphic patterns
    • G09G5/377Details of the operation on graphic patterns for mixing or overlaying two or more graphic patterns
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration, e.g. from bit-mapped to bit-mapped creating a similar image
    • G06T5/007Dynamic range modification
    • G06T5/008Local, e.g. shadow enhancement
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration, e.g. from bit-mapped to bit-mapped creating a similar image
    • G06T5/50Image enhancement or restoration, e.g. from bit-mapped to bit-mapped creating a similar image by the use of more than one image, e.g. averaging, subtraction
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10141Special mode during image acquisition
    • G06T2207/10144Varying exposure
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20004Adaptive image processing
    • G06T2207/20012Locally adaptive
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20021Dividing image into blocks, subimages or windows
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20172Image enhancement details
    • G06T2207/20208High dynamic range [HDR] image processing

Abstract

A method of generating a high dynamic range (HDR) image is provided that includes capturing a long exposure image and a short exposure image of a scene, computing a merging weight for each pixel location of the long exposure image based on a pixel value of the pixel location and a saturation threshold, and computing a pixel value for each pixel location of the HDR image as a weighted sum of corresponding pixel values in the long exposure image and the short exposure image, wherein a weight applied to a pixel value of the pixel location of the short exposure image and a weight applied to a pixel value of the pixel location in the pixel long exposure image are determined based on the merging weight computed for the pixel location and responsive to motion in a scene of the long exposure image and the short exposure image.

Description

    CROSS-REFERENCE TO RELATED APPLICATIONS
  • [0001]
    This application claims benefit of U.S. Provisional Patent Application Ser. No. 61/733,513 filed Dec. 5, 2012, which is incorporated herein by reference in its entirety. This application may be related to co-pending U.S. patent application Ser. No. ______, attorney docket number TI-73306, filed Dec. 5, 2013, which is incorporated herein by reference in its entirety
  • BACKGROUND OF THE INVENTION
  • [0002]
    1. Field of the Invention
  • [0003]
    Embodiments of the present invention generally relate to merging multiple exposures of a scene to generate a high dynamic range image.
  • [0004]
    2. Description of the Related Art
  • [0005]
    The demand for higher dynamic range is a driving force for various imaging applications such as, for example, security cameras and automobile cameras. Imaging devices such as video cameras may be equipped with high dynamic range (HDR) sensors. Non-HDR cameras take photographs with a limited exposure range, resulting in the loss of detail in bright or dark areas. Some HDR imaging devices may compensate for this loss of detail by capturing two or more images at different exposure levels and combining the images to produce images a broader tonal range than non-HDR devices. Merging multiple exposures preserves both the saturated and the shadow regions and thus provides a higher dynamic range than a single exposure.
  • [0006]
    There are several known techniques for generating an HDR image (also referred to as a wide dynamic range (WDR) image) from two or more exposures. In one technique, the exposures may be spatially interleaved. In some techniques, the imaging system merges multiple exposures and provides a native HDR Bayer image with a pixel depth ranging from 12 to 20 bits. In some techniques, the imaging system captures multiple temporally spaced exposures and these exposures are merged to form an HDR image in the imaging device receiving the multiple exposures. Whether the imaging system generates the HDR image or the imaging device generates the HDR image, tone mapping may need to be performed on the HDR image to permit processing of the HDR image in an imaging pipeline with a lesser pixel bit depth, e.g., 10 to 12 bits.
  • SUMMARY
  • [0007]
    Embodiments of the invention relate to methods and apparatus for generating a high dynamic range (HDR) image. In one aspect, a method of generating a high dynamic range (HDR) image is provided that includes capturing a long exposure image and a short exposure image of a scene, computing a merging weight for each pixel location of the long exposure image based on a pixel value of the pixel location and a saturation threshold, and computing a pixel value for each pixel location of the HDR image as a weighted sum of corresponding pixel values in the long exposure image and the short exposure image, wherein a weight applied to a pixel value of the pixel location of the short exposure image and a weight applied to a pixel value of the pixel location in the pixel long exposure image are determined based on the merging weight computed for the pixel location of the long exposure image and responsive to motion in a scene of the long exposure image and the short exposure image.
  • [0008]
    In one aspect, an apparatus configured to generate a high dynamic range (HDR) image is provided that includes means for capturing a long exposure image and a short exposure image of a scene, means for computing a merging weight for each pixel location of the long exposure image based on a pixel value of the pixel location and a saturation threshold; and means for computing a pixel value for each pixel location of the HDR image as a weighted sum of corresponding pixel values in the long exposure image and the short exposure image, wherein a weight applied to a pixel value of the pixel location of the short exposure image and a weight applied to a pixel value of the pixel location in the pixel long exposure image are determined based on the merging weight computed for the pixel location of the long exposure image and responsive to motion in a scene of the long exposure image and the short exposure image.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • [0009]
    Particular embodiments in accordance with the invention will now be described, by way of example only, and with reference to the accompanying drawings:
  • [0010]
    FIG. 1 is a block diagram of a digital system configured to perform local tone mapping on high dynamic range (HDR) images;
  • [0011]
    FIG. 2 is a block diagram illustrating data flow of image processing in an embodiment of the digital system of FIG. 1;
  • [0012]
    FIGS. 3A and 3B are block diagrams illustrating data flow of image processing in an embodiment of the digital system of FIG. 1;
  • [0013]
    FIG. 4 is a block diagram of a merge component in an embodiment of the digital system of FIG. 1;
  • [0014]
    FIG. 5 is a flow diagram of a method for merging a long exposure image and a short exposure image to generate an HDR image;
  • [0015]
    FIG. 6 is a flow diagram of a method for local tone mapping of an HDR image;
  • [0016]
    FIG. 7 is a graph illustrating weight calculation for ghosting artifact reduction in a merged image;
  • [0017]
    FIG. 8 is an example illustrating local tone mapping;
  • [0018]
    FIGS. 9A and 9B are graphs illustrating generation of local tone curves;
  • [0019]
    FIGS. 10A, 10B, and 10C are examples illustrating computation of distance weights and pixel intensity weights for local tone mapping;
  • [0020]
    FIG. 11 is an example of applying local tone mapping to an HDR image; and
  • [0021]
    FIG. 12 is a graph illustrating derivation of parameter values for determining merging weights.
  • DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION
  • [0022]
    Specific embodiments of the invention will now be described in detail with reference to the accompanying figures. Like elements in the various figures are denoted by like reference numerals for consistency. As used herein, an image may be a single still picture of a scene or may be a frame in a video stream.
  • [0023]
    In general, embodiments of the invention provide for merging of long exposure images and short exposure images to generate dynamic range (HDR) images. Generation of a merged (HDR) image includes computing weights for pixels of long and short exposure images and computing pixels values for the merged image as the weighted sum of corresponding pixels in the long and short exposure images.
  • [0024]
    Unless otherwise specified, for simplicity of explanation, embodiments are described herein in which pixels in an HDR image prior to tone mapping are assumed to be 16 bits. One of ordinary skill in the art, having benefit of the disclosure herein, will understand embodiments with differing pixel bit depths, both before and after tone mapping.
  • [0025]
    FIG. 1 shows a digital system suitable for an embedded system (e.g., a digital camera) configured to perform local tone mapping of an HDR image as described herein. In some embodiments, the digital system is also configured to merge a long exposure image and a short exposure image to generate the HDR image. The digital system includes a DSP-based image coprocessor (ICP) 102, a RISC processor 104, and a video processing engine (VPE) 106 that may be configured to perform noise filtering as described herein. The RISC processor 104 may be any suitably configured RISC processor. The VPE 106 includes a configurable video processing front-end (Video FE) 108 input interface used for video capture from imaging peripherals such as image sensors, video decoders, etc., a configurable video processing back-end (Video BE) 110 output interface used for display devices such as SDTV displays, digital LCD panels, HDTV video encoders, etc, and memory interface 124 shared by the Video FE 108 and the Video BE 110. The digital system also includes peripheral interfaces 112 for various peripherals that may include a multi-media card, an audio serial port, a Universal Serial Bus (USB) controller, a serial port interface, etc.
  • [0026]
    The Video FE 108 includes an image signal processor (ISP) 116, and a 3A statistics generator 118. The ISP 116 provides an interface to image sensors and digital video sources. More specifically, the ISP 116 may accept raw image/video data from an HDR sensor module 126 (e.g., CMOS or CCD) and can accept YUV video data in numerous formats. The ISP 116 also includes a parameterized image processing module with functionality to generate image data in a color format (e.g., RGB) from raw sensor data. The ISP 116 is customizable for each sensor type and supports video frame rates for preview displays of captured digital images and for video recording modes. The ISP 116 also includes, among other functionality, an image resizer, statistics collection functionality, and a boundary signal calculator. The 3A module 118 includes functionality to support control loops for auto focus, auto white balance, and auto exposure by collecting metrics on the raw image data from the ISP 116 or external memory. In one or more embodiments, the Video FE 108 is configured to perform local tone mapping of an HDR image as described herein. In some embodiments, the Video FE 108 is configured to generate the HDR image by merging a long exposure image and a corresponding short exposure image captured by the sensor module 126.
  • [0027]
    The Video BE 110 includes an on-screen display engine (OSD) 120 and a video analog encoder (VAC) 122. The OSD engine 120 includes functionality to manage display data in various formats for several different types of hardware display windows and it also handles gathering and blending of video data and display/bitmap data into a single display window before providing the data to the VAC 122 in a color space format (e.g., RGB, YUV, YCbCr). The VAC 122 includes functionality to take the display frame from the OSD engine 120 and format it into the desired output format and output signals required to interface to display devices. The VAC 122 may interface to composite NTSC/PAL video devices, S-Video devices, digital LCD devices, high-definition video encoders, DVI/HDMI devices, etc.
  • [0028]
    The memory interface 124 functions as the primary source and sink to modules in the Video FE 108 and the Video BE 110 that are requesting and/or transferring data to/from external memory. The memory interface 124 includes read and write buffers and arbitration logic.
  • [0029]
    The ICP 102 includes functionality to perform the computational operations required for compression and other processing of captured images. The video compression standards supported may include, for example, one or more of the JPEG standards, the MPEG standards, the H.26x standards, and the emerging HEVC standard. In one or more embodiments, the ICP 102 may be configured to perform at least some of the computational operations of local tone mapping of an HDR image. In some embodiments, the ICP 102 may be configured to perform at least some of the computational operations of merging a long exposure image and a short exposure image to generate the HDR image.
  • [0030]
    In some embodiments, the HDR sensor module 126 is configured to capture HDR images of a scene and provides these images to the VPE 106 at a suitable frame rate, e.g., 30 frames per second (fps). In such embodiments, the HDR sensor module 126 includes one or more suitable imaging sensors, e.g., CCD (charge-coupled device) or CMOS (complementary metal oxide semi-conductor) sensors. In some embodiments, the HDR sensor module is configured to capture a long exposure image and a short exposure image of a scene and provide these exposures to the VPE 106 at a suitable frame rate, e.g., 60 fps. In such embodiments, the HDR sensor module 126 includes one or more suitable imaging sensors, e.g., CCD or CMOS sensors.
  • [0031]
    In operation, in some embodiments, an HDR image of a scene is captured by the HDR sensor module 124 and provided to the video FE 108. In such embodiments, the Video FE de-compands the HDR image for further processing. In some embodiments, a long exposure image and a short exposure image of a scene are captured by the HDR sensor module 124 and provided to the video FE 108. In the latter embodiments, the video FE 108 merges the two images to form an HDR image of the scene. The merging may be performed as described herein. The video FE 108 converts the HDR image to the input format needed to perform video compression. Prior to the compression, local tone mapping as described herein may be applied to the HDR image as part of processing the image in the image pipeline of the video FE 108. The video data generated by the video FE 108 is stored in the external memory. The video data is then encoded, i.e., compressed. During the compression process, the video data is read from the external memory and the compression computations on this video data are performed by the ICP 102. The resulting compressed video data is stored in the external memory. The compressed video data is then read from the external memory, decoded, and post-processed by the video BE 110 to display the image/video sequence.
  • [0032]
    FIG. 2 is a block diagram illustrating data flow of image processing (the “image pipeline”) in an embodiment of the digital system of FIG. 1 when the HDR sensor system 126 is configured to capture an HDR image of a scene. FIGS. 3A and 3B are block diagrams illustrating data flow of the image pipeline in an embodiment of the digital system of FIG. 1 when the HDR sensor system 126 is configured to capture a long exposure image and a short exposure image of a scene. One of ordinary skill in the art will understand that similar functionality may also be present in other digital devices (e.g., a cell phone, tablet, PDA, etc.) capable of capturing HDR digital images and/or HDR digital video sequences. While much of the pipeline processing is shown in sequence, one of ordinary skill in the art will understand that image processing is parallel in nature and thus, each component in the pipeline does not necessarily receive an entire digital image before processing can begin. The functionality of many of the image pipeline components is well-known and is not described in detail. Note that input images to the image pipeline are Bayer pattern images. Thus each pixel has only one of four color components—a red component denoted as R, a blue component denoted as B, or one of two green components denoted as Gr and Gb.
  • [0033]
    As shown in FIG. 2, the HDR image from the HDR sensor module 126 is received by a defect correction component 200. The defect correction component 200 is configured to correct the values of defective pixels in the HDR image. As is well known, an image sensor may have some number of defective pixels which respond to light exposure differently than other pixels due to factors such as manufacturing faults or operating conditions. The correction may be performed using a look-up table (LUT) based technique or any other suitable technique. The defect corrected image is provided to the black level adjustment component 202.
  • [0034]
    The black level adjustment component 202 is configured to set sensor black to image black in the HDR image. That is, in order to optimize the dynamic range of the pixel values from the HDR sensor module 126, the pixels in the HDR image representing black are corrected since an image sensor may record some non-zero current at these pixel locations. Note that a black pixel should have a value of 0. The black level adjustment component 202 may adjust for this difference by subtracting offsets from each pixel value while clamping/clipping to zero to avoid a negative result. One simple way to calculate this adjustment is to take a picture of complete black, e.g., by leaving on the lens cap or camera cover. A separate black level adjustment value may be used for each color channel. The adjusted HDR image is provided to the noise filter component 204.
  • [0035]
    The noise filter component 204 is configured to remove various sources of noise in an HDR image, e.g., optical, electrical, digital and power, by averaging similar neighboring pixels. Typically, if the noise level is high, more weight is given to the average of similar neighbors. Conversely, if the noise level is low, more weight is given to the original pixel value. An Optical Electrical Conversion Function (OECF) chart captured using a uniform lighting source may be used to determine the noise level for different intensities. The 12 uniform gray patches on the OECF chart provide 12 power levels, which may then be used to arrange noise using either a linear or square-root model depending on the sensor and gain (or ISO) level. The filtered HDR image is provided to the lens shading correction component 206.
  • [0036]
    The lens shading correction component 206 is configured to compensate the HDR image for lens shading. Lens shading is the phenomenon that an image is bright in the center and decrease in brightness towards the edge of the field. Lens shading may be caused by factors such as irregularities in the optical properties of a lens associated with a digital image sensor or improper alignment between the Bayer color filter array and the lens. Any suitable technique for lens shading correction may be used. For example, a gain may be applied on a per-pixel basis to compensate for any decrease in brightness. The shading corrected HDR image is provided to the white balance component 208.
  • [0037]
    The white balance component 208 is configured to adjust the white pixels in an HDR digital image to compensate for color differences introduced by light sources, such as the differences between incandescent, fluorescent, natural light sources, XE strobe, and W-LED flash, as well as mixed light conditions. That is, the illumination during the recording of a scene in a digital image may be different from the illumination when viewing the final digital image. This difference may result in a different color appearance that may be seen, for example, as the bluish appearance of a face or the reddish appearance of the sky. Also, the sensitivity of each color channel varies such that grey or neutral colors may not be represented correctly. Any suitable white balance technique may be used. The white balanced HDR image is provided to both the 3A analysis component 210 (e.g., the 3A statistics generator 118) and the tone mapping component 212.
  • [0038]
    The 3A analysis component 210 is configured to collect metrics from the HDR image for auto focus, auto white balance, and auto exposure of subsequent images. The tone mapping component 212 is configured to perform a method for local tone mapping on the HDR image as described herein in reference to FIG. 6. The tone mapped image is provided to the remaining components 214 of the image pipeline for further processing to generate the final HDR image. The further processing may include, for example, RGB blending, gamma correction, conversion to the YCbCr color space, edge enhancement, contrast enhancement, and/or false chroma suppression.
  • [0039]
    As shown in FIG. 3A, the short exposure image from the HDR sensor module 126 is processed by the defect correction component 300, the black level adjustment component 302, the noise filter component 304, the lens shading component 306, and the white balance component 308 of the image pipeline and stored in memory 310. The processed short exposure image is also provided to the 3A analysis component 312. These components operate as described for similarly named components in FIG. 2.
  • [0040]
    As shown in FIG. 3B, the long exposure image from the HDR sensor module 126 is processed by the defect correction component 300, the black level adjustment component 302, the noise filter component 304, the lens shading component 306, and the white balance component 308 of the image pipeline and is provided to the 3A analysis component 312. These components operate as described for similarly named components in FIG. 2. The processed long exposure image is also provided to the merge component 314. The merge component 314 is configured to merge the long exposure image with the short exposure image stored in the memory 310 to generate an HDR image. The operation of the merge component 314 is described below in more detail in reference to the block diagram of FIG. 4 and the method of FIG. 5. The merged (HDR) image is provided to the tone mapping component 316.
  • [0041]
    The tone mapping component 316 is configured to perform a method for local tone mapping on the HDR image as described herein in reference to FIG. 6. The tone mapped image is provided to the remaining components 318 of the image pipeline for further processing to generate the final HDR image. The further processing may include, for example, RGB blending, gamma correction, conversion to the YCbCr color space, edge enhancement, contrast enhancement, and/or false chroma suppression.
  • [0042]
    FIG. 4 is a block diagram of the merge component 314 of FIG. 3B. The merge component 314 includes a gain adjust component 400, a weight computation component 402, and a merge component 404. The inputs are a long exposure image (processed as described above) and a corresponding short exposure image (processed as described above). The exposure times of the two images may set according to a ratio that is maintained as a power of 2. For example, the exposure time of the short exposure image may be two milliseconds (ms) and the exposure time of the long exposure image may be 31 ms, with a ratio of ceil(31/2)=16. In another example, the exposure time of the short exposure image may be 4 ms and the exposure time of the long exposure image may be 29 ms, with a ratio of ceil(29/4)=8.
  • [0043]
    In general, the gain adjust component 404 adjusts the gain difference between the long exposure image and the short exposure image. The gain adjustment is described in more detail in reference to the method of FIG. 5.
  • [0044]
    The weight computation component 402 determines the per-pixel weights that are to be applied to corresponding pixels of the long exposure and short exposure images (after the gain adjustment) to generate the HDR image. The per-pixel weights for the short exposure image and the per-pixel weights for the long exposure image are referred to herein as, respectively, Alpha_U(x,y) and Alpha_L(x,y). In addition, the weight computation component 402 determines motion adaptation weights to be used in the computation of the per-pixel weights for the long exposure image. The motion adaptation weights, the computation of which may be per pixel or block-based, are applied to reduce ghosting artifacts that may be introduced in the merged image due to motion in the scene during the time between capture of the long exposure and short exposure images. Determination of the per-pixel weights and the motion adaptation weights is described in more detail in reference to the method of FIG. 5.
  • [0045]
    The merge component 402 receives the per-pixel weights for the long exposure and short exposure images and the gain-adjusted images and computes each pixel M(x,y) of the merged image as per
  • [0000]

    M(x,y)=(L(x,y)×Alpha L(x,y)+U(x,y)×Alpha U(x,y))
  • [0000]
    where L(x,y) and U(x,y) are corresponding pixels in, respectively, the long exposure image and the short exposure image, Alpha_L(x,y) is the weight for L(x,y), and Alpha_U(x,y) is the weight for U(x,y).
  • [0046]
    FIG. 5 is a flow diagram of a method for generating a merged image from corresponding long exposure and short exposure images that may be performed by the merge component 314 of FIG. 3. Initially, the gain difference between the two images is adjusted 500. The amount of gain adjustment for the long exposure image, gain_long, is computed as
  • [0000]

    gain_long=(short exposure)/(long exposure)
  • [0000]
    where short exposure is the exposure time of the short exposure image and long exposure is the exposure time of the long exposure image. To perform the gain adjustment, the pixels in the long exposure image are multiplied by gain_long.
  • [0047]
    Per-pixel weights for pixels in the gain-adjusted long exposure image and for pixels in the short exposure image are then computed 502. First, a per-pixel HDR (merging) weight, referred to as alpha(x,y) herein, is computed as per
  • [0000]
    alpha ( x , y ) = { 1 L ( x , y ) > T af × L ( x , y ) + bf × L ( x , y ) L ( x , y ) < T
  • [0000]
    where x and y are pixel coordinates, L is a pixel value in the long exposure image, T is the saturation threshold, and the values of the parameters af and bf are derived based on T as described below.
  • [0048]
    The value of the saturation threshold T is the value of a maximum saturated pixel after gain_long applied. Assuming the pixel depth of the long exposure image is 16 bits, the value of a maximum saturated pixel is 216−1=65535. When gain_long is applied to the long exposure image, the value of a maximum saturated pixel becomes 65535/gain_long. For example, if gain_long=¼, the value of a maximum saturated pixel becomes 65535/4=16383. Thus, T is 16383.
  • [0049]
    FIG. 12 is a graph illustrating the derivation of the values of the parameters af and bf. In this graph, the x-axis is the pixel intensity in the long exposure image and the y axis is gain. The parameter V is a tuning parameter for the derivation, the value of which may be any suitable value between 0 and 1. The particular value of V is implementation dependent and may be determined empirically. The equation for the curve is
  • [0000]

    af 2 +bf.
  • [0000]
    At z=T/2, the equation for the point on the curve is
  • [0000]
    a ( T 2 ) 2 + b ( T 2 ) = V
  • [0000]
    and at z=T, the equation for the point on the curve is
  • [0000]

    aT 2 +bT=1.
  • [0000]
    Solving these equations,
  • [0000]
    af = ( 2 - 4 V ) T 2
  • [0000]
    gives the value of af, and
  • [0000]
    bf = ( 4 V - 1 ) T
  • [0000]
    gives the value of bf.
  • [0050]
    Motion adaptation weights, referred to as MAWeight herein, are also computed for ghosting artifact reduction. In some embodiments, a per-pixel motion adaptation weight is computed. In some embodiments, a motion adaptation weight for each pixel is computed based on a block of pixels around the pixel. Using per-pixel motion adaptation weights may be more accurate but computationally expensive while using block-based motion adaption weights may be much faster but less accurate. The motion adaptation weight for a particular pixel location (x,y), MAWeight(x,y), is computed as per
  • [0000]
    MAWeight ( x , y ) = { 1 D ( x , y ) < D 1 1 - ( D ( x , y ) - D 1 ) * slope D 1 D ( x , y )
  • [0000]
    where D(x,y) is a computed delta between values of corresponding pixels in the two images. FIG. 7 is a graph illustrating the weight calculation for ghosting artifact reduction. The values of D1 and slope may be determined empirically. When there is motion in the scene, D(x,y) becomes larger, which reduces MAWeight, which then reduces the amount of merging. Less merging helps avoid ghosting artifacts that may occur due to high motion.
  • [0051]
    For per-pixel ghosting artifact reduction, the value of D(x,y) is computed as
  • [0000]

    D(x,y)=|U(x,y)−L(x,y)|
  • [0000]
    where L(x,y) and U(x,y) are corresponding pixels in, respectively, the long exposure image and the short exposure image. For block based ghosting artifact reduction, assuming a block size of N×N where N=2n+1 (n=1,2,3, . . . ), the value of D(x,y) is computed as
  • [0000]

    D(x,y)=max−n≦i≦n(max−n≦j≦n |U(x+i,y+j)−L(x+i,y+j)|).
  • [0000]
    The value of n is implementation dependent and may be selected as a tradeoff between computation speed and accuracy.
  • [0052]
    The per-pixel weights for the long exposure image and the short exposure image are computed as per
  • [0000]

    Alpha L(x,y)=(1−alpha(x,y))*MAWeight(x,y)
  • [0000]

    Alpha U(x,y)=1−Alpha L(x,y)
  • [0053]
    The merged (HDR) image is then generated 504 using the per-pixel weights to combine corresponding pixel values from the gain-adapted long exposure image and the gain-adapted short exposure image. More specifically, each pixel M(x,y) of the merged image is computed as per
  • [0000]

    M(x,y)=(L(x,y)×Alpha L(x,y)+U(x,y)×Alpha U(x,y))
  • [0000]
    where L(x,y) and U(x,y) are corresponding pixels in, respectively, the long exposure image and the short exposure image, Alpha_L(x,y) is the weight for L(x,y), and Alpha_U(x,y) is the weight for U(x,y).
  • [0054]
    FIG. 6 is a flow diagram of a method for tone mapping of an HDR image that may be performed by the tone mapping component 212 of FIG. 2 or the tone mapping component 316 of FIG. 3. In general, the method takes the 16-bit linear data of the HDR image and adaptively maps the data into a smaller number of bits based on the scene content in the image. The number of bits to which each 16-bit pixel is mapped depends on the bit depth of the ISP 116. For simple of explanation, mapping from 16-bit data to 12-bit data is assumed. One of ordinary skill in the art will understand embodiments in which the smaller data size differs.
  • [0055]
    As shown in FIG. 6, initially a luminance image of the HDR image is generated 600. In some embodiments, the luminance image is down sampled to improve execution time. The amount of down sampling used is implementation dependent. For example, a Bayer image of size 1280×736 may be reduced to a luminance image of size 160×92.
  • [0056]
    White point adjustment is then performed 602 on the luminance image. White point adjustment in HDR images is important due to the wide dynamic range. For example, for 16-bit pixel depths, the dynamic range is [0 65535] where black is 0 and white is 65535. If the values of the white pixels in the image are not close to true white, i.e., 65535, low contrast may result in the tone-mapped image. The white point adjustment may be performed as follows. First, a histogram of the pixel values in the image is computed and the whitest pixel value, e.g., 65500, in the image is determined from the histogram. Some small percentage of the whiter pixels with values closest to the whitest pixel value, e.g., 2-5%, is then saturated to white, i.e., the values of these pixels are changed to 65535. Further, after the saturation process, every pixel in the image is scaled by the gain between the whitest original pixel value and true white. The percentage of the whitest pixels to be saturated is an implementation choice.
  • [0057]
    For example, assume a luminance image of size 160×92 and that 2% of the whiter pixels are to be saturated. The number of pixels in the image is 14720 and 2% is approximately 294 pixels. Approximately 294 pixels are identified for saturation by working backward from the identified whitest pixel value, e.g., 65500, to identify pixels with the next closest value and the next closest value, etc. until the pixel count is approximately 294. For example, suppose there are 100 pixels with a value of 65550, 100 pixels with a next whiter value of 65500, and 100 pixels with a next whiter value of 65495. The values of these 300 pixels are saturated by setting them to be true white, i.e., 65535. Further, every pixel in the image is scaled by the gain between the whitest pixel value identified, 65500, and true white, 65535, i.e., every pixel is multiplied by 65535/65500.
  • [0058]
    The luminance image is then divided 604 into overlapping blocks and a block mean pixel value is computed 606 for each of the blocks. The block size and the amount of overlap between the blocks is implementation dependent and may be chosen empirically, for example, based on a tradeoff between quality and performance. The block mean pixel value may be computed in any suitable way, e.g., as the average of the pixel values in the block or as a weighted average of the pixel values in the block. If a weighted average is used, the weight for each pixel may be based on the inverse of pixel intensity difference between the center pixel of the block and the pixel.
  • [0059]
    Local tone curves are then computed 608 for each block. These tone curves may be of any suitable length. For simplicity of explanation, the length of each local tone curve is assumed to be 256. One of ordinary skill in the art will understand embodiments with differing tone curve lengths. The computation of a local tone curve of length 256 is illustrated in the graphs of FIGS. 9A and 9B. To compute a tone curve for a block, three points on the curve are determined as illustrated in FIG. 9B. In this figure, CB is the block mean pixel value, TB=CB+delta_brightness is the target brightness, and LC and RC are, respectively, left and right contrast (i.e., the midpoint between 0 and CB and the midpoint between CB and 255, respectively). For purposes of determining the three points of a tone curve, LC=0 and RC=255. Thus, the three points of the local tone curve are [0 0], [CB TB] and [255, 255]. The 256 points on the tone curve are constructed from these three points using band limited interpolation and physicists' Hermite polynomials.
  • [0060]
    FIG. 9A illustrates the determination of the value of the delta_brightness to be added to the block mean pixel value to determine the target brightness TB. In this graph, the x axis is the block mean pixel value scaled from 16 bits to 8 bits and the y-axis is delta-brightness (gain). As can be seen from this graph, for scaled block mean pixel values less than the first knee point k1, the delta_brightness is the value of gain1. For scaled block mean pixel values larger than the second knee point k2, the delta_brightness is the value of gain2. For scaled block mean pixel values between the two knee points, the value of delta-brightness is chosen along the gain curve between the two knee points. The determination of delta_brightness values for particular block mean pixel values may be implemented as a lookup table (LUT) of 255 values derived based on the values of k1, k2, gain1, and gain2. The knee points and the gains may have any suitable values, which may be selected empirically.
  • [0061]
    After the local tone curves are constructed for each block, a gain map is computed 612 using the tone curves and weighted bilateral interpolation. The resulting gain map contains a gain corresponding to each pixel in the luminance image. The gain G(x,y) for a 16-bit luminance pixel X(x,y) in the luminance image with the value L is computed as
  • [0000]

    G(x,y)L out /L
  • [0000]
    where Lout is computed as a weighted sum of applying the tone curves of the four blocks having center points closest to X(x,y) to L. The four block centers closest to X(x,y) are referred to as the upper-left (UL) point, the upper-right (UR) point, the lower-left (LL) point, and the lower-right (LR) point herein. FIG. 8 is an example illustrating the locations of the center points relative to X(x,y). Each of the small circles in the image is a center point of a block. For odd block sizes, the center point is the center pixel in the block. For even block sizes, the center point is between the four center pixels in the block.
  • [0062]
    More specifically, Lout is computed as per
  • [0000]

    L outUL L out ULUR L out URLL L out LLLR L out LR
  • [0000]
    where Lout NN is the result of applying the tone curve of the block containing the NN neighboring center point of X(x,y) and βNN is a weight computed for this result, where NN={UL, UR, LL, LR}.
  • [0063]
    The value of Lout NN is computed as per
  • [0000]
    L out NN = LUT [ L floor ] NN W ceil + LUT [ L ceil ] NN W floor W ceil + W floor
  • [0000]
    where LUTNN is the tone mapping lookup table for the block containing the NN neighboring center point and Lfloor, Lceil, Wfloor, and Wceil are computed as follows. The floor and ceiling values of L are computed as per
  • [0000]
    L floor = L 256 ; 0 L floor < 254 L ceil = L floor + 1 ; 1 L floor < 255
  • [0000]
    and the floor and ceiling weights are computed as per
  • [0000]

    W floor =L−L floor*256
  • [0000]

    W ceil =L ceil*256−L
  • [0064]
    The value of βNN is computed as per
  • [0000]

    βNNdw NNαiw NN;0≦αdw NNiw NN≦1
  • [0000]
    where αdw NN is a distance weight based on the distance from the NN neighboring center point to the pixel X(x,y) and αiw NN is an intensity weight based on the difference in intensity between the mean pixel value of the block containing the NN neighboring center point and L.
  • [0065]
    Assuming that that the block size is odd and the center point is the center pixel of the block, the distance weight αdw NN is computed as per
  • [0000]

    αdw NN =|x−x NN ∥y−y NN|/λ
  • [0000]
    where (xNN, yNN) are the coordinates of the NN neighboring center point and λ is a normalization factor to scale the distance weight to be between 0 and 1 inclusive. If the block size is even, the center point is located between the four center pixels of the block, and the distance weight is computed based on the distances from the pixel X(x,y) to each of these four center pixels.
  • [0066]
    The intensity weight αiw NN is computed as per
  • [0000]

    αiw NN =LUT[|L−L NN|]
  • [0000]
    where LNN is the mean pixel value of the block containing the NN neighboring center point and the LUT is derived as per the graph of FIG. 10C. FIGS. 10A and 10B are examples illustrating, respectively, the computation of the distance weights and the intensity weights for X(x,y). In the graph of FIG. 10C, the x axis is |L−LNN| and the y-axis is the intensity weight (gain). As can be seen from this graph, for values of |L−LNN| less than the first knee point k1, the intensity weight is the value of gain1. For values of |L−LNN| larger than the second knee point k2, the intensity weight is the value of gain2. For values of |L−LNN| between the two knee points, the intensity weight is chosen along the gain curve between the two knee points. The determination of intensity weights for particular values of |L−LNN| may be implemented as a lookup table (LUT) of 255 values derived based on the values of k1, k2, gain1, and gain2. The knee points and the gains may have any suitable values, which may be selected empirically. In some embodiments, the values of the knee points and the gains are the same as those used to generate the LUT for determining the value of delta_brightness for particular block mean pixel values.
  • [0067]
    Referring again to FIG. 6, once the gain map is computed, the gains are applied 612 to the four Bayer channels of the HDR image. In embodiments where the luminance image is downsampled, the gain map may be upsampled to the original HDR image resolution. The gains may be applied to the four Bayer components of the corresponding pixels in the HDR image as per
  • [0000]

    Rbayer(x,y)=G(x,yRbayer(x,y)
  • [0000]

    Grbayer(x,y)=G(x,yGrbayer(x,y),
  • [0000]

    Gbbayer(x,y)=G(x,yGbbayer(x,y)
  • [0000]

    Bbayer(x,y)=G(x,yBbayer(x,y).
  • [0068]
    FIG. 11 is an example showing the effect of applying the above local tone mapping to an HDR image. The top image is the original 16-bit HDR image converted to a 12-bit image without tone mapping and the bottom image is the HDR image with local tone mapping applied. Note the improvement in brightness and contrast when the local tone mapping is applied.
  • OTHER EMBODIMENTS
  • [0069]
    While the invention has been described with respect to a limited number of embodiments, those skilled in the art, having benefit of this disclosure, will appreciate that other embodiments can be devised which do not depart from the scope of the invention as disclosed herein.
  • [0070]
    For example, embodiments have been described herein in which the local tone mapping applied to each pixel of an HDR image is based on four neighboring blocks of the pixel. One of ordinary skill in the art will understand embodiments of the invention in which more or fewer blocks may be used.
  • [0071]
    Embodiments of the methods described herein may be implemented in hardware, software, firmware, or any combination thereof. If completely or partially implemented in software, the software may be executed in one or more processors, such as a microprocessor, application specific integrated circuit (ASIC), field programmable gate array (FPGA), or digital signal processor (DSP). The software instructions may be initially stored in a computer-readable medium and loaded and executed in the processor. In some cases, the software instructions may also be sold in a computer program product, which includes the computer-readable medium and packaging materials for the computer-readable medium. In some cases, the software instructions may be distributed via removable computer readable media, via a transmission path from computer readable media on another digital system, etc. Examples of computer-readable media include non-writable storage media such as read-only memory devices, writable storage media such as disks, flash memory, memory, or a combination thereof.
  • [0072]
    It is therefore contemplated that the appended claims will cover any such modifications of the embodiments as fall within the true scope of the invention.

Claims (12)

    What is claimed is:
  1. 1. A method of generating a high dynamic range (HDR) image, the method comprising:
    capturing a long exposure image and a short exposure image of a scene;
    computing a merging weight for each pixel location of the long exposure image based on a pixel value of the pixel location and a saturation threshold; and
    computing a pixel value for each pixel location of the HDR image as a weighted sum of corresponding pixel values in the long exposure image and the short exposure image, wherein a weight applied to a pixel value of the pixel location of the short exposure image and a weight applied to a pixel value of the pixel location in the pixel long exposure image are determined based on the merging weight computed for the pixel location of the long exposure image and responsive to motion in a scene of the long exposure image and the short exposure image.
  2. 2. The method of claim 1, wherein computing a pixel value further comprises:
    computing a motion adaptation weight for each pixel location of the long exposure image;
    computing a weight for each pixel location of the long exposure image based on the merging weight computed for the pixel location of the long exposure image and the motion adaptation weight computed for each pixel location of the long exposure image; and
    computing a weight for each pixel location of the short exposure image by subtracting the weight computed for the pixel location in the long exposure image from one.
  3. 3. The method of claim 2, wherein computing a motion adaptation weight comprises:
    computing the motion adaptation weight for each pixel location of the long exposure image based on a delta between a pixel value of the pixel location in the long exposure image and a pixel value of the pixel location in the short exposure image.
  4. 4. The method of claim 2, wherein computing a motion adaptation weight comprises:
    computing the motion adaptation weight for each pixel location of the long exposure image based on deltas between pixel values of a block of pixel locations around the pixel location in the long exposure image and pixels values of a corresponding block of pixel locations around the pixel location in the short exposure image.
  5. 5. The method of claim 1, further comprising:
    adjusting gain difference between the long exposure image and the short exposure image.
  6. 6. The method of claim 1, further comprising:
    performing black level adjustment on the long exposure image and the short exposure image; and
    performing white balancing on the long exposure image and the short exposure image.
  7. 7. An apparatus configured to generate a high dynamic range (HDR) image, the apparatus comprising:
    means for capturing a long exposure image and a short exposure image of a scene;
    means for computing a merging weight for each pixel location of the long exposure image based on a pixel value of the pixel location and a saturation threshold; and
    means for computing a pixel value for each pixel location of the HDR image as a weighted sum of corresponding pixel values in the long exposure image and the short exposure image, wherein a weight applied to a pixel value of the pixel location of the short exposure image and a weight applied to a pixel value of the pixel location in the pixel long exposure image are determined based on the merging weight computed for the pixel location of the long exposure image and responsive to motion in a scene of the long exposure image and the short exposure image.
  8. 8. The apparatus of claim 7, wherein the means for computing a pixel value further comprises:
    means for computing a motion adaptation weight for each pixel location of the long exposure image;
    means for computing a weight for each pixel location of the long exposure image based on the merging weight computed for the pixel location of the long exposure image and the motion adaptation weight computed for each pixel location of the long exposure image; and
    means for computing a weight for each pixel location of the short exposure image by subtracting the weight computed for the pixel location in the long exposure image from one.
  9. 9. The apparatus of claim 8, wherein the means for computing a motion adaptation weight computes the motion adaptation weight for each pixel location of the long exposure image based on a delta between a pixel value of the pixel location in the long exposure image and a pixel value of the pixel location in the short exposure image,
  10. 10. The apparatus of claim 8, wherein the means for computing a motion adaptation weight computes the motion adaptation weight for each pixel location of the long exposure image based on deltas between pixel values of a block of pixel locations around the pixel location in the long exposure image and pixels values of a corresponding block of pixel locations around the pixel location in the short exposure image.
  11. 11. The apparatus of claim 7, further comprising:
    means for adjusting gain difference between the long exposure image and the short exposure image.
  12. 12. The apparatus of claim 7, further comprising:
    means for performing black level adjustment on the long exposure image and the short exposure image; and
    means for performing white balancing on the long exposure image and the short exposure image.
US14098230 2012-12-05 2013-12-05 Merging Multiple Exposures to Generate a High Dynamic Range Image Pending US20140152694A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US201261733513 true 2012-12-05 2012-12-05
US14098230 US20140152694A1 (en) 2012-12-05 2013-12-05 Merging Multiple Exposures to Generate a High Dynamic Range Image

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US14098230 US20140152694A1 (en) 2012-12-05 2013-12-05 Merging Multiple Exposures to Generate a High Dynamic Range Image
US14098243 US9437171B2 (en) 2012-12-05 2013-12-05 Local tone mapping for high dynamic range images

Publications (1)

Publication Number Publication Date
US20140152694A1 true true US20140152694A1 (en) 2014-06-05

Family

ID=50825015

Family Applications (2)

Application Number Title Priority Date Filing Date
US14098243 Active 2034-05-22 US9437171B2 (en) 2012-12-05 2013-12-05 Local tone mapping for high dynamic range images
US14098230 Pending US20140152694A1 (en) 2012-12-05 2013-12-05 Merging Multiple Exposures to Generate a High Dynamic Range Image

Family Applications Before (1)

Application Number Title Priority Date Filing Date
US14098243 Active 2034-05-22 US9437171B2 (en) 2012-12-05 2013-12-05 Local tone mapping for high dynamic range images

Country Status (1)

Country Link
US (2) US9437171B2 (en)

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140307129A1 (en) * 2013-04-15 2014-10-16 Htc Corporation System and method for lens shading compensation
US20150221071A1 (en) * 2014-02-05 2015-08-06 Panasonic Intellectual Property Management Co., Ltd. Image processing method and image processing device
US20150249795A1 (en) * 2014-02-28 2015-09-03 Samsung Electronics Co., Ltd. Imaging processing apparatus and method
US9148580B2 (en) 2013-07-16 2015-09-29 Texas Instruments Incorporated Transforming wide dynamic range images to reduced dynamic range images
US20160142659A1 (en) * 2014-11-13 2016-05-19 Samsung Electronics Co., Ltd. Pixel processing apparatus, and image signal processing apparatus and image processing system each including the same
US9692985B2 (en) * 2013-02-13 2017-06-27 Canon Kabushiki Kaisha Image processing apparatus and image processing method for tone control by applying different gain
US20170223267A1 (en) * 2016-02-03 2017-08-03 Texas Instruments Incorporated Image processing for wide dynamic range (wdr) sensor data
US20170237879A1 (en) * 2016-02-12 2017-08-17 Contrast Optical Design & Engineering, Inc. Color matching across multiple sensors in an optical system
WO2017139596A1 (en) * 2016-02-12 2017-08-17 Contrast Optical Design & Engineering, Inc. Devices and methods for high dynamic range video
WO2017201319A1 (en) * 2016-05-18 2017-11-23 Texas Instruments Incorporated Image data processing for multi-exposure wide dynamic range image data
WO2018031441A1 (en) * 2016-08-09 2018-02-15 Contrast, Inc. Real-time hdr video for vehicle control

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9307162B2 (en) * 2014-05-21 2016-04-05 Himax Imaging Limited Local enhancement apparatus and method to generate high dynamic range images by blending brightness-preserved and brightness-adjusted blocks
KR20160053632A (en) 2014-11-05 2016-05-13 삼성전자주식회사 Local tone mapping circuit, and mobile computing device including the same
US9652870B2 (en) * 2015-01-09 2017-05-16 Vixs Systems, Inc. Tone mapper with filtering for dynamic range conversion and methods for use therewith
CN104639845B (en) * 2015-03-17 2017-12-26 上海兆芯集成电路有限公司 High dynamic range image generating method and apparatus using the method
KR20160138685A (en) * 2015-05-26 2016-12-06 에스케이하이닉스 주식회사 Apparatus For Generating Low Complexity High Dynamic Range Image, and Method Thereof
WO2017002544A1 (en) * 2015-06-30 2017-01-05 オリンパス株式会社 Image processing apparatus and imaging system
JP2017038139A (en) * 2015-08-07 2017-02-16 キヤノン株式会社 Imaging apparatus, imaging apparatus control method, and program
CN105070273B (en) * 2015-09-02 2017-07-28 深圳市华星光电技术有限公司 Region luminance compensation method and design method Mura Mura pixel luminance

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070242900A1 (en) * 2006-04-13 2007-10-18 Mei Chen Combining multiple exposure images to increase dynamic range
US20080056707A1 (en) * 2006-08-31 2008-03-06 Micron Technology, Inc. Ambient infrared detection in solid state sensors
US20090087173A1 (en) * 2007-09-28 2009-04-02 Yun-Chin Li Image capturing apparatus with movement compensation function and method for movement compensation thereof
US20090180704A1 (en) * 2008-01-15 2009-07-16 Samsung Electronics Co., Ltd. Method and system for processing low-illuminance image
US20110292242A1 (en) * 2010-05-27 2011-12-01 Canon Kabushiki Kaisha User interface and method for exposure adjustment in an image capturing device
US20120008005A1 (en) * 2010-07-07 2012-01-12 Olympus Corporation Image processing apparatus, image processing method, and computer-readable recording medium having image processing program recorded thereon
US20120050557A1 (en) * 2010-09-01 2012-03-01 Qualcomm Incorporated High dynamic range image sensor
US20120127076A1 (en) * 2010-11-19 2012-05-24 Avago Technologies Ecbu Ip (Singapore) Pte. Ltd. Optical navigation with a dynamic squal threshold
US20120262600A1 (en) * 2011-04-18 2012-10-18 Qualcomm Incorporated White balance optimization with high dynamic range images
US20120288192A1 (en) * 2011-05-13 2012-11-15 Wolfgang Heidrich Color highlight reconstruction

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8345038B2 (en) * 2007-10-30 2013-01-01 Sharp Laboratories Of America, Inc. Methods and systems for backlight modulation and brightness preservation
US9055227B2 (en) 2010-03-17 2015-06-09 Texas Instruments Incorporated Scene adaptive brightness/contrast enhancement
JP6061488B2 (en) 2012-04-26 2017-01-18 キヤノン株式会社 Image processing apparatus and image processing method
US9142012B2 (en) * 2012-05-31 2015-09-22 Apple Inc. Systems and methods for chroma noise reduction
US9105078B2 (en) * 2012-05-31 2015-08-11 Apple Inc. Systems and methods for local tone mapping
US9743057B2 (en) * 2012-05-31 2017-08-22 Apple Inc. Systems and methods for lens shading correction

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070242900A1 (en) * 2006-04-13 2007-10-18 Mei Chen Combining multiple exposure images to increase dynamic range
US20080056707A1 (en) * 2006-08-31 2008-03-06 Micron Technology, Inc. Ambient infrared detection in solid state sensors
US20090087173A1 (en) * 2007-09-28 2009-04-02 Yun-Chin Li Image capturing apparatus with movement compensation function and method for movement compensation thereof
US20090180704A1 (en) * 2008-01-15 2009-07-16 Samsung Electronics Co., Ltd. Method and system for processing low-illuminance image
US20110292242A1 (en) * 2010-05-27 2011-12-01 Canon Kabushiki Kaisha User interface and method for exposure adjustment in an image capturing device
US20120008005A1 (en) * 2010-07-07 2012-01-12 Olympus Corporation Image processing apparatus, image processing method, and computer-readable recording medium having image processing program recorded thereon
US20120050557A1 (en) * 2010-09-01 2012-03-01 Qualcomm Incorporated High dynamic range image sensor
US20120127076A1 (en) * 2010-11-19 2012-05-24 Avago Technologies Ecbu Ip (Singapore) Pte. Ltd. Optical navigation with a dynamic squal threshold
US20120262600A1 (en) * 2011-04-18 2012-10-18 Qualcomm Incorporated White balance optimization with high dynamic range images
US20120288192A1 (en) * 2011-05-13 2012-11-15 Wolfgang Heidrich Color highlight reconstruction

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
Jacobs, K., Loscos, C., Ward, G., Automatic high-dynamic range image generation for dynamic scenes, IEEE Computer Graphics and Applications 28, 84-93, 2008 *

Cited By (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9692985B2 (en) * 2013-02-13 2017-06-27 Canon Kabushiki Kaisha Image processing apparatus and image processing method for tone control by applying different gain
US20140307129A1 (en) * 2013-04-15 2014-10-16 Htc Corporation System and method for lens shading compensation
US9681026B2 (en) * 2013-04-15 2017-06-13 Htc Corporation System and method for lens shading compensation
US9148580B2 (en) 2013-07-16 2015-09-29 Texas Instruments Incorporated Transforming wide dynamic range images to reduced dynamic range images
US9881357B2 (en) 2013-07-16 2018-01-30 Texas Instruments Incorporated Transforming wide dynamic range images to reduced dynamic range images
US20150221071A1 (en) * 2014-02-05 2015-08-06 Panasonic Intellectual Property Management Co., Ltd. Image processing method and image processing device
US9749506B2 (en) * 2014-02-05 2017-08-29 Panasonic Intellectual Property Management Co., Ltd. Image processing method and image processing device
US20150249795A1 (en) * 2014-02-28 2015-09-03 Samsung Electronics Co., Ltd. Imaging processing apparatus and method
US20160142659A1 (en) * 2014-11-13 2016-05-19 Samsung Electronics Co., Ltd. Pixel processing apparatus, and image signal processing apparatus and image processing system each including the same
US20170223267A1 (en) * 2016-02-03 2017-08-03 Texas Instruments Incorporated Image processing for wide dynamic range (wdr) sensor data
US9871965B2 (en) * 2016-02-03 2018-01-16 Texas Instruments Incorporated Image processing for wide dynamic range (WDR) sensor data
US20170237879A1 (en) * 2016-02-12 2017-08-17 Contrast Optical Design & Engineering, Inc. Color matching across multiple sensors in an optical system
WO2017139599A1 (en) * 2016-02-12 2017-08-17 Contrast Optical Design & Engineering, Inc. Combined hdr/ldr video streaming
WO2017139596A1 (en) * 2016-02-12 2017-08-17 Contrast Optical Design & Engineering, Inc. Devices and methods for high dynamic range video
US9948829B2 (en) * 2016-02-12 2018-04-17 Contrast, Inc. Color matching across multiple sensors in an optical system
WO2017201319A1 (en) * 2016-05-18 2017-11-23 Texas Instruments Incorporated Image data processing for multi-exposure wide dynamic range image data
WO2018031441A1 (en) * 2016-08-09 2018-02-15 Contrast, Inc. Real-time hdr video for vehicle control

Also Published As

Publication number Publication date Type
US9437171B2 (en) 2016-09-06 grant
US20140152686A1 (en) 2014-06-05 application

Similar Documents

Publication Publication Date Title
US7102669B2 (en) Digital color image pre-processing
US6201530B1 (en) Method and system of optimizing a digital imaging processing chain
US8760537B2 (en) Capturing and rendering high dynamic range images
US20090052797A1 (en) Imaging device, image processing device, image processing method, program for image processing method, and recording medium having program for image processing method recorded thereon
US7349574B1 (en) System and method for processing non-linear image data from a digital imager
US8259198B2 (en) System and method for detecting and correcting defective pixels in an image sensor
US20080259181A1 (en) Imaging apparatus, imaging method, integrated circuit, and storage medium
US20110090242A1 (en) System and method for demosaicing image data using weighted gradients
US20120026368A1 (en) Binning compensation filtering techniques for image signal processing
US20050123211A1 (en) Digital image processing system and method for processing digital images
US20080158396A1 (en) Image Signal Processor For CMOS Image Sensors
US20110091101A1 (en) System and method for applying lens shading correction during image processing
US20070183681A1 (en) Adaptive image filter for filtering image information
US20100271498A1 (en) System and method to selectively combine video frame image data
US20060232692A1 (en) Image pickup apparatus
US20080031538A1 (en) Adaptive spatial image filter for filtering image information
US20110205389A1 (en) Methods and Systems for Automatic White Balance
US20130016251A1 (en) Solid-state imaging device, image processing apparatus, and camera module
US20120081553A1 (en) Spatial filtering for image signal processing
US20080043120A1 (en) Image processing apparatus, image capture apparatus, image output apparatus, and method and program for these apparatus
US20100002104A1 (en) Image-signal processing apparatus and image-signal processing
US20100278423A1 (en) Methods and systems for contrast enhancement
US20110229019A1 (en) Scene Adaptive Brightness/Contrast Enhancement
US20080137947A1 (en) Image processing apparatus, image processing method, and program
US20120008015A1 (en) Image combining apparatus, image combining method and program product

Legal Events

Date Code Title Description
AS Assignment

Owner name: TEXAS INSTRUMENTS INCORPORATED, TEXAS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:NARASIMHA, RAJESH;BATUR, AZIZ UMIT;SIGNING DATES FROM 20131204 TO 20131205;REEL/FRAME:031771/0075