TW201517624A - Method and apparatus for generating high dynamic range images and computer program product - Google Patents

Method and apparatus for generating high dynamic range images and computer program product Download PDF

Info

Publication number
TW201517624A
TW201517624A TW103115135A TW103115135A TW201517624A TW 201517624 A TW201517624 A TW 201517624A TW 103115135 A TW103115135 A TW 103115135A TW 103115135 A TW103115135 A TW 103115135A TW 201517624 A TW201517624 A TW 201517624A
Authority
TW
Taiwan
Prior art keywords
image data
pixel
value
dynamic range
high dynamic
Prior art date
Application number
TW103115135A
Other languages
Chinese (zh)
Inventor
Yuan-Chih Peng
Original Assignee
Himax Imaging Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Himax Imaging Ltd filed Critical Himax Imaging Ltd
Publication of TW201517624A publication Critical patent/TW201517624A/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/50Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
    • G06T5/92
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60Control of cameras or camera modules
    • H04N23/68Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
    • H04N23/681Motion detection
    • H04N23/6811Motion detection based on the image signal
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70Circuitry for compensating brightness variation in the scene
    • H04N23/741Circuitry for compensating brightness variation in the scene by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/50Control of the SSIS exposure
    • H04N25/57Control of the dynamic range
    • H04N25/58Control of the dynamic range involving two or more exposures
    • H04N25/581Control of the dynamic range involving two or more exposures acquired simultaneously
    • H04N25/585Control of the dynamic range involving two or more exposures acquired simultaneously with pixels having different sensitivities within the sensor, e.g. fast or slow pixels or pixels having different sizes
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10016Video; Image sequence
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10024Color image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10141Special mode during image acquisition
    • G06T2207/10144Varying exposure
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20172Image enhancement details
    • G06T2207/20208High dynamic range [HDR] image processing

Abstract

A method for generating high dynamic range (HDR) images, comprising: capturing a spatially varying exposure image data of a scene by an image sensing array using a first exposure value and a second exposure value, wherein the first exposure value is larger than the second exposure value; re-sampling the spatially varying exposure image data to obtain a first image data of the scene corresponding to the first exposure value and second image data of the scene corresponding to the second exposure value; determining a motion index of each pixel of a HDR image data according to the first image data and the second image data; determining a pixel value of each pixel of the HDR image data according to a first pixel value of a corresponding pixel of the first image data, a second pixel value of a corresponding pixel of the second image data and the motion index; and outputting the HDR image data.

Description

產生高動態範圍影像的方法與裝置以及電腦程式產品 Method and apparatus for generating high dynamic range images and computer program products

本發明係有關於影像訊號處理技術,且特別有關於產生高動態範圍影像的技術。 The present invention relates to image signal processing techniques, and more particularly to techniques for producing high dynamic range images.

在影像處理技術中,影像中最高亮度與最低亮度的比值係定義為此影像的動態範圍。以互補式金氧半導體(Complementary Metal Oxide Semiconductor,CMOS)影像感測器為例,CMOS影像感測器所擷取之影像的動態範圍受雜訊底(noise floor)以及全井容量(full well capacity),因此,每像素可提供8位元(256階)亮度資訊的CMOS影像感測器所擷取之影像的動態範圍會遠低於人眼所感測之真實世界的動態範圍。CMOS影像感測器可調整曝光時間以及光敏度(photosensitivity)以符合所欲擷取之場景。儘管如此,若在所欲擷取之場景中同時有亮區域以及暗區域,例如具有非常大對比度之場景,則部份影像資訊可能無法清楚表示。舉例而言,在以高曝光量擷取之影像中,暗區域中的細節可清楚顯示但在亮區域中會有過曝現象。相反地,在以低曝光量擷取之影像中,亮區域中的細節可清楚顯示但暗區域會太暗且多雜訊。 In image processing technology, the ratio of the highest brightness to the lowest brightness in an image is defined as the dynamic range of the image. Taking a Complementary Metal Oxide Semiconductor (CMOS) image sensor as an example, the dynamic range of the image captured by the CMOS image sensor is affected by the noise floor and the full well capacity. Therefore, the dynamic range of the image captured by the CMOS image sensor that provides 8-bit (256-step) brightness information per pixel is much lower than the real-world dynamic range sensed by the human eye. The CMOS image sensor adjusts the exposure time and photosensitivity to match the scene you want to capture. However, if there are bright areas and dark areas in the scene to be captured, such as scenes with very large contrast, some of the image information may not be clearly indicated. For example, in images captured with high exposure, details in dark areas can be clearly displayed but overexposed in bright areas. Conversely, in images captured with low exposure, the details in the bright areas are clearly displayed but the dark areas are too dark and too noisy.

在擴大CMOS影像感測器之動態範圍的一方法中,會減少暗雜訊(dark noise)或增進全井容量。在擴大CMOS影像感測器之動態範圍的另一方法中,利用具有不同光敏度複數個像素單元在一次擷取中產生高動態範圍影像。除此之外,在擴大CMOS影像感測器之動態範圍的再一方法中,結合以不同曝光量擷取之複數筆影像資料而產生高動態範圍影像。儘管如此,在以上述方法擴大CMOS影像感測器之動態範圍時,還是需要考量其他問題,例如運動模糊(motion blur)、運動假影(motion artifacts)、處理時間增加和硬體成本增加等。 In a method of expanding the dynamic range of a CMOS image sensor, dark noise or full well capacity is reduced. In another method of expanding the dynamic range of a CMOS image sensor, a plurality of pixel cells having different photosensitivity are used to produce a high dynamic range image in one capture. In addition, in yet another method of expanding the dynamic range of the CMOS image sensor, a high dynamic range image is produced in combination with a plurality of image data captured at different exposures. Nonetheless, when expanding the dynamic range of CMOS image sensors in the above manner, other issues such as motion blur, motion artifacts, increased processing time, and increased hardware cost are still to be considered.

有鑑於此,本發明提供一種成本低效益高的方法以產生沒有運動模糊和運動假影的高動態範圍影像。 In view of this, the present invention provides a cost effective method to produce high dynamic range images without motion blur and motion artifacts.

在一實施例中,本發明提供一種產生高動態範圍影像的方法,包括:藉由一影像感測陣列,利用一第一曝光值以及一第二曝光值擷取一場景之一曝光隨空間變化之影像資料,其中該第一曝光值大於該第二曝光值;再取樣該曝光隨空間變化之影像資料以取得該場景對應該第一曝光值的一第一影像資料以及該場景對應該第二曝光值的一第二影像資料;根據該第一影像資料以及該第二影像資料,決定一高動態範圍影像資料之每一像素的一運動索引;根據該第一影像資料之一對應像素的第一像素值、該第二影像資料之一對應像素的第二影像值以及該運動索引,決定該高動態範圍影像資料之每一像素的一像素值;以及輸出該高動態範圍影像資料。 In one embodiment, the present invention provides a method for generating a high dynamic range image, comprising: capturing an exposure of a scene with a first exposure value and a second exposure value by an image sensing array; Image data, wherein the first exposure value is greater than the second exposure value; re-sampling the image data of the exposure space-dependent to obtain a first image data corresponding to the first exposure value of the scene and the scene corresponding to the second a second image data of the exposure value; determining, according to the first image data and the second image data, a motion index of each pixel of the high dynamic range image data; and corresponding to the pixel according to the first image data a pixel value, a second image value corresponding to the pixel of the second image data, and the motion index, determining a pixel value of each pixel of the high dynamic range image data; and outputting the high dynamic range image data.

在另一實施例中,本發明提供一種產生高動態範圍 影像的裝置,包括:一影像感測陣列,利用一第一曝光值以及一第二曝光值擷取一場景之一曝光隨空間變化之影像資料,其中該第一曝光值大於該第二曝光值;以及一影像處理器,耦接至該影像感測陣列,包括:一再取樣器,再取樣該曝光隨空間變化之影像資料以取得該場景對應該第一曝光值的一第一影像資料以及該場景對應該第二曝光值的一第二影像資料;一運動偵測器,根據該第一影像資料以及該第二影像資料,決定一高動態範圍影像資料之每一像素的一運動索引;以及一飽和偵測器,根據該第一影像資料之一對應像素的第一像素值、該第二影像資料之一對應像素的第二影像值以及該運動索引,決定該高動態範圍影像資料之每一像素的一像素值,並且輸出該高動態範圍影像資料。 In another embodiment, the present invention provides a high dynamic range The image sensing device includes: an image sensing array, wherein a first exposure value and a second exposure value are used to capture one of the scenes to expose the spatially varying image data, wherein the first exposure value is greater than the second exposure value And an image processor coupled to the image sensing array, comprising: a resampler, and sampling the spatially varying image data to obtain a first image data corresponding to the first exposure value of the scene and the The scene corresponds to a second image data of the second exposure value; a motion detector determines a motion index of each pixel of the high dynamic range image data according to the first image data and the second image data; a saturation detector, determining a high dynamic range image data according to a first pixel value corresponding to one of the first image data, a second image value corresponding to the pixel of the second image data, and the motion index One pixel value of one pixel, and the high dynamic range image data is output.

在再一實施例中,本發明提供一種電腦程式產品,其被一電子設備載入以使該電子設備執行一種產生高動態範圍影像的方法,包括:一第一程式碼,用於藉由一影像感測陣列,利用一第一曝光值以及一第二曝光值擷取一場景之一曝光隨空間變化之影像資料,其中該第一曝光值大於該第二曝光值;一第二程式碼,用於再取樣該曝光隨空間變化之影像資料以取得該場景對應該第一曝光值的一第一影像資料以及該場景對應該第二曝光值的一第二影像資料;一第三程式碼,用於根據該第一影像資料以及該第二影像資料,決定一高動態範圍影像資料之每一像素的一運動索引;一第四程式碼,用於根據該第一影像資料之一對應像素的第一像素值、該第二影像資料之一對應像素的第二影像值以及該運動索引,決定該高動態範圍影像資料之每一像素的一像素值;以及一第五程式碼,用於輸出該高動態範圍影像資料。 In still another embodiment, the present invention provides a computer program product loaded by an electronic device to cause the electronic device to perform a method for generating a high dynamic range image, comprising: a first code for The image sensing array uses a first exposure value and a second exposure value to capture one of the scenes to expose the spatially varying image data, wherein the first exposure value is greater than the second exposure value; a second code, And re-sampling the image data of the exposure according to the space to obtain a first image data corresponding to the first exposure value of the scene and a second image data corresponding to the second exposure value of the scene; a third code, And determining, according to the first image data and the second image data, a motion index of each pixel of the high dynamic range image data; and a fourth code for using the pixel corresponding to the first image data Determining, by the first pixel value, the second image value of the pixel corresponding to the second image data, and the motion index, determining a pixel value of each pixel of the high dynamic range image data; A fifth code for outputting the high dynamic range image data.

10‧‧‧產生高動態範圍影像的方法 10‧‧‧Methods for generating high dynamic range images

80‧‧‧產生高動態範圍影像的裝置 80‧‧‧Devices that produce high dynamic range images

310、320‧‧‧積分時間 310, 320‧‧‧ points time

800‧‧‧影像感測陣列 800‧‧‧Image Sensing Array

810‧‧‧影像處理器 810‧‧‧ image processor

811‧‧‧再取樣器 811‧‧‧Resampler

812‧‧‧運動偵測器 812‧‧‧ motion detector

813‧‧‧乘法器 813‧‧‧Multiplier

814‧‧‧混合器 814‧‧‧Mixer

815‧‧‧飽和偵測器 815‧‧‧Saturation detector

CIMD‧‧‧混合影像資料 CIMD‧‧‧ mixed image data

D‧‧‧像數值差 D‧‧‧like numerical difference

EV1‧‧‧第一曝光值 EV1‧‧‧ first exposure value

EV2‧‧‧第二曝光值 EV2‧‧‧second exposure value

GA‧‧‧曝光增益 GA‧‧‧ exposure gain

HIMD‧‧‧高動態範圍影像資料 HIMD‧‧‧High dynamic range image data

IMD1‧‧‧第一影像資料 IMD1‧‧‧ first image data

IMD2‧‧‧第二影像資料 IMD2‧‧‧Second imagery

MI‧‧‧運動索引 MI‧‧‧ Sports Index

MIMD‧‧‧增益影像資料 MIMD‧‧‧ Gain image data

P0、P1、P2、P3、P4、P5‧‧‧像素 P0, P1, P2, P3, P4, P5‧‧ ‧ pixels

P11-P15、P21-P25、P31-P35、P41-P45‧‧‧像素 P 11 -P 15 , P 21 -P 25 , P 31 -P 35 , P 41 -P 45 ‧‧ ‧ pixels

P1 11-P1 15、P1 21-P1 25、P1 31-P1 35、P1 41-P1 45‧‧‧像素 P 1 11 -P 1 15 , P 1 21 -P 1 25 , P 1 31 -P 1 35 , P 1 41 -P 1 45 ‧‧ ‧ pixels

P2 11-P2 15、P2 21-P2 25、P2 31-P2 35、P2 41-P2 45‧‧‧像素 P 2 11 -P 2 15 , P 2 21 -P 2 25 , P 2 31 -P 2 35 , P 2 41 -P 2 45 ‧‧ ‧ pixels

S1、S2、S3、S4‧‧‧距離 S1, S2, S3, S4‧‧‧ distance

S100、S200、S300、S400‧‧‧步驟 S100, S200, S300, S400‧‧‧ steps

SIMD‧‧‧曝光隨空間變化之影像資料 SIMD‧‧‧Exposure imagery with spatial variation

TH1、TH2‧‧‧閾值 TH1, TH2‧‧‧ threshold

Tre、Trs1、Trs2‧‧‧時間點 Tre, Trs1, Trs2‧‧‧

第1圖所示為根據本發明一實施例之產生高動態範圍影像之方法的流程圖。 1 is a flow chart of a method of generating a high dynamic range image in accordance with an embodiment of the present invention.

第2A圖所示為根據本發明一實施例之用以擷取曝光隨空間變化之影像資料的影像感測陣列的一示例性組成的示意圖。 FIG. 2A is a diagram showing an exemplary composition of an image sensing array for capturing image data of spatially varying exposure according to an embodiment of the invention.

第2B圖所示為根據本發明一實施例之用以擷取曝光隨空間變化之影像資料的影像感測陣列的另一示例性組成的示意圖。 FIG. 2B is a diagram showing another exemplary composition of an image sensing array for capturing image data of spatial variation of exposure according to an embodiment of the invention.

第3圖所示為根據本發明一實施例之不同曝光值的積分時間的時序圖。 Figure 3 is a timing diagram showing the integration time for different exposure values in accordance with an embodiment of the present invention.

第4圖所示為根據本發明一實施例之曝光隨空間變化之影像資料的示意圖。 Fig. 4 is a view showing image data of exposure varying with space according to an embodiment of the present invention.

第5A圖所示為根據本發明一實施例之對應第一曝光值之第一影像資料的示意圖。 FIG. 5A is a schematic diagram showing first image data corresponding to a first exposure value according to an embodiment of the invention.

第5B圖所示為根據本發明一實施例之對應第二曝光值之第二影像資料的示意圖。 FIG. 5B is a schematic diagram showing second image data corresponding to a second exposure value according to an embodiment of the invention.

第6圖所示為根據本發明一實施例之高動態範圍影像資料的示意圖。 Figure 6 is a schematic diagram of high dynamic range image data in accordance with an embodiment of the present invention.

第7圖所示為像素值差與運動索引之間之關係的示意圖。 Figure 7 is a diagram showing the relationship between the pixel value difference and the motion index.

第8圖所示為根據本發明一實施例之產生高動態範圍影像之裝置的示意圖。 Figure 8 is a schematic illustration of an apparatus for generating high dynamic range images in accordance with an embodiment of the present invention.

以下說明為本發明的實施例。其目的是要舉例說明本發明一般性的原則,不應視為本發明之限制,本發明之範圍當以申請專利範圍所界定者為準。 The following description is an embodiment of the present invention. The intent is to exemplify the general principles of the invention and should not be construed as limiting the scope of the invention, which is defined by the scope of the claims.

第1圖所示為根據本發明一實施例之產生高動態範圍影像的方法10的流程圖。首先,在步驟S100中,取得一場景對應第一曝光值EV1的第一影像資料IMD1以及該場景對應第二曝光值EV2的第二影像資料IMD2。在本說明書中,第一曝光值EV1大於第二曝光值EV2。第一影像資料IMD1以及第二影像資料IMD2係藉由再取樣該場景的曝光隨空間變化之影像資料,上述曝光隨空間變化之影像資料係由影像感測陣列(例如CMOS感測陣列)利用第一曝光值EV1以及第二曝光值EV2擷取而得。參照第2A圖,第2A圖所示為根據本發明一實施例之用以擷取曝光隨空間變化之影像資料的影像感測陣列的一示例性組成的示意圖。影像感測陣列覆蓋有Bayer式彩色濾波器陣列(Bayer color filter array)。如第2A圖所示,影像感測像素以不同曝光值(第一曝光值EV1以及第二曝光值EV2)感測該場景。須注意的是,本發明並不侷限於第2A圖所示的用以擷取曝光隨空間變化之影像資料的影像感測陣列的組成。舉例而言,第2B圖所示為根據本發明一實施例之用以擷取曝光隨空間變化之影像資料的影像感測陣列的另一示例性組成的示意圖。由於在包括CMOS感測陣列的影像裝置中沒有實體的快門,不同的曝光值係利用不同積分時間(電荷儲存時間)實現。第3圖所示為根據本發明一實施例之不同曝光值的積分時間的時序圖。第3圖所示為根據本發明一實施例之不同曝光值的積分時間的時序圖。所感測的資料係一列一列地從COMS感測陣列讀出,因 此,不同曝光值的積分時間被配置為結束於同一時間點但在不同時間點開始。在第3圖中,第一曝光值EV1的積分時間310開始於時間點Trs1並結束於時間點Tre,而第二曝光值EV2的積分時間320開始於時間點Trs2並結束於時間點Tre。舉例而言,在CMOS感測陣列的一列像素中,被配置為以第一曝光值感測資料的像素在時間點Trs1時被重設,而被配置為以第二曝光值感測資料的像素在時間點Trs2時被重設,然後該列像素之所有像素的感測資料在時間點Tre時被讀出。 1 is a flow chart of a method 10 of generating a high dynamic range image in accordance with an embodiment of the present invention. First, in step S100, the first image data IMD1 corresponding to the first exposure value EV1 of the scene and the second image data IMD2 corresponding to the second exposure value EV2 of the scene are obtained. In the present specification, the first exposure value EV1 is greater than the second exposure value EV2. The first image data IMD1 and the second image data IMD2 are obtained by resampling the image data of the exposure of the scene with spatial variation, and the image data of the exposure varying with space is utilized by the image sensing array (for example, CMOS sensing array). An exposure value EV1 and a second exposure value EV2 are obtained. Referring to FIG. 2A, FIG. 2A is a diagram showing an exemplary composition of an image sensing array for capturing image data of spatially varying exposure according to an embodiment of the invention. The image sensing array is covered with a Bayer color filter array. As shown in FIG. 2A, the image sensing pixels sense the scene with different exposure values (first exposure value EV1 and second exposure value EV2). It should be noted that the present invention is not limited to the composition of the image sensing array for capturing image data whose exposure varies with space as shown in FIG. 2A. For example, FIG. 2B is a diagram showing another exemplary composition of an image sensing array for capturing image data of spatial variation of exposure according to an embodiment of the invention. Since there is no physical shutter in an image device including a CMOS sensing array, different exposure values are achieved with different integration times (charge storage time). Figure 3 is a timing diagram showing the integration time for different exposure values in accordance with an embodiment of the present invention. Figure 3 is a timing diagram showing the integration time for different exposure values in accordance with an embodiment of the present invention. The sensed data is read out from the COMS sensing array in columns and columns. Thus, the integration time for different exposure values is configured to end at the same point in time but start at a different point in time. In FIG. 3, the integration time 310 of the first exposure value EV1 starts at the time point Trs1 and ends at the time point Tre, and the integration time 320 of the second exposure value EV2 starts at the time point Trs2 and ends at the time point Tre. For example, in a column of pixels of the CMOS sensing array, the pixels configured to sense the data with the first exposure value are reset at the time point Trs1, and the pixels configured to sense the data with the second exposure value. At the time point Trs2, it is reset, and then the sensing data of all the pixels of the column of pixels are read out at the time point Tre.

第4圖所示為根據本發明一實施例之曝光隨空間變化之影像資料SIMD的示意圖。曝光隨空間變化之影像資料SIMD由如第2A圖所配置之影像感測陣列所擷取,曝光隨空間變化之影像資料SIMD的每一像素包括對應第一曝光值EV1以及第二曝光值EV2其中之一的像素值。第5A圖所示為根據本發明一實施例之對應第一曝光值EV1之第一影像資料IMD1的示意圖。第5B圖所示為根據本發明一實施例之對應第二曝光值EV1之第二影像資料IMD2的示意圖。藉由再取樣第4圖所示的曝光隨空間變化之影像資料SIMD可取得第一影像資料IMD1以及第二影像資料IMD2。第一影像資料IMD1為對應第一曝光值EV1的影像資料矩陣,而第二影像資料IMD2為對應第二曝光值EV2的影像資料矩陣。舉例而言,第5A圖之第一影像資料IMD1中像素P 1 1,3的像素值以及第5B圖之第二影像資料IMD2中像素P 2 1,3的像素值分別由下列式子取得: ;以及 其中Sn表示像素Pn的中心位置與像素P2和P3的中心位置之間的距離,像素P2和P3的中心位置對應至像素P 1 1,3的中心位置以及P 2 1,3的中心位置,且V(P)表示像素P的像素值。須注意的是,本發明並不局限於上述再取樣。舉例而言,再取樣可根據用以擷取曝光隨空間變化之影像資料的影像感測陣列的配置而有所調整。 FIG. 4 is a schematic diagram showing an image data SIMD of exposure varying with space according to an embodiment of the invention. The image data of the exposure-space-changing image data SIMD is captured by the image sensing array configured as shown in FIG. 2A, and each pixel of the image data SIMD of the exposure-variant space includes a corresponding first exposure value EV1 and a second exposure value EV2. One of the pixel values. FIG. 5A is a schematic diagram showing the first image data IMD1 corresponding to the first exposure value EV1 according to an embodiment of the invention. FIG. 5B is a schematic diagram showing the second image data IMD2 corresponding to the second exposure value EV1 according to an embodiment of the invention. The first image data IMD1 and the second image data IMD2 can be obtained by resampling the image data SIMD of the exposure-variant image shown in FIG. The first image data IMD1 is a matrix of image data corresponding to the first exposure value EV1, and the second image data IMD2 is a matrix of image data corresponding to the second exposure value EV2. For example, the pixel values of the pixels P 1 1,3 in the first image data IMD1 in FIG. 5A and the pixel values of the pixels P 2 1,3 in the second image data IMD2 in the fifth image are respectively obtained by the following formula: ;as well as Wherein Sn represents the distance between the center position of the pixel P n and the center position of the pixels P 2 and P 3 , and the center positions of the pixels P 2 and P 3 correspond to the center position of the pixel P 1 1,3 and P 2 1, The center position of 3 , and V ( P ) represents the pixel value of the pixel P. It should be noted that the present invention is not limited to the above resampling. For example, resampling may be adjusted based on the configuration of the image sensing array used to capture image data that varies spatially with exposure.

接著,在步驟S200中,根據第一影像資料IMD1以及第二影像資料IMD2決定高動態範圍影像資料HIMD之每一像素的運動索引。第6圖所示為根據本發明一實施例之高動態範圍影像資料HIMD的示意圖。高動態範圍影像資料HIMD為高動態範圍影像資料的矩陣。高動態範圍影像資料HIMD、第一影像資料IMD1以及第二影像資料IMD2具有相同的尺寸。高動態範圍影像資料HIMD之每一像素的運動索引係根據高動態範圍影像資料HIMD之每一像素的像素值差以及閾值TH1和TH2決定。第7圖所示為像素值差D與運動索引MI之間之關係的示意圖。如第7圖所示,若高動態範圍影像資料HIMD一像素的像素值差小於或等於第一閾值TH1,此像素的運動索引為0。若高動態範圍影像資料HIMD一像素的像素值差大於或等於第二閾值TH2,此像素的運動索引為1。除此之外,若高動態範圍影像資料HIMD一像素的像素值差大於第一閾值TH1且小於第二閾值TH2,則此像素的運動索引為大於0且小於1的一數值,並且像素值差越大則運動索引越大。舉例而言,如第7圖所示,若高動態範圍影像資料HIMD一像素的像素值差X大於第一閾值TH1且小於第二閾值TH2,則此像素的運動索 引等於。閾值TH1與TH2可根據雜訊容忍度(noise tolerance)決定。雖然在第7圖中,閾值TH1與TH2之區間中的運動索引和像數值差之間的關係為線性,但本發明並不局限於此。 Next, in step S200, a motion index of each pixel of the high dynamic range image data HIMD is determined according to the first image data IMD1 and the second image data IMD2. FIG. 6 is a schematic diagram of a high dynamic range image data HIMD according to an embodiment of the invention. The high dynamic range image data HIMD is a matrix of high dynamic range image data. The high dynamic range image data HIMD, the first image data IMD1, and the second image data IMD2 have the same size. The motion index of each pixel of the high dynamic range image data HIMD is determined according to the pixel value difference of each pixel of the high dynamic range image data HIMD and the threshold values TH1 and TH2. Fig. 7 is a diagram showing the relationship between the pixel value difference D and the motion index MI. As shown in FIG. 7, if the pixel value difference of one pixel of the high dynamic range image data HIMD is less than or equal to the first threshold TH1, the motion index of the pixel is 0. If the pixel value difference of one pixel of the high dynamic range image data HIMD is greater than or equal to the second threshold TH2, the motion index of the pixel is 1. In addition, if the pixel value difference of one pixel of the high dynamic range image data HIMD is greater than the first threshold TH1 and less than the second threshold TH2, the motion index of the pixel is a value greater than 0 and less than 1, and the pixel value difference is The larger the motion index, the larger the index. For example, as shown in FIG. 7, if the pixel value difference X of one pixel of the high dynamic range image data HIMD is greater than the first threshold TH1 and smaller than the second threshold TH2, the motion index of the pixel is equal to . The thresholds TH1 and TH2 can be determined according to noise tolerance. Although in Fig. 7, the relationship between the motion index and the image value difference in the interval between the thresholds TH1 and TH2 is linear, the present invention is not limited thereto.

高動態範圍影像資料HIMD一像素的像素值差係根據第一影像資料IMD1以及第二影像資料IMD2決定。高動態範圍影像資料HIMD每一像素P i,j 的像數值差D i,j 係根據下列計算: The pixel value difference of one pixel of the high dynamic range image data HIMD is determined according to the first image data IMD1 and the second image data IMD2. The image value difference D i,j of each pixel P i,j of the high dynamic range image data HIMD is calculated according to the following:

其中GA表示一曝光增益並等於2(EV1-EV2)。由於第一影像資料IMD1與第二影像資料IMD2係根據不同曝光值取得,因此,當計算像數值差時,第二影像資料IMD2中的第二像素值必須乘上上述曝光增益(也就是2(EV1-EV2))。 Where GA represents an exposure gain and is equal to 2 (EV1-EV2) . Since the first image data IMD1 and the second image data IMD2 are obtained according to different exposure values, when calculating the image difference, the second pixel value in the second image data IMD2 must be multiplied by the exposure gain (that is, 2 ( EV1-EV2) ).

以第6圖之高動態範圍影像資料HIMD的像素P 3,3為例,像素值差D 3,3係根據下列計算: Taking the pixel P 3,3 of the high dynamic range image data HIMD of FIG. 6 as an example, the pixel value difference D 3, 3 is calculated according to the following:

在步驟S300中,高動態範圍影像資料HIMD之每一像素P i,j 的像素值V(P i,j )係根據對應第一像素值V(P 1 i,j )、對應第二像素值V(P 2 i,j )、運動索引M i,j 以及飽和閾值TH決定。像素值V(P i,j )係根據下列決定: In step S300, the pixel value V ( P i,j ) of each pixel P i,j of the high dynamic range image data HIMD is based on the corresponding first pixel value V ( P 1 i,j ) and the corresponding second pixel value. V ( P 2 i,j ), the motion index M i,j and the saturation threshold TH are determined. The pixel value V ( P i,j ) is determined according to the following:

若第一像素值V(P 1 i,j )大於或等於飽和閾值TH,也就 是說,像素P 1 i,j 有過曝現象,則像素值V(P i,j )等於第二像素值V(P 2 i,j )乘上曝光增益。曝光增益的值等於2(EV1-EV2)。另一方面,若第一像素值V(P 1 i,j )小於飽和閾值TH,也就是說,像素P 1 i,j 沒有過曝現象,則像素值V(P i,j )為第一像素值V(P 1 i,j )以及乘上曝光增益之第二像素值V(P 2 i,j )二者根據運動索引M i,j 的一組合,如上列式子所示。 If the first pixel value V ( P 1 i,j ) is greater than or equal to the saturation threshold TH, that is, the pixel P 1 i,j has an overexposure phenomenon, the pixel value V ( P i,j ) is equal to the second pixel value. V ( P 2 i,j ) is multiplied by the exposure gain. The value of the exposure gain is equal to 2 (EV1-EV2) . On the other hand, if the first pixel value V ( P 1 i,j ) is less than the saturation threshold TH, that is, the pixel P 1 i,j has no overexposure, the pixel value V ( P i,j ) is the first The pixel value V ( P 1 i,j ) and the second pixel value V ( P 2 i,j ) multiplied by the exposure gain are both according to a combination of the motion indices M i,j as shown in the above formula.

在步驟S400中,輸出高動態範圍影像資料HIMD。如上所述,在本發明中,當第一像素值V(P 1 i,j )小於飽和閾值TH時,運動索引M i,j 用來結合第一像素值V(P 1 i,j )以及乘上曝光增益GA之第二像素值V(P 2 i,j )以產生像素值V(P i,j ),而非在第一像素值V(P 1 i,j )小於飽和閾值TH時直接將像素值V(P i,j )設定為第一像素值V(P 1 i,j )。因此,當根據本發明產生高動態範圍影像資料HIMD時,可減少運動模糊和/或運動假影。 In step S400, the high dynamic range image data HIMD is output. As described above, in the present invention, when the first pixel value V ( P 1 i,j ) is smaller than the saturation threshold TH, the motion index M i,j is used to combine the first pixel value V ( P 1 i,j ) and Multiplying the second pixel value V ( P 2 i,j ) of the exposure gain GA to generate the pixel value V ( P i,j ), rather than when the first pixel value V ( P 1 i,j ) is less than the saturation threshold TH The pixel value V ( P i,j ) is directly set to the first pixel value V ( P 1 i,j ). Therefore, when high dynamic range image data HIMD is generated according to the present invention, motion blur and/or motion artifacts can be reduced.

第8圖所示為根據本發明一實施例之產生高動態範圍影像之裝置80的示意圖。產生高動態範圍影像之裝置80包括影像感測陣列800,例如CMOS感測陣列,以及耦接至影像感測陣列800的影像處理器810。影像處理器810包括再取樣器811、運動偵測器812、乘法器813、混合器814以及飽和偵測器815。影像處理器810中的模組可包括影像處理硬體、儲存於非暫時性電腦讀取媒介並可由資料處理器執行的軟體、或其任意組合,並被配置為執行上述功能。 Figure 8 is a schematic illustration of an apparatus 80 for generating high dynamic range images in accordance with an embodiment of the present invention. The device 80 for generating a high dynamic range image includes an image sensing array 800, such as a CMOS sensing array, and an image processor 810 coupled to the image sensing array 800. The image processor 810 includes a resampler 811, a motion detector 812, a multiplier 813, a mixer 814, and a saturation detector 815. The modules in image processor 810 can include image processing hardware, software stored in a non-transitory computer reading medium and executable by a data processor, or any combination thereof, and configured to perform the functions described above.

再取樣器811再取樣影像感測陣列800利用第一曝光值EV1以及第二曝光值EV2所擷取的一場景的曝光隨空間變化之影像資料SIMD,以取得該場景對應第一曝光值EV1的第一影像資 料IMD1以及該場景對應第二曝光值V2的第二影像資料IMD2。曝光隨空間變化之影像資料SIMD、再取樣方法、第一影像資料IMD1以及第二影像資料IMD2的細節已於上述因此為了說明簡潔起見不再復述。 The resampler 811 resamples the image sensing array 800 with the first exposure value EV1 and the second exposure value EV2 to capture the spatially varying image data SIMD of a scene to obtain the first exposure value EV1 corresponding to the scene. First image The IMD 1 and the scene corresponding to the second image data IMD2 of the second exposure value V2. The details of the exposure-to-spatial image data SIMD, the resampling method, the first image data IMD1, and the second image data IMD2 have been described above and will not be repeated for the sake of brevity.

乘法器813將第二影像資料IMD2之每一像素P 2 i,j 的像素值乘上曝光增益GA並產生增益影像資料MIMD。曝光增益的值等於2(EV1-EV2)。運動偵測器812、、混合器814以及飽和偵測器815根據第一影像資料IMD1以及第二影像資料IMD2決定高動態範圍影像資料HIMD之每一像素P i,j 的運動索引M i,j ,並將高動態範圍影像資料HIMD之每一像素P i,j 的運動索引M i,j 輸出至混合器814。決定運動索引的細節已於上述因此為了說明簡潔起見不再復述。 The multiplier 813 multiplies the pixel value of each pixel P 2 i,j of the second image data IMD2 by the exposure gain GA and generates the gain image data MIMD. The value of the exposure gain is equal to 2 (EV1-EV2) . The motion detector 812, the mixer 814, and the saturation detector 815 determine the motion index M i,j of each pixel P i,j of the high dynamic range image data HIMD according to the first image data IMD1 and the second image data IMD2 . And outputting the motion index M i,j of each pixel P i,j of the high dynamic range image data HIMD to the mixer 814. The details of determining the motion index have been described above and will not be repeated for the sake of brevity.

混合器814接收第一影像資料IMD1、增益影像資料MIMD以及高動態範圍影像資料HIMD之每一像素P i,j 的運動索引M i,j 然後產生混合影像資料CIMD。混合影像資料CIMD之每一像素P c i,j 的像素值等於第一影像資料IMD1中對應像素P 1 i,j 的像素值以及增益影像資料MIMD中對應像素P M i,j 的像素值根據對應運動索引M i,j 的結合,即,。飽和偵測器815接收混合影像資料CIMD以及增益影像資料MIMD,決定高動態範圍影像資料HIMD之每一像素P i,j 的像素值,以及輸出高動態範圍影像資料HIMD。高動態範圍影像資料HIMD每一像素P i,j 的像數差係根據下列決定: The mixer 814 receives the motion index M i,j of each pixel P i,j of the first image data IMD1, the gain image data MIMD, and the high dynamic range image data HIMD and then generates the mixed image data CIMD. The pixel value of each pixel P c i,j of the mixed image data CIMD is equal to the pixel value of the corresponding pixel P 1 i,j in the first image data IMD1 and the pixel value of the corresponding pixel P M i,j in the gain image data MIMD according to Corresponding to the combination of the motion index M i,j , ie, . The saturation detector 815 receives the mixed image data CIMD and the gain image data MIMD, determines the pixel value of each pixel P i,j of the high dynamic range image data HIMD, and outputs the high dynamic range image data HIMD. The difference in the number of pixels per pixel P i,j of the high dynamic range image data HIMD is determined according to the following:

影像處理器810的其他模組,例如色彩去馬賽克化模組,可更進一步處理高動態範圍影像資料HIMD。 Other modules of image processor 810, such as a color demosaicing module, can further process high dynamic range image data HIMD.

本發明之方法,或特定型態或其部份,可以以程式碼的型態存在。程式碼可以包含於實體媒體,如軟碟、光碟片、硬碟、或是任何其他電子設備或非暫時性之機器可讀取(如電腦可讀取)儲存媒體,亦或不限於外在形式之電腦程式產品,其中,當程式碼被機器,如電腦載入且執行時,此機器變成用以參與本發明之裝置或系統,且可執行本發明之方法步驟。程式碼也可以透過一些傳送媒體,如電線或電纜、光纖、或是任何傳輸型態進行傳送,其中,當程式碼被電子設備或機器,如電腦接收、載入且執行時,此機器變成用以參與本發明之系統或裝置。當在一般用途處理單元實作時,程式碼結合處理單元提供一操作類似於應用特定邏輯電路之獨特裝置。 The method of the invention, or a particular type or portion thereof, may exist in the form of a code. The code may be embodied in a physical medium such as a floppy disk, a CD, a hard disk, or any other electronic device or a non-transitory machine readable (eg computer readable) storage medium, or is not limited to an external form. A computer program product, wherein when the code is loaded and executed by a machine, such as a computer, the machine becomes a device or system for participating in the present invention and the method steps of the present invention can be performed. The code can also be transmitted over some transmission medium, such as wire or cable, fiber optics, or any transmission type, where the machine becomes available when the code is received, loaded, and executed by an electronic device or machine, such as a computer. To participate in the system or device of the present invention. When implemented in a general purpose processing unit, the code combination processing unit provides a unique means of operation similar to application specific logic.

在一實施例中,本發明提供一種電腦程式產品,其被一電子設備載入以使該電子設備執行一種產生高動態範圍影像的方法,包括:一第一程式碼,用於藉由一影像感測陣列,利用一第一曝光值以及一第二曝光值擷取一場景之一曝光隨空間變化之影像資料,其中該第一曝光值大於該第二曝光值;一第二程式碼,用於再取樣該曝光隨空間變化之影像資料以取得該場景對應該第一曝光值的一第一影像資料以及該場景對應該第二曝光值的一第二影像資料;一第三程式碼,用於根據該第一影像資料以及該第二影像資料,決定一高動態範圍影像資料之每一像素的一運動索引;一第四程式碼,用於根據該第一影像資料之一對應像素的第一像素值、該第二影像資料之一對應像素的第二影像值以及 該運動索引,決定該高動態範圍影像資料之每一像素的一像素值;以及一第五程式碼,用於輸出該高動態範圍影像資料。第一影像資料、第二影像資料、運動索引以及高動態範圍影像資料之每一像素的像素值的決定以於上列敘述,為說明簡潔起見於此不再復述。 In one embodiment, the present invention provides a computer program product loaded by an electronic device to cause the electronic device to perform a method for generating a high dynamic range image, comprising: a first code for using an image Sensing the array, using a first exposure value and a second exposure value to capture one of the scenes to expose the spatially varying image data, wherein the first exposure value is greater than the second exposure value; a second code is used And re-sampling the image data of the exposure according to the space to obtain a first image data corresponding to the first exposure value of the scene and a second image data corresponding to the second exposure value of the scene; a third code, Determining, according to the first image data and the second image data, a motion index of each pixel of the high dynamic range image data; and a fourth code for using the corresponding pixel according to the first image data a pixel value, a second image value corresponding to one of the second image data, and The motion index determines a pixel value of each pixel of the high dynamic range image data; and a fifth code for outputting the high dynamic range image data. The determination of the pixel value of each pixel of the first image data, the second image data, the motion index, and the high dynamic range image data is described above, and will not be repeated herein for the sake of brevity.

以上所述為實施例的概述特徵。所屬技術領域中具有通常知識者應可以輕而易舉地利用本發明為基礎設計或調整以實行相同的目的和/或達成此處介紹的實施例的相同優點。所屬技術領域中具有通常知識者也應了解相同的配置不應背離本創作的精神與範圍,在不背離本創作的精神與範圍下他們可做出各種改變、取代和交替。說明性的方法僅表示示範性的步驟,但這些步驟並不一定要以所表示的順序執行。可另外加入、取代、改變順序和/或消除步驟以視情況而作調整,並與所揭露的實施例精神和範圍一致。 The above is an overview feature of the embodiment. Those having ordinary skill in the art should be able to use the present invention as a basis for design or adaptation to achieve the same objectives and/or achieve the same advantages of the embodiments described herein. It should be understood by those of ordinary skill in the art that the same configuration should not depart from the spirit and scope of the present invention, and various changes, substitutions and substitutions can be made without departing from the spirit and scope of the present invention. The illustrative methods are merely illustrative of the steps, but are not necessarily performed in the order presented. The steps may be additionally added, substituted, changed, and/or eliminated, as appropriate, and are consistent with the spirit and scope of the disclosed embodiments.

10‧‧‧產生高動態範圍影像的方法 10‧‧‧Methods for generating high dynamic range images

S100、S200、S300、S400‧‧‧步驟 S100, S200, S300, S400‧‧‧ steps

Claims (12)

一種產生高動態範圍影像的方法,包括:藉由一影像感測陣列,利用一第一曝光值以及一第二曝光值擷取一場景之一曝光隨空間變化之影像資料,其中該第一曝光值大於該第二曝光值;再取樣該曝光隨空間變化之影像資料以取得該場景對應該第一曝光值的一第一影像資料以及該場景對應該第二曝光值的一第二影像資料;根據該第一影像資料以及該第二影像資料,決定一高動態範圍影像資料之每一像素的一運動索引;根據該第一影像資料之一對應像素的第一像素值、該第二影像資料之一對應像素的第二影像值以及該運動索引,決定該高動態範圍影像資料之每一像素的一像素值;以及輸出該高動態範圍影像資料。 A method for generating a high dynamic range image, comprising: capturing an image of a spatially varying image by using a first exposure value and a second exposure value by using an image sensing array, wherein the first exposure The value is greater than the second exposure value; the image data of the exposure varies with space is resampled to obtain a first image data corresponding to the first exposure value of the scene and a second image data corresponding to the second exposure value of the scene; Determining, according to the first image data and the second image data, a motion index of each pixel of the high dynamic range image data; and corresponding to the first pixel value of the pixel and the second image data according to the first image data Corresponding to a second image value of the pixel and the motion index, determining a pixel value of each pixel of the high dynamic range image data; and outputting the high dynamic range image data. 如申請專利範圍第1項所述之產生高動態範圍影像的方法,其中該高動態範圍影像資料之每一像素P i,j 的該運動索引M i,j 係根據下列決定:若該高動態範圍影像資料之每一像素P i,j 的像素值差小於或等於一第一閾值,該運動索引M i,j 為0;若該高動態範圍影像資料之每一像素P i,j 的該像素值差大於或等於一第二閾值,該運動索引M i,j 為1;以及若該高動態範圍影像資料之每一像素P i,j 的該像素值差大 於該第一閾值且小於該第二閾值,該運動索引M i,j 為大於0且小於1之一數值,其中該像素值差越大,則該運動索引M i,j 越大。 The method for generating a high dynamic range image as described in claim 1, wherein the motion index M i , j of each pixel P i , j of the high dynamic range image data is determined according to the following: The pixel value difference of each pixel P i , j of the range image data is less than or equal to a first threshold, and the motion index M i , j is 0; if the pixel of each high dynamic range image data P i , j The pixel value difference is greater than or equal to a second threshold, the motion index M i , j is 1; and if the pixel value difference of each pixel P i , j of the high dynamic range image data is greater than the first threshold and less than the The second threshold, the motion index M i , j is a value greater than 0 and less than 1, wherein the larger the pixel value difference is, the larger the motion index M i , j is. 如申請專利範圍第2項所述之產生高動態範圍影像的方法,其中該高動態範圍影像資料之每一像素P i,j 的該像素值差D i,j 係根據下列計算: 其中V(P)表示一像素P的像素值,GA表示一曝光增益並等於2(EV1-EV2),其中EV1表示該第一曝光值,EV2表示該第二曝光值。 The method for generating a high dynamic range image as described in claim 2, wherein the pixel value difference D i , j of each pixel P i , j of the high dynamic range image data is calculated according to the following: Where V ( P ) represents the pixel value of a pixel P , GA represents an exposure gain and is equal to 2 (EV1-EV2) , where EV1 represents the first exposure value and EV2 represents the second exposure value. 如申請專利範圍第3項所述之產生高動態範圍影像的方法,其中該高動態範圍影像資料之每一像素P i,j 的該像素值V(P i,j )係根據下列決定:若該第一像素值大於或等於一飽和閾值,該高動態範圍影像資料之每一像素P i,j 的該像素值V(P i,j )等於該第二像素值乘上該曝光增益;以及若該第一像素值小於一飽和閾值,該高動態範圍影像資料之每一像素P i,j 的該像素值V(P i,j )等於(1-M i,j )×(該第一像素值)+M i,j ×GA×(該第二像素值)。 The method for generating a high dynamic range image as described in claim 3, wherein the pixel value V ( P i , j ) of each pixel P i , j of the high dynamic range image data is determined according to the following: The first pixel value is greater than or equal to a saturation threshold, and the pixel value V ( P i , j ) of each pixel P i , j of the high dynamic range image data is equal to the second pixel value multiplied by the exposure gain; If the first pixel value is less than a saturation threshold, the pixel value V ( P i , j ) of each pixel P i , j of the high dynamic range image data is equal to (1- M i , j )× (the first Pixel value) + M i , j × GA × (this second pixel value). 一種產生高動態範圍影像的裝置,包括: 一影像感測陣列,利用一第一曝光值以及一第二曝光值擷取一場景之一曝光隨空間變化之影像資料,其中該第一曝光值大於該第二曝光值;以及一影像處理器,耦接至該影像感測陣列,包括:一再取樣器,再取樣該曝光隨空間變化之影像資料以取得該場景對應該第一曝光值的一第一影像資料以及該場景對應該第二曝光值的一第二影像資料;一運動偵測器,根據該第一影像資料以及該第二影像資料,決定一高動態範圍影像資料之每一像素的一運動索引;以及一飽和偵測器,根據該第一影像資料之一對應像素的第一像素值、該第二影像資料之一對應像素的第二影像值以及該運動索引,決定該高動態範圍影像資料之每一像素的一像素值,並且輸出該高動態範圍影像資料。 A device for generating high dynamic range images, comprising: An image sensing array, using a first exposure value and a second exposure value to capture one of a scene to expose spatially varying image data, wherein the first exposure value is greater than the second exposure value; and an image processor And coupled to the image sensing array, comprising: a resampler, and then sampling the image data of the exposure according to the space to obtain a first image data corresponding to the first exposure value of the scene and the scene corresponding to the second exposure a second image data of the value; a motion detector, determining a motion index of each pixel of the high dynamic range image data according to the first image data and the second image data; and a saturation detector, Determining a pixel value of each pixel of the high dynamic range image data according to the first pixel value of the corresponding pixel of the first image data, the second image value of the corresponding pixel of the second image data, and the motion index And outputting the high dynamic range image data. 如申請專利範圍第5項所述之產生高動態範圍影像的裝置,其中該運動偵測器根據下列決定該高動態範圍影像資料之每一像素P i,j 的該運動索引M i,j ;若該高動態範圍影像資料之每一像素P i,j 的像素值差小於或等於一第一閾值,該運動索引M i,j 為0;若該高動態範圍影像資料之每一像素P i,j 的該像素值差大於或等於一第二閾值,該運動索引M i,j 為1;以及若該高動態範圍影像資料之每一像素P i,j 的該像素值差大 於該第一閾值且小於該第二閾值,該運動索引M i,j 為大於0且小於1之一數值,其中該像素值差越大,則該運動索引M i,j 越大。 The apparatus for generating a high dynamic range image according to claim 5, wherein the motion detector determines the motion index M i , j of each pixel P i , j of the high dynamic range image data according to the following; If the pixel value difference of each pixel P i , j of the high dynamic range image data is less than or equal to a first threshold, the motion index M i , j is 0; if each pixel of the high dynamic range image data P i The pixel value difference of j is greater than or equal to a second threshold, the motion index M i , j is 1; and if the pixel value difference of each pixel P i , j of the high dynamic range image data is greater than the first The threshold value is less than the second threshold, and the motion index M i , j is a value greater than 0 and less than 1, wherein the larger the pixel value difference is, the larger the motion index M i , j is. 如申請專利範圍第6項所述之產生高動態範圍影像的裝置,其中該高動態範圍影像資料之每一像素P i,j 的該像素值差D i,j 係根據下列計算: 其中V(P)表示一像素P的像素值,GA表示一曝光增益並等於2(EV1-EV2),其中EV1表示該第一曝光值,EV2表示該第二曝光值。 The apparatus for generating a high dynamic range image as described in claim 6 wherein the pixel value difference D i , j of each pixel P i , j of the high dynamic range image data is calculated according to the following: Where V ( P ) represents the pixel value of a pixel P , GA represents an exposure gain and is equal to 2 (EV1-EV2) , where EV1 represents the first exposure value and EV2 represents the second exposure value. 如申請專利範圍第7項所述之產生高動態範圍影像的裝置,其中該飽和偵測器根據下列決定該高動態範圍影像資料之每一像素P i,j 的該像素值V(P i,j ):若該第一像素值大於或等於一飽和閾值,該高動態範圍影像資料之每一像素P i,j 的該像素值V(P i,j )等於該第二像素值乘上該曝光增益;以及若該第一像素值小於一飽和閾值,該高動態範圍影像資料之每一像素P i,j 的該像素值V(P i,j )等於(1-M i,j )×(該第一像素值)+M i,j ×GA×(該第二像素值)。 The apparatus for generating a high dynamic range image according to claim 7 , wherein the saturation detector determines the pixel value V ( P i of each pixel P i , j of the high dynamic range image data according to the following : j ): if the first pixel value is greater than or equal to a saturation threshold, the pixel value V ( P i , j ) of each pixel P i , j of the high dynamic range image data is equal to the second pixel value multiplied by the An exposure gain; and if the first pixel value is less than a saturation threshold, the pixel value V ( P i , j ) of each pixel P i , j of the high dynamic range image data is equal to (1- M i , j )× (The first pixel value) + M i , j × GA × (the second pixel value). 一種電腦程式產品,其被一電子設備載入以使該電 子設備執行一種產生高動態範圍影像的方法,包括:一第一程式碼,用於藉由一影像感測陣列,利用一第一曝光值以及一第二曝光值擷取一場景之一曝光隨空間變化之影像資料,其中該第一曝光值大於該第二曝光值;一第二程式碼,用於再取樣該曝光隨空間變化之影像資料以取得該場景對應該第一曝光值的一第一影像資料以及該場景對應該第二曝光值的一第二影像資料;一第三程式碼,用於根據該第一影像資料以及該第二影像資料,決定一高動態範圍影像資料之每一像素的一運動索引;一第四程式碼,用於根據該第一影像資料之一對應像素的第一像素值、該第二影像資料之一對應像素的第二影像值以及該運動索引,決定該高動態範圍影像資料之每一像素的一像素值;以及一第五程式碼,用於輸出該高動態範圍影像資料。 a computer program product that is loaded by an electronic device to make the electricity The sub-device performs a method for generating a high dynamic range image, comprising: a first code for capturing an exposure of a scene by using a first exposure value and a second exposure value by using an image sensing array Spatially varying image data, wherein the first exposure value is greater than the second exposure value; a second code for resampling the image data of the exposure spatially varying to obtain a first exposure value corresponding to the scene An image data and a second image data corresponding to the second exposure value of the scene; a third code for determining each of the high dynamic range image data according to the first image data and the second image data a motion index of the pixel; a fourth code, configured to determine, according to the first pixel value of the corresponding pixel of the first image data, the second image value of the corresponding pixel of the second image data, and the motion index a pixel value of each pixel of the high dynamic range image data; and a fifth code for outputting the high dynamic range image data. 如申請專利範圍第9項所述之電腦程式產品,其中該高動態範圍影像資料之每一像素P i,j 的該運動索引M i,j 係根據下列決定:若該高動態範圍影像資料之每一像素P i,j 的像素值差小於或等於一第一閾值,該運動索引M i,j 為0;若該高動態範圍影像資料之每一像素P i,j 的該像素值差大於或等於一第二閾值,該運動索引M i,j 為1;以及 若該高動態範圍影像資料之每一像素P i,j 的該像素值差大於該第一閾值且小於該第二閾值,該運動索引M i,j 為大於0且小於1之一數值,其中該像素值差越大,則該運動索引M i,j 越大。 The computer program product of claim 9, wherein the motion index M i , j of each pixel P i , j of the high dynamic range image data is determined according to the following: if the high dynamic range image data The pixel value difference of each pixel P i , j is less than or equal to a first threshold, and the motion index M i , j is 0; if the pixel value difference of each pixel P i , j of the high dynamic range image data is greater than Or equal to a second threshold, the motion index M i , j is 1; and if the pixel value difference of each pixel P i , j of the high dynamic range image data is greater than the first threshold and less than the second threshold, The motion index M i , j is a value greater than 0 and less than 1, wherein the larger the pixel value difference, the larger the motion index M i , j . 如申請專利範圍第10項所述之電腦程式產品,其中該高動態範圍影像資料之每一像素P i,j 的該像素值差D i,j 係根據下列計算: 其中V(P)表示一像素P的像素值,GA表示一曝光增益並等於2(EV1-EV2),其中EV1表示該第一曝光值,EV2表示該第二曝光值。 The computer program product of claim 10, wherein the pixel value difference D i , j of each pixel P i , j of the high dynamic range image data is calculated according to the following: Where V ( P ) represents the pixel value of a pixel P , GA represents an exposure gain and is equal to 2 (EV1-EV2) , where EV1 represents the first exposure value and EV2 represents the second exposure value. 如申請專利範圍第11項所述之電腦程式產品,其中該高動態範圍影像資料之每一像素P i,j 的該像素值V(P i,j )係根據下列決定:若該第一像素值大於或等於一飽和閾值,該高動態範圍影像資料之每一像素P i,j 的該像素值V(P i,j )等於該第二像素值乘上該曝光增益;以及若該第一像素值小於一飽和閾值,該高動態範圍影像資料之每一像素P i,j 的該像素值V(P i,j )等於(1-M i,j )×(該第一像素值)+M i,j ×GA×(該第二像素值)。 The computer program product of claim 11, wherein the pixel value V ( P i , j ) of each pixel P i , j of the high dynamic range image data is determined according to the following: if the first pixel The value is greater than or equal to a saturation threshold, and the pixel value V ( P i , j ) of each pixel P i , j of the high dynamic range image data is equal to the second pixel value multiplied by the exposure gain; and if the first The pixel value is less than a saturation threshold, and the pixel value V ( P i , j ) of each pixel P i , j of the high dynamic range image data is equal to (1- M i , j )×(the first pixel value)+ M i , j × GA × (the second pixel value).
TW103115135A 2013-10-31 2014-04-28 Method and apparatus for generating high dynamic range images and computer program product TW201517624A (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US14/068,473 US20150116525A1 (en) 2013-10-31 2013-10-31 Method for generating high dynamic range images

Publications (1)

Publication Number Publication Date
TW201517624A true TW201517624A (en) 2015-05-01

Family

ID=52994968

Family Applications (1)

Application Number Title Priority Date Filing Date
TW103115135A TW201517624A (en) 2013-10-31 2014-04-28 Method and apparatus for generating high dynamic range images and computer program product

Country Status (3)

Country Link
US (1) US20150116525A1 (en)
CN (1) CN104601905A (en)
TW (1) TW201517624A (en)

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9871965B2 (en) * 2016-02-03 2018-01-16 Texas Instruments Incorporated Image processing for wide dynamic range (WDR) sensor data
US10863105B1 (en) * 2017-06-27 2020-12-08 Amazon Technologies, Inc. High dynamic range imaging for event detection and inventory management
CN110892709B (en) * 2017-08-18 2021-06-08 富士胶片株式会社 Imaging device, method for controlling imaging device, and program for controlling imaging device
DE102017219307B4 (en) * 2017-10-27 2019-07-11 Siemens Healthcare Gmbh Method and system for compensating motion artifacts by machine learning
CN109936700B (en) * 2017-12-19 2020-12-29 群光电子股份有限公司 Image series flow image quality processing system and image series flow image quality processing method
US10437120B1 (en) * 2018-04-23 2019-10-08 Omnivision Technologies, Inc. Methods and systems for displaying high dynamic range images
JP6757392B2 (en) * 2018-11-21 2020-09-16 株式会社モルフォ Image generator, image generation method and image generation program
JP7281897B2 (en) * 2018-12-12 2023-05-26 キヤノン株式会社 IMAGING DEVICE, CONTROL METHOD AND PROGRAM THEREOF

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101614914B1 (en) * 2009-07-23 2016-04-25 삼성전자주식회사 Motion adaptive high dynamic range image pickup apparatus and method
US8810663B2 (en) * 2011-01-27 2014-08-19 Aptina Imaging Corporation Methods for motion correction for high-dynamic-range imaging systems
JP5791336B2 (en) * 2011-04-01 2015-10-07 キヤノン株式会社 Image processing apparatus and control method thereof
CN103002225B (en) * 2011-04-20 2017-04-12 高通科技公司 Multiple exposure high dynamic range image capture
CN103026171B (en) * 2011-05-27 2016-03-16 松下电器产业株式会社 Image processing apparatus and image processing method
JP5333522B2 (en) * 2011-06-06 2013-11-06 カシオ計算機株式会社 MOVIE GENERATION DEVICE, MOVIE GENERATION METHOD, AND PROGRAM
US8913153B2 (en) * 2011-10-06 2014-12-16 Aptina Imaging Corporation Imaging systems and methods for generating motion-compensated high-dynamic-range images
JP5898466B2 (en) * 2011-11-11 2016-04-06 キヤノン株式会社 Imaging device, control method thereof, and program
CN102572245B (en) * 2011-12-22 2014-12-17 深圳市赛格导航科技股份有限公司 Method and device for extending image dynamic ranges

Also Published As

Publication number Publication date
US20150116525A1 (en) 2015-04-30
CN104601905A (en) 2015-05-06

Similar Documents

Publication Publication Date Title
TW201517624A (en) Method and apparatus for generating high dynamic range images and computer program product
US20220044363A1 (en) Techniques for Controlled Generation of Training Data for Machine Learning Enabled Image Enhancement
Wei et al. Physics-based noise modeling for extreme low-light photography
US7626614B1 (en) Transfer function and high dynamic range images
US20170256036A1 (en) Automatic microlens array artifact correction for light-field images
JP4999763B2 (en) Imaging apparatus, imaging method, program, recording medium, and integrated circuit
WO2017016258A1 (en) Terminal and photographing method for terminal
TWI615027B (en) High dynamic range image generation method, imaging device, terminal device and imaging method
US10171743B2 (en) Image pickup apparatus, image processing apparatus, image processing method, and non-transitory computer-readable storage medium for improving quality of captured image
JP2013223043A (en) Light-receiving device and transmission system
JP2006157908A (en) Sensing apparatus, method for sensing energy field and camera
WO2017057048A1 (en) Image processing device, image processing method and program
US20160019681A1 (en) Image processing method and electronic device using the same
KR20140013591A (en) Apparatus and method for photographing image
CN112651911B (en) High dynamic range imaging generation method based on polarized image
KR20160063794A (en) Image photographing apparatus and control methods thereof
US9083887B2 (en) Image capture devices configured to generate compensation gains based on an optimum light model and electronic apparatus having the same
JP2022179514A (en) Control apparatus, imaging apparatus, control method, and program
TW201524212A (en) Image data aggregating high dynamic range imaging systems and associated methods
CN107147851B (en) Photo processing method and device, computer readable storage medium and electronic equipment
CN107211092A (en) Image capture with improved temporal resolution and perceptual image definition
US20170026558A1 (en) Digital photographing apparatus and digital photographing method
US20160080613A1 (en) Image processing apparatus, image processing method, and program
JP2016111568A (en) Image blur correction control device, imaging apparatus, control method and program thereof
JP2020021314A (en) Image processing system and image processing method