TW201526640A - High dynamic range image composition apparatus for carrying out exposure mapping based on individual pixel and the method thereof - Google Patents

High dynamic range image composition apparatus for carrying out exposure mapping based on individual pixel and the method thereof Download PDF

Info

Publication number
TW201526640A
TW201526640A TW102147879A TW102147879A TW201526640A TW 201526640 A TW201526640 A TW 201526640A TW 102147879 A TW102147879 A TW 102147879A TW 102147879 A TW102147879 A TW 102147879A TW 201526640 A TW201526640 A TW 201526640A
Authority
TW
Taiwan
Prior art keywords
pixel
image
exposure
exposure value
pixels
Prior art date
Application number
TW102147879A
Other languages
Chinese (zh)
Other versions
TWI551141B (en
Inventor
shi-jie Huang
Original Assignee
shi-jie Huang
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by shi-jie Huang filed Critical shi-jie Huang
Priority to TW102147879A priority Critical patent/TWI551141B/en
Publication of TW201526640A publication Critical patent/TW201526640A/en
Application granted granted Critical
Publication of TWI551141B publication Critical patent/TWI551141B/en

Links

Abstract

Disclosed is a high dynamic range image composition apparatus for carrying out exposure mapping based on individual pixel. It comprises an image capturing module, an image processing module, and a recursive control module. The image capturing module is used for acquiring a plurality of images having different exposure value respectively. The image processing module is used for performing exposure value tuning means so as to perform exposure synthesis for the pixels based on exposure value between pixels at the same position in images with close exposure values. Accordingly, synthesis image can be composed. The recursive control module is used for re-transmitting generated composite image to the image processing module to be processed until the image processing module only generates a composite image.

Description

基於個別像素進行曝光映射之高動態範圍影像合成裝置及其方法 High dynamic range image synthesizing device based on individual pixels for exposure mapping and method thereof

本發明係有關於一種高動態範圍影像合成裝置及其方法,尤指一種基於個別像素進行曝光映射之高動態範圍影像合成裝置及其方法。 The present invention relates to a high dynamic range image synthesizing apparatus and method thereof, and more particularly to a high dynamic range image synthesizing apparatus and method thereof for performing exposure mapping based on individual pixels.

由於感光元件於硬體上的限制,任何攝影設備均具有有限的動態影像範圍。在一幀影像之中,若亮部與暗部之間的亮度差距太大,在低動態範圍的情況下,將可能使得截取出的影像在亮部的位置呈現一片亮白、而暗部的位置全黑的情況,使得這些高光、低光區域中的影像細節(如輪廓、線條等資訊)遺失,造成影像無法忠實呈現真實畫面的問題。 Due to the limitations of the photosensitive element on the hardware, any photographic device has a limited dynamic image range. In a frame of image, if the brightness difference between the bright part and the dark part is too large, in the case of a low dynamic range, it may be possible to make the cut image appear bright and white at the position of the bright part, and the position of the dark part is full. In the case of black, the details of the images (such as outlines, lines, etc.) in these high-light, low-light areas are lost, causing the problem that the images cannot faithfully present the real picture.

為克服上述的問題,業界發展出了許多種可提高影像動態範圍的影像處理方式,經由這些影像處理方式產生的影像一般稱之為高動態範圍影像(High dynamic range imaging,HDRI)。一般而言,較為常見的高動態範圍成像技術,係藉由調整攝影設備的參數,對同一景象分別基於不同的曝光值進行取像,並將這些影像進行合成,藉由影像合成的方式使影像中的高光區及低光區都能保有細節、藉以忠實呈現影像的原貌,提高了影像中所能顯示的動態範圍。該項技術有助於在高反差的環境中進行清晰的攝影,或是用於焊接工人使用的護目面罩,於電焊接時藉由即時拍攝,將焊接的高反差影像進行合成處理以供操作者於顯示屏上找到焊點的位置,其應用範圍相當廣泛。 In order to overcome the above problems, the industry has developed a variety of image processing methods that can improve the dynamic range of images. The images generated by these image processing methods are generally referred to as High Dynamic Range Imaging (HDRI). In general, the more common high dynamic range imaging technology is to image the same scene based on different exposure values by adjusting the parameters of the photographic equipment, and synthesize these images to form images by image synthesis. The high-gloss area and low-light area in the middle can preserve the details, so as to faithfully present the original image, and improve the dynamic range that can be displayed in the image. This technology helps to perform clear photography in high contrast environments, or for eye protection masks used by welding workers. By welding, high-contrast images of welding are processed for immediate operation during electric welding. The location of the solder joints is found on the display and is used in a wide range of applications.

然而,上述的影像合成過程中,係基於鏡頭所拍攝的物件處於靜態的情況下合成影像,當鏡頭中拍攝的物件移動 時,每一圖像禎之間所述的物件將會產生些許偏移(視移動的速度及每秒取得圖像禎數量而定),而使得圖像禎與圖像禎之間略有差異,這類的偏移於影像合成中將有可能被視為雜訊(Noise),造成影像失真的問題。 However, in the image synthesis process described above, the image is synthesized based on the object captured by the lens while the object is being shot, and the object captured in the lens is moved. At the same time, the object between each image 将会 will produce a slight offset (depending on the speed of movement and the number of images taken per second), which makes the image 祯 and the image 略 slightly different, which The offset of the class may be considered as noise in the image synthesis, causing image distortion.

再者,由於每張影像中的所有像素僅採用同一個曝光值作為基準值進行合成,於色調映射(Tone Mapping)時,每一個像素上因其亮度(Quantigraphic Measure)、相機響應函數(Camera Response)以及雜訊(Noise)干擾程度的不同,具備極高或極低亮度的像素在影像合成後仍可能有部分的高光或低光區域因反差太大仍然不能清楚的表示細節,使得影像中即便藉由高動態範圍處理過後仍有部分細節無法表示,不能達到完全的色調映射,實有改進之必要。 Furthermore, since all pixels in each image are synthesized using only the same exposure value as a reference value, in tone mapping (Tone Mapping), each pixel has its brightness (Quantigraphic Measure) and camera response function (Camera Response). ) and the degree of noise interference, pixels with extremely high or very low brightness may still have some highlights or low-light areas after image synthesis. Because the contrast is too large, the details cannot be clearly expressed, so that even in the image After the high dynamic range processing, there are still some details that cannot be represented, and the complete tone mapping cannot be achieved, which is necessary for improvement.

本發明之目的,在於解決一般於合成高動態範圍影像時,因拍照時的延遲產生雜訊、以及色調映射(Tone Mapping)時可能導致影像中亮部或暗部區域細節遺失之問題。 The object of the present invention is to solve the problem that when a high dynamic range image is synthesized, noise is generated due to delay in photographing, and tone mapping may result in loss of details of bright or dark regions in the image.

為解決上述問題,本發明提供一種基於個別像素曝光映射之高動態範圍影像合成裝置,包含有:一影像擷取模組、一影像處理模組、以及一遞迴控制模組。該影像擷取模組係用以取得複數張分別具有不同曝光值之影像。該影像處理模組用以依據兩兩曝光值相近之影像中同一位置像素之間的曝光值進行曝光值調校手段對該等像素進行曝光合成,藉以組成合成影像。該遞迴控制模組,係將所產生之合成影像重新傳送至該影像處理模組進行處理,直到該影像處理模組僅產生一張合成影像。 To solve the above problems, the present invention provides a high dynamic range image synthesizing device based on an individual pixel exposure mapping, comprising: an image capturing module, an image processing module, and a recursive control module. The image capture module is configured to obtain a plurality of images having different exposure values. The image processing module is configured to perform exposure synthesis on the pixels according to the exposure values between the pixels at the same position in the images with similar exposure values, so as to form a composite image. The recursive control module retransmits the generated composite image to the image processing module for processing until the image processing module generates only one composite image.

進一步地,該曝光值調校手段係將兩兩曝光值相近之影像中同一位置像素之曝光值依據下述公式進行權重分配: Further, the exposure value adjustment means assigns the exposure value of the pixel at the same position in the image with similar exposure values to each other according to the following formula:

進一步地,該曝光合成係依據該曝光值調校手段中權重分配後的曝光值,代入下述公式產生一合成像素: Further, the exposure synthesis system generates a synthesized pixel according to the exposure value after the weight distribution in the exposure value adjustment means, and substitutes the following formula:

進一步地,所述之基於全像素曝光值映射之高動態範圍影像合成裝置更進一步包含有一延展合成模組,若該兩影像中的第N像素間之該曝光值差值大於一閾值時,則該延展合成模組產生一虛擬像素,並依據該兩影像中的第N像素及該虛擬像素之間的曝光值差值分別產生一第一修正像素,以及一第二修正像素,再依據該第一修正像素以及該第二修正像素之間的曝光值差值產生該合成像素。 Further, the high dynamic range image synthesizing device based on the full pixel exposure value mapping further includes an extended synthesis module, wherein if the difference between the exposure values of the Nth pixels in the two images is greater than a threshold, then The extended synthesis module generates a virtual pixel, and generates a first modified pixel and a second modified pixel according to the difference between the Nth pixel and the virtual pixel in the two images, and then according to the first The synthesized pixel is generated by a corrected pixel and a difference in exposure value between the second modified pixels.

進一步地,該虛擬像素之曝光值係為該兩影像中第N像素的曝光值之平均值。 Further, the exposure value of the virtual pixel is an average value of exposure values of the Nth pixel in the two images.

進一步地,該影像處理模組係依據每一該曝光值差值產生相對應之複數個映射函數,並依據所述之映射函數產生一查找表(look-up table,LUT),該影像處理模組係依據該兩影像中第N像素之間的曝光值差值代入該查找表以產生該合成像素。 Further, the image processing module generates a corresponding plurality of mapping functions according to each of the exposure value differences, and generates a look-up table (LUT) according to the mapping function, and the image processing module The group is substituted into the lookup table according to the difference in exposure values between the Nth pixels in the two images to generate the synthesized pixels.

本發明之另一目的,在於提供一種基於全像素曝光值映射之高動態範圍影像合成方法,該方法包含有以下步驟:(a)取得複數張分別具有不同曝光值之影像;(b)依據兩兩曝光值相近之影像中同一位置像素之間的藉由一曝光值調校手段,對該等像素進行曝光合成;(c)判斷若所述之合成影像張數不為1,則將所述之合成影像回傳至步驟(b)重新進行處理。 Another object of the present invention is to provide a high dynamic range image synthesis method based on full pixel exposure value mapping, the method comprising the steps of: (a) obtaining a plurality of images having different exposure values; (b) according to two Exposing the pixels to the pixels at the same position in the image with the same exposure value by an exposure value adjustment method; (c) determining that if the number of synthesized images is not 1, the The composite image is passed back to step (b) and processed again.

進一步地,該曝光值調校手段係依下述公式產生該合成像素: Further, the exposure value adjustment means generates the synthesized pixel according to the following formula:

進一步地,該像素合成程序包含有以下步驟:經由一權重分配程序產生一合成曝光值;以及依據該合成曝光值產生 一合成像素。 Further, the pixel synthesis program includes the steps of: generating a composite exposure value via a weight assignment program; and generating the synthesized exposure value according to the A synthetic pixel.

進一步地,該權重分配程序包含有以下步驟:測量曝光值較低之影像中第N像素的一第一曝光值,以及一第一權重係數;測量曝光值較高之影像中第N像素的一第二曝光值,以及一第二權重係數;以及將該第一曝光值乘以該第一權重係數並將該第二曝光值乘以該第二權重係數後相加,相加後除以該第一權重係數以及該第二權重係數之和以產生該合成曝光值,或可寫作如下公式: Further, the weight distribution program includes the steps of: measuring a first exposure value of the Nth pixel in the image with lower exposure value, and a first weight coefficient; measuring one of the Nth pixels in the image with higher exposure value a second exposure value, and a second weight coefficient; and multiplying the first exposure value by the first weight coefficient and multiplying the second exposure value by the second weight coefficient, adding, dividing by The sum of the first weighting factor and the second weighting factor to generate the composite exposure value, or can be written as follows:

進一步地,該第一權重係數以及該第二權重係數係為該兩影像中第N像素之亮度動態範圍取夏儂熵值(Shannon’s entropy)之反函數,或可寫作如下公式: Further, the first weight coefficient and the second weight coefficient are inverse functions of the luminance dynamic range of the Nth pixel in the two images by Shannon's entropy, or may be written as follows:

進一步地,該權重分配程序更進一步包含有以下步驟:若該兩影像中的第N像素間之該曝光值差值大於一閾值,則產生一虛擬像素,該虛擬像素之曝光值介於該兩影像中的第N像素之間;對該虛擬像素與兩影像中曝光值較低的第N像素執行該權重分配程序,以產生一第一修正像素;對該虛擬像素與兩影像中曝光值較高的第N像素執行該權重分配程序,以產生一第二修正像素;以及以該第一修正像素取代曝光值較低之影像中N像素,並以該第二修正像素取代曝光值較高之影像中第N像素,並執行該權重分配程序。 Further, the weight distribution program further includes the following steps: if the difference between the exposure values of the Nth pixels in the two images is greater than a threshold, a virtual pixel is generated, and the exposure value of the virtual pixel is between the two Between the Nth pixels in the image; performing the weight distribution process on the virtual pixel and the Nth pixel having lower exposure values in the two images to generate a first modified pixel; comparing the exposure value between the virtual pixel and the two images The high Nth pixel performs the weight assignment process to generate a second modified pixel; and replaces the N pixel in the image with the lower exposure value with the first modified pixel, and replaces the higher exposure value with the second modified pixel The Nth pixel in the image and the weight assignment procedure is executed.

進一步地,於該像素合成程序中更包含有以下步驟:依據每一該曝光值差值產生相對應之一映射函數,並依據所述之映射函數產生一查找表(look-up table,LUT);以及依據該兩影像中第N像素之間的曝光值差值代入該查找表,以產生該合成像素。 Further, the pixel synthesis program further includes the following steps: generating a corresponding mapping function according to each of the exposure value differences, and generating a look-up table (LUT) according to the mapping function. And substituting the lookup table according to the difference in exposure values between the Nth pixels in the two images to generate the synthesized pixel.

是以,本發明較上述先前技術具備以下有益效果: Therefore, the present invention has the following beneficial effects over the above prior art:

1.本發明之基於全像素曝光值映射之高動態範圍影像合成裝置及其方法,針對影像中每一個像素分別進行合成的方式,可完整顯示位於極亮、極暗處的影像細節。 The high dynamic range image synthesizing device and the method thereof based on the full pixel exposure value mapping of the present invention can completely display the image details located in the extremely bright and extremely dark portions by synthesizing each pixel in the image separately.

2.本發明之延展合成模組係可於兩影像中的像素之間其曝光值差值過高時產生一曝光值居中的虛擬像素,並將該虛擬像素與兩影像進行合成,進而降低高曝光值差值所產生的影像鋸齒化、或合成像素超出動態範圍的臨界點等問題。 2. The extended synthesis module of the present invention can generate a virtual pixel with an exposure value centered when the difference between the exposure values of the pixels in the two images is too high, and synthesize the virtual pixel and the two images, thereby reducing the height. The image resulting from the difference in exposure value is jagged, or the synthetic pixel is beyond the critical point of the dynamic range.

3.本發明更提供產生查找表之方式,藉此加速本發明整體演算法的效率,並能應用於動態影像如焊接畫面等進行高動態範圍影像合成處理,且不影響其畫面的流暢度。 3. The present invention further provides a method for generating a lookup table, thereby accelerating the efficiency of the overall algorithm of the present invention, and can be applied to a dynamic image such as a welding picture for high dynamic range image synthesis processing without affecting the smoothness of the picture.

100‧‧‧影像合成裝置 100‧‧‧Image synthesizer

10‧‧‧影像擷取模組 10‧‧‧Image capture module

20‧‧‧處理單元 20‧‧‧Processing unit

21‧‧‧影像處理模組 21‧‧‧Image Processing Module

22‧‧‧遞迴控制模組 22‧‧‧Return control module

23‧‧‧延展合成模組 23‧‧‧Extended Synthetic Module

S110至S160‧‧‧步驟 Steps S110 to S160‧‧

圖1係為本發明基於全像素曝光值映射之高動態範圍影像合成裝置之方塊圖。 1 is a block diagram of a high dynamic range image synthesizing apparatus based on full pixel exposure value mapping according to the present invention.

圖2係為本發明之影像合成示意圖。 2 is a schematic diagram of image synthesis of the present invention.

圖3-1及圖3-2係為本發明之影像合成輸出示意圖。 3-1 and 3-2 are schematic diagrams of the image synthesis output of the present invention.

圖4係為本發明之查找表示意圖。 4 is a schematic diagram of a lookup table of the present invention.

圖5至圖6係為本發明之方法流程圖。 5 to 6 are flow charts of the method of the present invention.

茲就本案之結構特徵暨操作方式,並配合圖示說明,謹述於后,俾提供審查參閱。 For the structural features and operation methods of this case, and with the illustrations, please refer to it later.

關於本發明之技術請參照「圖1」,係為本發明基於個別像素進行曝光映射之高動態範圍影像合成裝置之方塊圖,如圖所示:所述之影像合成裝置100包含有:一影像擷取模組10、以及一處理單元20。該影像擷取模組10係選自由照相機鏡頭、攝影機鏡頭、掃瞄機鏡頭所構成之群組,用於對景物進行取像。該影像擷取模組10係基於不同的曝光值(exposure value)對同一景物進行取像,從而取得複數張具有不同曝光值的影像。該處理單元20係包含有一影像處理模組21,以及一遞迴控制模組21。該處理 單元20係可為一攝像裝置(如照相機、攝影機、掃瞄機等)中的影像處理單元,或是裝設於上述攝像裝置中之一現場可程式化邏輯閘陣列(Field Programmable Gate Array,FPGA)晶片中之程式。於應用於桌上型電腦、手提電腦、行動裝置等實施態樣,該處理單元20可為設備中的運算器,則該影像處理模組21、該遞迴控制模組22可為繪圖軟體或影像處理軟體中之程式,於本發明中並不予以限制。 For the technology of the present invention, please refer to FIG. 1 , which is a block diagram of a high dynamic range image synthesizing device for performing exposure mapping based on individual pixels according to the present invention. As shown in the figure, the image synthesizing device 100 includes: an image. The module 10 and the processing unit 20 are captured. The image capturing module 10 is selected from the group consisting of a camera lens, a camera lens, and a scanner lens for capturing a scene. The image capturing module 10 captures the same scene based on different exposure values, thereby obtaining a plurality of images having different exposure values. The processing unit 20 includes an image processing module 21 and a recursive control module 21. This treatment The unit 20 can be an image processing unit in an imaging device (such as a camera, a camera, a scanner, etc.) or a Field Programmable Gate Array (FPGA) installed in the camera device. ) The program in the chip. The processing unit 20 can be an arithmetic unit in the device, and the image processing module 21 and the recursive control module 22 can be a drawing software or The program in the image processing software is not limited in the present invention.

本發明之術語「個別像素曝光映射」係如下所述:在高動態範圍影像的合成過程中,每張影像均包含有N個像素,當兩兩影像進行合成時,在該兩兩影像的相同位置上係具有相對應的像素(Pixel),為便於後續說明,將該等位於相同位置之像素併稱為像素對(Pixel Pair),本發明係依據每一組像素對之間的曝光值差值,依權重分配(Weighted Sum Method),進行高動態範圍影像合成,有別於過去整張影像中所有像素均採用相同曝光值的合成方式。透過個別像素曝光映射所得的像素係依據下述公式(1)產生: The term "individual pixel exposure mapping" of the present invention is as follows: in the process of synthesizing high dynamic range images, each image contains N pixels, and when the two images are combined, the same in the two images In the position, there are corresponding pixels (Pixel). For convenience of subsequent description, the pixels located at the same position are called Pixel Pairs, and the present invention is based on the difference of exposure values between each pair of pixels. The value, Weighted Sum Method, performs high dynamic range image synthesis, which is different from the previous method in which all pixels in the entire image use the same exposure value. The pixels obtained by the individual pixel exposure mapping are generated according to the following formula (1):

在公式(1)中,^q與^k均對應於影像中縱、橫位置上的一個座標,亦即所像素在影像上的座標,為簡化表示公式,在此且將^q與^k的座標省略。其中,P mapped 即為映射處理後之像素值;f代表攝像裝置的相機響應函數(camera response);^q代表該像素取其相機響應函數的返函數後,再除以已知的曝光度等所得到的亮度估測值;^k則為該像素的像素值取其反相機響應函數的反函數後,再除以影像進光亮所得到曝光度等級估測值,該曝光度等級^k與像素之曝光值EV之關係為^k=2EV,本發明的主要目的即是在於將兩張影像中各個像素的曝光值EV進行合成,以取得較為適切的曝光度等級^kIn formula (1), both ^ q and ^ k correspond to a coordinate in the vertical and horizontal positions in the image, that is, the coordinates of the pixel on the image, which is a simplified representation formula, where ^ q and ^ k The coordinates are omitted. Wherein, P mapped value of the pixel that is mapped treatment; F imaging device to the camera response function (camera response); ^ q representing the pixel whichever camera function in response to the return function, and then divided by the known exposure, etc. The obtained brightness estimation value; ^ k is the inverse function of the pixel value of the pixel after taking the inverse function of the inverse camera response function, and then dividing by the image into the light to obtain the exposure level estimation value, the exposure level ^ k and The relationship between the exposure values EV of the pixels is ^ k = 2 EV , and the main object of the present invention is to combine the exposure values EV of the respective pixels in the two images to obtain a more appropriate exposure level ^ k .

以下針對本發明所執行之影像處理程序作詳細之說明。 The image processing program executed by the present invention will be described in detail below.

為便於理解本發明之影像合成方式,請參照「圖2」,係為本發明之影像合成示意圖,如圖所示:該影像處理模組21係將兩兩曝光值相近之影像進行合成,若影像的總數為I,則該影像處理模組21進行合成 程序後將產生I-1張合成影像,而該遞迴控制模組22係將所產生之合成影像重新傳送至該影像處理模組21進行處理,經過I-1次的遞迴之後,最終僅會保留一張合成影像,即為經動態範圍壓縮後可保有所有細節之高動態範圍影像。 For the purpose of understanding the image synthesizing method of the present invention, please refer to FIG. 2, which is a schematic diagram of image synthesis according to the present invention. As shown in the figure, the image processing module 21 synthesizes images with similar exposure values. The total number of images is I, and the image processing module 21 performs synthesis. After the program, I-1 composite images will be generated, and the recursive control module 22 will retransmit the generated composite images to the image processing module 21 for processing. After I-1 recursive, finally only A composite image is retained, which is a high dynamic range image that retains all details after dynamic range compression.

更具體而言,該影像處理模組21於該影像擷取模組10取得之複數張影像中,選擇兩曝光值相近之影像,為便於說明,在此將該兩影像稱為影像i以及影像j,其中影像i的整體曝光值小於影像j的整體曝光值,且影像i中的第N個像素稱之為像素PNi,相對地,影像j中的第N個像素稱之為像素PNj,該像素PNi以及該像素PNj係為一組位於相同位置之像素對。承接前述步驟,該影像處理模組21計算像素PNi以及該像素PNj之間的一曝光值差值△EVi,j,並依據該曝光值差值△EVi,j以及該像素PNi、像素PNj之曝光值進行權重分配以產生一合成曝光值EVi,j,再將該合成曝光值EVi,j代入前述的公式(1)後即可取得映射產生的合成像素。當該影像處理模組21對N個像素對進行合成動作之後,便會產生N個合成像素,這些合成像素即可組合為高動態範圍影像。 More specifically, the image processing module 21 selects two images with similar exposure values in the plurality of images acquired by the image capturing module 10. For convenience of description, the two images are referred to as images i and images. j, wherein the overall exposure value of the image i is smaller than the overall exposure value of the image j, and the Nth pixel in the image i is referred to as a pixel P Ni , and the Nth pixel in the image j is referred to as a pixel P Nj . The pixel P Ni and the pixel P Nj are a pair of pixel pairs located at the same position. According to the foregoing steps, the image processing module 21 calculates an exposure value difference ΔEV i,j between the pixel P Ni and the pixel P Nj , and according to the exposure value difference ΔEV i,j and the pixel P Ni The exposure value of the pixel P Nj is weighted to generate a composite exposure value EV i,j , and the synthesized exposure value EV i,j is substituted into the above formula (1) to obtain the synthesized pixel generated by the mapping. After the image processing module 21 synthesizes the N pixel pairs, N synthesized pixels are generated, and the synthesized pixels can be combined into a high dynamic range image.

本發明之術語「權重分配」,係將兩像素(PNi、PNj)之曝光值依據一定比重進行分配,其分配方式係由下述公式(2)決定: The term "weight distribution" of the present invention distributes the exposure values of two pixels (P Ni , P Nj ) according to a certain specific gravity, and the distribution method is determined by the following formula (2):

在公式(2)中,經權重分配後之曝光值為EVi,j;而EVi、EVi分別代表兩像素本身的曝光值,而Ci、Cj為決定該兩像素的權重係數,且最後產生的曝光值為EVi,j會越接近權重係數較高者的曝光值。 In formula (2), the exposure value after weight assignment is EV i,j ; and EV i , EV i represent the exposure values of the two pixels themselves, and C i and C j are the weight coefficients for determining the two pixels. And the resulting exposure value is EV i,j will be closer to the exposure value of the higher weight coefficient.

然而,該權重係數Ci、Cj實際上必須依據該等像素之亮度的變動範圍而定,由於雜訊的干擾,在進行像素PNi以及該像素PNj的映射時會產生一個像素值可發散映射至多個像素值的問題,因而造成映射上的不確定性(uncertainty)。對此,本發明所採用之處理方法為求取該等像素之亮度的動態範圍,亦即將像素之亮度之最大值除以最小值後取其對數,並依據資訊理論中夏儂熵值(Shannon’s entropy)的模型求取該影像中動態範圍 的不確定性方程,該方程係選自由像素的曝光度、以及各個具有不同曝光值的影像像素之曝光值差值所組成之向量產生的樣本空間。在簡化攝像裝置的相機響應函數情況下,對該夏儂熵值取反函數,即可獲得兩影像各別之確定性函數(certainty functions),如公式(3),以像素PNi為例: However, the weight coefficients C i and C j must actually depend on the variation range of the brightness of the pixels. Due to the interference of the noise, a pixel value may be generated when mapping the pixel P Ni and the pixel P Nj . The problem of diverging maps to multiple pixel values, thus causing uncertainty on the map. In this regard, the processing method adopted by the present invention is to obtain the dynamic range of the brightness of the pixels, that is, the maximum value of the brightness of the pixel is divided by the minimum value and then take the logarithm thereof, and according to the information theory, the summer entropy value (Shannon's The model of entropy) obtains the uncertainty equation of the dynamic range in the image, which is selected from the sample space generated by the exposure of the pixel and the vector composed of the difference of the exposure values of the image pixels having different exposure values. In the case of simplifying the camera response function of the camera device, the inverse function of the Xia's entropy value is obtained, and the respective deterministic functions of the two images can be obtained, as in the formula (3), taking the pixel P Ni as an example:

其中,C即為權重係數;Yi係為將像素PNi的像素值取其相機響應函數的反函數,並除以其亮度及曝光度等級所得到的值,在數學意義上,Yi便是由相機響應函數反推取得的曝光度等級估測值^k i與該像素實際的曝光度等級之比值,或寫作: Wherein, C is the weight coefficient; Y i is the inverse value of the pixel value of the pixel P Ni as the inverse of the camera response function, and divided by the brightness and exposure level, in the mathematical sense, Y i Is the ratio of the exposure level estimate ^ k i obtained by the camera response function back to the actual exposure level of the pixel, or writing:

而HN係指在常態分布下,將Yi於向量ζ(定義為[亮度q,曝光值差值△EVi,j])的樣本空間中取其常態分佈之熵值。將所欲合成之兩像素的像素值代入該確定性函數後,可得到該像素在權重分配時的權重係數,並依據權重係數合成較為容易顯示其細節的曝光值。舉例而言,對一個較暗的區塊,其較曝光值低的像素的權重係數會相對於曝光值高的像素來得大,且在兩像素之間曝光值差值越高的情況下,彼此的權重係數間的差值也會提高。然上述方式僅供說明參考,依據相機響應函數的不同,亦可由其他數學模型推導得出不同的確定性方程,詳細可參照論文Technology and Society(ISTAS),2013 IEEE International Symposium on 27-29 June 2013,pages 98-106,High Dynamic Range Tone Mapping Based on Per-Pixel Exposure Mapping,而本發明僅討論相機響應函數為對數函數的狀況,其演算方法並非本發明所欲限制之。 H N refers to the entropy value of the normal distribution of Y i in the sample space of the vector ζ (defined as [brightness q, exposure value difference ΔEV i, j ]) under the normal distribution. After substituting the pixel values of the two pixels to be synthesized into the deterministic function, the weight coefficient of the pixel at the time of weight distribution can be obtained, and the exposure value which is easier to display the details can be synthesized according to the weight coefficient. For example, for a darker block, the weight coefficient of the pixel with lower exposure value is larger than that of the pixel with high exposure value, and the higher the difference between the exposure values between the two pixels, the mutual The difference between the weight coefficients will also increase. However, the above methods are for illustrative purposes only. Different deterministic equations can be derived from other mathematical models depending on the camera response function. For details, please refer to the paper Technology and Society (ISTAS), 2013 IEEE International Symposium on 27-29 June 2013 , pages 98-106, High Dynamic Range Tone Mapping Based on Per-Pixel Exposure Mapping, while the present invention only discusses the condition that the camera response function is a logarithmic function, and the calculation method is not intended to be limited by the present invention.

承上所述,在該兩影像中第N像素對之間的曝光值差值過高時,像素之間的權重係數便會產生較大的差距,此一結果可能導致產生得高動態範圍影像仍然無法顯示出完整的影像細節。請一併參照圖3-1及圖3-2,係為本發明之影像合成輸出示意圖,如圖所示:圖3-1及圖3-2分別 代表一個僅有一維的影像,其橫軸代表影像中含有600個像素,縱軸則表示該等像素的像素值,由圖3-1可見,當曝光值差值過高時,僅採用權重分配來進行像素的合成將導致像素值的不連續,使得影像上可能產生類似鋸齒狀的現象,而由圖3-2最後兩個弧狀段的像素中可見,一個較暗的像素Pi以及一個極亮的像素Pj,兩者所合成的合成像素仍然可能超過可顯示的動態範圍的臨界值。 As described above, when the difference between the exposure values of the Nth pixel pairs in the two images is too high, a large difference is generated between the weight coefficients of the pixels, and the result may result in a high dynamic range image. Still unable to display complete image details. Please refer to FIG. 3-1 and FIG. 3-2 together, which is a schematic diagram of the image synthesis output of the present invention, as shown in the figure: FIG. 3-1 and FIG. 3-2 respectively represent a one-dimensional image, and its horizontal axis. The representative image contains 600 pixels, and the vertical axis indicates the pixel values of the pixels. As shown in Figure 3-1, when the difference in exposure value is too high, only the weight distribution is used to perform pixel synthesis, which will result in pixel values. Continuous, so that the image may produce a jagged phenomenon, which is visible in the pixels of the last two arc segments of Figure 3-2, a darker pixel P i and an extremely bright pixel P j The synthesized pixels may still exceed the threshold of the dynamic range that can be displayed.

為解決上述問題,本發明之基於個別像素曝光映射之高動態範圍影像合成裝置100更進一步包含有一延展合成模組23,若該兩影像中的第N像素(像素PNi、像素PNj)之間的該曝光值差值大於一閾值時,該延展合成模組23產生一虛擬像素,並且將該像素PNi、像素PNj分別與該虛擬像素進行權重分配,以取得一第一修正像素,以及一第二修正像素,此時,若該第一修正像素及該第二修正像素之間的曝光值差值已減少至較合理的範圍,便可直接依據其曝光值差值產生該合成像素,若該第一修正像素及該第二修正像素之間的曝光值差值仍然過高,則該延展合成模組23再次產生虛擬像素並重複上述步驟直到曝光值差值進入合理範圍為止。所述之該閾值並無特定的數值,端視使用者對高動態範圍影像之細節呈現需求而定,若將該閾值降低,則可得到細節呈現品質較高的影像,但若閾值過低,反而可能導致影像模糊不清的問題,因此該閾值之大小須依照使用者需求設計在合理範圍內。於一較佳實施態樣中,該虛擬像素之曝光值係為該兩影像中第N像素的曝光值之平均值較佳,使產生的修正像素能最大限度地縮小曝光值差值,經此一程序處理後的影像等同於前述權重分配之延展,由圖3-1中可見,經該延展合成模組23處理後所產生的合成像素之間其像素值較為平滑,無鋸齒狀的現象,且由3-2中可見經該延展合成模組23處理後所產生的像素可有效地限制在動態範圍之內,而不會因曝光值差值過高導致影像的細節遺失等問題。 In order to solve the above problem, the high dynamic range image synthesizing device 100 based on the individual pixel exposure mapping further includes an extended synthesis module 23, if the Nth pixel (pixel P Ni , pixel P Nj ) in the two images When the difference between the exposure values is greater than a threshold, the extended synthesis module 23 generates a virtual pixel, and the pixels P Ni and P Nj are respectively weighted with the virtual pixel to obtain a first modified pixel. And a second modified pixel. At this time, if the difference between the exposure value between the first modified pixel and the second modified pixel has been reduced to a reasonable range, the synthesized pixel may be directly generated according to the difference of the exposure value. If the difference between the exposure values of the first modified pixel and the second modified pixel is still too high, the extended synthesis module 23 generates the virtual pixel again and repeats the above steps until the exposure value difference enters a reasonable range. There is no specific value for the threshold, and the user depends on the detailed presentation requirements of the high dynamic range image. If the threshold is lowered, the image with higher quality is obtained, but if the threshold is too low, On the contrary, it may cause the image to be blurred, so the threshold value must be designed within a reasonable range according to the user's needs. In a preferred embodiment, the exposure value of the virtual pixel is preferably an average value of the exposure values of the Nth pixel in the two images, so that the generated correction pixel can minimize the difference in the exposure value. The image processed by the program is equivalent to the extension of the weight distribution. As can be seen from FIG. 3-1, the pixel values generated by the synthetic pixels processed by the extended synthesis module 23 are relatively smooth and have no jaggedness. Moreover, it can be seen from 3-2 that the pixels generated by the extension synthesis module 23 can be effectively limited to the dynamic range without causing problems such as loss of details of the image due to excessive difference in exposure values.

上述基於權重分配的影像合成方式若以演算法執行,且該影像擷取模組取得的影像數目為I,則此一演算法的時間複雜度(Time complexity)為O(I2),亦即取得的影像張數越多,進行運算所需花費的時間越長;另一方面,若因曝光值差值過高使得該延展合成模組需要產生V個 虛擬像素時,演算法的整體時間複雜度更劣化為O((VI)2),若對不斷變化的動態影像(例如先前技術中所提及之焊接影像)進行處理時,將會使整體畫面的幀數(Frames per second,FPS)過低而無法順暢地顯示影像。是以,本發明另提供一種加速整體演算法的方式,該影像處理模組21係依據每一該曝光值差值產生相對應之一映射函數,並依據所述之映射函數產生一查找表(look-up table,LUT),且該影像處理模組21依據該兩影像中第N像素之間的曝光值差值代入該查找表,以產生該合成像素。請參照圖4,係為本發明之查找表示意圖。具體而言,該查找表的產生方式是在每次進行像素對的權重分配時,先將曝光值差值較高的像素對透過該延展合成模組23插入虛擬像素後進行合成,再將不同的曝光值差值所產生的映射結果歸納為一個映射函數,最後將所有映射函數組合於同一平面上即可完成圖4之查找表。實際使用時,該處理單元僅需而將兩兩該影像中各像素對的像素值以及曝光值差值帶入該查找表中,即可產生該合成像素,在理想狀態下可使演算法整體的時間複雜度優化至O(1),進而縮短了高動態範圍影像的合成處理時間。由於歸納取得該映射函數的方式係為該領域中通常知識者所熟習之技術,於本發明中便省略其詳細步驟。 If the image composition method based on the weight distribution is executed by an algorithm and the number of images acquired by the image capturing module is I, the time complexity of the algorithm is O(I 2 ), that is, The more the number of images obtained, the longer it takes to perform the calculation; on the other hand, if the difference between the exposure values is too high, the extended synthesis module needs to generate V virtual pixels, the overall time of the algorithm is complicated. The degree is further degraded to O((VI) 2 ). If the dynamic image (such as the welding image mentioned in the prior art) is processed, the frame number of the whole frame (FPS) will be made. It is too low to display images smoothly. Therefore, the present invention further provides a method for accelerating the overall algorithm. The image processing module 21 generates a corresponding mapping function according to each of the exposure value differences, and generates a lookup table according to the mapping function. The look-up table (LUT), and the image processing module 21 substitutes the difference between the exposure values of the Nth pixels in the two images into the lookup table to generate the synthesized pixels. Please refer to FIG. 4 , which is a schematic diagram of a lookup table of the present invention. Specifically, the lookup table is generated by separately inserting a pixel pair having a higher difference in exposure value into the virtual pixel through the extension synthesis module 23, and then synthesizing each time the weight distribution of the pixel pair is performed. The mapping result produced by the difference of the exposure values is summarized into a mapping function, and finally all the mapping functions are combined on the same plane to complete the lookup table of FIG. In actual use, the processing unit only needs to bring the pixel value and the exposure value difference of each pixel pair in the image into the lookup table to generate the synthesized pixel, and in an ideal state, the overall algorithm can be generated. The time complexity is optimized to O(1), which in turn reduces the synthesis processing time for high dynamic range images. Since the manner in which the mapping function is obtained is summarized as a technique familiar to those of ordinary skill in the art, detailed steps are omitted in the present invention.

以下說明本發明之基於全像素曝光值映射之高動態範圍影像合成方法,請參閱圖5至圖6,係為本發明之方法流程圖,如圖所示該方法包含有下述步驟:取得複數張分別具有不同曝光值之影像,該影像係包含有N個像素(S110)。其次,依據兩兩曝光值相近之影像中同一位置之第N像素係藉由一曝光值調校手段,對該等像素進行曝光合成,藉以組成合成影像(S120),其中該曝光值調校手段包含有以下步驟:經由一權重分配程序產生一合成曝光值(S121),以及依據該合成曝光值產生一合成像素(S122)。接續,將所產生之N個合成像素組成一合成影像(S130)。最後,判斷若所述之合成影像張數不為1,則將所述之合成影像回傳至步驟S120重新進行處理(S140)。藉由上述方法步驟,本發明可針對複數張影像中的同一位置的每一個像素對進行合成,最終取得一高動態範圍影像。 The following describes the high dynamic range image synthesis method based on the full pixel exposure value mapping of the present invention. Referring to FIG. 5 to FIG. 6 , it is a flowchart of the method of the present invention. As shown in the figure, the method includes the following steps: obtaining a complex number Each of the images has images of different exposure values, and the image contains N pixels (S110). Secondly, according to the N-th pixel of the same position in the image with similar exposure values, the pixels are subjected to exposure synthesis by an exposure value adjustment method to form a composite image (S120), wherein the exposure value adjustment means The method includes the steps of: generating a composite exposure value via a weight assignment process (S121), and generating a composite pixel based on the composite exposure value (S122). Next, the generated N synthesized pixels are combined into a composite image (S130). Finally, it is judged that if the number of synthesized images is not 1, the synthesized image is returned to step S120 and processed again (S140). By the above method steps, the present invention can synthesize each pixel pair at the same position in a plurality of images, and finally obtain a high dynamic range image.

以下進一步說明本發明之像素合成之詳細過程,該權重分配程序如下:於該同一位置像素中,測量曝光值較低之影像中的一第一曝光 值,以及一第一權重係數。測量曝光值較高之影像中的一第二曝光值,以及一第二權重係數。最後,將該第一曝光值乘以該第一權重係數並將該第二曝光值乘以該第二權重係數後相加,相加後除以該第一權重係數以及該第二權重係數之和以產生該合成曝光值,並依據該合成曝光值產生一合成像素。其中該第一權重係數以及該第二權重係數係為該兩影像中第N像素之亮度動態範圍取夏儂熵值(Shannon’s entropy)之反函數。 The detailed process of pixel synthesis of the present invention is further described below. The weight distribution procedure is as follows: in the same location pixel, a first exposure in an image with a lower exposure value is measured. Value, and a first weighting factor. A second exposure value in the image with a higher exposure value is measured, and a second weight coefficient is measured. Finally, multiplying the first exposure value by the first weight coefficient and multiplying the second exposure value by the second weight coefficient, adding, dividing by the first weight coefficient and the second weight coefficient And generating the synthesized exposure value and generating a synthesized pixel according to the synthesized exposure value. The first weight coefficient and the second weight coefficient are inverse functions of the luminance dynamic range of the Nth pixel in the two images by Shannon's entropy.

為避免由於曝光值差值過高而造成的影像不連續、細節遺失等問題,該權重分配程序更進一步包含有以下步驟:若該兩影像中的第N像素間之該曝光值差值大於一閾值時,則產生一虛擬像素,該虛擬像素之曝光值介於該兩影像中的第N像素之間,於一較佳實施態樣中,該虛擬像素之曝光值係為該兩影像中第N像素的曝光值之平均值。接著,對該虛擬像素與兩影像中曝光值較低的第N像素執行該權重分配程序,以產生一第一修正像素、對該虛擬像素與兩影像中曝光值較高的第N像素執行該權重分配程序,以產生一第二修正像素。最後,以該第一修正像素取代曝光值較低之影像中第N像素,並以該第二修正像素取代曝光值較高之影像中第N像素,並執行該權重分配程序。 In order to avoid problems such as image discontinuity and loss of detail due to excessive difference in exposure value, the weight distribution program further includes the following steps: if the difference between the exposure values of the Nth pixels in the two images is greater than one In the threshold, a virtual pixel is generated, and the exposure value of the virtual pixel is between the Nth pixels in the two images. In a preferred embodiment, the exposure value of the virtual pixel is the The average of the exposure values of the N pixels. Then, the weight distribution process is performed on the virtual pixel and the Nth pixel with lower exposure values in the two images to generate a first modified pixel, and the virtual pixel and the Nth pixel having higher exposure values in the two images are executed. The weight is assigned to generate a second corrected pixel. Finally, the Nth pixel in the image with the lower exposure value is replaced by the first modified pixel, and the Nth pixel in the image with the higher exposure value is replaced by the second modified pixel, and the weight distribution process is executed.

最後,為提升本發明之方法整體演算法之效率,於該像素合成程序中更包含有以下步驟:依據每一該曝光值差值產生相對應之一映射函數,並依據所述之映射函數產生一查找表(look-up table,LUT)。以及依據該兩影像中第N像素之間的曝光值差值代入該查找表,以產生該合成像素。 Finally, in order to improve the efficiency of the overall algorithm of the method of the present invention, the pixel synthesis program further includes the following steps: generating a corresponding mapping function according to each of the exposure value differences, and generating according to the mapping function; A look-up table (LUT). And substituting the lookup table according to the difference between the exposure values between the Nth pixels in the two images to generate the synthesized pixel.

綜上所述,本發明之基於全像素曝光值映射之高動態範圍影像合成裝置及其操作方法,係可透過影像擷取模組取得複數張分別具有不同曝光值之影像,再經由影像處理模組合成為細節呈現度較高的合成影像,經遞迴控制模組,最終產生一高動態範圍影像。此外,當兩影像中的像素之間其曝光值差值過高時,可經由延展合成模組插入虛擬像素,可避免終呈現的影像不連續或細節遺失之問題。本發明之方法可透過建立查找表加速整體演算法的運算,使本發明之基於全像素曝光值映射之高動態範圍影像合成裝置及其操作方法亦可針對連續變動的影像進行高動態範圍影像的合成處理,而不減低其畫面流暢程度。 In summary, the high dynamic range image synthesizing device based on the full pixel exposure value mapping of the present invention and the operating method thereof can obtain a plurality of images having different exposure values through the image capturing module, and then pass the image processing mode. The combination becomes a composite image with higher detail rendering, and is returned to the control module to finally generate a high dynamic range image. In addition, when the difference between the exposure values of the pixels in the two images is too high, the virtual pixels can be inserted through the extended synthesis module, thereby avoiding the problem that the final image is discontinuous or the details are lost. The method of the present invention can accelerate the operation of the overall algorithm by establishing a lookup table, so that the high dynamic range image synthesizing device based on the full pixel exposure value mapping of the present invention and the operation method thereof can also perform high dynamic range imaging for continuously changing images. Synthetic processing without reducing the fluency of its picture.

本發明已藉上述較佳具體例進行更詳細說明,惟本發明並不限定於上述所舉例之實施態樣,凡在本發明所揭示之技術思想範圍內,對該等結構作各種變化及修飾,該等變化及修飾仍屬本發明之範圍。 The present invention has been described in more detail with reference to the preferred embodiments described above, but the present invention is not limited to the embodiments described above, and various changes and modifications may be made to the structures within the scope of the technical idea disclosed herein. Such changes and modifications are still within the scope of the invention.

100‧‧‧影像合成裝置 100‧‧‧Image synthesizer

10‧‧‧影像擷取模組 10‧‧‧Image capture module

20‧‧‧處理單元 20‧‧‧Processing unit

21‧‧‧影像處理模組 21‧‧‧Image Processing Module

22‧‧‧遞迴控制模組 22‧‧‧Return control module

23‧‧‧延展合成模組 23‧‧‧Extended Synthetic Module

Claims (14)

一種基於個別像素曝光映射之高動態範圍影像合成裝置,包含有:一影像擷取模組,用以取得複數張分別具有不同曝光值之影像;一影像處理模組,用以依據兩兩曝光值相近之影像中同一位置像素之間的曝光值進行曝光值調校手段對該等像素進行曝光合成,藉以組成合成影像;以及一遞迴控制模組,係將所產生之合成影像重新傳送至該影像處理模組進行處理,直到該影像處理模組僅產生一張合成影像。 A high dynamic range image synthesizing device based on an individual pixel exposure mapping, comprising: an image capturing module for acquiring a plurality of images respectively having different exposure values; and an image processing module for using the two exposure values Exposure values between pixels in the same position in the similar image are subjected to exposure value adjustment means for combining and synthesizing the pixels to form a composite image; and a recursive control module retransmits the generated synthesized image to the The image processing module processes until the image processing module generates only one composite image. 如申請專利範圍第1項所述之影像合成裝置,其中曝光值調校手段係將兩兩曝光值相近之影像中同一位置像素之曝光值依據下述公式進行權重分配: The image synthesizing device according to claim 1, wherein the exposure value adjusting means assigns the exposure value of the pixel at the same position in the image with similar exposure values to each other according to the following formula: 如申請專利範圍第1項所述之影像合成裝置,其中該曝光合成係依據該曝光值調校手段中權重分配後的曝光值,代入下述公式產生一合成像素: The image synthesizing device according to claim 1, wherein the exposure synthesis system generates a synthesized pixel according to the exposure value after the weight distribution in the exposure value adjustment means, and substitutes the following formula: 如申請專利範圍第3項所述之影像合成裝置,更進一步包含有一延展合成模組,於兩兩該影像中同一位置像素間的曝光值差值大於一閾值時產生一虛擬像素,並依據二該影像中所述同一位置的像素及該虛擬像素之間的曝光值差值分別產生一第一修正像素,以及一第二修正像素,再依據該第一修正像素以及該第二修正像素之間的曝光值差值產生該合成像素。 The image synthesizing device of claim 3, further comprising an extended synthesis module, wherein a virtual pixel is generated when the difference between the exposure values of the pixels at the same position in the image is greater than a threshold value, and according to the second The difference between the pixels at the same position and the difference between the virtual pixels in the image respectively generates a first modified pixel and a second modified pixel, and then according to the first modified pixel and the second modified pixel. The difference in exposure value produces the synthesized pixel. 如申請專利範圍第4項所述之影像合成裝置,其中該虛擬像素之曝光值係為兩兩該影像中同一位置像素的曝光值之平均值。 The image synthesizing device of claim 4, wherein the exposure value of the virtual pixel is an average of exposure values of pixels at the same position in the two images. 如申請專利範圍第1至3項其中一項所述之影像合成裝置,其中該影像處理模組係依據曝光值調校手段產生相對應之複數個映射函數,並依據所述之映射函數產生一查找表(Look-up Table,LUT),該影像處理模組係 依據該影像中同一位置像素之間的曝光值差值代入該查找表以產生該合成像素。 The image synthesizing device according to any one of claims 1 to 3, wherein the image processing module generates a corresponding plurality of mapping functions according to an exposure value adjustment means, and generates a mapping function according to the mapping function. Look-up Table (LUT), the image processing module The lookup table is substituted into the lookup table according to the difference in exposure values between pixels in the same position in the image to generate the synthesized pixel. 一種基於個別像素進行曝光映射之高動態範圍影像合成方法,該方法包含有以下步驟:(a)取得複數張分別具有不同曝光值之影像;(b)兩兩曝光值相近之影像中同一位置像素係藉由一曝光值調校手段,對該等像素進行曝光合成,藉以組成合成影像;以及(c)判斷若所述之合成影像張數不為1,則將所述之合成影像回傳至步驟(b)重新進行處理。 A high dynamic range image synthesis method for performing exposure mapping based on individual pixels, the method comprising the steps of: (a) obtaining a plurality of images having different exposure values; (b) pixels of the same position in images having similar exposure values Performing exposure synthesis on the pixels by an exposure value adjustment method to form a composite image; and (c) determining that if the number of synthesized images is not 1, the composite image is transmitted back to Step (b) is processed again. 如申請專利範圍第5項所述之方法,其中該曝光值調校手段包含有以下步驟:經由一權重分配程序產生一合成曝光值;以及依據該合成曝光值產生該合成像素。 The method of claim 5, wherein the exposure value adjustment means comprises the steps of: generating a composite exposure value via a weight assignment procedure; and generating the composite pixel based on the composite exposure value. 如申請專利範圍第8項所述之方法,其中該曝光值調校手段係依下述公式產生該合成像素: The method of claim 8, wherein the exposure value adjustment means generates the synthesized pixel according to the following formula: 如申請專利範圍第8項所述之方法,其中該權重分配程序包含有以下步驟:於該同一位置像素中,測量曝光值較低之影像中該像素的一第一曝光值,以及一第一權重係數;於該同一位置像素中,測量曝光值較高之影像中該像素的一第二曝光值,以及一第二權重係數;以及將該第一曝光值乘以該第一權重係數並將該第二曝光值乘以該第二權重係數後相加並除以該第一權重係數以及該第二權重係數之和以產生該合成曝光值,或可寫作如下公式: The method of claim 8, wherein the weighting process comprises the steps of: measuring a first exposure value of the pixel in the image with a lower exposure value, and a first a weight coefficient; in the same position pixel, measuring a second exposure value of the pixel in the image with higher exposure value, and a second weight coefficient; and multiplying the first exposure value by the first weight coefficient and The second exposure value is multiplied by the second weight coefficient and added by the sum of the first weight coefficient and the second weight coefficient to generate the composite exposure value, or can be written as follows: 如申請專利範圍第10項所述之方法,其中該第一權重係數以及該第二 權重係數係為該兩影像中該同一位置像素分別之亮度動態範圍取夏儂熵值(Shannon’s entropy)之反函數,或可寫作如下公式: The method of claim 10, wherein the first weight coefficient and the second weight coefficient are the inverse of the brightness dynamic range of the same position pixel in the two images by Shannon's entropy Function, or can be written as follows: 如申請專利範圍第10項所述之方法,該權重分配程序更進一步包含有以下步驟:若二該影像中的同一位置像素間之曝光值差值大於一閾值,則產生一虛擬像素,該虛擬像素之曝光值介於二該影像中的該同一位置像素之曝光值之間;以及於該同一位置像素中,對該虛擬像素與二該影像中曝光值較低的像素執行該權重分配程序,產生一第一修正像素,對該虛擬像素與兩影像中曝光值較高的像素執行該權重分配程序,產生一第二修正像素;以及以該第一修正像素取代曝光值較低之影像中的像素,並以該第二修正像素取代曝光值較高之影像中的像素,並執行該權重分配程序。 The method of claim 10, further comprising the step of: generating a virtual pixel if the difference in exposure values between pixels in the same location in the image is greater than a threshold, the virtual pixel The exposure value of the pixel is between two exposure values of the same position pixel in the image; and in the same position pixel, the weight distribution process is performed on the virtual pixel and the pixel with lower exposure value in the image. Generating a first modified pixel, performing the weight assignment process on the virtual pixel and the pixel with higher exposure value in the two images to generate a second modified pixel; and replacing the image with the lower exposure value by the first modified pixel a pixel, and replacing the pixel in the image with the higher exposure value with the second modified pixel, and executing the weight assignment procedure. 如申請專利範圍第12項所述之方法,其中該虛擬像素之曝光值係為二該影像中同一位置像素的曝光值之平均值。 The method of claim 12, wherein the exposure value of the virtual pixel is an average of exposure values of pixels at the same position in the image. 如申請專利範圍第7項所述之方法,其中於該像素合成程序中更包含有以下步驟:依據每一該曝光值差值產生相對應之一映射函數,並依據所述之映射函數產生一查找表(Look-up Table,LUT);以及依據二該影像中同一位置像素之間的曝光值差值代入該查找表,以產生該合成像素。 The method of claim 7, wherein the pixel synthesis program further comprises the steps of: generating a corresponding mapping function according to each of the exposure value differences, and generating a mapping according to the mapping function; a look-up table (LUT); and substituting the look-up table according to the difference in exposure values between pixels at the same position in the image to generate the synthesized pixel.
TW102147879A 2013-12-24 2013-12-24 A high dynamic range image synthesizing apparatus and a method thereof for performing exposure mapping based on individual pixels TWI551141B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
TW102147879A TWI551141B (en) 2013-12-24 2013-12-24 A high dynamic range image synthesizing apparatus and a method thereof for performing exposure mapping based on individual pixels

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
TW102147879A TWI551141B (en) 2013-12-24 2013-12-24 A high dynamic range image synthesizing apparatus and a method thereof for performing exposure mapping based on individual pixels

Publications (2)

Publication Number Publication Date
TW201526640A true TW201526640A (en) 2015-07-01
TWI551141B TWI551141B (en) 2016-09-21

Family

ID=54197893

Family Applications (1)

Application Number Title Priority Date Filing Date
TW102147879A TWI551141B (en) 2013-12-24 2013-12-24 A high dynamic range image synthesizing apparatus and a method thereof for performing exposure mapping based on individual pixels

Country Status (1)

Country Link
TW (1) TWI551141B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9998720B2 (en) 2016-05-11 2018-06-12 Mediatek Inc. Image processing method for locally adjusting image data of real-time image
CN112135063A (en) * 2019-06-24 2020-12-25 聚晶半导体股份有限公司 Image output method and electronic device
TWI723435B (en) * 2019-06-24 2021-04-01 聚晶半導體股份有限公司 Image outputting method and electronic device

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI433533B (en) * 2010-11-22 2014-04-01 Altek Corp Image capturing device and image synthesis method thereof
JP5316625B2 (en) * 2011-11-11 2013-10-16 カシオ計算機株式会社 Image composition apparatus and program
JP5896788B2 (en) * 2012-03-07 2016-03-30 キヤノン株式会社 Image composition apparatus and image composition method

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9998720B2 (en) 2016-05-11 2018-06-12 Mediatek Inc. Image processing method for locally adjusting image data of real-time image
TWI639974B (en) * 2016-05-11 2018-11-01 聯發科技股份有限公司 Image processing method for locally adjusting image data of real-time image
CN112135063A (en) * 2019-06-24 2020-12-25 聚晶半导体股份有限公司 Image output method and electronic device
TWI723435B (en) * 2019-06-24 2021-04-01 聚晶半導體股份有限公司 Image outputting method and electronic device
CN112135063B (en) * 2019-06-24 2021-11-30 聚晶半导体股份有限公司 Image output method and electronic device
US11206359B2 (en) 2019-06-24 2021-12-21 Altek Semiconductor Corp. Image outputting method and electronic device

Also Published As

Publication number Publication date
TWI551141B (en) 2016-09-21

Similar Documents

Publication Publication Date Title
US9485435B2 (en) Device for synthesizing high dynamic range image based on per-pixel exposure mapping and method thereof
JP5102374B2 (en) Method and apparatus for moving blur and ghost prevention in an imaging system
JP5713752B2 (en) Image processing apparatus and control method thereof
CN104980652B (en) Image processing apparatus and image processing method
JP2004180308A (en) Method and apparatus for generating image used for expanded range panorama synthesis
CN104320575A (en) Image processing method and image processing apparatus for portable terminal
TWI697867B (en) Metering compensation method and related monitoring camera apparatus
CN108668093A (en) The generation method and device of HDR image
JP2011109638A (en) Image processing apparatus and image processing method
US11941791B2 (en) High-dynamic-range image generation with pre-combination denoising
JP4066803B2 (en) Image processing apparatus, image processing program, image processing method, and electronic camera
JP2014155001A (en) Image processing apparatus and image processing method
JP2015144475A (en) Imaging apparatus, control method of the same, program and storage medium
KR20150035315A (en) Method for generating a High Dynamic Range image, device thereof, and system thereof
TWI551141B (en) A high dynamic range image synthesizing apparatus and a method thereof for performing exposure mapping based on individual pixels
JP5713643B2 (en) IMAGING DEVICE, IMAGING DEVICE CONTROL METHOD, PROGRAM, AND STORAGE MEDIUM
JP5932392B2 (en) Image processing apparatus and image processing method
WO2016158490A1 (en) Projection system, projector device, imaging device, and program
KR20140072980A (en) Apparatus and method for generating a High Dynamic Range image using single image
JP5952574B2 (en) Image processing apparatus and control method thereof
JP6730423B2 (en) Image processing apparatus, image processing method, and image processing program
Wang et al. Low visual difference virtual high dynamic range image synthesizer from a single legacy image
KR20110095556A (en) Image projection apparatus and image correcting method thereof
JP6603557B2 (en) Image processing apparatus and image processing method
TWI475512B (en) Noise processing method of dynamic range image and image capturing device thereof

Legal Events

Date Code Title Description
MM4A Annulment or lapse of patent due to non-payment of fees