TWI513310B - Device and method for expanding dynamic range of camera - Google Patents

Device and method for expanding dynamic range of camera Download PDF

Info

Publication number
TWI513310B
TWI513310B TW102125130A TW102125130A TWI513310B TW I513310 B TWI513310 B TW I513310B TW 102125130 A TW102125130 A TW 102125130A TW 102125130 A TW102125130 A TW 102125130A TW I513310 B TWI513310 B TW I513310B
Authority
TW
Taiwan
Prior art keywords
circuit
image
dark
value
dynamic range
Prior art date
Application number
TW102125130A
Other languages
Chinese (zh)
Other versions
TW201503699A (en
Inventor
Shih Chang Hsia
Kuo Ting Tseng
Original Assignee
Univ Nat Yunlin Sci & Tech
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Univ Nat Yunlin Sci & Tech filed Critical Univ Nat Yunlin Sci & Tech
Priority to TW102125130A priority Critical patent/TWI513310B/en
Publication of TW201503699A publication Critical patent/TW201503699A/en
Application granted granted Critical
Publication of TWI513310B publication Critical patent/TWI513310B/en

Links

Landscapes

  • Studio Devices (AREA)

Description

攝影機動態範圍擴展裝置及方法Camera dynamic range expansion device and method

本發明係有關於一種攝影機動態範圍擴展裝置及方法,尤指一種利用一系統裝置經設計之演算法,便利生成高動態範圍圖像之攝影機動態範圍擴展裝置及方法。The present invention relates to a camera dynamic range expansion device and method, and more particularly to a camera dynamic range expansion device and method for facilitating generation of a high dynamic range image by using an algorithm designed by a system device.

按,數位攝影機被廣泛應用於生活各處,例如:行車紀錄器、監視器及數位相機…等。當攝影時,係由鏡頭將攝取之圖像投影到感光元件上,由於現有攝影機感光元件曝光設計,若拍攝處光源分佈極端時,常容易發生可見範圍縮窄情況,即整個畫面只有亮處部分呈現,另暗處部分則呈一片黑色,又或者整個畫面只有暗處部分呈現,而亮處部分呈一片白色,及亮度過暗區域。Press, digital cameras are widely used in life, such as: driving recorders, monitors and digital cameras...etc. When shooting, the image captured by the lens is projected onto the photosensitive element. Due to the exposure design of the existing camera photosensitive element, if the source of the light source is extremely extreme, the visible range is often narrowed, that is, the entire screen has only the bright part. Presented, the dark part is black, or the whole picture is only in the dark part, and the bright part is white, and the brightness is too dark.

當攝影畫面只有亮處部分呈現,另暗處部分呈一片黑色時,乃因攝影機之感測元件針對場景較亮區域曝光,由於該處亮度高,攝影機感測元件會選擇較短的曝光時間防止該處過度曝光,為了遷就亮部區域的短曝光時間,暗部區域因此曝光不足,而呈現一片黑色,以致亮度過暗區域,其圖像無法看清楚,反之,整個攝影畫面只有暗處部分呈現,而亮處部分呈一片白色,則係因為攝影機之感測元件針對場景較暗區域曝光,由於該處亮度不 足,攝影機之感測元件會選擇較長的曝光時間使得該處曝光充足,而為了遷就暗部區域的長曝光時間,亮部區域因此曝光過度,而呈現一片白色,導致亮度過亮區域之圖像無法清楚顯示,此即為低動態範圍〔Low Dynamic Range,LDR〕圖像。When the photographic picture is only partially illuminated, and the dark part is black, the sensing component of the camera is exposed to the brighter area of the scene. Because of the high brightness, the camera sensing component will select a shorter exposure time to prevent it. Excessive exposure, in order to accommodate the short exposure time of the bright area, the dark area is therefore underexposed, and appears a black, so that the brightness is too dark, the image can not be seen clearly, and vice versa, the entire photographic picture is only in the dark part, The bright part is white, because the sensing component of the camera is exposed to the darker area of the scene, because the brightness is not Foot, the sensing element of the camera will choose a longer exposure time to make the exposure sufficient, and in order to accommodate the long exposure time of the dark area, the bright area is overexposed and appears white, resulting in an image of the bright area It cannot be clearly displayed, this is the Low Dynamic Range (LDR) image.

為了克服該低動態範圍圖像問題,現有攝影業者常會於拍攝時,採取不同的曝光值設定,以拍攝出相同圖像,而不同感光程度之數個圖像後,再取出每一張細節充足的部分合成為同一張,由於係針對攝影景像各部位分別採取正確曝光時間以收集圖像資訊,故可以得到符合真實明暗度效果,然,此做法係須儲存多個圖像,而造成攝影機其圖像儲存空間浪費,另於進行合成處理時,容易受攝影業者技術良窳,而有品質上差異,且於進行合成處理上,更會耗費相當多時間,而不符合效率。In order to overcome this low dynamic range image problem, existing photographers often take different exposure value settings during shooting to capture the same image, and after several images with different sensitivity levels, take out each detail. The part is synthesized into the same piece. Since the correct exposure time is taken for each part of the photographic scene to collect the image information, the true darkness effect can be obtained. However, this method requires storing multiple images and causing the camera. The image storage space is wasted, and when the synthesis process is performed, it is easy to be technically good by the photographer, and there is a difference in quality, and it takes a considerable amount of time to perform the synthesis process, and does not conform to the efficiency.

緣是,本發明人有鑑於現有克服該低動態範圍圖像之方式,有上述諸多缺失處,乃藉其多年於相關領域的製造及設計經驗和知識的輔佐,並經多方巧思,針對現有攝影機之動態範圍擴展進行研發改良,而研創出本發明。The reason is that the present inventors have in view of the existing ways of overcoming the low dynamic range image, and the above-mentioned many disadvantages are supported by the manufacturing and design experience and knowledge of the related fields for many years, and through various ingenuity, The dynamic range expansion of the camera has been developed and improved, and the present invention has been developed.

本發明係有關於一種攝影機動態範圍擴展裝置及方法,其主要目的係為了提供一種利用一系統裝置經設計之演算法,便利生成高動態範圍圖像之攝影機動態範圍擴展裝置及方法。The present invention relates to a camera dynamic range expansion apparatus and method, and its main object is to provide a camera dynamic range expansion apparatus and method for facilitating generation of a high dynamic range image by using an algorithm designed by a system apparatus.

為了達到上述實施目的,本發明人乃研擬如下攝影機動態範圍擴展裝置,係主要包含有色彩空間轉換模組、參數統計電路、類型判斷電路、暗部補強電路、反相處理模組、混合電路及亮部提升電路,乃使該色彩空間轉換模組與該參數統計電路相耦接,且使該參數統計電路與該類型判斷電路相耦接,並使該 類型判斷電路分別與該暗部補強電路及該混合電路耦接,又使該暗部補強電路耦接於該混合電路,另使該色彩空間轉換模組與該反相處理模組相耦接,又使該反相處理模組耦接於該混合電路,復使該混合電路與該亮部提升電路相耦接。In order to achieve the above-mentioned implementation purposes, the inventors have developed the following camera dynamic range expansion device, which mainly includes a color space conversion module, a parameter statistics circuit, a type judgment circuit, a dark portion reinforcement circuit, an inversion processing module, a hybrid circuit, and The bright portion lifting circuit couples the color space conversion module with the parameter statistical circuit, and couples the parameter statistical circuit with the type determining circuit, and enables the The type judging circuit is coupled to the dark portion reinforcing circuit and the hybrid circuit, and the dark portion reinforcing circuit is coupled to the hybrid circuit, and the color space conversion module is coupled to the inversion processing module, and The inverting processing module is coupled to the hybrid circuit, and the hybrid circuit is coupled to the bright portion boosting circuit.

如上所述之攝影機動態範圍擴展裝置,其中,該攝影機動態範圍擴展裝置係進一步包含有自動增益控制電路,以與該亮部提升電路相耦接。The camera dynamic range expansion device as described above, wherein the camera dynamic range expansion device further comprises an automatic gain control circuit coupled to the bright portion boost circuit.

如上所述之攝影機動態範圍擴展裝置,其中,該反相處理模組係包含有相依序耦接之填滿電路、濾波電路及反相處理電路,乃使該色彩空間轉換模組與該填滿電路相耦接,另使該反相處理電路耦接於該混合電路。The camera dynamic range expansion device as described above, wherein the inversion processing module comprises a phase-coupling filling circuit, a filtering circuit and an inversion processing circuit, so that the color space conversion module is filled with The circuit is coupled to the circuit, and the inverting processing circuit is coupled to the hybrid circuit.

本發明係又進一步提出一種攝影機動態範圍擴展方法,係實施步驟係包含如下:A.圖像色彩空間轉換:係將攝影機拍攝下之圖像輸入一色彩空間轉換模組,以將紅、綠、藍三原色之色彩空間轉換成亮度、色差之色彩空間;B.圖像類型分類:繼將該原圖像之亮度值資料輸送往一參數統計電路進行直方圖統計分析,再由一類型判斷電路該原圖像分類為過暗型、過亮型或極端型其中之一;C.圖像暗部補強處理:再依該類型判斷電路判斷類型不同,將該過暗型與該極端型之圖像輸送往一暗部補強電路,以進行暗部補強作業,以獲得暗部補強圖像,再輸入一混合電路,若為過亮型圖像則直接輸入該混合電路;D.圖像反相處理:另由色彩空間轉換模組轉換之該 原圖像資料係同時輸送往一反相處理模組,以經該反相處理模組進行反相處理,獲得反相圖像,以輸入該混合電路;E.圖像混合處理:續由該混合電路將輸入之反相圖像及暗部補強圖像進行混合,以獲得混合圖像;F.圖像亮部提升處理:復由一亮部提升電路將該混合圖像進行亮部提升作業後,獲得高動態範圍圖像。The invention further proposes a camera dynamic range expansion method, the implementation steps are as follows: A. Image color space conversion: the image taken by the camera is input into a color space conversion module to red, green, The color space of the blue primary color is converted into the color space of brightness and chromatic aberration; B. Image type classification: following the brightness value data of the original image is sent to a parameter statistical circuit for histogram statistical analysis, and then a type judgment circuit The original image is classified into one of too dark type, over bright type or extreme type; C. Image dark part reinforcement processing: according to the type judgment circuit, the type of judgment is different, and the image of the over dark type and the extreme type is conveyed Reinforce the circuit to a dark part to perform the dark part reinforcement operation to obtain the dark part reinforcement image, and then input a hybrid circuit. If it is a super bright image, input the hybrid circuit directly; D. Image inversion processing: another color Space conversion module conversion The original image data is simultaneously sent to an inversion processing module for performing inversion processing through the inversion processing module to obtain an inverted image to input the hybrid circuit; E. Image mixing processing: continued by The hybrid circuit mixes the input inverted image and the dark portion enhanced image to obtain a mixed image; F. Image bright portion lifting process: after a bright portion lifting circuit performs the bright image lifting operation , get a high dynamic range image.

如上所述之攝影機動態範圍擴展方法,其中,該圖像類型分類步驟中之過暗型、過亮型及極端型係以Hi及Lo兩個參數值為分類基準,該Hi及Lo兩參數值公式如下: The camera dynamic range expansion method as described above, wherein the over-dark, over-bright, and extreme types in the image type classification step are based on two parameter values of Hi and Lo, and the Hi and Lo two parameter values are used. The formula is as follows:

其中,該h(i)代表直方圖灰階度i的統計像素量,Th1~Th4為四個自訂閥值,分別落在直方圖灰階度由小到大的四個位置,當Lo >2Hi 時,係為過暗型,又當Hi >2Lo 時,則為過亮型,另當時,即為極端型,該P total 為畫面總像素點數。Where h(i) represents the statistical pixel quantity of the histogram gray scale i, and Th1~Th4 are four custom thresholds, respectively falling in four positions of the histogram gray scale from small to large, when Lo > 2 Hi , the system is too dark, and when Hi > 2 Lo , it is too bright, another or When it is extreme, the P total is the total number of pixels of the picture.

如上所述之攝影機動態範圍擴展方法,其中,該類型判斷電路內係進一步包含相耦接之第一多工器、第二多工器及第三多工器,當類型判斷電路判斷Lo >2Hi 時,係透過對Hi左移1位元後得到2Hi,再將之與Lo做比較,若符合Lo >2Hi 則第一多工器選擇1,其序號〔type〕係輸出00代表過暗型,若不符合Lo >2Hi 則第一多工器選擇0,此時,輸出為第二多工器結果,該第二多工 器係透過判斷是否符合過亮型條件Hi >2Lo 來控制,若符合則第二多工器選擇1,其序號輸出為01代表過亮型,若不符合Hi >2Lo ,則第二多工器選擇0,此時,其輸出為第三多工器結果,該第三多工器係透過判斷是否符合極端型條件來控制,以上兩條件任一條件若符合,則第三多工器選擇1,此時,其序號輸出10代表極端型,如兩條件皆不符合則序號輸出11代表正常型。The camera dynamic range expansion method as described above, wherein the type judging circuit further comprises a first multiplexer, a second multiplexer and a third multiplexer coupled to each other, and the type judging circuit judges Lo >2 In Hi , the system shifts Hi by 1 bit to get 2Hi, and then compares it with Lo. If Lo > 2 Hi is met, the first multiplexer selects 1, and its serial number (type) is output 00 represents too dark. Type, if it does not meet Lo > 2 Hi , the first multiplexer selects 0. At this time, the output is the result of the second multiplexer, and the second multiplexer determines whether it meets the over-bright condition Hi > 2 Lo . Control, if it is met, the second multiplexer selects 1, the serial number output is 01 for over-bright type. If Hi > 2 Lo is not met, the second multiplexer selects 0. At this time, its output is the third multiplex. As a result, the third multiplexer determines whether it meets the extreme conditions. or To control, if any of the above two conditions are met, the third multiplexer selects 1. At this time, the serial number output 10 represents the extreme type. If both conditions are not met, the serial number output 11 represents the normal type.

如上所述之攝影機動態範圍擴展方法,其中,該圖像混合處理步驟中該反相圖像及暗部補強圖像係進行相乘運算,其相乘運算公式如下:y mix (i ,j )=y dark (i ,jy inv (i ,jk The camera dynamic range expansion method as described above, wherein the inverse image and the dark portion enhancement image are multiplied in the image mixing processing step, and the multiplication operation formula is as follows: y mix ( i , j )= y dark ( i , jy inv ( i , jk

其中,該y mix 係為混合運算後輸出值,y dark 為暗部補強圖像值,y inv 為反相圖像值,k值為輸出高動態範圍圖像之關鍵參數。The y mix is the output value after the hybrid operation, y dark is the dark portion to reinforce the image value, y inv is the inverted image value, and k is the key parameter for outputting the high dynamic range image.

如上所述之攝影機動態範圍擴展方法,其中,該圖像類型分類步驟中係進一步包含一非線性曲線方程式之運算,該非線性曲線方程式之公式如下: The camera dynamic range expansion method as described above, wherein the image type classification step further comprises an operation of a nonlinear curve equation, and the formula of the nonlinear curve equation is as follows:

其中,y inv 為反相值、y lpf 為原圖像資料輸入值。Where y inv is the inverse value and y lpf is the input value of the original image data.

如上所述之攝影機動態範圍擴展方法,其中,該圖像類型分類步驟中係進一步包含一補償方程式之運算,該補償方程式之公式如下: The camera dynamic range expansion method as described above, wherein the image type classification step further comprises an operation of compensating the equation, and the formula of the compensation equation is as follows:

其中,y dark 為暗部補強圖像輸出值,y in 為輸入圖像灰階值,Th dark 為自訂之閥值。Where y dark is the dark portion to reinforce the image output value, y in is the input image grayscale value, and Th dark is the custom threshold.

如上所述之攝影機動態範圍擴展方法,其中,該圖像亮部提升處理步驟之亮部提升運算公式如下: The camera dynamic range expansion method as described above, wherein the bright portion lifting operation formula of the image bright portion lifting processing step is as follows:

其中,y mix 為混合圖像值,y dark 為暗部補強圖像值,Th b 為閥值,另y HDR 為高動態範圍圖像值。Where y mix is the mixed image value, y dark is the dark portion to reinforce the image value, Th b is the threshold value, and y HDR is the high dynamic range image value.

如上所述之攝影機動態範圍擴展方法,其中,該暗部補強電路係進一步包含相耦接之多工器、暫存器及減法器,當暗部補強電路之多工器接收到由該類型判斷電路輸出之一過暗型序號時,多工器係選擇128作為閥值輸出,當暗部補強電路之多工器接收到由該類型判斷電路輸出之一極端型序號時,多工器係選擇50作為閥值輸出,其輸出值係被存進暫存器,接著由暫存器將該閥值送進減法器與一由該類型判斷電路輸入之圖像亮度值資料相減,並將相減結果右移4位元後與該輸入之圖像亮度值資料相加得到補強運算結果。The camera dynamic range expansion method as described above, wherein the dark portion reinforcing circuit further comprises a multiplexer coupled to the multiplexer, the register and the subtractor, and the multiplexer of the dark portion reinforcing circuit receives the output from the type determining circuit When one of the dark type numbers is over, the multiplexer selects 128 as the threshold value output. When the multiplexer of the dark portion reinforcing circuit receives an extreme type number output by the type determining circuit, the multiplexer selects 50 as the valve. Value output, the output value is stored in the register, and then the buffer sends the threshold to the subtractor and subtracts the image brightness value data input by the type judgment circuit, and subtracts the result right After shifting 4 bits, the image of the input brightness value is added to obtain the result of the reinforcement operation.

如上所述之攝影機動態範圍擴展方法,其中,該亮部提升電路係包含相耦接之一多工器、一比較器、一減法器、一乘法器及兩加法器,乃使減法器執行一由暗部補強電路輸入之暗部補強值 〔dark_enh〕減50動作,乘法器將減法器的結果與自身相乘,以得到暗部補強平方值〔dark_enh_mul〕,再使用右移8位元與右移10位元相加動作,以達成乘以0.005的值,之後將此值加在混合圖像之輸出值〔out_mix〕上,復待後方多工器〔MUX〕選擇輸出。The camera dynamic range expansion method as described above, wherein the bright portion lifting circuit comprises a phase coupled multiplexer, a comparator, a subtractor, a multiplier and two adders, so that the subtractor performs one Dark portion reinforcement value input by the dark portion reinforcement circuit [dark_enh] minus 50 action, the multiplier multiplies the result of the subtractor by itself to obtain the dark squared square value [dark_enh_mul], and then uses the right shift 8 bit and the right shift 10 bit to add the multiplication The value of 0.005 is then added to the output value [out_mix] of the mixed image, and the rear multiplexer [MUX] is selected to output.

藉此,當攝影機鏡頭拍攝下圖像,輸入本發明之色彩空間轉換模組後,經由參數統計電路、類型判斷電路、暗部補強電路、反相處理模組、混合電路、亮部提升電路及自動增益控制電路之運算,便可即時輸出高動態範圍〔HDR〕之圖像,不須等待人力進行合成處理之時間,符合實用效率,另亦可避免依賴人力合成處理上會因處理人員技術上良窳,產生之品質上差異性,且本發明於實施上只須單張圖像資料便可生成高動態範圍圖像,更可避免儲存多個圖像,造成之儲存空間不當浪費等情形者。Thereby, when the camera lens captures the image and inputs the color space conversion module of the present invention, the parameter statistics circuit, the type judgment circuit, the dark portion reinforcement circuit, the inversion processing module, the hybrid circuit, the bright portion lifting circuit, and the automatic The operation of the gain control circuit can instantly output images with high dynamic range (HDR), without waiting for the time for manual processing, which is in line with practical efficiency, and can also avoid relying on manual synthesis processing because of the technically good processing personnel.窳, the quality difference is generated, and the invention only needs a single image data to generate a high dynamic range image, and can avoid storing a plurality of images, resulting in improper storage space and the like.

(1)‧‧‧色彩空間轉換模組(1) ‧‧‧Color Space Conversion Module

(2)‧‧‧參數統計電路(2) ‧‧‧Parameter statistics circuit

(3)‧‧‧類型判斷電路(3) ‧‧‧ type judgment circuit

(4)‧‧‧暗部補強電路(4) ‧‧‧Dark part reinforcement circuit

(5)‧‧‧反相處理模組(5) ‧‧‧ reverse processing module

(51)‧‧‧填滿電路(51)‧‧‧ Filling the circuit

(52)‧‧‧濾波電路(52)‧‧‧Filter circuit

(53)‧‧‧反相處理電路(53)‧‧‧Inverting Processing Circuit

(6)‧‧‧混合電路(6) ‧‧‧Hybrid circuits

(7)‧‧‧亮部提升電路(7) ‧ ‧ bright boost circuit

(8)‧‧‧自動增益控制電路(8)‧‧‧Automatic gain control circuit

第一圖:本發明之架構圖First figure: the architecture diagram of the present invention

第二圖:本發明之流程圖Second figure: flow chart of the present invention

第三圖:本發明之Lo值統計電路圖The third figure: the Lo value statistical circuit diagram of the present invention

第四圖:本發明之類型判斷電路圖Fourth figure: the type judgment circuit diagram of the present invention

第五圖:本發明之暗部補強電路圖Figure 5: Reinforcement circuit diagram of the dark portion of the present invention

第六圖:本發明之亮部提升電路圖Figure 6: Bright circuit diagram of the bright part of the present invention

而為令本發明之技術手段及其所能達成之效果,能夠有更完整且清楚的揭露,茲詳細說明如下,請一併參閱揭露之 圖式及圖號:首先,請參閱第一圖所示,為本發明之攝影機動態範圍擴展裝置,係主要由色彩空間轉換模組〔RGB to YCbCr Module〕(1)、參數統計電路〔Parameter Statistics Circuit〕(2)、類型判斷電路〔Type Judgment Circuit〕(3)、暗部補強電路〔Dark Enhancement Circuit〕(4)、反相處理模組〔Inverse Module〕(5)、混合電路〔Mix Circuit〕(6)、亮部提升電路〔Bright Enhancement Circuit〕(7)及自動增益控制電路〔Automatic Gain Control Circuit〕(8)所組成;其中:該色彩空間轉換模組(1)係與該參數統計電路(2)相耦接,且使該參數統計電路(2)與該類型判斷電路(3)相耦接,並使該類型判斷電路(3)分別與該暗部補強電路(4)及該混合電路(6)耦接,又使該暗部補強電路(4)與該混合電路(6)耦接,另該反相處理模組(5)係包含相依序耦接之填滿電路(51)、濾波電路(52)及反相處理電路〔Inverse Processing Circuit〕(53),該填滿電路(51)係為2×2最大值填滿電路〔2×2Maximum Fill Circuit〕,該濾波電路(52)則為3×3低通濾波電路〔3×3 Lowpass Filter〕,乃使該反相處理模組(5)之填滿電路(51)與該色彩空間轉換模組(1)相耦接,又使該反相處理模組(5)之反相處理電路(53)與該混合電路(6)相耦接,復使該混合電路(6)與該亮部提升電路(7)耦接,且使該亮部提升電路(7)與自動增益控制電路(8)相耦接。In order to make the technical means of the present invention and the effects thereof can be more completely and clearly disclosed, the details are as follows, please refer to the disclosure. Figure and figure number: First, please refer to the first figure, which is the camera dynamic range expansion device of the present invention, which mainly consists of a color space conversion module [RGB to YCbCr Module] (1), parameter statistics circuit [Parameter Statistics] Circuit] (2), Type Judgment Circuit (3), Dark Enhancement Circuit (4), Inverse Module (5), Mix Circuit (Mix Circuit) 6), a Bright Enhancement Circuit (7) and an Automatic Gain Control Circuit (8); wherein: the color space conversion module (1) is associated with the parameter statistics circuit ( 2) coupled, and the parameter statistical circuit (2) is coupled to the type determining circuit (3), and the type determining circuit (3) and the dark portion reinforcing circuit (4) and the hybrid circuit ( 6) coupling, the dark portion reinforcing circuit (4) is coupled to the hybrid circuit (6), and the inverting processing module (5) includes a phase-parameterly coupled filling circuit (51) and a filter circuit. (52) and an inverse processing circuit (Inverse Processing Circuit) (53), The filling circuit (51) is a 2×2 maximum filling circuit [2×2 Maximum Fill Circuit], and the filtering circuit (52) is a 3×3 low-pass filter circuit (3×3 Lowpass Filter), The filling circuit (51) of the inverting processing module (5) is coupled to the color space conversion module (1), and the inverting processing circuit (53) of the inverting processing module (5) is The hybrid circuit (6) is coupled to the hybrid circuit (7) and coupled to the bright portion boosting circuit (7), and the bright portion boosting circuit (7) is coupled to the automatic gain control circuit (8) .

據此,請一併參閱第二圖所示,為本發明之攝影機動態範圍擴展裝置之實施步驟係包含如下:A.圖像色彩空間轉換:當攝影機之鏡頭拍攝下之原 圖像〔y lpf 〕輸入本發明之色彩空間轉換模組(1)後,即由本發明之色彩空間轉換模組(1)將紅、綠、藍三原色〔RGB〕之色彩空間轉換成亮度、色差〔YCbCr〕之色彩空間。Accordingly, please refer to the second figure, the implementation steps of the camera dynamic range expansion device of the present invention include the following: A. Image color space conversion: the original image taken by the camera lens [ y lpf After inputting the color space conversion module (1) of the present invention, the color space conversion module (1) of the present invention converts the color spaces of the three primary colors [RGB] of red, green and blue into the color of the brightness and the color difference [YCbCr]. space.

B.圖像類型分類:繼將該原圖像〔y lpf 〕亮度值〔Y〕資料輸送往參數統計電路(2)以進行直方圖統計分析〔Histogram Analysis〕,再由類型判斷電路(3)將該原圖像〔y lpf 〕分類為過暗型〔Dark Type〕、過亮型〔Bright Type〕或極端型〔Extreme Type〕三種類型其中之一,而該過暗型、過亮型及極端型係以Hi及Lo兩個參數值為分類基準,該Hi及Lo兩參數值公式如下: B. Image type classification: the original image [ y lpf ] brightness value [Y] data is sent to the parameter statistics circuit (2) for histogram statistical analysis [Histogram Analysis], and then by the type judgment circuit (3) The original image [ y lpf ] is classified into one of three types: Dark Type, Bright Type, or Extreme Type, and the over-dark, over-bright, and The extreme type is based on the Hi and Lo parameters. The Hi and Lo parameters are as follows:

其中,該h(i)代表直方圖〔histogram〕灰階度i的統 計像素量,Th1~Th4為四個自訂閥值,分別落在直方圖灰階度由小到大的四個位置,得到之Hi及Lo值可提供類型判斷電路(3)做為圖像類型之分類,該過暗型表示整個圖像偏暗,因此,Lo值會比Hi值高很多,當Lo >2Hi ,類型判斷電路(3)即判斷為過暗型,反之,過亮型則表示整個圖像偏亮,因此,Hi值會比Lo值高許多,當Hi >2Lo ,類型判斷電路(3)即判斷為過亮型,另極端型代表整個圖像過亮部分很多,過暗的部分亦很多,因此,Hi值與Lo值都會相當高,但彼此間的差異不會大於對方的兩倍, 故當時,類型判斷電路(3)即判斷為極端 型,其中,該P total 為畫面總像素點數。Where h(i) represents the statistical pixel quantity of the histogram gray scale i, and Th1~Th4 are four custom thresholds, respectively falling in four positions of the histogram gray scale from small to large. The obtained Hi and Lo values can provide the type judgment circuit (3) as the classification of the image type, and the over dark type indicates that the entire image is dark, so the Lo value is much higher than the Hi value, when Lo > 2 Hi , The type judging circuit (3) judges to be too dark, whereas the over-bright type indicates that the entire image is bright. Therefore, the Hi value is much higher than the Lo value. When Hi > 2 Lo , the type judging circuit (3) Judging as too bright, the other extremes represent a lot of bright parts of the entire image, too many parts of the dark, so the Hi value and Lo value will be quite high, but the difference between them is not more than twice the other, so when or The type judging circuit (3) judges to be an extreme type, wherein the P total is the total number of pixels of the picture.

C.圖像暗部補強處理:再依該類型判斷電路(3)判斷類型不同,將過暗型與極端型之圖像輸送往暗部補強電路(4)進行暗部補強〔Dark Enhancement〕動作,由於極端型之暗度小於過暗型,故在暗部補償時,係會以小於過暗型的參數進行暗部補償處理,以獲得暗部補強圖像〔y dark 〕,再輸入混合電路(6),若為過亮型圖像則直接輸送至混合電路(6)。C. Image dark portion reinforcement processing: According to the type judgment circuit (3), the type of judgment is different, and the image of the over dark type and the extreme type is sent to the dark portion reinforcing circuit (4) for dark enhancement (Dark Enhancement) action, due to extreme The darkness of the type is less than the over-dark type. Therefore, when the dark portion is compensated, the dark portion compensation processing is performed with the parameter smaller than the over-dark type to obtain the dark portion reinforcing image [ y dark ], and then input to the hybrid circuit (6). The over-bright image is sent directly to the hybrid circuit (6).

D.圖像反相處理:另由色彩空間轉換模組(1)轉換之該原圖像〔y lpf 〕資料係同時輸送往反相處理模組(5),以經反相處理模組(5)之填滿電路(51)、濾波電路(52)及反相處理電路(53)處理得到反相圖像〔y inv 〕以輸送至混合電路(6)。D. Image inversion processing: the original image [ y lpf ] data converted by the color space conversion module (1) is simultaneously sent to the inversion processing module (5) to be processed by the inversion processing module ( 5) the fill circuit (51), a filter circuit (52) processing circuit and an inverter (53) to give the inverted image] [y inv delivery to the mixing circuit (6).

E.圖像混合處理:續由混合電路(6)將輸入之反相圖像〔y inv 〕及暗部補強圖像〔y dark 〕進行混合,以獲得到修正後之混合圖像〔y mix 〕,該反相圖像〔y inv 〕及暗部補強圖像〔y dark 〕之混合處理,係將兩圖像做相乘運算,其相乘運算公式如下:y mix (i ,j )=y dark (i ,jy inv (i ,jk E. Image mixing processing: The mixed image ( y inv ) and the dark portion reinforcing image ( y dark ) are mixed by the mixing circuit (6) to obtain the corrected mixed image [ y mix ] The mixing process of the inverse image [ y inv ] and the dark portion enhancement image [ y dark ] is to multiply the two images by the following equation: y mix ( i , j )= y dark ( i , jy inv ( i , jk

其中,y mix 為混合運算後之輸出值,y dark 為暗部補強圖像值,y inv 為反相圖像值,k值為輸出高動態範圍圖像之關鍵參數,越大k值可獲得之高動態範圍〔High Dynamic Range,HDR〕程度越高,即暗部被提高越多亮部亦被降低越多之輸出結果,當使用反相圖像〔y inv 〕及暗部補強圖像〔y dark 〕相乘後,該混合圖像〔y mix 〕 暗處即會被提高亮度,而亮處亦會被減低亮度。Where y mix is the output value after the blending operation, y dark is the dark portion to reinforce the image value, y inv is the inverse image value, and k is the key parameter for outputting the high dynamic range image, the larger the k value is available The higher the dynamic range (High Dynamic Range, HDR), the more the dark portion is raised, the more the bright portion is also reduced. The more the output is, the inverse image [ y inv ] and the dark portion image ( y dark ) are used. After multiplying, the mixed image [ y mix ] will be brightened in the dark and the bright spot will be reduced.

F.圖像亮部提升處理:為防止於混合處理時,原亮度較高處被減弱到比原亮度較暗處還暗情況,本發明係在混和處理後,將該混合圖像〔y mix 〕輸送往亮部提升電路(7),以由亮部提升電路(7)執行亮部提升〔Bright Enhancement〕動作,以防止原亮較高處被過度減暗現象,其運算公式如下: F. Image brightening processing: In order to prevent the mixing process from being weakened, the original brightness is weakened to be darker than the original brightness, and the present invention is the mixed image after the mixing process [ y mix 〕 is sent to the bright portion lifting circuit (7) to perform the Bright Enhancement action by the bright portion lifting circuit (7) to prevent excessive darkening of the original bright portion, and the calculation formula is as follows:

其中,y mix 為該式主要處理對象,即混合處理後之該混合圖像值,y dark 為暗部補強圖像值,Th b 為閥值,另y HDR 為亮部提升後之高動態範圍圖像值,於該運算式中主要係針對暗部補強圖像〔y dark 〕中灰階值大於Th b 部位進行混合圖像〔y mix 〕之調整,而獲得之高動態範圍圖像〔y HDR 〕係會再輸送往自動增益控制電路(8),進行亮度之對比調整後輸出,於此,即可獲得經微調修正後之高動態範圍圖像輸出。Where y mix is the main processing object of the formula, that is, the mixed image value after the blending process, y dark is the dark portion reinforcing image value, Th b is the threshold value, and y HDR is the high dynamic range graph after the bright portion is raised. like value to the calculation formula for mainly reinforcing the dark portion of the image] [y dark gray level value greater than Th b adjust the mixed image portion [of] y mix, to obtain a high dynamic range image] [y HDR The system will then send it to the automatic gain control circuit (8) for brightness contrast adjustment and output, and then the fine-tuned high dynamic range image output can be obtained.

又為解決本發明之參數統計電路(2)於運算時,在直方圖中灰階值中間段部分過度提高情況,本發明係利用一非線性曲線方程式,其公式如下: In order to solve the problem that the parameter statistical circuit (2) of the present invention is excessively improved in the middle portion of the gray scale value in the histogram, the present invention utilizes a nonlinear curve equation, and the formula is as follows:

其中,y inv 為反相值、y lpf 為原圖像輸入值,於該非線性的曲線方程式中輸入值〔y lpf 〕越低,輸出值〔y inv 〕越高特點,當較低灰階度,便可產生較高的反相灰階值做相乘以提升亮度, 若原圖像輸入值〔y lpf 〕為0,雖會得到最大反相值255,由於本發明於混合電路(6)之混合圖像係由反相圖像及由原圖像〔y lpf 〕經暗部補強之暗部補強圖像〔y dark 〕相乘而來,當原圖像〔y lpf 〕輸入值為0,係造成其相乘之乘積為0,而亮度無法提升缺失,因此,為防止上述情況發生,本發明係使用下式公式進行補償處理: Where y inv is the inverse value and y lpf is the original image input value. In the nonlinear curve equation, the lower the input value [ y lpf ], the higher the output value [ y inv ], and the lower gray scale. , can generate a higher inverse gray scale value to multiply to increase the brightness, if the original image input value [ y lpf ] is 0, although the maximum inverse value of 255 will be obtained, because the present invention is in the hybrid circuit (6) the mixed image and the original image line by the inverter y lpf [image] [image reinforcing derived by multiplying] y dark dark dark portion of the reinforcing portion, when the original image is input [0] y lpf, resulting in system The multiplied product is 0, and the brightness cannot increase the missing. Therefore, in order to prevent the above, the present invention uses the following formula to perform the compensation process:

其中y dark 為暗部補強圖像輸出值,y in 為輸入圖像灰階值,式中的閥值〔Th dark 〕為可改變增亮的幅度,越高的閥值〔Th dark 〕使得低灰階度得到更高的增益,也使得處理範圍增寬、影響範圍增廣,而越低的閥值〔Th dark 〕則可收斂影響範圍,對於低灰階度的提升也較小,若輸入為過暗型圖像,表示圖像大部分都是低灰階度,此時會使用較大的閥值〔Th dark 〕來增大提升增益及處理範圍,本發明於過暗型圖像之閥值〔Th dark 〕數值設為128,若輸入為極端型圖像,極端型圖像中的低灰階度係較過暗型圖像少,此時若與過暗型圖像使用一樣閥值〔Th dark 〕會使圖像暗處被過度提高,故極端型圖像之閥值〔Th dark 〕應較過暗型圖像低。Where y dark is the dark part to reinforce the image output value, y in is the input image gray scale value, the threshold value in the formula [ Th dark ] is the amplitude that can change the brightness, and the higher the threshold value ( Th dark ) makes the low gray The higher the gain of the gradation, the wider the processing range and the wider the influence range, and the lower the threshold ( Th dark ) can converge the range of influence, and the increase for the low gray level is smaller, if the input is An over-dark image indicates that the image is mostly low-gray. In this case, a larger threshold ( Th dark ) is used to increase the gain and processing range. The present invention is a valve for over-dark images. The value of [ Th dark ] is set to 128. If the input is an extreme image, the low gray level in the extreme image is less than that of the dark image. If the value is the same as the over dark image, the threshold is used. [ Th dark ] will increase the darkness of the image too much, so the threshold value of the extreme image ( Th dark ) should be lower than that of the dark image.

本發明之條件判斷是依據兩個參數Lo及Hi,Lo值為直方圖〔histogram〕灰階度小於50的部分加總起來,其中灰階度小於15部分取兩倍加成。由於Lo與Hi值是統計整個圖像〔 frame〕後才可得到參數,依硬體上即時輸出要求,無法等到統計完當前圖像〔frame〕資料再做後續處理,故實作上本發明係使用前一張圖像〔frame〕統計資料當作當前圖像〔frame〕的Lo值。請一併參閱第三圖所示,本發明於電路執行上係先判斷輸入值是否小於15,若為是,則執行一次+2動作,否則繼續判斷輸入值是否小於50,若為是,則執行一次+1動作,重複以上動作直到垂直同步信號觸發,即代表一個圖像〔frame〕結束,此時先將累加值輸出作為Lo值供下一個圖像〔frame〕使用,再歸零累加器重新計算下一個Lo值。The condition of the present invention is judged according to two parameters Lo and Hi, and the Lo value is a histogram whose gray scale is less than 50, wherein the gray scale is less than 15 and the double is added. Since the Lo and Hi values are statistical images of the entire image [ After the frame], the parameters can be obtained. According to the real-time output requirement on the hardware, the current image [frame] data cannot be counted and then processed. Therefore, the present invention uses the previous image [frame] statistics. Make the Lo value of the current image [frame]. Please refer to the third figure, the invention first determines whether the input value is less than 15 in the circuit execution, and if so, performs a +2 action, otherwise it continues to determine whether the input value is less than 50, and if yes, then Perform a +1 action and repeat the above actions until the vertical sync signal is triggered, that is, the end of an image [frame]. At this time, the accumulated value output is used as the Lo value for the next image [frame], and then the zero accumulator is used. Recalculate the next Lo value.

又請一併參閱第四圖所示,本發明之類型判斷電路(3)係包含相耦接之第一多工器、第二多工器及第三多工器,當類型判斷電路(3)對過暗型條件判斷係為Lo >2Hi ,透過對Hi左移1位元〔bit〕後得到2Hi,再將之與Lo做比較,若符合Lo >2Hi 則第一多工器〔MUX 1〕選擇1,其序號〔type〕係輸出00代表過暗型;若不符合Lo >2Hi 則第一多工器〔MUX 1〕選擇0,此時,輸出為第二多工器〔MUX 2〕結果,第二多工器〔MUX 2〕是透過判斷是否符合過亮型條件Hi >2Lo 來控制,若符合則第二多工器〔MUX 2〕選擇1,其序號〔type〕輸出01代表過亮型,若不符合Hi >2Lo ,則第二多工器〔MUX 2〕選擇0,此時輸出為第三多工器〔MUX 3〕結果,第三多工器〔MUX 3〕是透過判斷是否符合極端型條件來控制,以上兩條件任一條件若符合則第三多工 器〔MUX 3〕選擇1,此時,序號〔type〕輸出10代表極端型,如 兩條件皆不符合,則序號〔type〕輸出11代表正常型。Please also refer to the fourth figure, the type judging circuit (3) of the present invention includes a first multiplexer, a second multiplexer and a third multiplexer coupled to each other, when the type judging circuit (3) The judgment of the over-dark condition is Lo > 2 Hi . After shifting Hi to the left by 1 bit [bit], 2Hi is obtained, and then compared with Lo. If Lo > 2 Hi is satisfied, the first multiplexer is used. MUX 1] select 1, whose serial number [type] is output 00 for over-dark type; if Lo > 2 Hi is not met, the first multiplexer [MUX 1] selects 0, at this time, the output is the second multiplexer [ MUX 2] As a result, the second multiplexer [MUX 2] is controlled by judging whether it is in compliance with the over-bright condition Hi > 2 Lo , and if it is met, the second multiplexer [MUX 2] selects 1, the serial number [type] Output 01 stands for over-bright type. If Hi > 2 Lo is not met, the second multiplexer [MUX 2] selects 0. At this time, the output is the result of the third multiplexer [MUX 3], and the third multiplexer [MUX] 3] is to judge whether it meets the extreme conditions or To control, if any of the above two conditions are met, the third multiplexer [MUX 3] selects 1. At this time, the serial number [type] output 10 represents the extreme type. If both conditions are not met, the serial number [type] is output. 11 represents the normal type.

另請一併參閱第五圖所示,本發明之暗部補強電路(4)係包含相耦接之多工器、暫存器及減法器,當暗部補強電路(4)之多工器接收到由類型判斷電路(3)輸出之過暗型序號〔type[1]〕為0,而極端型序號〔type[1]〕為1,當序號〔type[1]〕為0時,多工器〔MUX〕將選擇128作為閥值〔Th dark 〕輸出;當序號〔type[1]〕為1時,多工器〔MUX〕將選擇50作為閥值〔Th dark 〕輸出,所輸出值將被存進暫存器,接著將暫存器送進減法器與一由類型判斷電路輸入之圖像亮度值資料〔data_Y〕相減,並將相減結果右移4位元〔bit〕後與該輸入圖像亮度值資料〔data_Y〕相加得到補強運算結果,右移4位元〔bit〕目的在於取代於中,乘 0.005動作。Please also refer to the fifth figure. The dark portion reinforcing circuit (4) of the present invention comprises a multiplexer, a register and a subtractor coupled to each other, and the multiplexer of the dark portion reinforcing circuit (4) receives the multiplexer. The over-type number [type[1]] output by the type judgment circuit (3) is 0, and the extreme type number [type[1]] is 1, when the serial number [type[1]] is 0, the multiplexer [MUX] will select 128 as the threshold value ( Th dark ) output; when the serial number [type[1]] is 1, the multiplexer [MUX] will select 50 as the threshold value ( Th dark ) output, and the output value will be Stored in the scratchpad, then the scratchpad is sent to the subtractor and subtracted from the image brightness value data [data_Y] input by the type judging circuit, and the subtraction result is shifted right by 4 bits [bit] and The input image brightness value data [data_Y] is added to obtain the result of the reinforcement operation, and the right shift of 4 bits [bit] is intended to replace In the middle, multiply 0.005 action.

復請一併參閱第六圖所示,本發明之亮部提升電路(7)係主要由一多工器、一比較器、一減法器、一乘法器及兩個加法器組成,首先減法器部分係不斷執行由暗部補強電路(4)輸入之暗部補強值〔dark_enh〕減50動作,乘法器將減法器的結果與自身相乘得到暗部補強平方值〔dark_enh_mul〕為[15:0],之後再使用右移8位元〔bit〕與右移10位元〔bit〕相加動作來達成0.0039+0.009=0.0048,約略等於乘以0.005的值,之後將此值加在混合圖像之輸出值〔out_mix〕上,到此為止為亮部提升運算的部 分,待後方多工器〔MUX〕選擇輸出。電路中後方多工器〔MUX〕的選擇依據為判斷暗部補強值〔dark_enh〕是否小於50的比較器,比較器不斷執行暗部補強值〔dark_enh〕與50的比較動作,若暗部補強值〔dark_enh〕小於50代表此處不需做亮部提升,比較器輸出1,多工器直接選擇混合圖像之輸出值〔out_mix〕作為高動態範圍輸出值〔out_HDR〕之輸出,暗部補強值〔dark_enh〕大於等於50表示該位置需做亮部提升,此時,比較器輸出0,後方多工器〔MUX〕係選擇經亮部提升運算後結果輸出高動態範圍輸出值〔out_HDR〕。Referring to FIG. 6 together, the bright portion lifting circuit (7) of the present invention is mainly composed of a multiplexer, a comparator, a subtractor, a multiplier and two adders, and the first subtractor The part is continuously executed by the dark part reinforcement value (dark_enh) minus 50 input by the dark part reinforcement circuit (4), and the multiplier multiplies the result of the subtractor by itself to obtain the dark part reinforcement square value [dark_enh_mul] is [15:0], after which Then use the right shift 8-bit [bit] and the right shift 10-bit [bit] add action to achieve 0.0039 + 0.009 = 0.0048, approximately equal to multiply the value of 0.005, and then add this value to the output value of the mixed image. [out_mix], the part that is now the highlight promotion calculation Points, wait for the rear multiplexer [MUX] to select the output. The selection of the rear multiplexer [MUX] in the circuit is based on a comparator that determines whether the dark portion reinforcement value [dark_enh] is less than 50, and the comparator continuously performs the comparison of the dark portion reinforcement value [dark_enh] with 50, if the dark portion reinforcement value [dark_enh] Less than 50 means there is no need to do the bright part boost, the comparator output is 1, the multiplexer directly selects the output value of the mixed image [out_mix] as the output of the high dynamic range output value [out_HDR], and the dark portion reinforcement value [dark_enh] is larger than A value equal to 50 indicates that the position needs to be brightened. At this time, the comparator outputs 0, and the rear multiplexer [MUX] selects the high dynamic range output value (out_HDR) after the bright portion is boosted.

藉此,於攝影機鏡頭拍攝下圖像,輸入本發明之色彩空間轉換模組(1)後,經由參數統計電路(2)、類型判斷電路(3)、暗部補強電路(4)、反相處理模組(5)、混合電路(6)、亮部提升電路(7)及自動增益控制電路(8)之運算,便可即時輸出高動態範圍〔HDR〕之圖像,不須等待人力進行合成處理之時間,符合實用效率,另亦可避免依賴人力合成處理上會因處理人員技術上良窳,產生之品質上差異性,且本發明於實施上只須單張圖像資料便可生成高動態範圍圖像,更可避免儲存多個圖像,造成之儲存空間不當浪費等情形者。Thereby, the image is taken by the camera lens, and after inputting the color space conversion module (1) of the present invention, the parameter statistics circuit (2), the type judgment circuit (3), the dark portion reinforcement circuit (4), and the inversion processing are performed. Module (5), hybrid circuit (6), bright boost circuit (7) and automatic gain control circuit (8) can instantly output high dynamic range (HDR) images without waiting for human synthesis The processing time is in line with the practical efficiency, and it can also avoid the quality difference caused by the processing personnel in the human synthetic processing, and the invention can only generate a single image data in the implementation. The dynamic range image can avoid storing multiple images, causing improper storage space and other situations.

前述之實施例或圖式並非限定本發明之攝影機動態範圍擴展裝置及方法實施樣態,凡所屬技術領域中具有通常知識者所為之適當變化或修飾,皆應視為不脫離本發明之攝影機動態範圍擴展裝置及方法專利範疇。The foregoing embodiments or drawings are not intended to limit the implementation of the camera dynamic range expansion apparatus and method of the present invention. Any changes or modifications made by those skilled in the art should be regarded as not departing from the camera dynamics of the present invention. Range extension device and method patent category.

綜上所述,本發明實施例確能達到所預期功效,又其所揭露之具體構造,不僅未曾見諸於同類產品中,亦未曾公開於申請前,誠已完全符合專利法之規定與要求,爰依法提出發明專利之申請,懇請惠予審查,並賜准專利,則實感德便。In summary, the embodiments of the present invention can achieve the expected functions, and the specific structures disclosed therein have not been seen in similar products, nor have they been disclosed before the application, and have fully complied with the requirements and requirements of the Patent Law. If you apply for an invention patent in accordance with the law, you are welcome to review it and grant a patent.

(1)‧‧‧色彩空間轉換模組(1) ‧‧‧Color Space Conversion Module

(2)‧‧‧參數統計電路(2) ‧‧‧Parameter statistics circuit

(3)‧‧‧類型判斷電路(3) ‧‧‧ type judgment circuit

(4)‧‧‧暗部補強電路(4) ‧‧‧Dark part reinforcement circuit

(5)‧‧‧反相處理模組(5) ‧‧‧ reverse processing module

(51)‧‧‧填滿電路(51)‧‧‧ Filling the circuit

(52)‧‧‧濾波電路(52)‧‧‧Filter circuit

(53)‧‧‧反相處理電路(53)‧‧‧Inverting Processing Circuit

(6)‧‧‧混合電路(6) ‧‧‧Hybrid circuits

(7)‧‧‧亮部提升電路(7) ‧ ‧ bright boost circuit

(8)‧‧‧自動增益控制電路(8)‧‧‧Automatic gain control circuit

Claims (13)

一種攝影機動態範圍擴展裝置,係主要包含有色彩空間轉換模組、參數統計電路、類型判斷電路、暗部補強電路、反相處理模組、混合電路及亮部提升電路,乃使該色彩空間轉換模組與該參數統計電路相耦接,且使該參數統計電路與該類型判斷電路相耦接,並使該類型判斷電路分別與該暗部補強電路及該混合電路耦接,又使該暗部補強電路耦接於該混合電路,另使該色彩空間轉換模組與該反相處理模組相耦接,又使該反相處理模組耦接於該混合電路,復使該混合電路與該亮部提升電路相耦接。A camera dynamic range expansion device mainly comprises a color space conversion module, a parameter statistics circuit, a type judgment circuit, a dark portion reinforcement circuit, an inverse processing module, a hybrid circuit and a bright portion lifting circuit, so that the color space conversion mode The group is coupled to the parameter statistic circuit, and the parameter statistic circuit is coupled to the type judging circuit, and the type judging circuit is respectively coupled to the dark portion reinforcing circuit and the hybrid circuit, and the dark portion reinforcing circuit is further coupled The color space conversion module is coupled to the inversion processing module, and the inversion processing module is coupled to the hybrid circuit to reproduce the hybrid circuit and the bright portion. The boosting circuit is coupled. 如申請專利範圍第1項所述攝影機動態範圍擴展裝置,其中,該攝影機動態範圍擴展裝置係進一步包含有自動增益控制電路,以與該亮部提升電路相耦接。The camera dynamic range expansion device of claim 1, wherein the camera dynamic range expansion device further comprises an automatic gain control circuit coupled to the bright portion boost circuit. 如申請專利範圍第1項所述攝影機動態範圍擴展裝置,其中,該反相處理模組係包含有相依序耦接之2×2最大值填滿電路、3×3低通濾波電路及反相處理電路,乃使該色彩空間轉換模組與該填滿電路相耦接,另使該反相處理電路耦接於該混合電路。The camera dynamic range expansion device according to claim 1, wherein the reverse processing module comprises a 2×2 maximum filling circuit, a 3×3 low-pass filter circuit, and a phase inversion. The processing circuit is configured to couple the color space conversion module to the filling circuit, and the inverting processing circuit is coupled to the hybrid circuit. 一種攝影機動態範圍擴展方法,係實施步驟係包含如下:A.圖像色彩空間轉換:係將攝影機拍攝下之圖像輸入一色彩空間轉換模組,以將紅、綠、藍三原色之色彩空間轉換成亮度、色差之色彩空間;B.圖像類型分類:繼將該原圖像之亮度值資料輸送往一參數統計電路進行直方圖統計分析,再由一類型判斷電路將該原圖像分類為過暗型、過亮型或極端型其中之一; C.圖像暗部補強處理:再依該類型判斷電路判斷類型不同,將該過暗型與該極端型之圖像輸送往一暗部補強電路,以進行暗部補強作業,以獲得暗部補強圖像後,再輸入一混合電路,若為過亮型圖像則直接輸入該混合電路;D.圖像反相處理:另由色彩空間轉換模組轉換之該原圖像資料係同時輸送往一反相處理模組,以經該反相處理模組進行反相處理,獲得反相圖像,以輸入該混合電路;E.圖像混合處理:續由該混合電路將輸入之反相圖像及暗部補強圖像進行混合,以獲得混合圖像;F.圖像亮部提升處理:復由一亮部提升電路將該混合圖像進行亮部提升作業後,獲得高動態範圍圖像。A camera dynamic range expansion method, the implementation steps include the following: A. Image color space conversion: the image taken by the camera is input into a color space conversion module to convert the color space of the three primary colors of red, green and blue. Color space of brightness and chromatic aberration; B. Image type classification: following the brightness value data of the original image is sent to a parameter statistical circuit for histogram statistical analysis, and then the original image is classified by a type judgment circuit as One of too dark, over bright or extreme; C. Image dark portion reinforcement processing: according to the type judgment circuit, the type of judgment is different, and the image of the over-dark type and the extreme type is sent to a dark portion reinforcing circuit to perform the dark portion reinforcing operation to obtain the dark portion reinforcing image. , then input a hybrid circuit, if it is a super bright image, directly input the hybrid circuit; D. Image inversion processing: the original image data converted by the color space conversion module is simultaneously sent to an inversion Processing the module, performing inversion processing through the inversion processing module to obtain an inverted image to input the hybrid circuit; E. Image mixing processing: continuing to input the inverted image and the dark portion by the hybrid circuit The reinforced image is mixed to obtain a mixed image; F. The image bright portion lifting process is performed by a bright portion lifting circuit to perform a bright portion lifting operation to obtain a high dynamic range image. 如申請專利範圍第4項所述攝影機動態範圍擴展方法,其中,該攝影機動態範圍擴展方法係進一步包含有自動增益控制電路,以與該亮部提升電路相耦接,係使經亮部提升電獲得之高動態範圍圖像,再輸送往自動增益控制電路,進行亮度之對比調整後輸出。The camera dynamic range expansion method according to claim 4, wherein the camera dynamic range expansion method further comprises an automatic gain control circuit coupled to the bright portion lifting circuit to increase the brightness of the bright portion. The obtained high dynamic range image is sent to the automatic gain control circuit for contrast adjustment of the brightness and output. 如申請專利範圍第4項所述攝影機動態範圍擴展方法,其中,該圖像類型分類步驟中之過暗型、過亮型及極端型係以Hi及Lo兩個參數值為分類基準,該Hi及Lo兩參數值公式如下: 其中,該h(i)代表直方圖灰階度i的統計像素量, Th1~Th4為四個自訂閥值,分別落在直方圖灰階度由小到大的四個位置,當Lo >2Hi 時,係為過暗型,又當Hi >2Lo 時, 則為過亮型,另當時,即為極端型, 該P total 為畫面總像素點數。The camera dynamic range expansion method according to claim 4, wherein the over-dark, over-bright, and extreme types in the image type classification step are based on two parameter values of Hi and Lo, and the Hi And Lo two parameter value formula is as follows: Where h(i) represents the statistical pixel quantity of the histogram gray scale i, and Th1~Th4 are four custom thresholds, respectively falling in four positions of the histogram gray scale from small to large, when Lo > 2 Hi , the system is too dark, and when Hi > 2 Lo , it is too bright, another or When it is extreme, the P total is the total number of pixels of the picture. 如申請專利範圍第6項所述攝影機動態範圍擴展方法,其中,該類型判斷電路內係進一步包含相耦接之第一多工器、第二多工器及第三多工器,當類型判斷電路判斷Lo >2Hi 時,係透過對Hi左移1位元後得到2Hi,再將之與Lo做比較,若符合Lo >2Hi 則第一多工器選擇1,其序號〔type〕係輸出00代表過暗型,若不符合Lo >2Hi 則第一多工器選擇0,此時,輸出為第二多工器結果,該第二多工器係透過判斷是否符合過亮型條件Hi >2Lo 來控制,若符合則第二多工器選擇1,其序號輸出為01代表過亮型,若不符合Hi >2Lo ,則第二多工器選擇0,此時,其輸出為第三多工器結果,該 第三多工器係透過判斷是否符合極端型條件來控制,以上兩條件任一條件若符合,則第三多工 器選擇1,此時,其序號輸出10代表極端型,如兩條件皆不符合則序號輸出11代表正常型。The camera dynamic range expansion method according to claim 6, wherein the type judging circuit further comprises a first multiplexer, a second multiplexer and a third multiplexer coupled to each other, when the type is judged When the circuit judges Lo > 2 Hi , it obtains 2Hi by shifting Hi by 1 bit, and then compares it with Lo. If Lo > 2 Hi is met, the first multiplexer selects 1, and its serial number is [type]. The output 00 represents an over-dark type. If Lo > 2 Hi is not met, the first multiplexer selects 0. At this time, the output is the second multiplexer result, and the second multiplexer determines whether the over-elastic condition is met. Hi > 2 Lo to control, if it is met, the second multiplexer selects 1, the serial number output is 01 for over-bright type, if it does not meet Hi > 2 Lo , the second multiplexer selects 0, at this time, its output For the third multiplexer result, the third multiplexer determines whether it meets the extreme condition or To control, if any of the above two conditions are met, the third multiplexer selects 1. At this time, the serial number output 10 represents the extreme type. If both conditions are not met, the serial number output 11 represents the normal type. 如申請專利範圍第4項所述攝影機動態範圍擴展方法,其中,該圖像混合處理步驟中該反相圖像及暗部補強圖像係進行相乘運算,其相乘運算公式如下:y mix (i ,j )=y dark (i ,jy inv (i ,jk 其中,該y mix 係為混合運算後輸出值,y dark 為暗部補強圖像值,y inv 為反相圖像值,k值為輸出高動態範圍圖像之關鍵參數。The camera dynamic range expansion method according to claim 4, wherein the image mixing processing step performs the multiplication operation on the inverted image and the dark portion image, and the multiplication formula is as follows: y mix ( i , j )= y dark ( i , jy inv ( i , jk where y mix is the output value after mixing operation, y dark is the dark portion to reinforce the image value, and y inv is the inverse image The image value, k value is the key parameter for outputting a high dynamic range image. 如申請專利範圍第4項所述攝影機動態範圍擴展方法,其中,該圖像類型分類步驟中係進一步包含一非線性曲線方程式之運算,該非線性曲線方程式之公式如下: 其中,y inv 為反相值、y lpf 為原圖像資料輸入值。The camera dynamic range expansion method according to claim 4, wherein the image type classification step further comprises an operation of a nonlinear curve equation, and the formula of the nonlinear curve equation is as follows: Where y inv is the inverse value and y lpf is the input value of the original image data. 如申請專利範圍第4項所述攝影機動態範圍擴展方法,其中,該圖像類型分類步驟中係進一步包含一補償方程式之運算,該補償方程式之公式如下: 其中,y dark 為暗部補強圖像輸出值,y in 為輸入圖像灰階值,Th dark 為自訂之閥值。The camera dynamic range expansion method according to claim 4, wherein the image type classification step further comprises an operation of compensating an equation, and the formula of the compensation equation is as follows: Where y dark is the dark portion to reinforce the image output value, y in is the input image grayscale value, and Th dark is the custom threshold. 如申請專利範圍第4項所述攝影機動態範圍擴展方法,其中,該圖像亮部提升處理步驟之亮部提升運算公式如下: 其中,y mix 為混合圖像值,y dark 為暗部補強圖像值,Th b 為閥值,另y HDR 為高動態範圍圖像值。The camera dynamic range expansion method according to claim 4, wherein the bright portion lifting operation formula of the image bright portion lifting processing step is as follows: Where y mix is the mixed image value, y dark is the dark portion to reinforce the image value, Th b is the threshold value, and y HDR is the high dynamic range image value. 如申請專利範圍第11項所述攝影機動態範圍擴展方法,其中,該暗部補強電路係進一步包含相耦接之多工器、暫存 器及減法器,當暗部補強電路之多工器接收到由該類型判斷電路輸出之一過暗型序號時,多工器係選擇128作為閥值輸出,當暗部補強電路之多工器接收到由該類型判斷電路輸出之一極端型序號時,多工器係選擇50作為閥值輸出,其輸出值係被存進暫存器,接著由暫存器將該閥值送進減法器與一由該類型判斷電路輸入之圖像亮度值資料相減,並將相減結果右移4位元後與該輸入之圖像亮度值資料相加得到補強運算結果。The camera dynamic range expansion method according to claim 11, wherein the dark portion reinforcing circuit further comprises a multiplexer coupled to the temporary storage And the subtractor, when the multiplexer of the dark portion reinforcing circuit receives an over-dark serial number outputted by the type judging circuit, the multiplexer selects 128 as the threshold value output, when the multiplexer of the dark portion reinforcing circuit receives When the type judging circuit outputs an extreme type number, the multiplexer selects 50 as the threshold value output, and the output value is stored in the register, and then the threshold is sent to the subtractor by the register. The image brightness value data input by the type judging circuit is subtracted, and the subtraction result is shifted right by 4 bits and added to the input image brightness value data to obtain a reinforcing operation result. 如申請專利範圍第11項所述攝影機動態範圍擴展方法,其中,該亮部提升電路係包含相耦接之一多工器、一比較器、一減法器、一乘法器及兩加法器,乃使減法器執行一由暗部補強電路輸入之暗部補強值〔dark_enh〕減50動作,乘法器將減法器的結果與自身相乘,以得到暗部補強平方值〔dark_enh_mul〕,再使用右移8位元與右移10位元相加動作,以達成乘以0.005的值,之後將此值加在混合圖像之輸出值〔out_mix〕上,復待後方多工器〔MUX〕選擇輸出。The camera dynamic range expansion method according to claim 11, wherein the bright portion lifting circuit comprises a multiplexer coupled to the multiplexer, a comparator, a subtractor, a multiplier and two adders, The subtractor performs a dark portion enhancement value [dark_enh] input by the dark portion reinforcement circuit minus 50, and the multiplier multiplies the result of the subtractor with itself to obtain a dark portion enhancement square value [dark_enh_mul], and then uses the right shift 8-bit element. Adding a 10-bit right shift to achieve a value multiplied by 0.005, then adding this value to the output value of the mixed image [out_mix], and returning the rear multiplexer [MUX] to select the output.
TW102125130A 2013-07-12 2013-07-12 Device and method for expanding dynamic range of camera TWI513310B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
TW102125130A TWI513310B (en) 2013-07-12 2013-07-12 Device and method for expanding dynamic range of camera

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
TW102125130A TWI513310B (en) 2013-07-12 2013-07-12 Device and method for expanding dynamic range of camera

Publications (2)

Publication Number Publication Date
TW201503699A TW201503699A (en) 2015-01-16
TWI513310B true TWI513310B (en) 2015-12-11

Family

ID=52718581

Family Applications (1)

Application Number Title Priority Date Filing Date
TW102125130A TWI513310B (en) 2013-07-12 2013-07-12 Device and method for expanding dynamic range of camera

Country Status (1)

Country Link
TW (1) TWI513310B (en)

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020027606A1 (en) * 1998-10-19 2002-03-07 Orly Yadid-Pecht Optical imager using a method for adaptive real-time expanding of the dynamic range
US20050243177A1 (en) * 2003-04-29 2005-11-03 Microsoft Corporation System and process for generating high dynamic range video
US20070146502A1 (en) * 2005-12-23 2007-06-28 Magnachip Semiconductor Ltd Image sensor and method for controlling image brightness distribution therein
US20080136953A1 (en) * 2004-12-07 2008-06-12 Bright Imaging Ltd. Method And Apparatus For Imaging Of Scenes Having Large Intensity Variance
US20090141151A1 (en) * 2007-11-29 2009-06-04 Samsung Electronics Co., Ltd. Dynamic range enhancement method and apparatus
TW201033939A (en) * 2009-03-13 2010-09-16 Micro Star Int Co Ltd Dark part exposure compensation method using single image to simulate high dynamic range and image processing apparatus thereof
TW201223293A (en) * 2010-08-25 2012-06-01 Dolby Lab Licensing Corp Extending image dynamic range
US20120162463A1 (en) * 2007-01-23 2012-06-28 Nikon Corporation Image processing device, electronic camera, image processing method, and image processing program
TW201233159A (en) * 2010-09-30 2012-08-01 Apple Inc High dynamic range transition

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020027606A1 (en) * 1998-10-19 2002-03-07 Orly Yadid-Pecht Optical imager using a method for adaptive real-time expanding of the dynamic range
US20050243177A1 (en) * 2003-04-29 2005-11-03 Microsoft Corporation System and process for generating high dynamic range video
US20080136953A1 (en) * 2004-12-07 2008-06-12 Bright Imaging Ltd. Method And Apparatus For Imaging Of Scenes Having Large Intensity Variance
US20070146502A1 (en) * 2005-12-23 2007-06-28 Magnachip Semiconductor Ltd Image sensor and method for controlling image brightness distribution therein
US20120162463A1 (en) * 2007-01-23 2012-06-28 Nikon Corporation Image processing device, electronic camera, image processing method, and image processing program
US20090141151A1 (en) * 2007-11-29 2009-06-04 Samsung Electronics Co., Ltd. Dynamic range enhancement method and apparatus
TW201033939A (en) * 2009-03-13 2010-09-16 Micro Star Int Co Ltd Dark part exposure compensation method using single image to simulate high dynamic range and image processing apparatus thereof
TW201223293A (en) * 2010-08-25 2012-06-01 Dolby Lab Licensing Corp Extending image dynamic range
TW201233159A (en) * 2010-09-30 2012-08-01 Apple Inc High dynamic range transition

Also Published As

Publication number Publication date
TW201503699A (en) 2015-01-16

Similar Documents

Publication Publication Date Title
US10021313B1 (en) Image adjustment techniques for multiple-frame images
CN102801918B (en) Picture pick-up device and the method for controlling this picture pick-up device
TWI416940B (en) Image processing apparatus and image processing program
KR101388130B1 (en) Image processing apparatus and image processing method
US9426381B1 (en) Method and/or apparatus for implementing high dynamic range image processing in a video processing system
US20120127336A1 (en) Imaging apparatus, imaging method and computer program
JP2007173933A (en) Digital camera, gain calculation apparatus, and gain calculation method
CN101304482A (en) Method and apparatus for reducing flicker of image sensor
JP2011003048A (en) Image processing apparatus and image processing program
JP2014153959A (en) Image processing device, image processing method, program, and storage medium
JP2013121132A (en) Imaging apparatus and imaging method
JP5804857B2 (en) Image processing apparatus, image processing method, and program
JP5238931B2 (en) HDR image generation method and digital imaging device using the method
JP6611543B2 (en) Image processing apparatus, image processing method, and program
JP2016134862A (en) Imaging device, imaging method, image display device, and program
JP5854716B2 (en) Image processing apparatus, image processing method, and program
JP2015139082A (en) Image processor, image processing method, program and electronic apparatus
JP5591026B2 (en) Imaging apparatus and control method thereof
TWI513310B (en) Device and method for expanding dynamic range of camera
JP2006333113A (en) Imaging device
JP2012134651A (en) Image processing device, method, and program
JP2003223636A (en) Image processor
JP6200687B2 (en) Image processing apparatus and image processing method
JP2007235421A (en) Imaging apparatus
JP2004222160A (en) Digital camera