TW202127857A - White balance adjustment method, image processing device and image processing system - Google Patents

White balance adjustment method, image processing device and image processing system Download PDF

Info

Publication number
TW202127857A
TW202127857A TW109100807A TW109100807A TW202127857A TW 202127857 A TW202127857 A TW 202127857A TW 109100807 A TW109100807 A TW 109100807A TW 109100807 A TW109100807 A TW 109100807A TW 202127857 A TW202127857 A TW 202127857A
Authority
TW
Taiwan
Prior art keywords
image
light source
source information
image areas
areas
Prior art date
Application number
TW109100807A
Other languages
Chinese (zh)
Other versions
TWI723729B (en
Inventor
黃宜瑾
利建宏
許銀雄
Original Assignee
宏碁股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 宏碁股份有限公司 filed Critical 宏碁股份有限公司
Priority to TW109100807A priority Critical patent/TWI723729B/en
Application granted granted Critical
Publication of TWI723729B publication Critical patent/TWI723729B/en
Publication of TW202127857A publication Critical patent/TW202127857A/en

Links

Images

Landscapes

  • Image Processing (AREA)
  • Color Television Image Signal Generators (AREA)
  • Processing Of Color Television Signals (AREA)

Abstract

A white balance adjustment method, an image processing device and an image processing system are provided. A raw image is received. The raw image is segmented into multiple first image blocks according to brightness information of the raw image. A plurality of local illuminant information respectively corresponding to the first image blocks is generated for each first image block. The raw image is segmented into multiple second image blocks, and mixed illuminant information respectively corresponding to the second image blocks is generated according to at least one of the local illuminant information corresponded to the second image blocks. Pixels values within the second image blocks are respectively adjusted according to the mixed illuminant information of the second image blocks to obtain a white balance image.

Description

白平衡調整方法、影像處理裝置與影像處理系統White balance adjustment method, image processing device and image processing system

本發明是有關於一種影像處理方法,且特別是有關於一種白平衡調整方法、影像處理裝置與影像處理系統。The present invention relates to an image processing method, and more particularly to a white balance adjustment method, an image processing device and an image processing system.

隨著科技的發展,各式各樣的智慧型影像擷取裝置,舉凡平板型電腦、個人數位化助理、及智慧型手機等,已成為現代人不可或缺的工具。其中,高階款的智慧型影像擷取裝置所搭載的相機鏡頭已經與傳統消費型相機不相上下,甚至可以取而代之,少數高階款更具有接近數位單眼的畫素和畫質或者是提供更為進階的功能和效果。With the development of technology, various smart image capture devices, such as tablet computers, personal digital assistants, and smart phones, have become indispensable tools for modern people. Among them, the high-end smart image capture device is equipped with camera lenses that are comparable to traditional consumer cameras, and can even be replaced. A few high-end models have more pixel and image quality close to digital monocular or provide more advanced The functions and effects of the order.

拍攝者可能使用影像擷取裝置在不同的地點或時間進行拍攝,因而影像擷取裝置將於不同的光源環境下拍攝影像。拍攝環境裡的光源將直接影響擷取影像中被拍攝目標的呈現顏色。於一應用情境中,傷者可能對傷口或患部拍攝影像並將拍攝影像提供給醫療單位進行診斷,倘若影像中的色彩資訊反應於拍攝環境的光源而發生失真,則將可能無法依據拍攝影像進行正確診斷。對此,一般影像擷取裝置所採用的自動白平衡(auto white balance,AWB)演算法主要是在影像感測器擷取到場景的影像後,利用場景的灰階內容來進行白平衡調整,以穩定地顯示場景的彩色內容。然而,於光源複雜的環境中,由於同一被攝目標上的不同位置可能被不同的混合光源照射,因此要十分精確地還原被攝目標的色彩資訊是相對不容易的。舉例而言,輔助光源(例如閃光燈)的使用將使得白平衡調整後的影像還是可能存在局部或整體色偏的現象。因此,如何藉由更佳的白平衡調整方法來避免影像色偏,實乃本領域技術人員所努力的方向之一。The photographer may use the image capturing device to shoot at different places or times, so the image capturing device will shoot images under different light source environments. The light source in the shooting environment will directly affect the color of the target in the captured image. In an application scenario, the injured may take images of the wound or the affected area and provide the taken images to the medical unit for diagnosis. If the color information in the image is distorted due to the light source in the shooting environment, it may not be correct based on the captured image. diagnosis. In this regard, the auto white balance (AWB) algorithm used by general image capture devices mainly uses the grayscale content of the scene to adjust the white balance after the image sensor captures the image of the scene. To stably display the color content of the scene. However, in an environment with complex light sources, since different positions on the same subject may be illuminated by different mixed light sources, it is relatively difficult to restore the color information of the subject very accurately. For example, the use of an auxiliary light source (such as a flash) will cause the image after the white balance adjustment to still have local or overall color shift. Therefore, how to avoid image color cast by a better white balance adjustment method is actually one of the directions that those skilled in the art are striving for.

有鑑於此,本發明提出一種白平衡調整方法、影像處理裝置以及影像處理系統,其可根據影像中不同的影像區域來適應性地調整影像的白平衡,進而達到高品質的影像輸出。In view of this, the present invention provides a white balance adjustment method, an image processing device, and an image processing system, which can adaptively adjust the white balance of the image according to different image regions in the image, thereby achieving high-quality image output.

本發明實施例提供一種白平衡調整方法。所述方法包括下列步驟。接收一原始影像。依據原始影像的亮度資訊,將原始影像劃分為多個第一影像區域。針對各第一影像區域,產生分別對應至第一影像區域的多個區域光源資訊。將原始影像劃分為多個第二影像區域,並依據第二影像區域所對應的區域光源資訊其中至少一產生分別對應至第二影像區域的多個混合光源資訊。依據第二影像區域的混合光源資訊分別修正第二影像區域內的多個像素值而獲取白平衡影像。The embodiment of the present invention provides a white balance adjustment method. The method includes the following steps. Receive an original image. According to the brightness information of the original image, the original image is divided into a plurality of first image regions. For each first image area, multiple area light source information corresponding to the first image area is generated. The original image is divided into a plurality of second image areas, and a plurality of mixed light source information respectively corresponding to the second image area is generated according to at least one of the area light source information corresponding to the second image area. According to the mixed light source information of the second image area, a plurality of pixel values in the second image area are respectively corrected to obtain a white balance image.

本發明實施例提供一種影像處理裝置,其包括儲存裝置以及處理器。儲存裝置儲存有多個模組。處理器耦接儲存裝置,經配置而執行所述模組以執行下列步驟。接收一原始影像。依據原始影像的亮度資訊,將原始影像劃分為多個第一影像區域。針對各第一影像區域,產生分別對應至第一影像區域的多個區域光源資訊。將原始影像劃分為多個第二影像區域,並依據第二影像區域所對應的區域光源資訊其中至少一產生分別對應至第二影像區域的多個混合光源資訊。依據第二影像區域的混合光源資訊分別修正第二影像區域內的多個像素值而獲取白平衡影像。An embodiment of the present invention provides an image processing device, which includes a storage device and a processor. The storage device stores multiple modules. The processor is coupled to the storage device and is configured to execute the module to perform the following steps. Receive an original image. According to the brightness information of the original image, the original image is divided into a plurality of first image regions. For each first image area, multiple area light source information corresponding to the first image area is generated. The original image is divided into a plurality of second image areas, and a plurality of mixed light source information respectively corresponding to the second image area is generated according to at least one of the area light source information corresponding to the second image area. According to the mixed light source information of the second image area, a plurality of pixel values in the second image area are respectively corrected to obtain a white balance image.

本發明實施例提供一種影像處理系統,其包括影像感測器、儲存裝置以及處理器。儲存裝置儲存有多個模組。處理器耦接影像感測器與儲存裝置,經配置而執行所述模組以執行下列步驟。接收一原始影像。依據原始影像的亮度資訊,將原始影像劃分為多個第一影像區域。針對各第一影像區域,產生分別對應至第一影像區域的多個區域光源資訊。將原始影像劃分為多個第二影像區域,並依據第二影像區域所對應的區域光源資訊其中至少一產生分別對應至第二影像區域的多個混合光源資訊。依據第二影像區域的混合光源資訊分別修正第二影像區域內的多個像素值而獲取白平衡影像。An embodiment of the present invention provides an image processing system, which includes an image sensor, a storage device, and a processor. The storage device stores multiple modules. The processor is coupled to the image sensor and the storage device, and is configured to execute the module to perform the following steps. Receive an original image. According to the brightness information of the original image, the original image is divided into a plurality of first image regions. For each first image area, multiple area light source information corresponding to the first image area is generated. The original image is divided into a plurality of second image areas, and a plurality of mixed light source information respectively corresponding to the second image area is generated according to at least one of the area light source information corresponding to the second image area. According to the mixed light source information of the second image area, a plurality of pixel values in the second image area are respectively corrected to obtain a white balance image.

基於上述,於本發明的實施例中,原始影像可分割為多個影像區域而依據不同的混合光源資訊來進行白平衡校正,且這些混合光源資訊也是對不同影像區域進行估測而產生。藉此,本發明實施例可以有效地校正原始影像中反應於多種環境光源而起的色偏現象,以執行更精準的影像白平衡校正。Based on the above, in the embodiment of the present invention, the original image can be divided into a plurality of image areas to perform white balance correction based on different mixed light source information, and the mixed light source information is also generated by estimating different image areas. In this way, the embodiment of the present invention can effectively correct the color shift phenomenon in the original image that is reflected in a variety of ambient light sources, so as to perform more accurate image white balance correction.

為讓本發明之上述特徵和優點能更明顯易懂,下文特舉實施例,並配合所附圖式作詳細說明如下。In order to make the above-mentioned features and advantages of the present invention more comprehensible, the following specific embodiments are described in detail in conjunction with the accompanying drawings.

本發明的部份實施例接下來將會配合附圖來詳細描述,以下的描述所引用的元件符號,當不同附圖出現相同的元件符號將視為相同或相似的元件。這些實施例只是本發明的一部份,並未揭示所有本發明的可實施方式。更確切的說,這些實施例只是本發明的專利申請範圍中的方法、裝置與系統的範例。Part of the embodiments of the present invention will be described in detail in conjunction with the accompanying drawings. The reference symbols in the following description will be regarded as the same or similar elements when the same symbol appears in different drawings. These embodiments are only a part of the present invention, and do not disclose all the possible implementation modes of the present invention. To be more precise, these embodiments are just examples of methods, devices, and systems within the scope of the patent application of the present invention.

圖1是根據本發明一實施例所繪示的影像處理系統的方塊圖,但此僅是為了方便說明,並不用以限制本發明。首先圖1先介紹影像處理系統之所有構件以及配置關係,詳細功能將配合圖2一併揭露。FIG. 1 is a block diagram of an image processing system according to an embodiment of the present invention, but this is only for convenience of description, and is not intended to limit the present invention. First, Figure 1 first introduces all the components and configuration relationships of the image processing system. Detailed functions will be disclosed in conjunction with Figure 2.

請參照圖1,影像處理系統100包括影像感測器110以及影像處理裝置120,其中影像處理裝置120包括儲存裝置122以及處理器124。在本實施例中,影像處理系統100可以是將影像感測器110以及影像處理裝置120整合為單一裝置(all-in-one)的影像擷取裝置,例如是具有鏡頭的數位相機、單眼相機、數位攝影機、智慧型手機、平板電腦等等。在另一實施例中,影像處理裝置120可以是個人電腦、筆記型電腦、智慧型手機、平板電腦等具有影像處理功能的電子裝置,並且經由通訊介面(未繪示)以有線或無線的方式接收影像感測器110所拍攝到的影像。1, the image processing system 100 includes an image sensor 110 and an image processing device 120, where the image processing device 120 includes a storage device 122 and a processor 124. In this embodiment, the image processing system 100 may be an image capturing device that integrates the image sensor 110 and the image processing device 120 into a single device (all-in-one), such as a digital camera with a lens or a single-lens camera. , Digital cameras, smart phones, tablets, etc. In another embodiment, the image processing device 120 may be an electronic device with image processing functions such as a personal computer, a notebook computer, a smart phone, a tablet computer, etc., and may be wired or wireless via a communication interface (not shown) The image captured by the image sensor 110 is received.

在本實施例中,影像感測器110為包括透鏡以及感光元件的攝像鏡頭。感光元件用以感測進入透鏡的光線強度,進而產生影像。感光元件可以例如是電荷耦合元件(charge coupled device,CCD)、互補性氧化金屬半導體(complementary metal-oxide semiconductor,CMOS)元件或其他元件。鏡頭所擷取到的影像將成像於感測元件並且轉換成數位訊號,以輸出至處理器124。In this embodiment, the image sensor 110 is an imaging lens including a lens and a photosensitive element. The photosensitive element is used to sense the intensity of light entering the lens to generate an image. The photosensitive element may be, for example, a charge coupled device (CCD), a complementary metal-oxide semiconductor (CMOS) element, or other elements. The image captured by the lens will be imaged on the sensing element and converted into a digital signal for output to the processor 124.

儲存裝置122用以儲存影像、程式碼等資料,其可以例如是任意型式的固定式或可移動式隨機存取記憶體(random access memory,RAM)、唯讀記憶體(read-only memory,ROM)、快閃記憶體(flash memory)、硬碟或其他類似裝置、積體電路及其組合。The storage device 122 is used to store images, program codes and other data, and it can be, for example, any type of fixed or removable random access memory (RAM), read-only memory (ROM). ), flash memory, hard disk or other similar devices, integrated circuits and combinations thereof.

處理器124用以控制影像處理系統100的構件之間的作動,其可以例如是中央處理單元(central processing unit,CPU)、圖形處理單元(graphic processing unit,GPU),或是其他可程式化之一般用途或特殊用途的微處理器(microprocessor)、數位訊號處理器(digital signal processor,DSP)、影像訊號處理器(image signal processor,ISP)、可程式化控制器、特殊應用積體電路(application specific integrated circuits,ASIC)、可程式化邏輯裝置(programmable logic device,PLD)或其他類似裝置或這些裝置的組合。The processor 124 is used to control the actions between the components of the image processing system 100, and it can be, for example, a central processing unit (CPU), a graphics processing unit (GPU), or other programmable ones. General purpose or special purpose microprocessor (microprocessor), digital signal processor (digital signal processor, DSP), image signal processor (image signal processor, ISP), programmable controller, special application integrated circuit (application specific integrated circuits (ASIC), programmable logic device (PLD) or other similar devices or a combination of these devices.

以下即列舉實施例說明針對影像處理系統100調整白平衡的方法的詳細步驟。在以下的實施例中將以影像處理系統100實作成影像擷取裝置來進行說明,而處理器124可以影像訊號處理器來實現,其作用是針對前端的影像感測器110的輸出訊號進行處理,以在不同條件下還原出場景的細節。The following is an example to illustrate the detailed steps of the method for adjusting the white balance of the image processing system 100. In the following embodiments, the image processing system 100 will be implemented as an image capture device for illustration, and the processor 124 can be implemented by an image signal processor, and its role is to process the output signal of the front-end image sensor 110 , In order to restore the details of the scene under different conditions.

圖2為根據本發明之一實施例所繪示的白平衡調整方法的流程圖。本實施例的方法適用於圖1中的影像處理系統100,以下即搭配影像處理系統100中的各項元件說明本實施例方法的詳細流程。FIG. 2 is a flowchart of a method for adjusting white balance according to an embodiment of the present invention. The method of this embodiment is applicable to the image processing system 100 in FIG.

請同時參照圖1以及圖2,首先,於步驟S201,影像處理裝置120的處理器124接收一原始影像。在本實施例中,由於處理器124則是以影像訊號處理器來實現,因此第一影像可以是處理器124即時地自影像感測器110接收其所擷取到的影像序列中的其中一張影像。然而,在另一實施例中,第一影像可以是處理器124直接或是間接透過其它裝置而取得到外部的影像感測器110所擷取到的影像序列中的其中一張影像。Please refer to FIG. 1 and FIG. 2 at the same time. First, in step S201, the processor 124 of the image processing device 120 receives an original image. In this embodiment, since the processor 124 is implemented as an image signal processor, the first image can be one of the image sequences captured by the processor 124 that it receives from the image sensor 110 in real time. Images. However, in another embodiment, the first image may be one of the images in the image sequence captured by the external image sensor 110 obtained by the processor 124 directly or indirectly through other devices.

於步驟S202,處理器124依據原始影像的亮度資訊,將原始影像劃分為多個第一影像區域。詳細而言,原始影像是由陣列排列的多個像素組成,每一像素的像素值包括對應至多個色彩空間分量的多個像素分量。一般而言,影像感測器110所產生的像素分量分別為對應至紅色通道的R分量、對應至綠色通道的G分量以及對應至藍色通道的B分量。處理器124將各像素的RGB分量轉換為各像素的亮度值,此亮度值例如是YCbCr色彩空間裡的亮度分量(即Y分量)、YUV色彩空間裡的亮度分量(即Y分量),或是HSV色彩空間裡的亮度分量(即V分量),本發明對此不限制。In step S202, the processor 124 divides the original image into a plurality of first image regions according to the brightness information of the original image. In detail, the original image is composed of a plurality of pixels arranged in an array, and the pixel value of each pixel includes a plurality of pixel components corresponding to a plurality of color space components. Generally speaking, the pixel components generated by the image sensor 110 are the R component corresponding to the red channel, the G component corresponding to the green channel, and the B component corresponding to the blue channel. The processor 124 converts the RGB components of each pixel into the brightness value of each pixel. The brightness value is, for example, the brightness component in the YCbCr color space (ie Y component), the brightness component in the YUV color space (ie Y component), or The brightness component (that is, the V component) in the HSV color space is not limited by the present invention.

於一實施例中,依據原始影像中這些像素的亮度值是否大於一臨界值,處理器124可將原始影像中的像素劃分為多個第一影像區域。詳細而言,處理器124可逐一判斷這些像素的亮度值是否大於一臨界值,並將彼此連通且亮度值位於同一亮度區間的像素劃分為一個第一影像區域。換言之,多個彼此連通且亮度值皆大於臨界值的像素將被劃分至相同的第一影像區域。同理,多個彼此連通且亮度值皆不大於臨界值的像素也將被劃分至相同的第一影像區域。換言之,原始影像中的這些第一影像區域可分成兩類別,一個類別是亮度值大於臨界值的第一影像區域,另一個類別是亮度值不大於臨界值的第一影像區域。並且,同一第一影像區域裡的像素具有連通性。舉例而言,原始影像可能包括亮度值大於臨界值的2個第一影像區域以及亮度值不大於臨界值的3個第一影像區域。In one embodiment, the processor 124 may divide the pixels in the original image into a plurality of first image regions according to whether the brightness values of the pixels in the original image are greater than a threshold value. In detail, the processor 124 may determine whether the brightness values of these pixels are greater than a critical value one by one, and divide the connected pixels with the brightness values in the same brightness interval into a first image area. In other words, multiple connected pixels with brightness values greater than the threshold will be divided into the same first image area. In the same way, a plurality of pixels that are connected to each other and whose brightness value is not greater than the critical value will also be divided into the same first image area. In other words, the first image areas in the original image can be divided into two categories, one category is the first image area with a brightness value greater than the threshold, and the other category is the first image area with the brightness value not greater than the threshold. Moreover, the pixels in the same first image area have connectivity. For example, the original image may include two first image regions with a brightness value greater than a threshold value and three first image regions with a brightness value not greater than the threshold value.

需說明的是,於一實施例中,用以劃分第一影像區域的臨界值可依據原始影像的整體亮度資訊來決定。例如,處理器124可先找出原始影像中的最大亮度值與最小亮度值,並取最大亮度值與最小亮度值的平均值作為劃分第一影像區域的臨界值。此外,於一實施例中,用以劃分第一影像區域的臨界值的數目可以是一個以上。It should be noted that, in one embodiment, the threshold for dividing the first image area can be determined based on the overall brightness information of the original image. For example, the processor 124 may first find the maximum brightness value and the minimum brightness value in the original image, and take the average value of the maximum brightness value and the minimum brightness value as the critical value for dividing the first image area. In addition, in one embodiment, the number of thresholds used to divide the first image area may be more than one.

舉例而言,圖3是依照本發明一實施例的將原始影像劃分為多個第一影像區域的示意圖。請參照圖3,為了方便說明,圖3以原始影像Img-raw包括8*6的像素為例進行說明,但本領域技術人員可將相同的步驟與方法實施於不同影像尺寸的原始影像上。於圖3中,各像素位置上的數值代表中各像素的亮度值。處理器124可依據原始影像Img-raw中各像素的亮度值獲取最大亮度值‘200’以及最小亮度值‘30’,並計算出亮度臨界值為‘115’。接著,處理器124逐一判斷各像素的亮度值是否大於亮度臨界值為‘115’。像是,像素P11的亮度值‘30’不大於亮度臨界值為‘115’,但像素P26的亮度值‘145’大於亮度臨界值為‘115’。基此,處理器124可將原始影像Img-raw劃分為兩個第一影像區域Z1與Z2。於圖3的範例中,第一影像區域Z1包括22個像素(例如像素P11),而第一影像區域Z2包括26個像素(例如像素P26)。For example, FIG. 3 is a schematic diagram of dividing an original image into a plurality of first image regions according to an embodiment of the invention. Please refer to FIG. 3. For the convenience of description, FIG. 3 takes the original image Img-raw including 8*6 pixels as an example for description. However, those skilled in the art can implement the same steps and methods on original images of different image sizes. In FIG. 3, the numerical value at each pixel position represents the brightness value of each pixel in. The processor 124 can obtain the maximum brightness value of "200" and the minimum brightness value of "30" according to the brightness value of each pixel in the original image Img-raw, and calculate the brightness threshold value of "115". Then, the processor 124 judges one by one whether the brightness value of each pixel is greater than the brightness threshold value '115'. For example, the brightness value '30' of the pixel P11 is not greater than the brightness threshold value '115', but the brightness value of the pixel P26 is greater than the brightness threshold value '115'. Based on this, the processor 124 can divide the original image Img-raw into two first image regions Z1 and Z2. In the example of FIG. 3, the first image area Z1 includes 22 pixels (for example, the pixel P11), and the first image area Z2 includes 26 pixels (for example, the pixel P26).

回到圖2的流程,於步驟S203,處理器124針對各第一影像區域,產生分別對應至第一影像區域的多個區域光源資訊。亦即,處理器124針對每一個第一影像區域獨立估測出對應的區域光源資訊。於一實施例中,區域光源資訊可包括色溫值。於一實施例中,區域光源資訊可包括對應至RGB通道的RGB成分比例(例如,RGB三色光的光強度比例)。舉例而言,某一第一影像區域的區域光源資訊可為對應至RGB通道的(0.5, 0.4, 0.2)。於一實施例中,處理器124可依據各個第一影像區域中的至少一子區域內的像素資訊產生第一影像區域的區域光源資訊。上述子區域可以是一個以上,且其尺寸可視實際應用而設置。Returning to the flow of FIG. 2, in step S203, the processor 124 generates a plurality of area light source information corresponding to the first image area for each first image area. That is, the processor 124 independently estimates the corresponding area light source information for each first image area. In one embodiment, the area light source information may include color temperature values. In one embodiment, the area light source information may include the ratio of RGB components corresponding to the RGB channels (for example, the light intensity ratio of the RGB three-color light). For example, the area light source information of a certain first image area may be (0.5, 0.4, 0.2) corresponding to the RGB channels. In one embodiment, the processor 124 may generate the area light source information of the first image area according to the pixel information in at least one sub-area of each first image area. There may be more than one sub-areas, and the size can be set according to actual applications.

於步驟S204,處理器124將原始影像劃分為多個第二影像區域,並依據第二影像區域所對應的區域光源資訊其中至少一產生分別對應至第二影像區域的多個混合光源資訊。詳細而言,假設原始影像包括(M*P)*(N*Q)個像素,處理器124將原始影像劃分為M*N個第二影像區域,且第二影像區域各自包括P*Q個像素。第二影像區域的尺寸與數目可視實際需求而設置,本發明對此不限制。由此可知,單一個第二影像區域可能包括一個以上的第一影像區域內的像素。若單一個第二影像區域包括相異的第一影像區域內的像素,第二影像區域將可對應至二筆以上的區域光源資訊。若單一個第二影像區域包括單一個第一影像區域內的像素,第二影像區域將對應至一筆區域光源資訊。針對各個第二影像區域,處理器124可依據第二影像區域各自對應的一筆或多筆區域光源資訊來產生混合光源資訊。當某一第二影像區域對應至多筆區域光源資訊(亦即涉及不同的環境光源),處理器124可對多筆區域光源資訊進行統計運算或加權處理而產生此第二影像區域的混合光源資訊。當某一第二影像區域只對應至一筆區域光源資訊(亦即涉及單一的環境光源),處理器124可直接將區域光源資訊作為此第二影像區域的混合光源資訊。In step S204, the processor 124 divides the original image into a plurality of second image regions, and generates a plurality of mixed light source information corresponding to the second image region according to at least one of the regional light source information corresponding to the second image region. In detail, assuming that the original image includes (M*P)*(N*Q) pixels, the processor 124 divides the original image into M*N second image regions, and the second image regions each include P*Q pixels. Pixels. The size and number of the second image area can be set according to actual requirements, and the present invention is not limited thereto. It can be seen that a single second image area may include more than one pixel in the first image area. If a single second image area includes pixels in different first image areas, the second image area can correspond to more than two pieces of area light source information. If a single second image area includes pixels in a single first image area, the second image area will correspond to a piece of area light source information. For each second image area, the processor 124 may generate mixed light source information according to one or more pieces of area light source information corresponding to each of the second image areas. When a certain second image area corresponds to multiple pieces of area light source information (that is, different ambient light sources are involved), the processor 124 may perform statistical calculations or weighting processing on the multiple pieces of area light source information to generate mixed light source information for the second image area . When a certain second image area only corresponds to a piece of area light source information (that is, a single ambient light source is involved), the processor 124 can directly use the area light source information as the mixed light source information of the second image area.

於步驟S205,處理器124依據第二影像區域的混合光源資訊分別修正第二影像區域內的多個像素值而獲取白平衡影像。於一實施例中,處理器124可直接依據混合光源資訊調整第二影像區域內的多個像素值而獲取白平衡影像。於一實施例中,處理器124可依據混合光源資訊決定白平衡增益參數(例如R通道增益參數、G通道增益參數、B通道增益參數),並依據白平衡增益參數調整第二影像區域內的多個像素值而獲取白平衡影像。更具體而言,某一第二影像區域內的像素的RGB分量可基於混合光源資訊進行調整,以達到白平衡調整的目的。藉此,在本實施例中,由於原始影像有考慮到不同光源而進行進行區域化的白平衡調整,而使得處理器124所產生並且輸出後的白平衡影像影像可達到更為精準的白平衡表現。In step S205, the processor 124 respectively corrects a plurality of pixel values in the second image area according to the mixed light source information of the second image area to obtain a white balance image. In one embodiment, the processor 124 may directly adjust a plurality of pixel values in the second image area according to the mixed light source information to obtain a white balance image. In one embodiment, the processor 124 may determine white balance gain parameters (such as R channel gain parameters, G channel gain parameters, and B channel gain parameters) according to the mixed light source information, and adjust the white balance gain parameters in the second image area according to the white balance gain parameters. Multiple pixel values to obtain a white balance image. More specifically, the RGB components of the pixels in a certain second image area can be adjusted based on the mixed light source information to achieve the purpose of white balance adjustment. Therefore, in this embodiment, since the original image is adjusted for the regionalized white balance in consideration of different light sources, the white balance image generated and output by the processor 124 can achieve a more accurate white balance. Performance.

為了更清楚明瞭說明,圖4是依照本發明一實施例的白平衡調整方法的流程圖。以下將以圖4來針對上述流程的一種實施細節加以說明。本實施例的方法適用於圖1中的影像處理系統100,以下即搭配影像處理系統100中的各項元件說明本實施例方法的詳細流程。For a clearer description, FIG. 4 is a flowchart of a white balance adjustment method according to an embodiment of the present invention. Hereinafter, FIG. 4 will be used to describe an implementation detail of the above process. The method of this embodiment is applicable to the image processing system 100 in FIG.

請同時參照圖1與圖4,於步驟S401,處理器124接收一原始影像。於步驟S402,處理器124依據原始影像的亮度資訊,將原始影像劃分為多個第一影像區域。接著,於本實施例中,處理器124將依據各第一影像區域中的至少一子區域產生第一影像區域的區域光源資訊。需說明的是,上述取子區域的方式將依據於第一影像區域的形狀是否為矩形而有所區別。1 and 4 at the same time, in step S401, the processor 124 receives an original image. In step S402, the processor 124 divides the original image into a plurality of first image regions according to the brightness information of the original image. Then, in this embodiment, the processor 124 will generate the area light source information of the first image area according to at least one sub-area in each first image area. It should be noted that the above method of taking the sub-regions will be different depending on whether the shape of the first image region is rectangular.

於步驟S403,處理器124判斷第一影像區域是否為矩形。於本實施例中,若第一影像區域其中之一為矩形(步驟S403判斷為是),於步驟S404,處理器124將第一影像區域其中之一劃分為子區域中的多個第二子區域,更依這些據第二子區域分別產生對應的多個參考光源資訊,並藉由統計這些參考光源資訊來產生第一影像區域其中之一的區域光源資訊其中之一。於此,這些第二子區域的數量多於1個。另一方面,若第一影像區域其中之一並非為矩形(步驟S403判斷為否),於步驟S405,處理器124自第一影像區域其中之一取出子區域中的第一子區域,並依據第一子區域產生第一影像區域其中之一的區域光源資訊其中之一。於此,此第一子區域的數量為1個。In step S403, the processor 124 determines whether the first image area is rectangular. In this embodiment, if one of the first image areas is rectangular (Yes in step S403), in step S404, the processor 124 divides one of the first image areas into a plurality of second sub-regions. The area further generates a plurality of corresponding reference light source information according to the second sub-areas, and generates one of the area light source information of one of the first image areas by counting the reference light source information. Here, the number of these second sub-regions is more than one. On the other hand, if one of the first image areas is not a rectangle (No in step S403), in step S405, the processor 124 extracts the first sub-area in the sub-area from one of the first image areas, and according to The first sub-region generates one of the regional light source information of one of the first image regions. Here, the number of the first sub-region is one.

於本實施例中,針對矩形的第一影像區域產生區域光源的步驟S404可實施為子步驟S4041~S4043。於步驟S4041,處理器124將第一影像區域劃分為多個第二子區域。這些第二子區域的尺寸可為w*h,可視實際應用而設置。換言之,處理器124可將矩形的第一影像區域切分為多個第二子區域。In this embodiment, the step S404 of generating an area light source for the rectangular first image area can be implemented as sub-steps S4041 to S4043. In step S4041, the processor 124 divides the first image area into a plurality of second sub-areas. The size of these second sub-regions can be w*h, which can be set according to actual applications. In other words, the processor 124 may divide the rectangular first image area into a plurality of second sub-areas.

於步驟S4042,處理器124利用一神經網路模型依據第二子區域分別預測多個參考光源資訊。詳細而言,處理器124逐一將這些第二子區域輸入至一個經訓練的神經網路模型,以分別預測出這些第二子區域各自的參考光源資訊。舉例而言,一個矩形的第一影像區域可切分為n個第二子區域,則處理器將利用經訓練的神經網路模型分別預測出n筆參考光源資訊。此神經網路模型可為依據訓練資料進行機器學習而事先建構的機器學習模型,其可儲存於儲存裝置122中。換言之,神經網路模型的模型參數(例如神經網路層數目與各神經網路層的權重等等)可經由事前訓練而決定並儲存於儲存裝置122中。此神經網路模型例如是遞歸神經網絡(RNN)模型。在神經網路模型訓練階段,多張尺寸為w*h的訓練影像資料被賦予真實光源資訊,以依據訓練影像資料的像素資訊與對應的真實光源資訊建構出神經網路模型。然而,於其他實施例中,處理器124也可利用其他光源估測演算法而依據這些第二子區域內的像素資訊估測出參考光源資訊。In step S4042, the processor 124 uses a neural network model to respectively predict a plurality of reference light source information according to the second sub-region. In detail, the processor 124 inputs these second sub-regions one by one to a trained neural network model to predict the respective reference light source information of the second sub-regions respectively. For example, a rectangular first image area can be divided into n second sub-areas, and the processor will use the trained neural network model to respectively predict n pieces of reference light source information. The neural network model can be a machine learning model constructed in advance for machine learning based on training data, and it can be stored in the storage device 122. In other words, the model parameters of the neural network model (such as the number of neural network layers and the weight of each neural network layer, etc.) can be determined through pre-training and stored in the storage device 122. This neural network model is, for example, a recurrent neural network (RNN) model. In the neural network model training phase, multiple training image data of size w*h are assigned real light source information, and a neural network model is constructed based on the pixel information of the training image data and the corresponding real light source information. However, in other embodiments, the processor 124 may also use other light source estimation algorithms to estimate the reference light source information based on the pixel information in the second sub-regions.

於步驟S4043,處理器124藉由統計第二子區域的參考光源資訊來產生區域光源資訊。於一實施例中,處理器124可計算這些參考光源資訊的平均值來產生區域光源資訊。舉例而言,處理器124可平均n個第二子區域的n個R光強度比例而獲取區域光源資訊中的R光強度比例。依此類推,處理器124可透過平均運算而獲取區域光源資訊(即RGB光強度比例)。In step S4043, the processor 124 generates area light source information by counting the reference light source information of the second sub-area. In one embodiment, the processor 124 may calculate the average value of the reference light source information to generate the area light source information. For example, the processor 124 may average the n R light intensity ratios of the n second sub-regions to obtain the R light intensity ratio in the regional light source information. By analogy, the processor 124 can obtain the area light source information (that is, the RGB light intensity ratio) through averaging.

此外,於本實施例中,針對非矩形的第一影像區域產生區域光源的步驟S405可實施為子步驟S4051~S4052。於步驟S4051,處理器124自第一影像區域取出一第一子區域。舉例而言,圖5是依照本發明一實施例的自第一影像塊取第一子區域的示意圖。請參照圖5,沿用圖3的原始影像Img-raw,原始影像Img-raw可基於亮度資訊而被區分為第一影像區域Z1與Z2。處理器124判斷第一影像區域Z1並非為矩形區域,並自第一影像區域Z1取出2*3的第一子區域B1。相似的,處理器124判斷第一影像區域Z2並非為矩形區域,並自第一影像區域Z2取出2*3的第一子區域B2。然而,圖5僅為一範例,處理器124也可分別自擷取第一影像區域Z1與Z2的其他局部區域來獲取第一子區域,圖5並非用以限定本發明。In addition, in this embodiment, the step S405 of generating an area light source for the non-rectangular first image area can be implemented as sub-steps S4051 to S4052. In step S4051, the processor 124 extracts a first sub-region from the first image region. For example, FIG. 5 is a schematic diagram of taking a first sub-region from a first image block according to an embodiment of the invention. Please refer to FIG. 5, following the original image Img-raw of FIG. 3, the original image Img-raw can be divided into first image regions Z1 and Z2 based on brightness information. The processor 124 determines that the first image area Z1 is not a rectangular area, and extracts the 2*3 first sub-area B1 from the first image area Z1. Similarly, the processor 124 determines that the first image area Z2 is not a rectangular area, and extracts the 2*3 first sub-area B2 from the first image area Z2. However, FIG. 5 is only an example, and the processor 124 may also obtain the first sub-areas from other partial areas of the captured first image areas Z1 and Z2, respectively. FIG. 5 is not used to limit the present invention.

於步驟S4052,處理器124利用一神經網路模型依據依據第一子區域預測區域光源資訊。舉例而言,處理器124可將圖5所示的第一子區域B1輸入至神經網路模型來預測出第一影像區域Z1的區域光源資訊。步驟S4052的操作相似於前述使用神經網路模型預測參考光源資訊的步驟S4042。基於上述可知,於本實施例中,第一影像區域的區域光源資訊是基於將至少一子區域輸入至神經網路模型進行預測而產生,且至少一子區域包括w*h個像素,w與h分別為大於1的整數。In step S4052, the processor 124 uses a neural network model to predict regional light source information based on the first sub-region. For example, the processor 124 may input the first sub-area B1 shown in FIG. 5 to the neural network model to predict the area light source information of the first image area Z1. The operation of step S4052 is similar to the aforementioned step S4042 of using the neural network model to predict the reference light source information. Based on the above, in this embodiment, the area light source information of the first image area is generated based on inputting at least one sub-area into the neural network model for prediction, and at least one sub-area includes w*h pixels, w and h is an integer greater than one, respectively.

步驟S403、步驟S404以及步驟S405將被重複執行,以使處理器124產生原始影像中所有的第一影像區域的區域光源資訊。之後,於步驟S406,處理器124將原始影像劃分為多個第二影像區域。接著,處理器124將依序針對第二影像區域產生對應的混合光源資訊。於本實施例中,處理器124判斷第二影像區域其中之一是否對應至兩筆以上的區域光源資訊。若第二影像區域其中之一對應至兩筆以上的區域光源資訊,處理器124依據像素數量比例對第二影像區域其中之一所對應的區域光源資訊其中至少二進行加權運算而產生第二影像區域其中之一的混合光源資訊其中之一。相反的,若第二影像區域其中之一並未對應至兩筆以上的區域光源資訊,處理器124依據第二影像區域其中之一所對應的區域光源資訊其中一產生第二影像區域其中之一的混合光源資訊其中之一。Step S403, step S404, and step S405 will be repeatedly executed, so that the processor 124 generates the area light source information of all the first image regions in the original image. After that, in step S406, the processor 124 divides the original image into a plurality of second image regions. Then, the processor 124 will sequentially generate corresponding mixed light source information for the second image area. In this embodiment, the processor 124 determines whether one of the second image areas corresponds to more than two pieces of area light source information. If one of the second image areas corresponds to more than two pieces of area light source information, the processor 124 performs a weighting operation on at least two of the area light source information corresponding to one of the second image areas according to the ratio of the number of pixels to generate a second image One of the mixed light source information of one of the areas. On the contrary, if one of the second image areas does not correspond to more than two pieces of area light source information, the processor 124 generates one of the second image areas according to one of the area light source information corresponding to one of the second image areas One of the mixed light source information.

如圖4所示,於步驟S407,處理器124判斷第二影像區域是否對應至兩筆以上的區域光源資訊。若第二影像區域對應至兩筆以上的區域光源資訊(步驟S407為是),於步驟S408,處理器124對第二影像區域所對應的兩筆以上的區域光源資訊進行加權運算而產生混合光源資訊。上述像素數量比例為第二影像區域其中之一內對應至區域光源資訊中的第一區域光源資訊的像素數量與第二影像區域其中之一內對應至區域光源資訊中的第二區域光源資訊的像素數量之間的比例。若第二影像區域並未對應至兩筆以上的區域光源資訊(步驟S407為否),於步驟S409,處理器124依據第二影像區域所對應的一筆區域光源資訊產生混合光源資訊。As shown in FIG. 4, in step S407, the processor 124 determines whether the second image area corresponds to more than two pieces of area light source information. If the second image area corresponds to more than two pieces of area light source information (Yes in step S407), in step S408, the processor 124 performs a weighting operation on the two or more pieces of area light source information corresponding to the second image area to generate a mixed light source News. The aforementioned pixel number ratio is the number of pixels in one of the second image areas corresponding to the first area light source information in the area light source information and one of the second image areas corresponding to the second area light source information in the area light source information The ratio between the number of pixels. If the second image area does not correspond to more than two pieces of area light source information (No in step S407), in step S409, the processor 124 generates mixed light source information according to a piece of area light source information corresponding to the second image area.

舉例而言,圖6A是依照本發明一實施例的計算多個第二影像區域的混合光源資訊的示意圖。請參照圖6A,原始影像Img-raw將被分割為4個第二影像區域G11、G12、G21、G22。沿用圖3與圖5的原始影像Img-raw,原始影像Img-raw可基於亮度資訊而被區分為兩個第一影像區域Z1與Z2,因而各像素位置具有對應的區域光源資訊。同時參照圖5與圖6A,處理器124可依據第一子區域B1而預測出區域光源資訊SP1,並依據第一子區域B2而預測出區域光源資訊SP2。因此,第二影像區域G11裡的像素P11因為屬於第一影像區域Z1而對應至區域光源資訊SP1。第二影像區域G11裡的像素P34因為屬於第一影像區域Z2而對應至區域光源資訊SP2。For example, FIG. 6A is a schematic diagram of calculating mixed light source information of multiple second image regions according to an embodiment of the present invention. Referring to FIG. 6A, the original image Img-raw will be divided into four second image regions G11, G12, G21, and G22. Following the original image Img-raw of FIGS. 3 and 5, the original image Img-raw can be divided into two first image regions Z1 and Z2 based on brightness information, so that each pixel position has corresponding regional light source information. Referring to FIGS. 5 and 6A at the same time, the processor 124 can predict the area light source information SP1 according to the first sub-area B1, and predict the area light source information SP2 according to the first sub-area B2. Therefore, the pixel P11 in the second image area G11 corresponds to the area light source information SP1 because it belongs to the first image area Z1. The pixel P34 in the second image area G11 corresponds to the area light source information SP2 because it belongs to the first image area Z2.

針對第二影像區域G11,對應於區域光源資訊SP1的像素數量為11且對應於區域光源資訊SP2的像素數量為1。因此,處理器124將以11比1的像素數量比例計算第二影像區域G11的混合光源資訊。詳細而言,處理器124將以權重因子11/12乘上區域光源資訊SP1並以權重因子1/12乘上區域光源資訊SP2,並將上述兩個相乘結果加總來產生第二影像區域G11的混合光源資訊。For the second image area G11, the number of pixels corresponding to the area light source information SP1 is 11 and the number of pixels corresponding to the area light source information SP2 is 1. Therefore, the processor 124 will calculate the mixed light source information of the second image area G11 at a ratio of 11 to 1 pixel number. In detail, the processor 124 multiplies the area light source information SP1 by a weighting factor of 11/12 and multiplies the area light source information SP2 by a weighting factor of 1/12, and adds the above two multiplication results to generate the second image area. G11 mixed light source information.

相似的,針對第二影像區域G12,對應於區域光源資訊SP1的像素數量為8且對應於區域光源資訊SP2的像素數量為4。因此,處理器124將以2比1的像素數量比例計算第二影像區域G12的混合光源資訊。詳細而言,處理器124將以權重因子2/3乘上區域光源資訊SP1並以權重因子1/3乘上區域光源資訊SP2,並將上述兩個相乘結果加總來產生第二影像區域G12的混合光源資訊。針對第二影像區域G21,處理器124將以1比3的像素數量比例計算第二影像區域G21的混合光源資訊。需注意的是,由於第二影像區域G22並未對應至兩筆以上的區域光源資訊,因此處理器124直接將區域光源資訊SP2作為第二影像區域G22的混合光源資訊。Similarly, for the second image area G12, the number of pixels corresponding to the area light source information SP1 is 8 and the number of pixels corresponding to the area light source information SP2 is 4. Therefore, the processor 124 calculates the mixed light source information of the second image area G12 at a ratio of 2 to 1 pixel number. In detail, the processor 124 multiplies the area light source information SP1 by a weighting factor of 2/3 and multiplies the area light source information SP2 by a weighting factor of 1/3, and sums the two multiplication results to generate the second image area. G12 mixed light source information. For the second image area G21, the processor 124 calculates the mixed light source information of the second image area G21 at a ratio of 1:3. It should be noted that since the second image area G22 does not correspond to more than two pieces of area light source information, the processor 124 directly uses the area light source information SP2 as the mixed light source information of the second image area G22.

步驟S407、步驟S408以及步驟S409將可能被重複執行,以使處理器124產生原始影像中所有的第二影像區域的混合光源資訊。接著,於步驟S410,處理器124依據第二影像區域的混合光源資訊分別修正第二影像區域內的多個像素值而獲取白平衡影像。Step S407, step S408, and step S409 may be repeatedly executed, so that the processor 124 generates mixed light source information of all the second image regions in the original image. Next, in step S410, the processor 124 respectively corrects a plurality of pixel values in the second image area according to the mixed light source information of the second image area to obtain a white balance image.

舉例而言,圖6B與圖6C是依照本發明一實施例的調整原始影像的像素值的示意圖。同時參照圖6A至圖6C,假設處理器124可分別針對第二影像區域G11、G12、G21、G22產生4筆混合光源資訊MP11、MP12、MP21、MP22。處理器124將依據混合光源資訊MP11調整第二影像區域G11內的像素值。像是,像素P11的RGB分量(r11 ,g11 ,b11 )將依據混合光源資訊MP11而調整為白平衡影像Img-B中的RGB分量(r’11 ,g’11 ,b’11 ),假設混合光源資訊MP11為RGB三色光的光強度比例(RMP11, GMP11, BMP11 ),則式(1)~(3)計算如下所示。 r’11 =r11 -r11* RMP11 式(1) g’11 =g11 -g11* GMP11 式(3) b’11 =b11 -b11* BMP11 式(2)For example, FIGS. 6B and 6C are schematic diagrams of adjusting the pixel value of the original image according to an embodiment of the invention. 6A to 6C, it is assumed that the processor 124 can generate 4 pieces of mixed light source information MP11, MP12, MP21, and MP22 for the second image regions G11, G12, G21, and G22, respectively. The processor 124 adjusts the pixel value in the second image area G11 according to the mixed light source information MP11. Such, the RGB components of the pixels P11 (r 11, g 11, b 11) will be adjusted according to the source information MP11 mixing balance of the RGB components of the image Img-B (r '11, g' 11 , b '11) , Assuming that the mixed light source information MP11 is the light intensity ratio of the RGB three-color light (R MP11, G MP11, B MP11 ), the formulas (1)~(3) are calculated as follows. r 'g 11 = r 11 -r 11 * R MP11 formula (1)' 11 = g 11 -g 11 * G MP11 formula (3) b '11 = b 11 -b 11 * B MP11 formula (2)

然而,式(1)~(3)僅為一示範例,於其他實施例中,處理器124可依據其他算法而依據混合光源資訊MP11來產生白平衡校正後的像素值。However, equations (1) to (3) are only an example. In other embodiments, the processor 124 may generate white balance corrected pixel values according to the mixed light source information MP11 according to other algorithms.

依此類推,處理器124將依據混合光源資訊MP12、MP21、MP22分別調整第二影像區域G12、G21、G22內的像素值。像是,像素P16的RGB分量(r16 ,g16 ,b16 )將依據混合光源資訊MP12而調整為白平衡影像Img-B中的RGB分量(r’16 ,g’16 ,b’16 ),而像素P48的RGB分量(r48 ,g48 ,b48 )將依據混合光源資訊MP22而調整為白平衡影像Img-B中的RGB分量(r’48 ,g’48 ,b’48 )。By analogy, the processor 124 will adjust the pixel values in the second image regions G12, G21, and G22 according to the mixed light source information MP12, MP21, and MP22, respectively. Like, RGB components of the pixel (r 16, g 16, b 16) P16 would be adjusted according to the source information MP12 mixing balance of the RGB components of the image Img-B (r '16, g' 16 , b '16) , the RGB components of the pixels P48 (r 48, g 48, b 48) the hybrid light source information based on the adjusted RGB components MP22 (r '48, g' 48 , b '48) the white balance of the image Img-B.

綜上所述,於本發明的實施例中,原始影像可分割為多個影像區域而依據不同的混合光源資訊來進行白平衡校正,且這些混合光源資訊也是對不同影像區域進行估測而產生。藉此,本發明實施例可以有效地校正原始影像中反應於多種環境光源而起的色偏現象,以執行更精準的影像白平衡校正。本發明所提出的白平衡調整方法及其影像處理裝置與系統,其可針對多光源或光源複雜的場景來適應性地調整影像的白平衡,進而達到高品質的影像輸出。In summary, in the embodiment of the present invention, the original image can be divided into a plurality of image areas to perform white balance correction based on different mixed light source information, and the mixed light source information is also generated by estimating different image areas . In this way, the embodiment of the present invention can effectively correct the color shift phenomenon in the original image that is reflected in a variety of ambient light sources, so as to perform more accurate image white balance correction. The white balance adjustment method and the image processing device and system proposed by the present invention can adaptively adjust the white balance of the image for scenes with multiple light sources or complex light sources, thereby achieving high-quality image output.

雖然本發明已以實施例揭露如上,然其並非用以限定本發明,任何所屬技術領域中具有通常知識者,在不脫離本發明的精神和範圍內,當可作些許的更動與潤飾,故本發明的保護範圍當視後附的申請專利範圍所界定者為準。Although the present invention has been disclosed in the above embodiments, it is not intended to limit the present invention. Anyone with ordinary knowledge in the relevant technical field can make some changes and modifications without departing from the spirit and scope of the present invention. The protection scope of the present invention shall be subject to those defined by the attached patent application scope.

100:影像處理系統 110:影像感測器 120:影像處理裝置 122:儲存裝置 124:處理器 Img-raw:原始影像 Img-raw:白平衡影像 P11、P16、P26、P34、P48:像素 Z1、Z2:第一影像區域 B1、B2:第一子區域 G11、G12、G21、G22:第二影像區域 S201~S205、S401~S410:步驟100: image processing system 110: Image sensor 120: Image processing device 122: storage device 124: processor Img-raw: raw image Img-raw: White balance image P11, P16, P26, P34, P48: pixels Z1, Z2: the first image area B1, B2: the first sub-area G11, G12, G21, G22: second image area S201~S205, S401~S410: steps

圖1是根據本發明一實施例的影像處理系統的方塊圖。 圖2是依照本發明一實施例的白平衡調整方法的流程圖。 圖3是依照本發明一實施例的將原始影像劃分為多個第一影像區域的示意圖。 圖4是依照本發明一實施例的白平衡調整方法的流程圖。 圖5是依照本發明一實施例的自第一影像塊取子區域的示意圖。 圖6A是依照本發明一實施例的計算多個第二影像區域的混合光源資訊的示意圖。 圖6B與圖6C是依照本發明一實施例的調整原始影像的像素值的示意圖。FIG. 1 is a block diagram of an image processing system according to an embodiment of the invention. FIG. 2 is a flowchart of a white balance adjustment method according to an embodiment of the invention. FIG. 3 is a schematic diagram of dividing an original image into a plurality of first image regions according to an embodiment of the invention. FIG. 4 is a flowchart of a white balance adjustment method according to an embodiment of the invention. FIG. 5 is a schematic diagram of taking a sub-region from a first image block according to an embodiment of the invention. FIG. 6A is a schematic diagram of calculating mixed light source information of a plurality of second image regions according to an embodiment of the present invention. 6B and 6C are schematic diagrams of adjusting the pixel value of the original image according to an embodiment of the invention.

S201~S205:步驟S201~S205: steps

Claims (10)

一種白平衡調整方法,所述方法包括: 接收一原始影像; 依據該原始影像的亮度資訊,將該原始影像劃分為多個第一影像區域: 針對各該些第一影像區域,產生分別對應至該些第一影像區域的多個區域光源資訊; 將該原始影像劃分為多個第二影像區域,並依據該些第二影像區域所對應的該些區域光源資訊其中至少一產生分別對應至該些第二影像區域的多個混合光源資訊:以及 依據該些第二影像區域的該些混合光源資訊分別修正該些第二影像區域內的多個像素值而獲取一白平衡影像。A white balance adjustment method, the method includes: Receive an original image; According to the brightness information of the original image, the original image is divided into a plurality of first image regions: For each of the first image areas, generating a plurality of area light source information corresponding to the first image areas; Dividing the original image into a plurality of second image areas, and generating a plurality of mixed light source information corresponding to the second image areas according to at least one of the area light source information corresponding to the second image areas: and According to the mixed light source information of the second image areas, the pixel values in the second image areas are respectively corrected to obtain a white balance image. 如申請專利範圍第1項所述的白平衡調整方法,其中依據該原始影像的亮度資訊,將該原始影像劃分為該些第一影像區域的步驟包括: 依據該原始影像中多個像素的亮度值是否大於一臨界值,將該原始影像中的該些像素劃分為該些第一影像區域, 其中,針對各該些第一影像區域,產生分別對應至該些第一影像區域的該些區域光源資訊的步驟包括: 依據各該些第一影像區域中的該至少一子區域產生該些第一影像區域的該些區域光源資訊。According to the white balance adjustment method described in item 1 of the scope of patent application, the step of dividing the original image into the first image regions according to the brightness information of the original image includes: Dividing the pixels in the original image into the first image regions according to whether the brightness values of the pixels in the original image are greater than a critical value, Wherein, for each of the first image areas, the step of generating the area light source information corresponding to the first image areas respectively includes: The area light source information of the first image areas is generated according to the at least one sub-area in each of the first image areas. 如申請專利範圍第2項所述的白平衡調整方法,其中依據各該些第一影像區域中的該至少一子區域產生該些第一影像區域的該些區域光源資訊的步驟包括: 分別判斷該些第一影像區域是否為矩形;以及 若該些第一影像區域其中之一並非為矩形,自該些第一影像區域其中之一取出該至少一子區域中的一第一子區域,並依據該第一子區域產生該些第一影像區域其中之一的該些區域光源資訊其中之一,其中該第一子區域的數量為1。According to the white balance adjustment method described in item 2 of the scope of patent application, the step of generating the area light source information of the first image areas according to the at least one sub-area in each of the first image areas includes: Respectively determine whether the first image areas are rectangular; and If one of the first image regions is not a rectangle, a first subregion of the at least one subregion is taken out from one of the first image regions, and the first subregions are generated according to the first subregion. One of the area light source information of one of the image areas, wherein the number of the first sub-area is one. 如申請專利範圍第3項所述的白平衡調整方法,其中依據各該些第一影像區域中的該至少一子區域產生該些第一影像區域的該些區域光源資訊的步驟更包括: 若該些第一影像區域其中之一為矩形,將該些第一影像區域其中之一劃分為該至少一子區域中的多個第二子區域,依據該些第二子區域分別產生多個參考光源資訊,並藉由統計該些參考光源資訊來產生該些第一影像區域其中之一的該些區域光源資訊其中之一,其中該些第二子區域的數量多於1。According to the white balance adjustment method described in claim 3, the step of generating the area light source information of the first image areas according to the at least one sub-area in each of the first image areas further includes: If one of the first image areas is rectangular, divide one of the first image areas into a plurality of second sub-areas in the at least one sub-areas, and generate a plurality of second sub-areas respectively according to the second sub-areas Reference light source information, and generate one of the area light source information of one of the first image areas by counting the reference light source information, wherein the number of the second sub-areas is more than one. 如申請專利範圍第2項所述的白平衡調整方法,其中該些第一影像區域的該些區域光源資訊是基於將該至少一子區域輸入至一神經網路模型進行預測而產生,且該至少一子區域包括w*h個像素,w與h分別為大於1的整數。In the white balance adjustment method described in claim 2, wherein the area light source information of the first image areas is generated based on inputting the at least one sub-area into a neural network model for prediction, and the At least one sub-region includes w*h pixels, and w and h are integers greater than one, respectively. 如申請專利範圍第1項所述的白平衡調整方法,其中將該原始影像劃分為該些第二影像區域,並依據該些第二影像區域所對應的該些區域光源資訊其中至少一產生分別對應至該些第二影像區域的多個混合光源資訊的步驟包括: 判斷該些第二影像區域其中之一是否對應至兩筆以上的該些區域光源資訊;以及 若該些第二影像區域其中之一對應至兩筆以上的該些區域光源資訊,依據一像素數量比例對該些第二影像區域其中之一所對應的該些區域光源資訊其中至少二進行加權運算而產生該些第二影像區域其中之一的該些混合光源資訊其中之一。According to the white balance adjustment method described in claim 1, wherein the original image is divided into the second image regions, and at least one of the regional light source information corresponding to the second image regions is generated respectively The steps of corresponding to the plurality of mixed light source information of the second image areas include: Determining whether one of the second image areas corresponds to more than two pieces of light source information of the areas; and If one of the second image areas corresponds to more than two pieces of the area light source information, at least two of the area light source information corresponding to one of the second image areas are weighted according to a ratio of the number of pixels One of the mixed light source information of one of the second image regions is generated by operation. 如申請專利範圍第6項所述的白平衡調整方法,其中該像素數量比例為該些第二影像區域其中之一內對應至該些區域光源資訊中的第一區域光源資訊的像素數量與該些第二影像區域其中之一內對應至該些區域光源資訊中的第二區域光源資訊的像素數量之間的比例。For the white balance adjustment method described in item 6 of the scope of patent application, the ratio of the number of pixels is the number of pixels corresponding to the first area light source information in the area light source information and the number of pixels in one of the second image areas The ratio between the number of pixels corresponding to the second area light source information in the area light source information in one of the second image areas. 如申請專利範圍第6項所述的白平衡調整方法,其中將該原始影像劃分為該些第二影像區域,並依據該些第二影像區域所對應的該些區域光源資訊其中至少一產生分別對應至該些第二影像區域的多個混合光源資訊的步驟更包括: 若該些第二影像區域其中之一並未對應至兩筆以上的該些區域光源資訊,依據該些第二影像區域其中之一所對應的該些區域光源資訊其中一產生該些第二影像區域其中之一的該些混合光源資訊其中之一。According to the white balance adjustment method described in item 6 of the scope of patent application, the original image is divided into the second image areas, and at least one of the area light source information corresponding to the second image areas is generated respectively The step of corresponding to the plurality of mixed light source information of the second image areas further includes: If one of the second image areas does not correspond to more than two pieces of the area light source information, generate the second images according to one of the area light source information corresponding to one of the second image areas One of the mixed light source information in one of the areas. 一種影像處理裝置,包括: 一儲存裝置,儲存有多個模組;以及 一處理器,耦接該儲存裝置,經配置而執行該些模組以: 接收一原始影像; 依據該原始影像的亮度資訊,將該原始影像劃分為多個第一影像區域: 針對各該些第一影像區域,產生分別對應至該些第一影像區域的多個區域光源資訊; 將該原始影像劃分為多個第二影像區域,並依據該些第二影像區域所對應的該些區域光源資訊其中至少一產生分別對應至該些第二影像區域的多個混合光源資訊:以及 依據該些第二影像區域的該些混合光源資訊分別修正該些第二影像區域內的多個像素值而獲取一白平衡影像。An image processing device, including: A storage device storing multiple modules; and A processor, coupled to the storage device, is configured to execute the modules to: Receive an original image; According to the brightness information of the original image, the original image is divided into a plurality of first image regions: For each of the first image areas, generating a plurality of area light source information corresponding to the first image areas; Dividing the original image into a plurality of second image areas, and generating a plurality of mixed light source information corresponding to the second image areas according to at least one of the area light source information corresponding to the second image areas: and According to the mixed light source information of the second image areas, the pixel values in the second image areas are respectively corrected to obtain a white balance image. 一種影像處理系統,包括: 一影像感測器,用以擷取影像; 一儲存裝置,儲存有多個模組;以及 一處理器,耦接該影像感測器與該儲存裝置,經配置而執行該些模組以: 接收一原始影像; 依據該原始影像的亮度資訊,將該原始影像劃分為多個第一影像區域: 針對各該些第一影像區域,產生分別對應至該些第一影像區域的多個區域光源資訊; 將該原始影像劃分為多個第二影像區域,並依據該些第二影像區域所對應的該些區域光源資訊其中至少一產生分別對應至該些第二影像區域的多個混合光源資訊:以及 依據該些第二影像區域的該些混合光源資訊分別修正該些第二影像區域內的多個像素值而獲取一白平衡影像An image processing system, including: An image sensor for capturing images; A storage device storing multiple modules; and A processor, coupled to the image sensor and the storage device, is configured to execute the modules to: Receive an original image; According to the brightness information of the original image, the original image is divided into a plurality of first image regions: For each of the first image areas, generating a plurality of area light source information corresponding to the first image areas; Dividing the original image into a plurality of second image areas, and generating a plurality of mixed light source information corresponding to the second image areas according to at least one of the area light source information corresponding to the second image areas: and According to the mixed light source information of the second image areas, the pixel values in the second image areas are respectively corrected to obtain a white balance image
TW109100807A 2020-01-09 2020-01-09 White balance adjustment method, image processing device and image processing system TWI723729B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
TW109100807A TWI723729B (en) 2020-01-09 2020-01-09 White balance adjustment method, image processing device and image processing system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
TW109100807A TWI723729B (en) 2020-01-09 2020-01-09 White balance adjustment method, image processing device and image processing system

Publications (2)

Publication Number Publication Date
TWI723729B TWI723729B (en) 2021-04-01
TW202127857A true TW202127857A (en) 2021-07-16

Family

ID=76604595

Family Applications (1)

Application Number Title Priority Date Filing Date
TW109100807A TWI723729B (en) 2020-01-09 2020-01-09 White balance adjustment method, image processing device and image processing system

Country Status (1)

Country Link
TW (1) TWI723729B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI796922B (en) * 2022-01-04 2023-03-21 宏碁股份有限公司 Image chromatic aberration removal method, image processing device and image processing system

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI796922B (en) * 2022-01-04 2023-03-21 宏碁股份有限公司 Image chromatic aberration removal method, image processing device and image processing system

Also Published As

Publication number Publication date
TWI723729B (en) 2021-04-01

Similar Documents

Publication Publication Date Title
AU2019326496B2 (en) Method for capturing images at night, apparatus, electronic device, and storage medium
CN104349066B (en) A kind of method, apparatus for generating high dynamic range images
US9288392B2 (en) Image capturing device capable of blending images and image processing method for blending images thereof
US9367916B1 (en) Method and system of run-time self-calibrating lens shading correction
EP2529554A1 (en) Iteratively denoising color filter array images
US9319653B2 (en) White balance compensation method and electronic apparatus using the same
US20150365612A1 (en) Image capture apparatus and image compensating method thereof
US11164290B2 (en) Method and system for image blurring processing
TWI383673B (en) Method and device for adjusting weighting values in light metering
WO2023016320A1 (en) Image processing method and apparatus, and device and medium
WO2020093653A1 (en) Color adjustment method, color adjustment device, electronic device and computer-readable storage medium
TW201830337A (en) Method and device for performing automatic white balance on an image
CN113132695A (en) Lens shadow correction method and device and electronic equipment
TWI723729B (en) White balance adjustment method, image processing device and image processing system
CN113271450B (en) White balance adjusting method, image processing device and image processing system
US11869224B2 (en) Method and system for establishing light source information prediction model
TW201830335A (en) Method and device for determining an illuminant of an image and for chromatic adaptation of an image
TWI723435B (en) Image outputting method and electronic device
US20140192222A1 (en) White balance adjusting method with scene detection and device thereof
TWI524279B (en) Image capturing device and method for shading compensation thereof
TWI796922B (en) Image chromatic aberration removal method, image processing device and image processing system
CN111614865B (en) Multi-shot brightness synchronization method, equipment, device and storage medium
TW202211160A (en) Dual sensor imaging system and calibration method thereof
CN113259634B (en) Light source information prediction model establishing method and system
TWI790732B (en) Image correction method and image correction device