TWI514290B - Hand detection method and image procressing apparatus - Google Patents
Hand detection method and image procressing apparatus Download PDFInfo
- Publication number
- TWI514290B TWI514290B TW103115194A TW103115194A TWI514290B TW I514290 B TWI514290 B TW I514290B TW 103115194 A TW103115194 A TW 103115194A TW 103115194 A TW103115194 A TW 103115194A TW I514290 B TWI514290 B TW I514290B
- Authority
- TW
- Taiwan
- Prior art keywords
- pixel
- edge
- image
- pixels
- value
- Prior art date
Links
Landscapes
- Image Analysis (AREA)
Description
本發明是有關於一種偵測方法及影像處理裝置,且特別是有關於一種手部偵測方法及影像處理裝置。The present invention relates to a detection method and an image processing apparatus, and more particularly to a hand detection method and an image processing apparatus.
隨著科技不斷進步,人機介面的發展也因各種先進技術與創新應用而日新月異,利用手勢辨識人機介面來操控各種裝置越來越受到大眾所重視。With the continuous advancement of technology, the development of human-machine interface has been changing with various advanced technologies and innovative applications. The use of gestures to identify human-machine interfaces to manipulate various devices has received increasing attention from the public.
傳統的解決方案往往需要在使用者手指上配置感應器等輔助工具,此舉雖然可以增加手部偵測的準確性,但是亦增加使用者的負擔。另一較佳的方式直接以影像處理的方式分析使用者的手部移動,據以操控裝置。過去的手勢偵測與辨識方式,不外乎需要建立訓練資料庫,透過例如是哈爾特徵(Haar-like feature)、馬可夫鏈(Markov chain)等比對方式來偵測手部,或者利用膚色轉換的方式將背景去除掉,只保留具有相似膚色的區域,再透過外觀(contour)或凸殼(convex hull)等邊緣偵測方式來進行手部辨識。Traditional solutions often require the use of auxiliary tools such as sensors on the user's fingers. This can increase the accuracy of hand detection, but it also increases the burden on the user. Another preferred way is to directly analyze the movement of the user's hand in an image processing manner, thereby manipulating the device. In the past, gesture detection and recognition methods required the establishment of a training database to detect the hand, such as the Haar-like feature, the Markov chain, or the like. The way of conversion removes the background, leaving only areas with similar skin tones, and then hand recognition through edge detection or convex hull.
然而,前述的比對方式需要大量資料庫的建立以及較長的比對時間,因此需要搭配高處理效能的裝置來進行辨識處理,不易實作於低成本的電子消費產品。此外,用膚色辨識的方式容易被光源及手部周圍的相似膚色區域影響且偵測距離有限。另一方面,為避免背景的邊緣資訊影響判斷,使用邊緣偵測的方式絕大部份都需在乾淨的背景下進行操作,不符合實際的使用情境。However, the foregoing comparison method requires a large number of databases to be established and a long comparison time. Therefore, it is necessary to perform identification processing with a device with high processing performance, which is not easy to implement in a low-cost electronic consumer product. In addition, the way of skin color recognition is easily affected by the light source and similar skin color areas around the hand and the detection distance is limited. On the other hand, in order to avoid the influence of the edge information of the background, most of the methods using edge detection need to operate in a clean background, which does not conform to the actual use situation.
有鑑於此,本發明提供一種手部偵測方法及影像處理裝置,其可以低成本的數位實作方式有效率地偵測出手部的特徵。In view of the above, the present invention provides a hand detection method and an image processing apparatus that can efficiently detect a feature of a hand in a low-cost digital implementation.
本發明提出的手部偵測方法,適用於影像處理裝置,包括下列步驟:接收輸入影像;針對輸入影像,進行邊緣偵測處理,以產生邊緣影像;針對邊緣影像,進行二值化處理,以產生二值化邊緣影像,其中二值化邊緣影像包括具有第一像素值的多個邊緣畫素以及具有第二像素值的多個非邊緣畫素;自二值化邊緣影像的所述邊緣畫素中,搜尋起始畫素以及終點畫素;判斷起始畫素與終點畫素之間的下方區域的像素值是否符合關聯於輸入影像的動態膚色區間;以及當起始畫素與終點畫素之間的下方區域的像素值符合動態膚色區間時,判斷起始畫素與終點畫素之間的上方區域是否具有多個手指特徵,據以確認手部的存在。The hand detection method provided by the present invention is applicable to an image processing apparatus, and includes the following steps: receiving an input image; performing edge detection processing on the input image to generate an edge image; and performing binarization processing on the edge image to Generating a binarized edge image, wherein the binarized edge image includes a plurality of edge pixels having a first pixel value and a plurality of non-edge pixels having a second pixel value; the edge painting of the binarized edge image In the prime, search for the starting pixel and the end pixel; determine whether the pixel value of the lower region between the starting pixel and the ending pixel conforms to the dynamic skin color interval associated with the input image; and when the starting pixel and the ending pixel are drawn When the pixel value of the lower region between the elements matches the dynamic skin color interval, it is determined whether the upper region between the start pixel and the end pixel has a plurality of finger features, thereby confirming the presence of the hand.
本發明另提出的影像處理裝置,包括儲存單元以及一或多個處理單元,其中處理單元耦接儲存單元。儲存單元用以記錄 多個模組,而處理單元用以存取並執行記錄在儲存單元中的模組。所述模組包括影像接收模組、邊緣偵測模組、二值化模組、搜尋模組、膚色判斷模組以及手部特徵判斷模組。影像接收模組用以接收輸入影像。邊緣偵測模組用以針對輸入影像進行邊緣偵測處理,以產生邊緣影像。二值化模組用以針對邊緣影像進行二值化處理,以產生二值化邊緣影像,其中二值化邊緣影像包括具有第一像素值的多個邊緣畫素以及具有第二像素值的多個非邊緣畫素。搜尋模組用以自邊緣影像的所述邊緣畫素中,搜尋起始畫素以及終點畫素。膚色判斷模組用以判斷起始畫素與終點畫素之間的下方區域的像素值是否符合關聯於輸入影像的動態膚色區間。當膚色判斷模組判斷起始畫素與終點畫素之間的下方區域的像素值符合動態膚色區間時,手部特徵判斷模組用以判斷於起始畫素與終點畫素之間的上方區域是否具有多個手指的特徵,據以確認手部的存在。The image processing device of the present invention further includes a storage unit and one or more processing units, wherein the processing unit is coupled to the storage unit. Storage unit for recording A plurality of modules, and the processing unit is configured to access and execute the modules recorded in the storage unit. The module includes an image receiving module, an edge detecting module, a binarization module, a searching module, a skin color determining module, and a hand feature determining module. The image receiving module is configured to receive an input image. The edge detection module performs edge detection processing on the input image to generate an edge image. The binarization module is configured to perform binarization processing on the edge image to generate a binarized edge image, wherein the binarized edge image includes a plurality of edge pixels having a first pixel value and a plurality of second pixel values Non-edge pixels. The search module is configured to search for the starting pixel and the end pixel from the edge pixels of the edge image. The skin color judging module is configured to determine whether the pixel value of the lower region between the starting pixel and the end pixel meets the dynamic skin color interval associated with the input image. When the skin color judgment module determines that the pixel value of the lower region between the start pixel and the end pixel meets the dynamic skin color interval, the hand feature determination module is configured to determine the upper portion between the start pixel and the end pixel Whether the area has the characteristics of a plurality of fingers is used to confirm the presence of the hand.
基於上述,本發明所提出的手部偵測方法及影像處理裝裝置先將輸入影像進行單一梯度的邊緣偵測而產生邊緣影像後,透過動態膚色濾波程序的結果來設定邊緣二值化門檻值,進而對邊緣影像進行二值化處理以產生二值化邊緣影像。接著,可自二值化邊緣影像找尋起始畫素與終點畫素,並且判斷其下方區域是否符合關聯於輸入影像的動態膚色區間,再自其上方區域尋找三個峰谷組合以判斷是否具有手指特徵,據以確認手部的存在,並且初步地定位出手部的位置。在無需訓練資料庫比對以及純膚色 辨識的情況下,本發明所提出的手部偵測方法及其裝置不僅可達到手部偵測,更可達到即時影像或視訊處理的效能,以運用於低成本的消費性電子產品上,增強本發明在實際應用中的適用性。Based on the above, the hand detection method and the image processing device of the present invention first perform edge detection on a single gradient of the input image to generate an edge image, and then set the edge binarization threshold value through the result of the dynamic skin color filter program. Then, the edge image is binarized to generate a binarized edge image. Then, the starting pixel and the ending pixel can be searched from the binarized edge image, and whether the lower region conforms to the dynamic skin color interval associated with the input image, and then three peaks and valleys are searched from the upper region to determine whether there is The finger feature is used to confirm the presence of the hand and to initially locate the position of the hand. No need to train database comparisons and pure skin tone In the case of identification, the hand detection method and device provided by the invention can not only achieve hand detection, but also achieve the performance of instant image or video processing, and is applied to low-cost consumer electronic products, and enhanced. The applicability of the invention in practical applications.
為讓本發明的上述特徵和優點能更明顯易懂,下文特舉實施例,並配合所附圖式作詳細說明如下。The above described features and advantages of the invention will be apparent from the following description.
100‧‧‧影像處理裝置100‧‧‧Image processing device
102‧‧‧儲存單元102‧‧‧ storage unit
104‧‧‧處理單元104‧‧‧Processing unit
110‧‧‧影像接收模組110‧‧‧Image receiving module
120‧‧‧邊緣偵測模組120‧‧‧Edge Detection Module
130‧‧‧二值化模組130‧‧‧ Binarization Module
140‧‧‧搜尋模組140‧‧‧Search Module
150‧‧‧膚色判斷模組150‧‧‧ skin color judgment module
160‧‧‧手部特徵判斷模組160‧‧‧Hand feature judgment module
S202~S212‧‧‧手部偵測方法的流程S202~S212‧‧‧Hand flow detection method
Sb 、Eb 、Ss 、Es ‧‧‧畫素S b , E b , S s , E s ‧ ‧ pixels
F d 、B ‧‧‧距離 F d, B ‧‧‧ from
400‧‧‧直方圖400‧‧‧Histogram
圖1繪示依據本發明一實施例之影像處理裝置的方塊圖。FIG. 1 is a block diagram of an image processing apparatus according to an embodiment of the invention.
圖2繪示依據本發明一實施例之手部偵測方法的流程圖。2 is a flow chart of a method for detecting a hand according to an embodiment of the invention.
圖3(a)~3(b)繪示依據本發明一實施例之二值化邊緣影像。3(a)-3(b) illustrate binarized edge images in accordance with an embodiment of the present invention.
圖4繪示依據本發明一實施例之邊緣畫素的數量相對於畫素列的直方圖。4 is a histogram of the number of edge pixels relative to a pixel column, in accordance with an embodiment of the present invention.
本發明的部份實施例接下來將會配合附圖來詳細描述,以下的描述所引用的元件符號,當不同附圖出現相同的元件符號將視為相同或相似的元件。這些實施例只是本發明的一部份,並未揭示所有本發明的可實施方式。更確切的說,這些實施例只是本發明的專利申請範圍中的裝置與方法的範例。The components of the present invention will be described in detail in the following description in conjunction with the accompanying drawings. These examples are only a part of the invention and do not disclose all of the embodiments of the invention. Rather, these embodiments are merely examples of devices and methods within the scope of the patent application of the present invention.
圖1繪示依據本發明一實施例之一種影像處理裝置的方塊示意圖,但此僅是為了方便說明,並不用以限制本發明。首先 圖1先介紹影像處理裝置的所有構件以及配置關係,詳細功能將配合圖2一併揭露。1 is a block diagram of an image processing apparatus according to an embodiment of the present invention, but is for convenience of description and is not intended to limit the present invention. First of all FIG. 1 first introduces all components and configuration relationships of the image processing apparatus, and detailed functions will be disclosed together with FIG.
請參照圖1,本實施例的影像處理裝置100可針對一輸入影像進行手部偵測,以從輸入影像定位出手部位置。影像處理裝置100可以為個人電腦、筆記型電腦、平板電腦、數位相機、智慧型手機、掃描機等具有影像處理功能的電子裝置,本發明不在此設限。影像處理裝置100包括儲存單元102及一或多個處理單元104,其功能分述如下。Referring to FIG. 1 , the image processing apparatus 100 of the present embodiment can perform hand detection on an input image to locate a hand position from the input image. The image processing apparatus 100 can be an electronic device having an image processing function such as a personal computer, a notebook computer, a tablet computer, a digital camera, a smart phone, or a scanner, and the present invention is not limited thereto. The image processing apparatus 100 includes a storage unit 102 and one or more processing units 104, the functions of which are described below.
儲存單元102例如是任意型式的固定式或可移動式隨機存取記憶體(Random Access Memory,RAM)、唯讀記憶體(Read-Only Memory,ROM)、快閃記憶體(Flash memory)、硬碟或其他類似裝置或這些裝置的組合,而用以記錄可由處理單元104執行的多個模組,這些模組可載入處理單元104以對輸入影像進行手部偵測。The storage unit 102 is, for example, any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory (Flash memory), hard A disc or other similar device or combination of these devices is used to record a plurality of modules executable by processing unit 104 that can be loaded into processing unit 104 for hand detection of the input image.
處理單元104例如是中央處理單元(Central Processing Unit,CPU),或是其他可程式化之一般用途或特殊用途的微處理器(Microprocessor)、數位訊號處理器(Digital Signal Processor,DSP)、可程式化控制器、特殊應用積體電路(Application Specific Integrated Circuits,ASIC)、可程式化邏輯裝置(Programmable Logic Device,PLD)或其他類似裝置或這些裝置的組合。處理單元104耦接至儲存單元102,其可存取並執行記錄在儲存單元102中的模組。The processing unit 104 is, for example, a central processing unit (CPU), or other programmable general purpose or special purpose microprocessor (Microprocessor), digital signal processor (DSP), programmable Controllers, Application Specific Integrated Circuits (ASICs), Programmable Logic Devices (PLDs), or other similar devices or combinations of these devices. The processing unit 104 is coupled to the storage unit 102, which can access and execute the modules recorded in the storage unit 102.
上述模組包括影像接收模組110、邊緣偵測模組120、二值化模組130、搜尋模組140、膚色判斷模組150以及手部特徵判斷模組160。這些模組例如是電腦程式,其可載入處理單元104,從而對輸入影像執行手部偵測。以下即列舉實施例說明影像處理裝置100執行影像對比強化處理的詳細步驟。The module includes an image receiving module 110, an edge detecting module 120, a binarization module 130, a search module 140, a skin color determining module 150, and a hand feature determining module 160. These modules are, for example, computer programs that can be loaded into the processing unit 104 to perform hand detection on the input image. Hereinafter, the detailed steps of the image processing apparatus 100 performing the image contrast enhancement processing will be described with reference to the embodiments.
圖2繪示依據本發明一實施例之手部偵測方法的流程圖。本實施例的方法適用於圖1的影像處理裝置100,以下即搭配影像處理裝置100中的各項元件說明本發明之手部偵測方法的詳細步驟。在本實施例中,將以影像處理裝置100耦接於具有攝像鏡頭與電荷耦合元件(charge coupled device,CCD)影像感測器等影像擷取裝置為例以進行說明。在另一實施例中,影像擷取裝置亦可內建於影像處理裝置100,本發明不在此設限。因此,在執行手部偵測方法之前,影像處理裝置100的儲存單元102已儲存預先自影像擷取裝置所擷取之包含多張影像的視訊串流。2 is a flow chart of a method for detecting a hand according to an embodiment of the invention. The method of the present embodiment is applicable to the image processing apparatus 100 of FIG. 1, and the detailed steps of the hand detection method of the present invention are described below in conjunction with the components in the image processing apparatus 100. In the embodiment, the image processing device 100 is coupled to an image capturing device having an imaging lens and a charge coupled device (CCD) image sensor as an example for description. In another embodiment, the image capturing device may also be built in the image processing device 100, and the present invention is not limited thereto. Therefore, before the hand detection method is performed, the storage unit 102 of the image processing apparatus 100 has stored the video stream containing a plurality of images captured from the image capturing device in advance.
請參照圖2,首先,影像接收模組110接收輸入影像(步驟S202)。詳言之,輸入影像包括多個輸入畫素,且這些輸入畫素是以行與列的方式排列成一個矩陣。在本實施例中,輸入影像為符合常用於描述數位影像訊號之YCbCr格式的彩色影像,因此每一輸入畫素包括Y像素值、Cb像素值以及Cr像素值。然而,在其它實施例中,輸入影像亦可以為位於其它色域的影像,本發明不在此設限。在此,將Y像素值、Cb像素值以及Cr像素值定義為輸入畫素的「成份值」,其中Y像素值定義為「亮度值」,並且 Cb像素值以及Cr像素值定義為「色彩資訊」。Referring to FIG. 2, first, the image receiving module 110 receives an input image (step S202). In detail, the input image includes a plurality of input pixels, and the input pixels are arranged in a matrix in a row and column manner. In this embodiment, the input image is a color image conforming to the YCbCr format commonly used to describe a digital image signal, and thus each input pixel includes a Y pixel value, a Cb pixel value, and a Cr pixel value. However, in other embodiments, the input image may also be an image located in another color gamut, and the present invention is not limited thereto. Here, the Y pixel value, the Cb pixel value, and the Cr pixel value are defined as the "component value" of the input pixel, wherein the Y pixel value is defined as the "luminance value", and The Cb pixel value and the Cr pixel value are defined as "color information".
接著,邊緣偵測模組120針對輸入影像,進行邊緣偵測處理,以產生邊緣影像(步驟S204)。一般在進行影像邊緣偵測時,往往需同時計算水平方向以及垂直方向的梯度(gradient)。此外,光源或背景往往會造成影像中的物件具有邊緣模糊的情況,而影像再經過縮小後同樣地會造成邊緣模糊的情況。由於本發明是針對手部進行偵測,而手部的垂直特徵較為明顯,因此為了可在遠距離偵測得到邊緣特徵,在本實施例中的邊緣偵測模組120將根據每一輸入畫素的成份值,利用垂直索貝爾濾波器(Vertical Sobel Filter)對輸入影像進行垂直方向的邊緣偵測。Then, the edge detection module 120 performs edge detection processing on the input image to generate an edge image (step S204). Generally, when performing image edge detection, it is often necessary to simultaneously calculate the horizontal and vertical gradients. In addition, the light source or background tends to cause the edges of the image to be blurred, and the image is similarly reduced to cause edge blurring. Since the present invention is directed to the detection of the hand, and the vertical features of the hand are more obvious, in order to detect the edge features at a long distance, the edge detection module 120 in this embodiment will draw according to each input. The component value of the element is the vertical edge detection of the input image using a vertical Sobel filter.
詳言之,在本實施例中,邊緣偵測模組120先擷取輸入影像的每一輸入畫素的成份值,亦即Y像素值、Cb像素值以及Cr像素值。接著,邊緣偵測模組120可根據方程式(1)~方程式(3)分別針對每一輸入畫素的成份值進行垂直邊緣偵測處理,以產生成份導數值:
此外,為了能偵測到位於遠距離的手部特徵,對於每一輸入畫素,邊緣偵測模組120將對成份導數值進行加總,以產生邊緣偵測值。在此對成份導數值進行加總的目地在於增加邊緣強度的資訊,其可以方程式(4)來表示:E (i ,j )=EY (i ,j )+ECb (i ,j )+ECr (i ,j ) 方程式(4)其中E (i ,j )為座標位於(i ,j )的輸入畫素所對應的邊緣偵測值。之後,邊緣偵測模組120則是利用上述邊緣偵測值E (i ,j )來產生邊緣影像。In addition, in order to detect a hand feature located at a long distance, for each input pixel, the edge detection module 120 will sum the component derivative values to generate an edge detection value. The purpose of summing the component derivative values here is to increase the information of the edge intensity, which can be expressed by equation (4): E ( i , j ) = EY ( i , j ) + ECb ( i , j ) + ECr ( i, j) equation (4) where E (i, j) is located at coordinate (i, j) of the input pixel corresponding to the edge detection value. Then, the edge detection module 120 uses the edge detection value E ( i , j ) to generate an edge image.
接著,二值化模組130針對邊緣影像,進行二值化處理,以產生二值化邊緣影像(步驟S206)。換言之,二值化模組130將以一個二值化的門檻值α來二值化邊緣影像,據以產生二值化邊緣影像。在本實施例中,其可以方程式(5)來表示:
值得注意的是,為了保留邊緣影像中手部的梯度以及有效濾除背景的資訊,二值化的門檻值α關係到手部的邊緣特徵是否明顯,以及雜訊是否太多而造成無法辨識或是誤判等情況。當α值越低,邊緣影像的邊緣資訊越明顯,然而其也越容易連結到非手部的其它邊緣區域,背景雜訊也越多,不利於後續的分析;反之,當α值越高,部份的邊源資訊將被濾除掉,而手部的邊緣特徵亦有可能不連續。基此,在本實施例中,影像處理裝置100更包括動態膚色濾波模組(未繪示),其可先根據輸入影像進行動態膚色濾波程序,以適應性地針對不同拍攝環境來調整二值化的門檻值α。It is worth noting that in order to preserve the gradient of the hand in the edge image and effectively filter the background information, the binning threshold α is related to whether the edge feature of the hand is obvious, and whether the noise is too much to be recognized or Misjudgment and other circumstances. When the value of α is lower, the edge information of the edge image is more obvious, but the easier it is to connect to other edge regions of the non-hand, the more background noise is, which is not conducive to subsequent analysis; otherwise, when the value of α is higher, Some of the edge source information will be filtered out, and the edge features of the hand may also be discontinuous. In this embodiment, the image processing apparatus 100 further includes a dynamic skin color filtering module (not shown), which can first perform a dynamic skin color filtering process according to the input image to adaptively adjust the binary value for different shooting environments. The threshold value is α.
詳言之,動態膚色濾波程序在於過濾輸入影像中非膚色區域。在此必須先說明的是,本發明並非以膚色來判斷手部的型態,因此即便拍攝主體受到光源影響或是背景有相似的膚色而造成膚色區域的手部特徵破裂或是不完整,並不影響手部偵測方法的結果。首先,動態膚色濾波模組可根據輸入影像的輸入畫素之色彩資訊(Cb像素值以及Cr像素值)來建構Cb動態膚色區間MinCr <Cr <MaxCr 以及Cr動態膚色區間MinCb <Cb <MaxCb ,其MinCb 、MaxCb 、MinCr 以及MaxCr 分別為Cb動態膚色區間的最小值、最大值以及Cr動態膚色區間的最小值、最大值,AvgCr 以及AvgCr 分別為所述輸入像素的Cb像素值以及Cr像素值的平均值,並且滿足方程式(6)~(9):MinCr =AvgCr 方程式(6)In particular, the dynamic skin tone filter program filters out non-skinned areas in the input image. It must be noted here that the present invention does not judge the shape of the hand by the skin color, so that even if the subject is affected by the light source or the background has a similar skin color, the hand features of the skin color region are broken or incomplete, and Does not affect the results of the hand detection method. First, the dynamic skin color filter module can construct a Cb dynamic skin color interval MinCr < Cr < MaxCr and a Cr dynamic skin color interval MinCb < Cb < MaxCb according to the input color information (Cb pixel value and Cr pixel value) of the input image. MinCb , MaxCb , MinCr, and MaxCr are the minimum and maximum values of the Cb dynamic skin color interval and the minimum and maximum values of the Cr dynamic skin color interval, respectively. AvgCr and AvgCr are the Cb pixel values of the input pixel and the average of the Cr pixel values, respectively. Value, and satisfy equations (6)~(9): MinCr = AvgCr equation (6)
MaxCr =AvgCr +40 方程式(7) MaxCr = AvgCr +40 Equation (7)
Ma.xCb =AvgCb ×1.1 方程式(8) Ma.xCb = AvgCb × 1.1 Equation (8)
MinCb =MaxCb -40 方程式(9)接著,動態膚色濾波模組即可根據輸入畫素的Cb像素值以及Cr像素值是否同時符合Cb動態膚色區間以及Cr動態膚色區間來判斷所述輸入畫素為膚色畫素或是非膚色畫素。 MinCb = MaxCb -40 Equation (9) Next, the dynamic skin tone filtering module can determine the input pixel according to whether the Cb pixel value of the input pixel and the Cr pixel value simultaneously meet the Cb dynamic skin color interval and the Cr dynamic skin color interval. Skin color pixels or non-skin pigments.
在本實施例中,二值化模組130將透過動態膚色濾波程序的結果來推算輸入畫面中前後背景的亮度值,進而設定二值化的門檻值α,故在此將二值化的門檻值α稱為為「邊緣二值化門檻值」。二值化模組130將參考影像亮度平均值、膚色亮度平均值以及非膚色亮度平均值等三個數值來設定邊緣二值化門檻值。在此,影像亮度平均值為所有輸入畫素的亮度值(Y像素值)的平均值;膚色亮度平均值為膚色畫素的亮度值的平均值;非膚色亮度平均值為非膚色畫素的亮度值的平均值。在本實施例中,二值化模組130可先找出參考影像亮度平均值、膚色亮度平均值以及非膚色亮度平均值中的最小值Minlum
、中間值Midlum
以及最大值Maxlum
後,依照方程式(10)取得邊緣二值化門檻值α,以使邊緣特徵較為明顯:
在二值化模組130產生二值化邊緣影像後,影像處理裝置100將利用手掌的掌心朝影像擷取裝置比出「5」的姿勢時手指與手掌微張的特性來搜尋找出手部的位置。首先,搜尋模組140自二值化邊緣影像的邊緣畫素中,搜尋起始畫素以及終點畫素(步驟S208)。詳言之,搜尋模組140可自二值化邊緣影像中,由上而下以及由左而右找出第一邊緣畫素以及第二邊緣畫素,並且判斷第一邊緣畫素與第二邊緣畫素之間的距離是否大於手掌寬度門檻值以及判斷第一邊緣畫素與第二邊緣畫素之間是否無其它邊緣畫素。在此設置手掌寬度門檻值的原因在於若第一邊緣畫素與第二邊緣畫素之間的距離太小,其對應的梯度會不清楚。此外,由於手掌的掌心垂直資訊較小,因此在進行二值化處理後所產生的邊緣畫素較少。因此,當搜尋模組140判斷第一邊緣畫素與第二邊緣畫素之間的距離大於手掌寬度門檻值以及第一邊緣畫素與第二邊緣畫素之間無其它邊緣畫素時,搜尋模組140將設定第一邊緣畫素以及第二邊緣畫素分別為起始畫素以及終點畫素;反之,搜尋模組140則會從二值化邊緣影像中繼續往右,再往下尋找新的第一邊緣畫素以及新的第二邊緣畫素。以圖(3)a的二值化邊緣影像300為例,搜尋模組140搜尋的起始畫素以及終點畫素可以分別為Sb 以及Eb ,其中Sb 與Eb 之間的距離為B 。After the binarization module 130 generates the binarized edge image, the image processing device 100 searches for the hand by using the characteristics of the finger and the palm of the hand when the palm of the palm is compared with the image capturing device in the posture of "5". s position. First, the search module 140 searches for the starting pixel and the end pixel from the edge pixels of the binarized edge image (step S208). In detail, the search module 140 can find the first edge pixel and the second edge pixel from top to bottom and from left to right from the binarized edge image, and determine the first edge pixel and the second edge. Whether the distance between the edge pixels is greater than the palm width threshold value and whether there is no other edge pixel between the first edge pixel and the second edge pixel. The reason why the palm width threshold is set here is that if the distance between the first edge pixel and the second edge pixel is too small, the corresponding gradient will be unclear. In addition, since the vertical information of the palm of the palm is small, the edge pixels generated after the binarization processing are small. Therefore, when the search module 140 determines that the distance between the first edge pixel and the second edge pixel is greater than the palm width threshold and that there is no other edge pixel between the first edge pixel and the second edge pixel, the search The module 140 sets the first edge pixel and the second edge pixel as the starting pixel and the ending pixel respectively; otherwise, the searching module 140 continues from the binarized edge image to the right, and then looks down. New first edge pixels and new second edge pixels. Taking the binarized edge image 300 of FIG. 3 as an example, the starting pixel and the ending pixel searched by the search module 140 may be S b and E b , respectively, where the distance between S b and E b is B.
接著,膚色判斷模組150判斷起始畫素與終點畫素之間的下方區域的像素值是否符合關聯於輸入影像的動態膚色區間 (步驟S210)。在本實施例中,在此的下方區域可以為起始畫素與終點畫素之間的下方所形成的一個正方形區域,然而本發明不在此設限。當膚色判斷模組150判斷此下方區域所對應輸入像素的成份值絕大部份符合動態膚色區間時,即代表此下方區域所對應輸入像素具有大量的膚色成份,也就是極有可能為手掌區域,因此手部特徵判斷模組160可在後續的步驟S212中做更進一步的特徵判斷。當膚色判斷模組150判斷此下方區域所對應輸入像素的成份值絕大部份不符合動態膚色區間時,即代表起始畫素與終點畫素並非為手掌的特徵,因此影像處理裝置100重新執行步驟S208,使搜尋模組140重新搜尋新的起始畫素以及新的終點畫素。Next, the skin color determination module 150 determines whether the pixel value of the lower region between the start pixel and the end pixel meets the dynamic skin color interval associated with the input image. (Step S210). In the present embodiment, the lower area here may be a square area formed below the start pixel and the end pixel, but the present invention is not limited thereto. When the skin color determination module 150 determines that the component value of the input pixel corresponding to the lower region conforms to the dynamic skin color interval, it means that the input pixel corresponding to the lower region has a large amount of skin color components, that is, it is highly likely to be a palm region. Therefore, the hand feature determination module 160 can perform further feature determination in the subsequent step S212. When the skin color determination module 150 determines that the component values of the input pixels corresponding to the lower region do not conform to the dynamic skin color interval, that is, the start pixel and the end pixel are not features of the palm, so the image processing apparatus 100 re Step S208 is executed to enable the search module 140 to re-search for a new starting pixel and a new ending pixel.
另一方面,當膚色判斷模組150判斷起始畫素與終點畫素之間的下方區域的像素值符合動態膚色區間時,手部特徵判斷模組160判斷於起始畫素與終點畫素之間的上方區域是否具有多個手指的特徵,據以確認手部的存在(步驟S212)。換言之,當膚色判斷模組150判斷起始畫素與終點畫素之間的下方區域極有可能為手掌的特徵時,手部特徵判斷模組160可更進一步地判斷起始畫素與終點畫素之間的上方區域是否為手指的特徵。由於一般人類的手指最高高度不會超過手掌寬度的1.1倍,因此以圖3(b)的二值化邊緣影像300為例,手部特徵判斷模組160可從起始畫素Sb 與終點畫素Eb 分別往上Fd 的距離尋找畫素Ss 與畫素Es ,其中F d =1.1×B 。而在本實施例中,起始畫素Sb 、終點畫素Eb 、畫素Ss 與畫素Es 之間所形成的區域302即為前述的上方區域。On the other hand, when the skin color determination module 150 determines that the pixel value of the lower region between the start pixel and the end pixel meets the dynamic skin color interval, the hand feature determination module 160 determines the start pixel and the end pixel. Whether or not the upper region has a feature of a plurality of fingers is used to confirm the presence of the hand (step S212). In other words, when the skin color determination module 150 determines that the lower region between the start pixel and the end pixel is highly likely to be a feature of the palm, the hand feature determination module 160 can further determine the start pixel and the end point. Whether the upper area between the elements is a feature of the finger. Since the height of the finger of a general human does not exceed 1.1 times the width of the palm, the hand feature determination module 160 can take the starting pixel S b and the end point by taking the binarized edge image 300 of FIG. 3(b) as an example. The distance between the pixels E b and the upper F d is to find the pixel S s and the pixel E s , where F d =1.1× B . In the present embodiment, the region 302 formed between the start pixel S b , the end pixel E b , the pixel S s and the pixel E s is the aforementioned upper region.
接著,手部特徵判斷模組160將判斷上方區域是否具有手指的特徵。詳言之,手部特徵判斷模組160將判斷上方區域內每一列畫素為峰點(peak)畫素列或是谷點(valley)畫素列及其配置的位置是否分別對應於手指的指尖特徵或是手指的指縫特徵。在本實施例中,手部特徵判斷模組160將先在上方區域內尋找是否存在三個峰谷(peak-valley)組合。在此的峰谷組合為兩相鄰的峰點畫素列間具有一定的距離,並且其之間存在谷點畫素列。由圖3(b)可看出,若手部特徵判斷模組160找到三個指尖的特徵,只要分別在各個指尖的兩側找到一個指縫的特徵,即可完成判斷。由於在手掌微張的狀態下,大姆指距離較遠,而小指的高度與其它手指的長度差距較大,並不為本實施例的判斷條件。換句話說,手部特徵判斷模組160在找到峰點畫素列時,判斷兩相鄰的峰點畫素之間的距離是否不大於寬度門檻值並且其之間是否存在谷點畫素列。在本實施例中,寬度門檻值可以為1/4手部寬度。以下將針對尋找三個峰谷組合的步驟做更進一步的說明。Next, the hand feature determination module 160 will determine whether the upper region has the characteristics of the finger. In detail, the hand feature determination module 160 determines whether each column of the pixel in the upper region is a peak pixel column or a valley pixel column and whether the position of the column corresponds to the finger. Fingertip features or finger joint features of a finger. In this embodiment, the hand feature determination module 160 will first look in the upper region for the presence of three peak-valley combinations. The combination of peaks and valleys here has a certain distance between two adjacent peak pixel columns, and there is a valley pixel column between them. As can be seen from FIG. 3(b), if the hand feature determination module 160 finds the features of the three fingertips, the judgment can be completed as long as the features of one finger seam are found on both sides of each fingertip. Since the thumb is far away in the state of the palm of the hand, and the height of the little finger is far from the length of the other fingers, it is not the judgment condition of the embodiment. In other words, when the peak feature determination module 160 finds the peak pixel column, it is determined whether the distance between two adjacent peak pixels is not greater than the width threshold and whether there is a valley pixel between them. . In this embodiment, the width threshold value may be 1/4 hand width. The following is a further explanation of the steps for finding a combination of three peaks and valleys.
首先,手部特徵判斷模組160可根據圖3(b)中的上方區域302中由下方往上(即,線段往線段的方向)進行判斷。手部特徵判斷模組160可累計上方區域302的水平投影的每一列有多少個邊緣畫素,並且記錄於向量His [0:m ],其中m 即為上方區域302的寬度。為方便明瞭,向量His [0:m ]可以圖4的直方圖400來表示,其中直方圖400的橫軸為向量His [0:m ]的索引(index),也就是畫素列的編號,直方圖400的縱軸為邊緣畫素的數量。手部 特徵判斷模組160可從向量His [0:m ]尋找是否具有大於上方區域302高度例如是0.8倍以上的最大值。若沒有,則手部特徵判斷模組160往線段的方向繼續向上尋找;若有並且其最大值為向量His [0:m ]中索引為k 的數值(也就是說最大值為His [k ]),則手部特徵判斷模組160可從向量His [0:m ]中取得大於His [k ]×0.8的數值N1 所對應的索引,並且記錄於向量P 中。假設在本實施例中,向量His [0:m ]中大於N1 所對應的索引為2、3、6以及9,則向量P =(2,3,6,9)。在此,向量P 所記錄的索引為前述峰點畫素列的編號,也就是關聯於指尖的特徵。另一方面,手部特徵判斷模組160又從向量His [0:m ]中取得小於例如是His [k ]×0.5的數值N2 所對應的索引,並且記錄於向量V 中。假設在本實施例中,向量His [0:m ]中小於N2 所對應的索引為0、1、4、5、7、8、10以及11,則向量V =(0,1,4,5,7,8,10,11)。在此向量V 所記錄的索引為前述谷點畫素列的編號,也就是關聯於指縫的特徵。First, the hand feature determination module 160 can be up from the bottom (ie, the line segment according to the upper region 302 in FIG. 3(b). To the line segment The direction) is judged. The hand feature determination module 160 can accumulate how many edge pixels are in each column of the horizontal projection of the upper region 302 and record it in the vector His [0: m ], where m is the width of the upper region 302. For convenience, the vector His [0: m ] can be represented by the histogram 400 of FIG. 4, wherein the horizontal axis of the histogram 400 is the index of the vector His [0: m ], that is, the number of the pixel column. The vertical axis of the histogram 400 is the number of edge pixels. The hand feature determination module 160 can look up from the vector His [0: m ] whether it has a maximum value greater than, for example, 0.8 times or more the height of the upper region 302. If not, the hand feature determination module 160 goes to the line segment. The direction continues to look upward; if it is and its maximum value is the value of the index in the vector His [0: m ], which is index k (that is, the maximum value is His [ k ]), the hand feature determination module 160 can learn from the vector. His [0: m] is greater than the acquired His [k] corresponding to the value of the index N 1 × 0.8, and the recording medium P in the vector. It is assumed that in the present embodiment, the index corresponding to the larger value of N 1 in the vector His [0: m ] is 2, 3, 6, and 9, and the vector P = (2, 3, 6, 9). Here, the index recorded by the vector P is the number of the aforementioned peak pixel column, that is, the feature associated with the fingertip. On the other hand, the hand feature judging module 160 takes an index corresponding to the value N 2 smaller than, for example, His [ k ] × 0.5 from the vector His [0: m ], and records it in the vector V. It is assumed that in the present embodiment, the index corresponding to less than N 2 in the vector His [0: m ] is 0, 1, 4, 5, 7, 8, 10, and 11, and the vector V = (0, 1, 4, 5,7,8,10,11). The index recorded in this vector V is the number of the aforementioned valley pixel column, that is, the feature associated with the finger seam.
手部特徵判斷模組160可根據向量P 以及向量V 來判斷上方區域302內是否確實存在指尖與指縫的特徵,也就是判斷起始畫素Sb 與終點畫素Eb 是否存在三個峰谷組合。詳言之,手部特徵判斷模組160可從向量P 中,以每兩相鄰的元素(element)為一組,尋找兩相鄰的元素之間的數值是否存在於向量V 之中。舉例而言,向量P =(2,3,6,9)中的2與3為連續數值,因此兩數值之間並無任何向量V 的元素。另一方面,3與6之間的數值為4以及5,其存在於向量V 之中;6與9之間的數值為7以及8,其亦存在於向 量V 之中。The hand feature judging module 160 can determine whether the fingertip and the finger seam are actually present in the upper region 302 according to the vector P and the vector V , that is, whether the starting pixel S b and the end pixel E b are present three. Peak and valley combination. In detail, the hand feature determination module 160 can search for a value between two adjacent elements in the vector V from the vector P in groups of two adjacent elements. For example, 2 and 3 of the vector P = (2, 3, 6, 9) are continuous values, so there is no element of vector V between the two values. On the other hand, the values between 3 and 6 are 4 and 5, which exist in the vector V ; the values between 6 and 9 are 7 and 8, which are also present in the vector V.
接著,手部特徵判斷模組160判斷每兩個相鄰的峰點畫素列之間的距離是否皆不大於寬度門檻值。在此範例中,由於m =11,手部寬度為12,因此寬度門檻值則將設定為3,也就是1/4手部寬度。在此,手部特徵判斷模組160則會判斷出3與6之間以及6與9之間皆不大於3。當兩個相鄰的峰點畫素之間的距離皆不大於寬度門檻值時,手部特徵判斷模組160接著判斷最左的峰點畫素列與最右的峰點畫素列的兩側是否存在至少一個谷點畫素列。若是,手部特徵判斷模組160可判斷出上方區域302符合三個峰谷組合,以確認手部的存在。舉例來說,在本實施例中,手部特徵判斷模組160可判斷出向量P =(2,3,6,9)的元素2的左邊具有向量V 的元素0以及1,並且向量P 的元素9的右邊具有向量V 的元素10以及11。Next, the hand feature determination module 160 determines whether the distance between each two adjacent peak pixel columns is not greater than the width threshold. In this example, since m = 11 and the hand width is 12, the width threshold will be set to 3, which is 1/4 hand width. Here, the hand feature determination module 160 determines that there is no more than 3 between 3 and 6 and between 6 and 9. When the distance between two adjacent peak pixels is not greater than the width threshold, the hand feature determination module 160 then determines two of the leftmost peak pixel column and the rightmost peak pixel column. Whether there is at least one valley element column on the side. If so, the hand feature determination module 160 can determine that the upper region 302 conforms to the three peak-to-valley combinations to confirm the presence of the hand. For example, in this embodiment, the hand feature determination module 160 can determine that the left side of the element 2 of the vector P = (2, 3, 6, 9) has the elements 0 and 1 of the vector V , and the vector P The right side of element 9 has elements 10 and 11 of vector V.
在一實施例中,在手部特徵判斷模組160在步驟S212中確認手部的存在後,可根據手指特徵於二值化邊緣影像的所在位置定位手部。也就是說,二值化邊緣影像的峰點畫素即對應於手部的指尖特徵,而谷點畫素即對應於手部的指縫特徵。在一實施例中,當手部特徵判斷模組160定位手部後,可將其進行標示,並且影像處理裝置100可重新執行步驟S208,以判斷輸入影像是否存在另一個手部特徵。此外,在一實施例中,在手部特徵判斷模組160在步驟S212中確認手部不存在後,影像處理裝置100亦會重新執行步驟S208,以繼續搜尋手部特徵。In an embodiment, after the hand feature determination module 160 confirms the presence of the hand in step S212, the hand can be positioned according to the position of the finger to binarize the edge image. That is to say, the peak pixel of the binarized edge image corresponds to the fingertip feature of the hand, and the valley pixel corresponds to the finger joint feature of the hand. In an embodiment, after the hand feature determination module 160 locates the hand, it can be marked, and the image processing apparatus 100 can re-execute step S208 to determine whether there is another hand feature in the input image. In addition, in an embodiment, after the hand feature determination module 160 confirms that the hand does not exist in step S212, the image processing apparatus 100 re-executes step S208 to continue searching for the hand feature.
在一實施例中,影像處理裝置100可在後續對手部的特徵進行追蹤。舉例來說,若手部保持朝上的狀態,因手部的移動範圍有限,因此搜尋模組140可縮小搜尋的空間與時間,並且無須再重新執行動態膚色濾波程序,手部特徵判斷模組160即可重新對手部進行定位。In an embodiment, the image processing device 100 can track the features of subsequent opponents. For example, if the hand is kept facing upward, the search module 140 can narrow the space and time of the search because the movement range of the hand is limited, and the dynamic skin color filtering program does not need to be re-executed, and the hand feature determination module 160 You can reposition your opponent.
綜上所述,本發明所提出的手部偵測方法及影像處理裝裝置先將輸入影像進行單一梯度的邊緣偵測而產生邊緣影像後,透過動態膚色濾波程序的結果來設定邊緣二值化門檻值,進而對邊緣影像進行二值化處理以產生二值化邊緣影像。接著,可自二值化邊緣影像找尋起始畫素與終點畫素,並且判斷其下方區域是否符合關聯於輸入影像的動態膚色區間,再自其上方區域尋找三個峰谷組合以判斷是否具有手指特徵,據以確認手部的存在,並且初步地定位出手部的位置。在無需訓練資料庫比對以及純膚色辨識的情況下,本發明所提出的手部偵測方法及其裝置不僅可達到手部的偵測,更可達到即時影像或視訊處理的效能,以運用於低成本的消費性電子產品上,增強本發明在實際應用中的適用性。In summary, the hand detection method and the image processing device of the present invention first perform edge detection on a single gradient of the input image to generate an edge image, and then set edge binarization through the result of the dynamic skin color filter program. The threshold value, and then the edge image is binarized to produce a binarized edge image. Then, the starting pixel and the ending pixel can be searched from the binarized edge image, and whether the lower region conforms to the dynamic skin color interval associated with the input image, and then three peaks and valleys are searched from the upper region to determine whether there is The finger feature is used to confirm the presence of the hand and to initially locate the position of the hand. In the case of no training database comparison and pure skin color recognition, the hand detection method and device provided by the invention can not only achieve hand detection, but also achieve the performance of instant image or video processing, and use The applicability of the present invention in practical applications is enhanced on low cost consumer electronics.
雖然本發明已以實施例揭露如上,然其並非用以限定本發明,任何所屬技術領域中具有通常知識者,在不脫離本發明的精神和範圍內,當可作些許的更動與潤飾,故本發明的保護範圍當視後附的申請專利範圍所界定者為準。Although the present invention has been disclosed in the above embodiments, it is not intended to limit the present invention, and any one of ordinary skill in the art can make some changes and refinements without departing from the spirit and scope of the present invention. The scope of the invention is defined by the scope of the appended claims.
S202~S212‧‧‧手部偵測方法的流程S202~S212‧‧‧Hand flow detection method
Claims (10)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
TW103115194A TWI514290B (en) | 2014-04-28 | 2014-04-28 | Hand detection method and image procressing apparatus |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
TW103115194A TWI514290B (en) | 2014-04-28 | 2014-04-28 | Hand detection method and image procressing apparatus |
Publications (2)
Publication Number | Publication Date |
---|---|
TW201541368A TW201541368A (en) | 2015-11-01 |
TWI514290B true TWI514290B (en) | 2015-12-21 |
Family
ID=55220516
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
TW103115194A TWI514290B (en) | 2014-04-28 | 2014-04-28 | Hand detection method and image procressing apparatus |
Country Status (1)
Country | Link |
---|---|
TW (1) | TWI514290B (en) |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
TW201120681A (en) * | 2009-12-10 | 2011-06-16 | Tatung Co | Method and system for operating electric apparatus |
WO2012139241A1 (en) * | 2011-04-11 | 2012-10-18 | Intel Corporation | Hand gesture recognition system |
TWM468724U (en) * | 2013-08-23 | 2013-12-21 | Univ Kun Shan | Automatic optical detection device for fingertip locus tracing and recording |
EP2680228A1 (en) * | 2012-06-25 | 2014-01-01 | Softkinetic Software | Improvements in or relating to three dimensional close interactions. |
-
2014
- 2014-04-28 TW TW103115194A patent/TWI514290B/en not_active IP Right Cessation
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
TW201120681A (en) * | 2009-12-10 | 2011-06-16 | Tatung Co | Method and system for operating electric apparatus |
WO2012139241A1 (en) * | 2011-04-11 | 2012-10-18 | Intel Corporation | Hand gesture recognition system |
EP2680228A1 (en) * | 2012-06-25 | 2014-01-01 | Softkinetic Software | Improvements in or relating to three dimensional close interactions. |
TWM468724U (en) * | 2013-08-23 | 2013-12-21 | Univ Kun Shan | Automatic optical detection device for fingertip locus tracing and recording |
Also Published As
Publication number | Publication date |
---|---|
TW201541368A (en) | 2015-11-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP4756705B2 (en) | Biometric imaging system and method | |
TWI499966B (en) | Interactive operation method of electronic apparatus | |
Raghavendra et al. | A low-cost multimodal biometric sensor to capture finger vein and fingerprint | |
WO2010032126A2 (en) | A vein pattern recognition based biometric system and methods thereof | |
US20180150699A1 (en) | Object recognition device | |
CN103870071B (en) | One kind touches source discrimination and system | |
TWI603270B (en) | Method and apparatus for detecting person to use handheld device | |
US10423824B2 (en) | Body information analysis apparatus and method of analyzing hand skin using same | |
KR101745601B1 (en) | method for finger counting by using image processing and apparatus adopting the method | |
CN108734069B (en) | Method and device for calculating quality score of finger vein image | |
Donida Labati et al. | A scheme for fingerphoto recognition in smartphones | |
TWI428807B (en) | Optical coordinate input device and coordinate calculation method thereof | |
US20160140762A1 (en) | Image processing device and image processing method | |
JP2024109741A (en) | Slap Segmentation of Contactless Fingerprint Images | |
JP5254897B2 (en) | Hand image recognition device | |
JP2004303014A (en) | Gesture recognition device, its method and gesture recognition program | |
KR101281461B1 (en) | Multi-touch input method and system using image analysis | |
JP2018142828A (en) | Deposit detector and deposit detection method | |
TWI606405B (en) | Image processing method and image processing system | |
TWI514290B (en) | Hand detection method and image procressing apparatus | |
KR101681197B1 (en) | Method and apparatus for extraction of depth information of image using fast convolution based on multi-color sensor | |
JP2012003724A (en) | Three-dimensional fingertip position detection method, three-dimensional fingertip position detector and program | |
TWI507919B (en) | Method for tracking and recordingfingertip trajectory by image processing | |
TW201401187A (en) | Virtual touch method using fingertip detection and system thereof | |
TWI615780B (en) | Fingerprint image processing method and device |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
MM4A | Annulment or lapse of patent due to non-payment of fees |