經濟部智慧財產局員工消費合作社印製 五、發明說明（i ) 【本發明之領域】 本發明係有關影像辨識之技術領域，尤指一種多層次 臉部影像辨識方法及系統。 【本發明之背景】 目前在正面臉部影像之辨識上，雖有數種辨識方式可 供採用，例如特徵模式、樣板匹配與類神經網路模式等， 然而前述習知之正面臉部影像辨識技術仍無法滿足實際之 而要，例如在以特徵攫取的方法做臉部辨識時，將會面臨 三個問題.：（1)應該選擇什麼做為特徵？（2)應該有多少 特徵才足夠？（3)特徵之攫取並不容易。 而樣板匹配的方法運算量較大，當比對的影像增多 時，其所需時間也相對的大幅增加；且如何定義適當的匹 配度（相似度）評定標準也是此種方法有待解決的問題。 另，應用類神經網路自動學f辨識的困難是在於臉部影像 《變化複雜且細微，所以結構簡單的類神經網路不易分析 出各影像之間的特徵差異。因此，如何能夠正確並有效地 達成正面臉部影像之辨識，實為_騎解決之課題。 發明人爰因於此，本於積極發明之精神，亟思一種可 以解決上述問題之「多層次臉部影像辨識系統」，幾經研 咒實驗終至完成此項新穎進步之發明。 【本發明之概述】 ·1--------1T---------## (請先閱讀背面之注意事項再填寫本頁)Printed by the Consumer Cooperative of Intellectual Property Bureau of the Ministry of Economic Affairs 5. Description of the Invention (i) [Field of the Invention] The present invention relates to the technical field of image recognition, especially a multi-level facial image recognition method and system. [Background of the present invention] At present, in the recognition of frontal facial images, although there are several recognition methods available, such as feature mode, template matching, and neural network-like modes, etc., the aforementioned conventional frontal facial image recognition technology is still It cannot meet the actual requirements. For example, when face recognition is performed by feature extraction, three problems will be faced. (1) What should be selected as a feature? (2) How many characteristics should be sufficient? (3) The capture of features is not easy. The method of template matching has a large amount of calculations. When the number of images to be compared increases, the time required for it is relatively large. How to define an appropriate evaluation standard for similarity (similarity) is also a problem to be solved by this method. In addition, the difficulty in applying automatic neural network-like identification for facial recognition is that the facial images are complex and subtle, so a simple neural network with a simple structure cannot easily analyze the feature differences between the images. Therefore, how to correctly and effectively recognize the frontal facial image is really a problem to be solved. Because of this, the inventor, in the spirit of active invention, urgently thought of a "multi-level facial image recognition system" that can solve the above problems. After several cursing experiments, this novel and progressive invention was completed. [Overview of the invention] · 1 -------- 1T --------- ## (Please read the notes on the back before filling this page)
477959 經濟部智慧財產局員工消費合作社印製 A7 五、發明說明（2， 本發明之目的係在提供一種多層次臉邵影像辨識方法 及系統，以降低辨識所需之資料比對量，大幅提高辨識速 度。 依據本發明之一特色，所提出之多層次臉部影像辨識 方去係包括下述之步驟：（A )輸入臉部原始影像； (B )進行前置處理，以將該原始影像裁剪成僅包含完整 的臉部影像；（C)將該臉部影像分解成^^種解析度，每 種解析度個有Μ個頻道，其中N^2且2，使該臉部影 像分解成N X Μ張子影像；（D )在學習階段時，以常態 表情的正面臉部影像為學習影像，而將學習影像分解後之 子影像分別輸入至Ν X Μ個自我構圖類神經網路以進行無 監督式歸類學習，且當類神經網路完成預定的學習週期， 將學習影像之子影像再一次分別輸入至該Μ個已經完成學 習的類神經網路，以使每一個類神經網路均產生一個優勝 單元；以及（Ε )在測試階段時，將測試影像分解，以由 最低解析度Μ個子影像開始，分別輸入至其對應的Μ個自 我構圖類神經網路，產生Μ個優勝單元，以進行決策辨 識，經由計算該Μ個優勝單元與其相對應自我構圖類神於 網路上各學習影像的優勝單元之距離，找出可能為優勝= 之候選者，如候選者僅有一人，則此人就是優勝 〃 可，決策 過程在此完成，如果候選者不只一位，則保留這些候選者 留待更高一層解析度時再做決策。 依據本發明之另一特色’所提出之多層次臉部影像辨 識系統，包括有影像輸入機制、前置處理機制、分解機制 ------------裝--------訂--------.^^1 (請先閱讀背面之注意事項再填寫本頁) 477959 經濟部智慧財產局員工消費合作社印製 A7 五、發明說明（3 ) 以及複數個自我構圖類神經網路，該影像輸入機制係〆 輸入臉部原始影像；該前置處理機制係用以將該原始 裁剪成僅包含完整的臉部影像；該分解機制係將該臉^影 像分解成N種解析度，每種解析度個有M個頻道，其中I -2且Mg2，使該臉部影像分解成ΝχΜ張子影像；當在 子ό階段時，係以常態表情的正面臉部影像為學習影像， 而將學習影像，分解後之子影像分別輸入至NX Μ個自我構 圖類神經網路以進行無監督式歸類學習，且當類神經網路 完成預定的學習週期，將學習影像之子影像再一次分別輸 入至該Μ個已經完成學習的類神經網路，以使每一個類神 經網路均產生一個優勝單元；而在測試階段時，將測試影 像分解，以由最低解析度Μ個子影像開始，分別輸入至其 對應的Μ個自我構圖類神經網路，產生μ個優勝單元，以 進行決策辨識，經由計算該Μ個優勝單元與其相對應自我 構圖類神經網路上各學習影像的優勝單元之距離，找出可 能為優勝者之候選者，如候選者僅有一人，則此人就是優 勝者，決策過程在此完成，如果候選者不只一位，則保留 這些候選者留待更高一層解析度時再做決策。 由於本發明設計新穎，能提供產業上利用，且確有增 進功效，故依法申請專利。 為使貴審查委員能進一步瞭解本發明之結構、特徵 及其目的，茲附以圖式及較佳具體實施例之詳細說明如 后： 本紙張尺度適用中國國家標準（CNS)A4規格（210 X 297公釐） i—丨丨丨丨丨丨丨—裝i·丨丨丨丨丨訂-丨------* (請先閲讀背面之注意事項再填寫本頁) 477959 A7 B7 經濟部智慧財產局員工消費合作社印製 第1圖 竭2圖 第3圖 第4圖 第5圖 第6圖 第7圖 第8圖 五、發明說明（4 ) 【圖式簡單説明 係本發明之多層次臉部影像辨識系統之架構圖。 係正交鏡像濾通器分解頻率平面分割圖。 係正交鏡像滤通器之分解過程示意圖。 係三層解析度正交鏡像濾通器分解頻率平面分割 圖0 係三層解析度正交鏡像濾通器之分解過程示音 圖。 " 係本發明之多層次臉部影像辨識系統在學習階段 之架構圖。 係SOM類神經網路之架構圖。 係本發明之多層次臉部影像辨識系統在測試階段 之架構圖。 圖號説明】 1 1 )臉部原始影像 （12 )完整臉部影像 101)〜（104) (111)〜（114)子影像 121 )〜（124 )子影像 2 1 ) S Ο Μ類神經網路 （2 2 )特徵圖 【較佳具體實施例之詳細説明】 有關本發明之多層次臉部影像辨識方法及系統之一較 佳貫施例’請先參照第1圖所示之系統流程圖，其中，系 統知入^的資料是以攝影機拍攝參與實驗者的臉部影像， 以輸入臉部原始影像η (步騾SU )。步驟S12則進行前 本紙張尺度適用中國國家標準（CNS)A4規格（210 X 297公釐） ------------裝--------訂--------- (請先閲讀背面之注意事項再填寫本頁) 477959 經濟部智慧財產局員工消費合作社印製 A7 五、發明說明（夕） 置處理，以將原始影像丨丨裁剪成僅包含完整的臉部影像 1 2 ’並將影像調整為例如丨2 8 X丨2 8的影像。 於步騾S 1 3中，係將該臉部影像丨2分解成至少二種解 析度，每種解析度個有複數個頻道，於本較佳實施例中， 係將該臉部影像12分解成三種解析度，每種解析度個有四 個頻道，因此，共可分解成12張子影像1〇1〜1〇4， 111〜114及 121 〜124 〇 前述步騾S 13對臉部影像丨2之分解係以正交鏡像濾通 焱（Quadrature Mirror Fillter，QMF )進行，該濾通 器為採用微波分解法（wavelet decomp osition)，具可 分離特性而且可完美重組影像。第2圖表示正交鏡像濾通 裔刀解的頻率平面分割圖。第3圖則表示正交鏡像濾通器 之分解過程。 第4圖顯7F在本發明之系統中係以正交鏡像滤通器將 影像分解成三種解析度，每種解析度均有四個頻道。而以 表示這1 2個子影像，下標表示原始影像經i 次分解所得之子影像，每次分解時，只對高一層解析度的 低頻子影像，再做分解。因為辽,_1影像包含這一層解 析度的低頻與高頻資料，而册· !，机影像只剩餘邊緣 線及特別突出的亮點。當ζ·=厂2,3時，子影像之解析度分 別為64 X 64、32><32及16)< 16。第5圖即顯示前述三層 解析度、四個頻道的正交鏡像濾通器之分解情況。 再凊參照第1圖及第6圖所示，在學習階段時，每人以 係以一張常態表情的正面臉部影像為系統的學習影像，而 本紙張尺度適用中國國家標準（CNS)A4規格（210 X 297公釐） I I I I I --------^» — — — — — 1 — (請先閲讀背面之注意事項再填寫本頁) 477959 A7 ____Β7____ 五、發明說明（6) 將學習影像分解後之子影像1 〇 1〜丨〇4，η卜丨丨4及 121〜124分別輸入至12個自我構圖（Self-〇rganizing (請先閲讀背面之注意事項再填寫本頁)477959 Printed by the Consumer Cooperative of the Intellectual Property Bureau of the Ministry of Economic Affairs A7 V. Invention Description (2, The purpose of the present invention is to provide a multi-level face image recognition method and system to reduce the amount of data comparison required for identification and greatly improve Recognition speed. According to a feature of the present invention, the proposed multi-level facial image recognition method includes the following steps: (A) inputting the original image of the face; (B) performing pre-processing to the original image Cropped to include only the complete face image; (C) Decomposes the face image into ^^ resolutions, each of which has M channels, of which N ^ 2 and 2, decompose the face image into NX Μ Zhang sub-images; (D) During the learning phase, the front facial image of the normal expression is used as the learning image, and the sub-images after the learning image is decomposed are input to NX self-composition neural networks for unsupervised Classify learning, and when the neural network completes a predetermined learning cycle, the child images of the learning image are input again to the M neural networks that have completed learning, so that each class of gods A winning unit is generated through the network; and (E) during the test phase, the test image is decomposed, starting with the lowest resolution M sub-images, and inputted to its corresponding M self-composing neural network to generate M Winning units for decision recognition. By calculating the distance between the M winning units and their corresponding self-composition-like winning units of each learning image on the network, find out the candidates that may be winning = if there are only candidates If there is one person, this person is the winner. The decision-making process is completed here. If there is more than one candidate, these candidates are reserved for a higher level of resolution before making a decision. According to another feature of the present invention, 'Proposed Multi-level facial image recognition system, including image input mechanism, pre-processing mechanism, and decomposition mechanism ------------ install -------- order ------- -. ^^ 1 (Please read the notes on the back before filling out this page) 477959 Printed by the Consumers' Cooperative of the Intellectual Property Bureau of the Ministry of Economic Affairs A7 V. Invention Description (3) and a number of self-composition neural networks, the image input Mechanism Into the original image of the face; the pre-processing mechanism is used to crop the original to include only the complete face image; the decomposition mechanism is to decompose the face image into N resolutions, each resolution has M Channels, of which I -2 and Mg2, decompose the facial image into N × M sub-images; when in the sub-phase, the front facial image with normal expression is used as the learning image, and the learning image is decomposed into the child image Input to NX M self-composing neural networks for unsupervised classification learning, and when the class neural network completes a predetermined learning cycle, the child images of the learning image are again input to the M already completed learning Neural-like network, so that each neural-like network generates a winning unit; in the test phase, the test image is decomposed, starting with the lowest resolution M sub-images and input to its corresponding M self-composition Neural network-like, generating μ winning units for decision recognition. By calculating the M winning units and their corresponding self-compositions of learning images on the neural network The distance of the winning unit to find the candidate that may be the winner. If there is only one candidate, this person is the winner. The decision-making process is completed here. If there is more than one candidate, these candidates are retained for higher Make decisions at one level of resolution. Since the invention is novel in design, can provide industrial use, and does have an increasing effect, it has applied for a patent in accordance with the law. In order to enable your reviewing committee to further understand the structure, characteristics and purpose of the present invention, the drawings and detailed descriptions of the preferred embodiments are attached as follows: This paper size applies the Chinese National Standard (CNS) A4 specification (210 X 297 mm) i— 丨 丨 丨 丨 丨 丨 丨 —install i · 丨 丨 丨 丨 Order- 丨 ------ * (Please read the notes on the back before filling this page) 477959 A7 B7 Ministry of Economy Printed by the Intellectual Property Bureau's Consumer Cooperatives Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Figure 7 Figure 8 Figure 5. Explanation of the invention (4) [Schematic description of the diagram is a multi-level of the invention Architecture diagram of facial image recognition system. This is an orthogonal frequency filter decomposed frequency plane segmentation diagram. It is a schematic diagram of the decomposition process of the orthogonal mirror filter. This is a three-layer resolution orthogonal mirror filter decomposition frequency plane segmentation. Figure 0 is a three-layer resolution orthogonal mirror filter decomposition process audio diagram. " It is a structural diagram of the multi-level facial image recognition system of the present invention in the learning stage. It is the architecture diagram of SOM neural network. It is a structural diagram of the multi-level facial image recognition system of the present invention in the test phase. Drawing number description] 1 1) Face original image (12) Complete face image 101) ~ (104) (111) ~ (114) sub-image 121) ~ (124) sub-image 2 1) S Ο M neural network Road (2 2) Feature Map [Detailed description of the preferred embodiment] For a preferred embodiment of the multi-level facial image recognition method and system of the present invention, please refer to the system flowchart shown in FIG. 1 first. Among them, the system knows that the data of the participant's face is taken by a camera to input the original face image η (step 骡 SU). Step S12 is carried out before this paper size applies the Chinese National Standard (CNS) A4 specification (210 X 297 mm) ------------ installation -------- order ---- ----- (Please read the precautions on the back before filling out this page) 477959 Printed by A7, Consumer Cooperatives, Intellectual Property Bureau, Ministry of Economic Affairs V. Invention Description (Even) Processing to crop the original image The complete facial image 1 2 ′ and adjust the image to, for example, 丨 2 8 X 丨 2 8 image. In step S1 3, the facial image 2 is decomposed into at least two resolutions, each of which has a plurality of channels. In the preferred embodiment, the facial image 12 is decomposed. It has three resolutions, and each resolution has four channels. Therefore, it can be decomposed into 12 sub-images 101 ~ 104, 111 ~ 114, and 121 ~ 124. The aforementioned step 13S 13 pairs of facial images 丨The decomposition of 2 is performed by the Quadrature Mirror Fillter (QMF). The filter uses a wavelet decomp osition, which has separable characteristics and can perfectly reconstruct the image. Figure 2 shows the frequency plane segmentation of the orthogonal image filtering solution. Figure 3 shows the decomposition process of the orthogonal mirror filter. Figure 4 shows that in the system of the present invention, the image is decomposed into three resolutions by orthogonal mirror filters, each resolution having four channels. The sub-images are represented by, and the subscript indicates that the sub-images of the original image are decomposed i times. Each time, only the low-frequency sub-images with a higher resolution are decomposed. Because the Liaoning_1 image contains low-frequency and high-frequency data of this layer of resolution, the book image only has edge lines and particularly prominent bright points. When ζ · = factory 2, 3, the resolutions of the sub-images are 64 × 64, 32 > < 32 and 16) < 16, respectively. Figure 5 shows the decomposition of the aforementioned three-layer, four-channel orthogonal mirror filter. Referring again to Figures 1 and 6, during the learning phase, each person uses a positive facial image based on a normal expression as the system's learning image, and this paper scale applies Chinese National Standard (CNS) A4 Specifications (210 X 297 mm) IIIII -------- ^ »— — — — — 1 — (Please read the notes on the back before filling out this page) 477959 A7 ____ Β7 ____ 5. Description of the invention (6) will Learn the child images after image decomposition 1 〇1 ～ 丨 〇4, η 卜 丨 丨 4 and 121 ～ 124 respectively into 12 self-composition (Self-〇rganizing (Please read the precautions on the back before filling this page)
Map，SOM)類神經網路21以進行無監督式歸類學習 (步騾S 1 4 )，產生特徵圖2 2。 第7圖説明前述S Ο Μ類神經網路2 1之架構，其中，輸 入層用以表示網路的輸入變數，即輸入的臉部影像向量。 因此，共12個經正交鏡像滤通器分解後之子影像，將分別 輸入至1 2個S ΟΜ類神經網路2 1中學習。輸出層用以表示 網路的輸出變數’也就是臉部子影像經S 〇 Μ類神經網路 21映射後於特徵圖22上之位置。聯結層是每個輸出單元 與輸入單元相聯結的加權値所構成的向量。當網路學習完 成之後，輸出單元相鄰近者會具有相似的聯結加權値。 再請參照第1圖及第6圖所示，當s Ο Μ類神經網路2 1 完成預定的學習週期，將每人的學習影像之子影像再一次 分別輸入至1 2個已經完成學習的s ΟΜ類神經網路2 1，以 使每人在每一個S Ο Μ類神經網路2 1均產生一個優勝單元 (步騾S 1 5 )。 經濟部智慧財產局員工消費合作社印製 在測試階段時，請參照第1圖及第8圖所示，係將測試 影像分解，以由最低解析度四個子影像…丨〜丨〇4開始，分 別輸入至其對應的四個S ΟΜ類神經網路2 1，產生四個優 勝單元（步騾S 1 6 )，步驟S 1 7進行決策辨識，其首先計 算這四個網路優勝單元與其相對應S ΟΜ類神經網路2 1上 各學習影像的優勝單元之距離，以距離的倒數做為相似程 度的參考値。保留相似値大於篩選門檻値的候選者，如果 本紙張尺度關家標準（CNS)A4規格（210 X 297^釐) 一 477959Map (SOM) -like neural network 21 to perform unsupervised classification learning (step S 1 4) to generate feature maps 2 2. Figure 7 illustrates the architecture of the aforementioned SOM neural network 21, in which the input layer is used to represent the input variables of the network, that is, the input facial image vectors. Therefore, a total of 12 child images decomposed by the orthogonal mirror filter will be input to 12 S OM neural networks 21 for learning. The output layer is used to represent the output variable of the network, that is, the position of the facial sub-image on the feature map 22 after being mapped by the SOM neural network 21. The connection layer is a vector formed by the weighted unitary between each output unit and the input unit. After the network learning is completed, the neighbors of the output unit will have similar connection weights. Please refer to FIG. 1 and FIG. 6 again, when the s OM neural network 2 1 completes the predetermined learning cycle, the child images of each person's learning image are input again to the 12 completed s OM-type neural network 21, so that each person generates a winning unit at each SOC-type neural network 21 (step S1 5). When printed by the Consumer Cooperative of the Intellectual Property Bureau of the Ministry of Economic Affairs during the test phase, please refer to Figures 1 and 8 to decompose the test image to start with four sub-images with the lowest resolution ... 丨 ~ 丨 〇4, respectively Input to its corresponding four S OM-type neural networks 2 1 to generate four winning units (steps S 1 6), and step S 1 7 performs decision recognition, which first calculates the four network winning units corresponding to them The distance between the winning units of each learning image on the S OM-type neural network 21 is based on the inverse of the distance as a reference for similarity. Candidates who are similar (greater than the screening threshold) are retained. If this paper size is based on the CNS A4 specification (210 X 297 ^ cent)-477959
經濟部智慧財產局員工消費合作社印製 五、發明說明（？） 大於門摇値的候選者僅有一人，則此人就是優勝者，決策 過程在此完成。如果大於門檻値的候選者不只一位，則保 貿這些候選者留待更高一層解析度時再做決策。亦即對較 高解析度之四個子影像11^^4同樣進行步騾Sl6及S1 7 <處理，經處理後如果大於門檻値的候選者僅有一人，則 此人就是優勝者，決策過程即在此完成。反之，則需再對 最鬲解析度之四個子影像1 2 1〜1 24同樣進行步騾s丨6及 S 1 7 <處理，俾以找出優勝者，而達成臉部影像之辨識。 由以上之説明可知，本發明之多層次臉部影像辨識方 法及系統係以應用多解析度分解與類神經網路配合多層決 策方法，達成臉部影像辨識功能。藉由正交鏡像濾通器之 多頻道分解的特性將原始影像分解成四個子影像，每一子 影像：資料量僅為原始影像的四分之一，可降低類神經網 路運算量，提昇學習效率。應用s〇M類神經網路無監督 式歸類學習的特性，不f由人為方式選定及抽取特徵，可 以避免特徵選取不當或不足，應用多層決策方法可降低資 料比對量，大幅提高辨識速度。 一综上所陳，本發明無論就目的、手段及功效，在在均 顯π其迴異於習知技術之特徵，為臉部影像辨識之設計上. 的一大突破，誠為一具產業上利用性、新穎性及進步性之 發明’應符合專射請要件，爰依法提出申請。 本紙張尺度剌中國國家規格⑽x 297q公爱； Μ-----—t--------- (請先閱讀背面之注意事項再填寫本頁)Printed by the Consumer Cooperatives of the Intellectual Property Bureau of the Ministry of Economic Affairs. 5. Description of Invention (?) There is only one candidate who is larger than the door shaker, then this person is the winner, and the decision-making process is completed here. If there is more than one candidate above the threshold, then these candidates will be left to a higher resolution before making a decision. That is, the four sub-images 11 ^^ 4 of higher resolution are also processed in steps S16 and S1 7 < after processing, if there is only one candidate that is larger than the threshold 値, then this person is the winner, and the decision-making process That's it. On the other hand, the four sub-images 1 2 1 to 1 24 with the highest resolution need to be processed in the same way as steps s 6 and S 1 7 < to find the winner and achieve the recognition of the facial image. From the above description, it can be known that the multi-level facial image recognition method and system of the present invention apply multi-resolution decomposition and neural network-like coordination with multi-layer decision-making methods to achieve the facial image recognition function. The original image is decomposed into four sub-images by the characteristics of multi-channel decomposition of the orthogonal mirror filter, each sub-image: the amount of data is only one quarter of the original image, which can reduce the amount of neural network operations and improve Learning efficiency. Applying the characteristics of unsupervised classification learning of SOM-type neural networks, artificial selection and extraction of features can avoid improper or insufficient feature selection. Application of multi-layer decision-making methods can reduce the amount of data comparison and greatly improve the recognition speed. . In summary, the present invention, no matter in terms of purpose, means and effect, shows π its characteristics that are different from the conventional technology, and is a design for facial image recognition. A major breakthrough, sincerely an industry Inventions that are usable, novel, and progressive should meet the requirements for exclusive shooting and apply in accordance with the law. The size of this paper (Chinese national specifications) x 297q public love; Μ -----— t --------- (Please read the precautions on the back before filling this page)