TWI689285B - Facial symmetry detection method and system thereof - Google Patents

Abstract

A facial symmetry detection method includes a detecting step and a determining step, wherein the detecting step includes a pre-processing step, a feature extracting step and a feature selecting step. The pre-processing step captures an image by an image capture device, and pre-processes the image so as to produce a post-processing image. The feature extracting step extracts a plurality of image features from the post-processing image so as to produce an image feature set. The image feature set includes a plurality of feature symmetry indexes and a plurality of blocking similarities. The feature selecting step selects a part of image feature from the image feature set so as to produce a determining feature set and enter the determining feature set into a classifier. The determining step produces a determining result according to the determining feature set. The determining result is divided into an unsymmetrical state and a normal state. Therefore, an accuracy rate of the facial symmetry detection method and a system thereof are increased.

Description

Face symmetry detection method and system

The invention relates to a face stroke detection method and system, in particular to a face stroke detection method and system judged by complex feature symmetry index and complex feature block similarity.

Since the most obvious feature of face stroke is skewed eyes and mouth, the traditional stroke detection method and its system only judge the left-right symmetry of the facial features of the original image. Therefore, the traditional stroke detection method and its system The accuracy is low, which causes patients with facial strokes to miss the best time to see a doctor.

In view of this, it is very important to develop a face stroke detection method and system with high accuracy.

Therefore, the object of the present invention is to provide a face stroke detection method and system, which can be judged by complex feature symmetry index and complex feature block similarity, which can improve the accuracy of face stroke detection method and system Avoid patients with facial stroke delaying medical treatment due to misjudgment.

According to one embodiment of the method aspect of the present invention, a face stroke detection method is provided, which includes a detection step and a judgment step, wherein the detection step includes a pre-processing step, a feature extraction step, and a feature selection step. The pre-processing step is to capture image data through an image capturing device, and perform pre-processing on the image data to obtain a processed image. The feature extraction step is to extract features from the processed image to generate an image feature set. The image feature set includes a complex feature symmetry index and a complex feature block similarity. The feature selection step is to perform feature selection on the image feature set to form a judgment feature set, and input the judgment feature set to the classifier. The judgment step is that the classifier makes a judgment based on the judgment feature set to generate a judgment result, and the judgment result is classified into a stroke state or a normal state.

Thus, the face stroke detection method of the present invention improves the accuracy of the face stroke detection method by inputting the judgment feature set selected from the image feature set to the classifier.

According to the face stroke detection method described in the preceding paragraph, the pre-processing step includes a face detection step, a normalization processing step, a feature point detection step, and a correction processing step. The face detection step is to perform face detection on the image data and capture the inner face image. The normalization processing step is to normalize the inner face image to obtain a normalized inner face image. The feature point detection step is to perform feature point detection on the normalized inner face image to obtain a face image within the feature point, and the face image within the feature point includes a plurality of face feature points. The correction processing step uses at least two of the face feature points to correct the face image in the feature point to obtain the processed image.

According to the face stroke detection method described in the preceding paragraph, the feature point detection is based on a regression tree to obtain the face image within the feature points, and the number of facial feature points can be 60.

According to the face stroke detection method described in the previous paragraph, the complex feature symmetry index includes mouth slope, mouth area ratio, mouth distance ratio, two-eye distance ratio, and two-eye area ratio, and the complex feature block similarity includes Eye color similarity index, eye three-valued similarity index, plural eye Jiabai similarity index, mouth color similarity index, mouth three-valued similarity index, and plural mouth Jiabai similarity index.

According to the face stroke detection method described in the preceding paragraph, the classifier may be a support vector machine, a random forest, or a Bayesian classifier.

According to the face stroke detection method described in the previous paragraph, it further includes a modeling step. The modeling steps include database creation step, pre-training processing step, training feature extraction step and training feature selection step. The database creation step is to establish a stroke detection database, and the stroke detection database may include plural stroke images and plural normal images. The pre-training processing step is to perform pre-training processing on each stroke image or each normal image to form a stroke detection image after processing. The training feature extraction step is to perform training feature extraction on the processed stroke detection image to generate a stroke detection feature set. The stroke detection feature set may include a complex training symmetry index and a complex training block similarity. The training feature selection step is to perform training feature selection on the stroke detection feature set to generate a judgment feature set, and use the judgment feature set to train a classifier.

According to the face stroke detection method described in the previous paragraph, the training feature selection is to use a random generation step-by-step floating algorithm to generate a judgment feature set.

According to one embodiment of the structural aspect of the present invention, a face stroke detection system is provided, which includes an image capture device and a processor, wherein the processing The device includes a pre-processing module, a feature extraction module and a feature selection module. The pre-processing module performs pre-processing on the image data to obtain the processed image. The feature extraction module performs feature extraction on the processed image to generate an image feature set. The feature selection module performs feature selection on the image feature set to form a judgment feature set, and inputs the judgment feature set to the classifier to generate a judgment result.

Thus, the feature selection module of the face stroke detection system of the present invention improves the accuracy of the face stroke detection system by inputting the judgment feature set selected from the image feature set to the classifier.

According to the facial stroke detection system described in the preceding paragraph, it further includes a stroke detection database, and the stroke detection database includes plural stroke images and plural normal images.

According to the face stroke detection system described in the preceding paragraph, the image capturing device may be a camera, and the classifier may be a support vector machine, random forest, or Bayesian classifier.

s100‧‧‧Face stroke detection method

s110‧‧‧detection steps

s111‧‧‧Pre-processing steps

s1111‧‧‧Face detection steps

s1112‧‧‧Regulation processing steps

s1113‧‧‧Feature point detection steps

s1114‧‧‧ Calibration process steps

s112‧‧‧Feature extraction step

s113‧‧‧Feature selection steps

s120‧‧‧judgment step

s130‧‧‧Modeling steps

s131‧‧‧ database creation steps

s132‧‧‧Pre-processing steps

s1321‧‧‧Training face detection steps

314‧‧‧Right mouth block

400‧‧‧ facial stroke detection system

410‧‧‧Image capture device

420‧‧‧ processor

421‧‧‧Pre-processing module

422‧‧‧Feature extraction module

423‧‧‧Feature selection module

424‧‧‧Classifier

430‧‧‧Stroke detection database

P ₂₈ , P ₂₉ , P ₃₃ , P ₃₆ , P ₃₇ , P ₃₈ , P ₃₉ , P ₄₀ , P ₄₁ , P ₄₂ , P ₄₃ , P ₄₄ , P ₄₅ , P ₄₆ , P ₄₇ , P ₄₈ , P ₄₉ , P ₅₀ , P ₅₁ , P ₅₂ , P ₅₃ , P ₅₄ , P ₅₅ , P ₅₆ , P ₅₇ , P ₅₈ , P ₅₉ ‧‧‧ facial feature points

s1322‧‧‧Training normalization processing steps

s1323‧‧‧Training feature detection steps

s1324‧‧‧Training correction process steps

s133‧‧‧Training feature extraction step

s134‧‧‧Training feature selection steps

310a‧‧‧Eye block

310b‧‧‧ mouth block

311‧‧‧Left eye block

312‧‧‧Right eye block

313‧‧‧ Left mouth block

f ₁ ‧‧‧ first center point

f ₂ ‧‧‧second center point

f ₃ ‧‧‧ third center point

roi _LE ‧‧‧ Starting point of left eye block

roi _RE ‧‧‧ Starting point of right eye block

roi _LM ‧‧‧ Starting point of left mouth block

roi _RM ‧‧‧ Starting point of the right mouth block

P _LE ‧‧‧ Reference point of the left eye block

P _RE ‧‧‧Right-eye block reference point

M ‧‧‧ vertical line

FIG. 1 shows a flowchart of steps of a face stroke detection method according to an embodiment of a method aspect of the present invention; FIG. 2 shows steps of processing steps before a face stroke detection method according to the embodiment of FIG. 1 Flowchart; FIG. 3 shows a schematic diagram of the feature points of the face image within the feature points of the feature point detection step of the processing step before the face stroke detection method according to the embodiment of FIG. 1; FIG. 4 is a schematic diagram of the feature points of the processed image before the processing steps of the face stroke detection method according to the embodiment of FIG. 1; FIG. 5 is a processed image of the face stroke detection method according to the embodiment of FIG. 4 A schematic diagram of the characteristic points of the eye block and the mouth block of FIG. 6 is a flow chart showing the steps of a method for detecting a face stroke according to another embodiment of a method aspect of the present invention; Figure 6 is a flow chart of the pre-training processing steps of the modeling step of the face stroke detection method according to the embodiment of Figure 6; Figure 8 is a block diagram of a face stroke detection system according to one embodiment of a structural aspect of the present invention. Schematic diagram; and FIG. 9 shows a block diagram of a face stroke detection system according to another embodiment of a structural aspect of the present invention.

FIG. 1 shows a flowchart of steps of a facial stroke detection method s100 according to an embodiment of a method aspect of the present invention. It can be seen from FIG. 1 that the facial stroke detection method s100 includes a detection step s110 and a determination step s120.

In detail, the detection step s110 includes a pre-processing step s111, a feature extraction step s112, and a feature selection step s113, wherein the pre-processing step s111 is to capture image data through the image capturing device 410 (labeled in FIG. 8), and The image data is pre-processed to obtain the processed image. The feature extraction step s112 performs feature extraction on the processed image to generate an image feature set. The image feature set includes a complex feature symmetry index and a complex number Feature block similarity, and feature selection step s113 performs feature selection on the image feature set to form a judgment feature set, and inputs the judgment feature set to the classifier 424 (labeled in FIG. 8). The judgment step s120 is that the classifier 424 makes a judgment based on the judgment feature set to generate a judgment result, and the judgment result is classified into a stroke state or a normal state. Therefore, by inputting the judgment feature set formed in the feature selection step s113 to the classifier 424, the accuracy of the facial stroke detection method s100 is improved.

FIG. 2 shows a flowchart of the process step s111 before the face stroke detection method s100 according to the embodiment of FIG. 1, FIG. 3 shows a process step s111 before the face stroke detection method s100 according to the embodiment of FIG. Feature point detection step s1113 is a schematic diagram of the feature points of the face image within the feature points. FIG. 4 is a schematic diagram of the feature points of the processed image before the process step s111 before the face stroke detection method s100 according to the embodiment of FIG. 1. As can be seen from FIG. 2, FIG. 3 and FIG. 4, the pre-processing step s111 may include a face detection step s1111, a normalization processing step s1112, a feature point detection step s1113, and a correction processing step s1114. The face detection step s1111 can perform face detection on the image data to capture the inner face image, where the inner face image can be the face image captured after face detection, and the face detection method can be Histogram of Oriented Gradient (HOG). The normalization processing step s1112 is to normalize the inner face image to obtain the normalized inner face image, and its role is to adjust the size of the inner face image, and the normalization method may be adjacent interpolation (Nearest Neighbor Interpolation). The feature point detection step s1113 is to perform feature point detection on the normalized inner face image to obtain the face image within the feature point. The face image within the feature point includes a plurality of face feature points. The method of feature point detection may use a regression tree-based method (Ensemble of Regression Trees; ERT), and the number of facial feature points can be 60. The correction processing step s1114 uses at least two of the face feature points to correct the face image in the feature point to obtain a processed image, wherein the correction method may be based on the slope of the left eye head and the right eye head in the face feature points To perform correction, that is, facial feature points ( P ₃₉ , P ₄₂ ), first calculate the slope of the head of the two eyes, and then calculate the correction rotation angle of the face image within the feature point according to the slope of the head of the two eyes to correct the face image within the feature point, thus Produce processed images. The calculation of the slope of the eyes of the two eyes complies with equation (1).

為臉部特徵點P ₄₂之y軸座標，

為臉部特徵點P ₃₉之y軸座標，

為臉部特徵點P ₄₂之x軸座標，

為臉部特徵點P ₃₉之x軸座標。校正旋轉角度的計算符合式(2)。 Among them, EyeM is the slope of the eyes of both eyes,

Is the y- axis coordinate of facial feature point P ₄₂ ,

Is the y- axis coordinate of facial feature point P ₃₉ ,

Is the x- axis coordinate of facial feature point P ₄₂ ,

It is the x- axis coordinate of facial feature point P ₃₉ . The calculation of the correction rotation angle complies with equation (2).

表示為臉部特徵點P _i 的x座標，

表示為臉部特徵點P _i 的y座標，其中i可為0-59，在往後段落中將不再贅述。 Where angle is the corrected rotation angle. It must be mentioned that in the subsequent paragraphs,

Expressed as the x coordinate of the facial feature point P _i ,

It is expressed as the y coordinate of the facial feature point P _i , where i can be 0-59, and will not be described in detail in the following paragraphs.

Please refer to Figure 1, Figure 3 and Figure 4, the image feature set generated in the feature extraction step s112 may include complex feature symmetry index and complex feature block similarity, wherein complex feature symmetry index includes mouth Slope, mouth area ratio, mouth distance ratio, two-eye distance ratio, and two-eye area ratio, and the complex feature block similarity may include eye color similarity index, eye ternary similarity index, complex eye Jia Bai Number, mouth color similarity index, mouth ternary similarity index, and plural mouth Jiabai similarity index.

The slope may be a two part nozzle mouth feature point of the face feature point, i.e. the slope face feature point (P _54, P _48), calculating the slope of the mouth according to formula (3).

Among them, MouthM is the slope of the mouth.

The mouth area ratio can be generated by calculating the area of the left mouth and the area of the right mouth, and the area of the left mouth can pass through the facial feature points ( P ₄₈ , P ₄₉ , P ₅₀ , P ₅₁ , P ₅₇ , P ₅₈ , P ₅₉ ), and the area of the right mouth can be calculated through the facial feature points ( P ₅₁ , P ₅₂ , P ₅₃ , P ₅₄ , P ₅₅ , P ₅₆ , P ₅₇ ). The area calculation conforms to equation (4).

Among them, A _LM is the area of the left mouth, and the calculation of the area of the right mouth conforms to equation (5).

Among them, A _RM is the area of the mouth on the right, and the ratio of the area of the mouth conforms to equation (6).

Among them, ratio _MA is the ratio of mouth area.

The mouth distance ratio can be generated by the average distance of the left mouth feature point of the face feature point and the average distance of the right mouth feature point, through the face feature point ( P ₄₉ , P ₅₉ ) and the face characteristic point (P _50, P ₅₈₎ can be drawn from the left side of the mouth the average feature points through the face feature point (P _52, P ₅₆₎ and the face feature point (P _53, P ₅₅₎ can be drawn from the right side of the mouth portion The average distance of the feature points, and the average distance of the feature points of the left mouth comply with equation (7).

Among them, D _LM is the average distance of the left mouth feature points, D ( P ₄₉ , P ₅₉ ) is the Euclid distance of the face feature points ( P ₄₉ , P ₅₉ ), and D ( P ₅₀ , P ₅₈ ) is the face The Euclidean distance of the feature points ( P ₅₀ ,P ₅₈ ) of the central part, and the average distance of the feature points of the mouth on the right accord with equation (8).

Among them, D _RM is the average distance of the feature points on the right mouth, D ( P ₅₂ , P ₅₆ ) is the Euclidean distance of the face feature points ( P ₅₂ , P ₅₆ ), and D ( P ₅₃ , P ₅₅ ) is the face The Euclid distance of the characteristic points ( P ₅₃ , P ₅₅ ), and the ratio of the mouth distance conforms to equation (9).

Among them, ratio _MD is the mouth distance ratio.

In order to calculate the ratio of the distance between two eyes , the average distance between the feature points of the left eye ( P ₃₇ , P ₄₁ ) and the feature points of the face ( P ₃₈ , P ₄₀ ) can be obtained, and the feature points of the left eye ( P ₄₃ , P ₄₇ ) and the facial feature points ( P ₄₄ , P ₄₆ ) obtain the average distance of the right eye feature point, thereby generating the ratio of the distance between the two eyes, and the average distance of the left eye feature point conforms to equation (10).

Among them, D _LE is the average distance of the left eye feature points, D ( P ₃₇ , P ₄₁ ) is the Euclidean distance of the face feature points ( P ₃₇ , P ₄₁ ), and D ( P ₃₈ , P ₄₀ ) is the face The Euclidean distance of the feature points ( P ₃₈ , P ₄₀ ) and the average distance of the feature points of the right eye comply with equation (11).

Among them, D _RE is the average distance of the feature points of the right eye, D ( P ₄₃ , P ₄₇ ) is the Euclid distance of the face feature points ( P ₄₃ , P ₄₇ ), and D ( P ₄₄ , P ₄₆ ) is the face characteristic point (P _44, P ₄₆₎ of the Euclidean distance, while the two distance ratio according to formula (12).

Among them, ratio _ED is the ratio of the distance between two eyes.

The ratio of the area of the two eyes can be generated by calculating the area of the left eye and the area of the right eye, and the area of the left eye can pass through the feature points of the face (P ₃₆ , P ₃₇ , P ₃₈ , P ₃₉ , P ₄₀ , P ₄₁ ) For calculation, the area of the right eye can be calculated through facial feature points (P ₄₂ , P ₄₃ , P ₄₄ , P ₄₅ , P ₄₆ , P ₄₇ ), and the calculation of the area of the left eye conforms to equation (13).

Among them, A _LE is the area of the left eye, and the calculation of the area of the right eye conforms to equation (14).

Among them, A _RE is the area of the right eye, and the ratio of the area of the two eyes conforms to equation (15).

Among them, ratio _EA is the ratio of the area of the two eyes.

FIG. 5 is a schematic diagram showing the feature points of the eye block 310a and the mouth block 310b of the processed image according to the face stroke detection method s100 according to the embodiment of FIG. 4, as can be seen from FIG. 5, the processed image can be more Including the image of the eye block 310a and the image of the mouth block 310b, wherein the eye block 310a may include the left eye block 311 and the right eye block 312, and the mouth block 310b may include the left mouth block 313 and Right mouth block 314.

，右眼區塊基準點P _RE 的座標可表示為

，並進而計算左眼區塊起始點roi _LE 及右眼區塊起始點roi _RE 之座標位置，左眼區塊311及右眼區塊312的大小可為35×35，因此左眼區塊起始點roi _LE 的座標可表示為

，右眼區塊起始點roi _RE 的座標可表示為

。 The eye color similarity index, the eye ternary similarity index, and the complex eye Jiabo similarity index of the complex feature block similarity can be calculated from the image of the left eye block 311 and the image of the right eye block 312 , Where the left-eye block 311 can include the left-eye block reference point P _LE and the left-eye block starting point roi _LE , and the right-eye block 312 includes the right-eye block reference point P _RE and the right-eye block starting point roi _RE . In order to obtain the image of the left-eye block 311 and the image of the right-eye block 312, first, the shortest distance between the facial feature point P ₃₉ and the vertical line M can be found by the vertical line M of the facial feature point P ₂₈ the second center point of the shortest distance f ₂ of a center point F ₁ and the facial feature points P ₄₂ and the vertical line M and the reference point P _LE block left x coordinate of the center point may be a first coordinate x ₁ as F reference, the reference point P _RE block right x-coordinate of the center point can be a second coordinate x F ₂ as a reference, and the reference point P _LE left block and the right block of the reference point P _RE y coordinate can face The y coordinate of the feature point P ₂₉ is used as a reference, so the coordinate of the reference point P _LE of the left eye block can be expressed as

, The coordinates of the reference point P _RE of the right eye block can be expressed as

, And then calculate the coordinate positions of the left eye block starting point roi _LE and the right eye block starting point roi _RE . The size of the left eye block 311 and the right eye block 312 can be 35×35, so the left eye area The coordinates of the block start point roi _LE can be expressed as

, The coordinates of the starting point of the right eye block roi _RE can be expressed as

.

，右嘴區塊起始點roi _RM 的座標可表 示為

。 The mouth color similarity index, mouth ternary similarity index and complex mouth Jiabai similarity index of the complex feature block similarity can be calculated from the image of the left mouth block 313 and the image of the right mouth block 314 , The center point of the facial feature point P ₃₃ and the facial feature point P ₅₁ can be the third center point f ₃ , the left mouth block starting point roi _LM and the right mouth block starting point roi _RM are the third center f y coordinates of the point ₃ as a reference, the starting point of the left nozzle block roi _LM x-coordinate of the reference point P x is the face feature point coordinates _50, and the right nozzle block roi _RM starting x-coordinate of the reference point Is the x coordinate of the facial feature point P ₅₂ , the size of the left mouth block 313 and the right mouth block 314 can be 20×20, so the coordinates of the starting point roi _LM of the left mouth block can be expressed as

, The coordinates of the roi _RM starting point of the right mouth block can be expressed as

.

The eye color similarity index is the result of evaluating the similarity index of the image of the left eye block 311 and the image of the right eye block 312, and the mouth color similarity index is the image of the left mouth block 313 and the right mouth The image of block 314 is the result of evaluating the similarity index. The similarity index evaluation conforms to equation (16).

為輸入影像G ₁之平均值、

為輸入影像G ₂之平均值，

為輸入影像G ₁之標準差，

為輸入影像G ₂之標準差，

為共變異數。值得一提的是，在計算相似度指標評估前，需先將二相似度指標評估的輸入影像中之一者進行左右映射後再進行運算。 Among them, G ₁ and G ₂ are the input images evaluated by the second similarity index for similarity comparison. The input images evaluated by the two similarity indexes may be the image of the left-eye block 311 and the image of the right-eye block 312, respectively. Or the images of the left mouth block 313 and the right mouth block 314, respectively, SSIM ( G ₁ , G ₂ ) is the similarity index, C ₁ and C ₂ are constants, C ₁ can be 6.5025, C ₂ can be 58.5225,

Is the average value of the input image G ₁ ,

Is the average value of the input image G ₂ ,

Is the standard deviation of the input image G ₁ ,

Is the standard deviation of the input image G ₂ ,

Is the covariance. It is worth mentioning that before calculating the evaluation of the similarity index, it is necessary to map one of the input images evaluated by the second similarity index to the left and right before performing the calculation.

The three-valued eye similarity index is to perform three-value localization on the image of the left-eye block 311 and the image of the right-eye block 312, respectively. The three-valued left-eye block image and the partially three-valued right-eye block image are evaluated for similarity indicators to generate an eye tri-valued similarity index. The mouth three-valued similarity index is to perform the local three-valued image of the left mouth block 313 and the right mouth block 314 respectively, and then to the left mouth block image and the local three-valued image after the local three-valued After that, the image of the right mouth block is evaluated for the similarity index, so as to generate the ternary similarity index of the mouth. The purpose of local triangulation of the eye block 310a and the mouth block 310b is to reduce the influence caused by different light rays, to effectively reduce noise and enhance texture features, and the method of local triangulation conforms to the formula ( 17) and formula (18).

And

Among them, LTP _R,N ( u,v ) is the result of local three-valued, R,N means that there are N adjacent pixel values on the circle of radius R , n _c means the pixel of the center point ( u,v ) Value, n _i is the pixel value of the i- th neighboring point, t is the threshold value, the threshold value can be 5, so the interval will be in [ n _c - t, n _c + t ], s ( x ) is the neighboring point The result after local trinization. In other words, when n _{i is} greater than n _c + t , then s ( x )=1, and when n _i falls within the interval, then s ( x )=0, when n _{i is} less than n _c - t , then s ( x )=-1.

The eye Jabber similarity index is to convert the image of the left-eye block 311 and the image of the right-eye block 312 into a Jabber filter to generate a complex left-eye texture feature map and a complex right-eye texture feature map, and then The result of the similarity index evaluation on the complex left eye texture feature map and the complex right eye texture feature map, in which the left eye texture feature map enhances the inconspicuous areas in the image of the left eye block 311, while the right eye Texture feature map is to enhance the right eye area The unobvious area in the image of block 312. The mouth Jabber similarity index is to convert the image of the left mouth block 313 and the image of the right mouth block 314 to a Jabber filter to form a complex left mouth texture feature map and a complex right mouth texture feature map, and then The result of the similarity index evaluation on the complex left mouth texture feature map and the complex right mouth texture feature map, where the left mouth texture feature map is to enhance the inconspicuous areas in the image of the left mouth block 313, while the right mouth The texture feature map is to enhance the inconspicuous areas in the image of the right mouth block 314. The conversion of the Jarber filter will be consistent with equation (19).

G _θ,s ( x,y )= ∫∫ φ _s,θ ( x,y ). f ( x,y ) dxdy formula (19).

Where G _θ,s ( x,y ) is the output image after the input image is converted by the Jabe filter, φ _s,θ ( x,y ) is the Jabe filter, s is the scale, s is greater than or equal to 0 and Less than or equal to 4. θ is an angle, θ is greater than or equal to 0 and less than or equal to 7, thereby obtaining 40 different scales and angles of Jarber filters. f ( x,y ) is the input image converted by the Jarber filter, which can be the image of the left-eye block 311, the image of the right-eye block 312, the image of the left-mouth block 313 or the image of the right-mouth block 314 .

Please refer to Table 1. From Table 1, it can be seen that the image feature set can include the mouth slope, the ratio of the area of the two eyes, the ratio of the distance between the two eyes, the color similarity index of the eye, the ternary similarity index of the eye, and the eye Jia Bai Sex index, mouth area ratio, mouth distance ratio, mouth color similarity index, mouth ternary similarity index, and mouth Jiabo similarity index, among which eye Jiabo similarity index and mouth Jiabo The number of similarity indexes can be 40 each, so the number of features in the image feature set can be 89. The feature selection step s113 of the facial stroke detection method s100 may perform feature selection on the image feature set to form The judgment feature set is judged, and the judgment feature set is input to the classifier 424 to generate a judgment result. The classifier 424 may be a support vector machine, random forest, or Bayesian classifier, and different classifiers 424 may have different sets of judgment features.

FIG. 6 is a flowchart showing the steps of a facial stroke detection method s100 according to another embodiment of a method aspect of the present invention. It can be seen from FIG. 6 that the facial stroke detection method s100 includes a modeling step s130, a detection step s110, and a judgment step s120.

Please refer to FIG. 1, FIG. 2, FIG. 3, FIG. 4, and FIG. 5, in the embodiment of FIG. 6, the detection step s110 and the judgment step s120 are the same as those of FIG. 1, FIG. 2, The corresponding steps in Figure 3, Figure 4 and Figure 5 are the same and will not be repeated here. The modeling step s130 includes a database creation step s131, Pre-training processing step s132, training feature extraction step s133 and training feature selection step s134, wherein database creation step s131 is used to create stroke detection database 430 (labeled in FIG. 9), stroke detection database 430 contains complex stroke images And a plurality of normal images, the pre-training processing step s132 performs pre-training processing on each stroke image or each normal image to form a processed stroke detection image, and the training feature extraction step s133 performs training feature extraction on the processed stroke detection image to generate a stroke Detection feature set. The stroke detection feature set includes complex training symmetry index and complex training block similarity. The method of training feature extraction is the same as the feature extraction method, which is not repeated here, so the number of features in the stroke detection feature set can be At 89, the training feature selection step s134 is to perform training feature selection on the stroke detection feature set to generate a judgment feature set, and use the judgment feature set to train the classifier 424.

FIG. 7 shows a flowchart of the pre-training processing step s132 of the modeling step s130 of the face stroke detection method s100 according to the embodiment of FIG. 6. The pre-training processing step s132 may include the training face detection step s1321, training The normalization processing step s1322, the training feature point detection step s1323, and the training correction processing step s1324. The training face detection step s1321 is to perform face detection on stroke images or normal images to capture the training inner face image. The face detection method is the same as the face detection step s1111 in FIG. 2 and will not be repeated here. . The training normalization processing step s1322 performs normalization processing on the training inner face image to obtain the training normalization inner face image, and its function and normalization processing method are the same as those of the normalization processing step s1112 in FIG. 2 and will not be repeated here. Training feature point detection step s1323 is for training The normalized inner face image is subjected to feature point detection to obtain the face image within the training feature point. The face image within the training feature point includes a plurality of face feature points. The feature point detection method and the feature point detection step s1113 in FIG. 2 The same, not repeat them here. The training correction processing step s1324 uses at least two of the facial feature points to correct the face image in the training feature point to obtain the processed stroke detection image. The correction method is the same as the correction processing step s1114 in FIG. 2 and will not be repeated here.

The training feature selection method in the training feature selection step s134 may be a randomly generated progressive floating algorithm. The training feature selection step s134 is to combine a randomly generated step-by-step floating algorithm with the classifier 424 to select the feature set most suitable for the classifier 424 from the stroke detection feature set to form a judgment feature set, and use the judgment feature set to train the classifier 424. In this way, the number of features in the judgment feature set can be reduced to reduce the pre-training processing and the recognition time of the classifier 424. That is to say, the face stroke detection method s100 can be applied to different classifiers 424, and according to the different classifiers 424, different judgment feature sets can be optionally taken out so that the face stroke detection method s100 is applied to different classifications The device 424 can have good accuracy. The random generation step-by-step floating algorithm may include a generation step (Generation), an inclusion step (Inclusion) and an exclusion step (Exclusion).

The generating step is to randomly select k features from the stroke detection feature set as the test feature set, the number of features of the stroke detection feature set is D , and the number of features of the stroke detection feature set may be 89, but the stroke detection feature set is not selected The D - k features are candidate features, and the set formed by the candidate features is the candidate feature set.

The inclusion step is to find the best test features from the D - k candidate features, and incorporate the best test features into the test feature set to form the best test feature set. The best test feature set enables the classifier 424 to have the highest Accuracy, the operation included in the step conforms to equation (20).

Among them, T ⁺ is the best test feature, A is the stroke detection feature set, B _k is the test feature set, k is the dimension, k can be 2-15, α is the candidate feature, and J ( B _k + α ) is The classifier 424 uses the feature set B _k + α to perform the accuracy of judgment. It is worth mentioning that, after executing step into operation, wherein the test operation performed after the step into a set of _{k +1} B for testing the operational characteristics of the step is not performed into the test set B _k plus the best characterized T ⁺ The formed set, and the dimension of the test feature set will increase, that is, B _{k +1} =B _k +T ⁺ , and k = k +1, and perform the exclusion step.

The exclusion step is to find the worst test feature from the test feature set, so that the test feature set discards the worst test feature to form the best test feature set, and the operation of the exclusion step conforms to equation (21).

Where, T ^- is the worst test feature, β is one of the test feature sets, and J ( B _k - β ) is the accuracy when the classifier 424 performs the judgment using the feature set B _k - β . It is worth mentioning that, after executing the step of calculating excluded, when J (B _k ^- T -) is greater than J (B _{k -1),} represents discarded T ^- high degree of accuracy, B _{k -1} = _{^{B k - T -, k =}} k -1, when T ^- T ⁺ simultaneously with the same phase, i.e., representatives in the steps and procedures that identify the characteristics of the T ^- is discarded and characterized ψ = ψ +1, sequentially exclusion step, where J (B _k ^- T -) B _k is set using the feature classifier 424 - T ^- the time of executing the accuracy of determination, J (B _{k -1)} is the set B _{k -1} 424 using the feature classifier accuracy when performing determination, ψ is characteristic of the number of discarded; if J (B _k ^- T -) is smaller than J (B _{k -1),} representative of the feature set B _k ^- T - B _{k -1} is not more than a set of further features Better. If the sum of the dimension k and the number of discarded features ψ is not equal to the number of features D of the stroke detection feature set, that is, k + ψ ≠ D , then the inclusion step is performed, and when the sum of the dimension k and the number of discarded features ψ is equal to the stroke When detecting the feature number D of the feature set, that is, k + ψ = D , B _{k is} the judgment feature set.

Please refer to Table 2. Table 2 shows the accuracy comparison table when the classifier 424 of the face stroke detection method s100 is a support vector machine. The first embodiment is to train the classifier 424 using a randomly generated sequential forward floating algorithm The first comparative example uses a stroke detection feature set as a judgment feature set to train a classifier, and the second comparative example uses a progressive search algorithm to train a classifier. It can be seen from Table 2 that the number of features in the judgment feature set of the first embodiment is 53, and the accuracy can reach 100%, that is, the random stroke generation algorithm is used to train the classifier 424 to make the face stroke detection method s100 have a better The accuracy is high and the number of required features is small, that is to say, the pre-training processing and the recognition time of the classifier 424 of the first embodiment are lower than those of the first and second comparative examples.

Please refer to Table 3, which shows the accuracy comparison table of the face stroke detection method s100 when the classifier 424 is a random forest. The second embodiment is to train the classifier 424 using a randomly generated sequential forward floating algorithm. The third comparative example is to use the stroke detection feature set as a judgment feature set to train a classifier, and the fourth comparative example is to use a progressive search algorithm to train the classifier. It can be seen from Table 3 that the accuracy of the second embodiment is higher than the accuracy of the third comparative example and the fourth comparative example, that is to say, the classifier 424 can be trained with a random generation and sequential forward floating algorithm to have higher accuracy.

Please refer to Table 4. Table 4 shows a comparison table of the accuracy of the face stroke detection method s100 classifier 424 when it is a Bayesian classifier. The third embodiment is to train a classifier using a randomly generated sequential forward floating algorithm 424. The fifth comparative example is to train the classifier using the stroke detection feature set as the judgment feature set, and the sixth comparative example is to train the classifier using a progressive search algorithm. It can be seen from Table 4 that the accuracy of the third embodiment is higher than that of the fifth comparative example, and the third embodiment is the same as the sixth comparative example, but the number of features used in the third embodiment is lower than that of the sixth comparative example To determine the number of features, so the recognition time of the classifier 424 of the third embodiment can be lower than that of the sixth comparative example.

Table 4

FIG. 8 is a block diagram of a facial stroke detection system 400 according to an embodiment of a structural aspect of the present invention. As can be seen from FIG. 8, the face stroke detection system 400 includes an image capturing device 410 and a processor 420, wherein the image capturing device 410 can be used to capture image data, and the processor 420 is electrically connected to the image capturing device 410.

In detail, the processor 420 may include a pre-processing module 421, a feature extraction module 422, a feature selection module 423, and a classifier 424. The pre-processing module 421 performs pre-processing on the image data to obtain processed images. The feature extraction module 422 performs feature extraction on the processed image to generate an image feature set. The feature selection module 423 performs feature selection on the image feature set to form a judgment feature set, and inputs the judgment feature set to the classifier 424 to generate a judgment result. The judgment result may be classified into a stroke state or a normal state. In this way, the facial stroke detection system 400 can have a high judgment accuracy, so as to avoid the miscalculation that leads to the prime time for the treatment of facial stroke patients to delay treatment.

FIG. 9 is a block diagram of a face stroke detection system 400 according to another embodiment of a structural aspect of the present invention. As can be seen from FIG. 9, the facial stroke detection system 400 may include an image capturing device 410, a processor 420, and a stroke detection database 430, where the stroke detection database 430 may include plural stroke images and plural normal images.

Please refer to FIG. 6 and FIG. 8 together. In the embodiment of FIG. 9, the image capturing device 410 and the processor 420 have the same structure as the corresponding ones in FIG. 8 and will not be repeated here. In particular, the pre-processing module 421 of the processor 420 can be used to perform a pre-training processing step s132. The pre-training processing step s132 performs pre-training processing on each stroke image or each normal image to form a post-processing stroke detection image. The feature extraction module 422 of the processor 420 can be used to perform the training feature extraction step s133. The training feature extraction step s133 performs training feature extraction on the processed stroke detection image to generate a stroke detection feature set. The feature selection module 423 of the processor 420 It can be used to perform the training feature selection step s134, which performs training feature selection on the stroke detection feature set to generate a judgment feature set, and uses the judgment feature set to train the classifier 424.

In order to facilitate the user to perform face stroke detection and improve the accuracy of the face stroke detection system 400 at any time, the face stroke detection system 400 can be applied to a computer or a mobile phone, and the image capture device 410 of the face stroke detection system 400 can be The camera and the classifier 424 used by it can be a support vector machine, random forest or Bayesian classifier, by which the user can detect facial strokes at any time and reduce the probability of misjudgment.

In summary, the face stroke detection method and face stroke detection system of the present invention can provide the following advantages:

(1) The accuracy of the face stroke detection method and its system can be improved by the judgment feature set formed by feature selection for the image feature set.

(2) By randomly generating a sequential forward floating algorithm combined with a classifier, a judgment feature set with the smallest number of features can be found, thereby reducing the number of features in the judgment feature set to reduce the recognition time of the classifier.

Although the present invention has been disclosed as above in the embodiments, it is not intended to limit the present invention. Any person who is familiar with this art can make various changes and modifications within the spirit and scope of the present invention, so the protection of the present invention The scope shall be determined by the scope of the attached patent application.

s100: Face stroke detection method

s110: detection step

s111: pre-processing steps

s112: feature extraction step

s113: feature selection step

s120: judgment step

Claims (9)

A face symmetry detection method, including: a detection step, including: a pre-processing step, an image data is captured by an image capturing device, and a pre-processing is performed on the image data to obtain a post-processing Image; a feature extraction step, a feature extraction is performed on the processed image to generate an image feature set, the image feature set includes a complex feature symmetry index and a complex feature block similarity, the complex feature symmetry index includes One mouth slope, one mouth area ratio, one mouth distance ratio, one or two eyes distance ratio and one or two eyes area ratio, the complex feature block similarity includes one eye color similarity index, one eye three value Similarity index, complex eye Jiabo similarity index, one mouth color similarity index, one mouth ternary similarity index, and plural mouth Jiabai similarity index; and a feature selection step The image feature set performs a feature selection to form a judgment feature set, and inputs the judgment feature set to a classifier; and a judgment step, the classifier generates a judgment result based on the judgment feature set, and the judgment result is divided into It is an asymmetrical state or a normal state. The face symmetry detection method as described in item 1 of the patent scope, wherein the pre-processing step includes: a face detection step, which is to perform a face detection on the image data and acquire an inner face image A normalization processing step, which performs the normalization process on the inner face image to obtain a normalized inner face image; a feature point detection step, which performs a feature point detection on the normalized inner face image, To obtain a face image within a feature point, the face image within the feature point including a plurality of face feature points; and a correction processing step, using at least two of the face feature points to correct the face image within the feature point to obtain The processed image. The face symmetry detection method as described in item 2 of the patent application range, wherein the feature point detection is based on a regression tree to obtain the face image within the feature point, and the number of face feature points is 60. The face symmetry detection method as described in item 1 of the patent application scope, wherein the classifier is a support vector machine, a random forest or a Bayesian classifier. The face symmetry detection method as described in item 1 of the patent application scope further includes: A modeling step, including: a database creation step, creating a face symmetry detection database, the face symmetry detection database includes complex face asymmetric images and complex normal images; a pre-training processing step, system Perform a pre-training process on each of the face asymmetric images or each of the normal images to form a processed face symmetry detection image; a training feature extraction step, perform a training on the processed face symmetry detection image Feature extraction to generate a face symmetry detection feature set, which includes complex training symmetry index and complex training block similarity; and a training feature selection step for the face symmetry detection The feature set performs a training feature selection to generate the judgment feature set, and uses the judgment feature set to train the classifier. The face symmetry detection method as described in item 5 of the patent application scope, wherein the training feature selection is to use a random generation step-by-step floating algorithm to generate the judgment feature set. A face symmetry detection system, including: an image capturing device for capturing an image data; and A processor, electrically connected to the image capturing device, and including a classifier, and the processor includes: a pre-processing module that performs a pre-processing on the image data to obtain a processed image; a feature The extraction module performs a feature extraction on the processed image to generate an image feature set. The image feature set includes a complex feature symmetry index and a complex feature block similarity. The complex feature symmetry index includes a mouth Slope, one-mouth area ratio, one-mouth distance ratio, one-two-eye distance ratio and one-two-eye area ratio, the complex feature block similarity includes one-eye color similarity index, one-eye three-valued similarity Index, complex eye Jabber similarity index, one mouth color similarity index, one mouth ternary similarity index, and multiple mouth Jabber similarity index; and a feature selection module for this image feature The collection performs a feature selection to form a judgment feature set, and inputs the judgment feature set to the classifier to generate a judgment result. The face symmetry detection system as described in item 7 of the patent application scope further includes a face symmetry detection database, and the face symmetry detection database includes complex face asymmetric images and complex normal images. A face symmetry detection system as described in item 7 of the patent application, wherein the image capture device is a camera, and the classifier is a support vector machine, a random forest, or a Bayesian classifier.

Images