CN110633634A - Face type classification method, system and computer readable storage medium for traditional Chinese medicine constitution - Google Patents
Face type classification method, system and computer readable storage medium for traditional Chinese medicine constitution Download PDFInfo
- Publication number
- CN110633634A CN110633634A CN201910729339.8A CN201910729339A CN110633634A CN 110633634 A CN110633634 A CN 110633634A CN 201910729339 A CN201910729339 A CN 201910729339A CN 110633634 A CN110633634 A CN 110633634A
- Authority
- CN
- China
- Prior art keywords
- face
- chinese medicine
- traditional chinese
- classification
- classified
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 239000003814 drug Substances 0.000 title claims abstract description 91
- 238000000034 method Methods 0.000 title claims abstract description 64
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 92
- 238000012549 training Methods 0.000 claims abstract description 73
- 230000001815 facial effect Effects 0.000 claims abstract description 31
- 238000010801 machine learning Methods 0.000 claims abstract description 30
- 238000013145 classification model Methods 0.000 claims abstract description 27
- 238000003066 decision tree Methods 0.000 claims description 24
- 238000000605 extraction Methods 0.000 claims description 12
- 238000007637 random forest analysis Methods 0.000 claims description 7
- 210000002837 heart atrium Anatomy 0.000 claims description 6
- 210000000216 zygoma Anatomy 0.000 claims description 6
- 238000012706 support-vector machine Methods 0.000 claims description 5
- 238000012360 testing method Methods 0.000 description 11
- 210000001061 forehead Anatomy 0.000 description 10
- 230000006870 function Effects 0.000 description 10
- 230000008569 process Effects 0.000 description 10
- 230000002787 reinforcement Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 7
- 238000012545 processing Methods 0.000 description 7
- 230000011218 segmentation Effects 0.000 description 5
- 239000000654 additive Substances 0.000 description 4
- 230000000996 additive effect Effects 0.000 description 4
- 230000006399 behavior Effects 0.000 description 4
- 230000003993 interaction Effects 0.000 description 4
- 238000013473 artificial intelligence Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000007635 classification algorithm Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000007812 deficiency Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000002372 labelling Methods 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 238000007477 logistic regression Methods 0.000 description 2
- 230000008450 motivation Effects 0.000 description 2
- 238000003058 natural language processing Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000001737 promoting effect Effects 0.000 description 2
- 238000013138 pruning Methods 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 229940079593 drug Drugs 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000001575 pathological effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000010187 selection method Methods 0.000 description 1
- 210000000697 sensory organ Anatomy 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
- 230000009278 visceral effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- Biomedical Technology (AREA)
- Theoretical Computer Science (AREA)
- Medical Informatics (AREA)
- Multimedia (AREA)
- Public Health (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Primary Health Care (AREA)
- Epidemiology (AREA)
- Pathology (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a face type classification method, a face type classification system and a computer-readable storage medium for traditional Chinese medicine constitution, wherein the method comprises the following steps: acquiring a face sample image for training, and constructing a feature library; training by adopting a machine learning algorithm according to the face sample images in the feature library to obtain a face type classification model about the traditional Chinese medicine constitution; acquiring a face image to be classified; classifying the face images to be classified by adopting the face classification model about the traditional Chinese medicine constitution to obtain and display corresponding classification result information; the invention can automatically output the classification result of the facial form of the traditional Chinese medicine constitution according to the facial image data with discrimination; compared with the traditional manual classification mode, the face type classification method for the traditional Chinese medicine constitutions has high automation degree, and effectively saves labor cost; meanwhile, the method is not influenced by artificial subjective factors, and the classification accuracy is high.
Description
Technical Field
The present invention relates to the field of machine learning and image processing technologies, and in particular, to a method, a system, and a computer-readable storage medium for classifying facial forms of constitutions in traditional Chinese medicine.
Background
The existing traditional Chinese medicine technology can preliminarily carry out rough visceral, meridian and pathological location dialectical positioning on the patients by accurately distinguishing the traditional Chinese medicine constitution types of the patients, provides basis for clinical medication and health care, combines the anthropometry, carries out objectification on facial features, and researches on the facial features through the processes of clinical case collection, expert labeling and the like.
Image recognition technology at present is generally divided into face recognition and commodity recognition, and the face recognition is mainly applied to security inspection, identity verification and mobile payment; the commodity identification is mainly applied to the commodity circulation process, in particular to the field of unmanned retail such as unmanned goods shelves and intelligent retail cabinets.
Therefore, in the current artificial intelligence environment, a face classification method for traditional Chinese medicine constitutions based on machine learning is urgently needed to replace the artificial classification method of traditional doctors.
Disclosure of Invention
In order to solve at least one of the above technical problems, the present invention provides a face classification method, system and computer-readable storage medium for constitutions in traditional Chinese medicine.
In order to achieve the above object, the present invention provides a face classification method for constitutions in traditional Chinese medicine, comprising:
acquiring a face sample image for training, and constructing a feature library;
training by adopting a machine learning algorithm according to the face sample images in the feature library to obtain a face type classification model about the traditional Chinese medicine constitution;
acquiring a face image to be classified;
and classifying the face images to be classified by adopting the face classification model about the traditional Chinese medicine constitution to obtain and display corresponding classification result information.
In this scheme, acquire the face sample image of training usefulness to construct the feature library, still include:
acquiring a face sample image, performing feature extraction and stereo matching, and performing binocular vision three-dimensional reconstruction on the face sample image to obtain a face sample model;
receiving a face type class label which is marked on the face sample image and is related to the traditional Chinese medicine constitution;
measuring n key points of the face under the face sample model and
selecting m face features with discriminative power by using a feature selection mode, and establishing a corresponding relation between the m face features and a face type label related to the traditional Chinese medicine constitution to obtain a training sample;
a plurality of training samples are collected and the feature library is constructed.
In this scheme, acquire the face image of treating categorised, still include:
acquiring a face image to be classified, performing feature extraction and stereo matching, and performing binocular vision three-dimensional reconstruction on the face image to obtain a face model;
and acquiring the face features of the face model to be classified.
Further, the method for classifying the face image to be classified by using the face classification model about the traditional Chinese medicine constitution to obtain and display corresponding classification result information further comprises the following steps:
and classifying the face images to be classified by adopting the face classification model related to the traditional Chinese medicine constitution according to the face characteristics to obtain and display corresponding classification result information.
Preferably, the face features are one or more of forehead width, cheekbone width, chin width, forehead length, atrium length, lower court length, eye width, nose length, nose width, nose height, human middle length, lip width, upper lip height, lower lip height, and chin.
Preferably, the machine learning algorithm is one or more of a random forest algorithm, a gradient elevator algorithm, a K-nearest neighbor algorithm, a support vector machine algorithm, an AdaBoost algorithm and a classification decision tree algorithm.
In a second aspect of the present invention, a face classification system for constitutions in traditional Chinese medicine is further provided, which includes: a memory and a processor, wherein the memory includes a face type classification method program related to the physique of traditional Chinese medicine, and the face type classification method program related to the physique of traditional Chinese medicine realizes the following steps when being executed by the processor:
acquiring a face sample image for training, and constructing a feature library;
training by adopting a machine learning algorithm according to the face sample images in the feature library to obtain a face type classification model about the traditional Chinese medicine constitution;
acquiring a face image to be classified;
and classifying the face images to be classified by adopting the face classification model about the traditional Chinese medicine constitution to obtain and display corresponding classification result information.
In this scheme, acquire the face sample image of training usefulness to construct the feature library, still include:
acquiring a face sample image, performing feature extraction and stereo matching, and performing binocular vision three-dimensional reconstruction on the face sample image to obtain a face sample model;
receiving a face type class label which is marked on the face sample image and is related to the traditional Chinese medicine constitution;
measuring n key points of the face under the face sample model and
selecting m face features with discriminative power by using a feature selection mode, and establishing a corresponding relation between the m face features and a face type label related to the traditional Chinese medicine constitution to obtain a training sample;
a plurality of training samples are collected and the feature library is constructed.
In this scheme, acquire the face image of treating categorised, still include:
acquiring a face image to be classified, performing feature extraction and stereo matching, and performing binocular vision three-dimensional reconstruction on the face image to obtain a face model;
and acquiring the face features of the face model to be classified.
In a third aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a program of a face classification method for constitutions in traditional Chinese medicine, and the program of the face classification method for constitutions in traditional Chinese medicine is executed by a processor to implement the above steps of the face classification method for constitutions in traditional Chinese medicine.
The method comprises the steps of obtaining a face sample image for training and constructing a feature library; training by adopting a machine learning algorithm according to the face sample images in the feature library to obtain a face type classification model about the traditional Chinese medicine constitution; acquiring a face image to be classified; and classifying the face images to be classified by adopting the face classification model about the traditional Chinese medicine constitution to obtain and display corresponding classification result information. The invention can automatically output the classification result of the facial form of the traditional Chinese medicine constitution according to the facial image data with discrimination. Compared with the traditional artificial (traditional Chinese medicine expert) classification mode, the facial type classification method for the traditional Chinese medicine constitutions has high automation degree, and effectively saves labor cost; meanwhile, the method is not influenced by artificial subjective factors, and the classification accuracy is high.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
FIG. 1 is a flow chart of a method for classifying facial forms of constitutions in accordance with the present invention;
FIG. 2 is a diagram illustrating the classification principle of the SVM-based algorithm according to the present invention;
FIG. 3 illustrates a classification diagram based on a decision tree algorithm of the present invention;
FIG. 4 is a flow chart of the method for constructing the feature library in the face classification process of the constitutions in traditional Chinese medicine;
FIG. 5 is a block diagram of a face classification system according to the present invention;
FIG. 6 is a flow chart of a method for classifying facial forms of constitutions in traditional Chinese medicine according to an embodiment of the present invention;
FIG. 7 is a diagram illustrating five key points of a face according to an embodiment of the present invention;
fig. 8 is a schematic diagram illustrating a face feature according to an embodiment of the present invention.
Detailed Description
In order that the above objects, features and advantages of the present invention can be more clearly understood, a more particular description of the invention will be rendered by reference to the appended drawings. It should be noted that the embodiments and features of the embodiments of the present application may be combined with each other without conflict.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, however, the present invention may be practiced in other ways than those specifically described herein, and therefore the scope of the present invention is not limited by the specific embodiments disclosed below.
FIG. 1 is a flow chart of a method for classifying facial forms of constitutions in accordance with the present invention.
As shown in fig. 1, a first aspect of the present invention provides a method for classifying facial forms of constitutions in traditional Chinese medicine, comprising:
s102, acquiring a face sample image for training, and constructing a feature library;
s104, training by adopting a machine learning algorithm according to the face sample images in the feature library to obtain a face classification model about the traditional Chinese medicine constitution;
s106, obtaining a face image to be classified;
and S108, classifying the face images to be classified by adopting the face type classification model related to the traditional Chinese medicine constitution to obtain and display corresponding classification result information.
It should be noted that the technical solution of the present invention can be operated in a terminal device such as a PC, a mobile phone, a PAD, and the like.
It should be noted that machine learning (machine learning) is a multi-domain cross discipline, and relates to multiple disciplines such as probability theory, statistics, algorithm complexity, and the like. It is specialized to study how computers simulate or implement human learning behavior, and it is able to discover and mine the potential value contained in the data. Machine learning has become a branch of artificial intelligence, and potential rules of data are discovered and mined through a self-learning algorithm, so that unknown data are predicted. Machine learning has been widely used in the fields of computer science research, natural language processing, machine vision, speech, games, and the like. The methods of machine learning are mainly divided into three types: supervised learning, unsupervised learning, and reinforcement learning.
The training data in supervised learning is labeled with classes. Supervised learning builds a model by using training data with class targets, and unknown data can be predicted by the trained model. For example, the machine learning algorithm used for handwritten number recognition belongs to supervised learning, and before training a model, a number table represented by the picture needs to be defined so that a computer can extract better feature similarity marks from data. Supervised learning can be divided into classification and regression.
Reinforcement learning is the process of building a system to improve the performance of the system during interaction with the environment.
The current state information of the environment may comprise a feedback signal by which the current state information may be compared to the current state information
The system performs an evaluation to improve the system. Through interaction with the environment, the system can obtain a series of behaviors through reinforcement learning, and forward feedback is maximized through the design of an incentive system. Reinforcement learning is often used in the field of games, such as go games, where the system determines the next step according to the current game status on the board, and the win or loss at the end of the game is used as the motivation signal.
Unsupervised learning deals with class labels or general trend ambiguity of data, and the unsupervised learning can search for potential regularity in the data without knowing the class labels and outputting scalar quantities and without feedback signals. Unsupervised learning can be divided into clustering and dimensionality reduction.
It should be noted that the machine learning algorithm used in the present invention may be one or more of a random forest algorithm, a gradient elevator algorithm, a K-nearest neighbor algorithm, a support vector machine algorithm, an AdaBoost algorithm, and a classification decision tree algorithm. But is not limited thereto.
Random forest is a supervised learning method, which combines several decision trees to obtain more accurate and stable prediction. Inputting: training a data set, and the number T of sample subsets;
and (3) outputting: and (5) a final strong classifier. Randomly extracting m sample points from an original sample set to obtain a training set; a CART decision tree is trained by using a training set, wherein in the training process, a segmentation rule for each node is to randomly select k features from all the features and then select an optimal segmentation point from the k features to divide left and right subtrees. If the leaf node is a classification algorithm, the predicted final class is the class with the largest vote number in the leaf node where the sample point is located; in the case of a regression algorithm, the final class is the mean of the leaf nodes to which the sample point belongs.
The gradient hoist algorithm (lightGBM) is a gradient Boosting framework that uses decision trees based on learning algorithms. It contains two key points: light is a lightweight class, and GBM is a gradient hoist. The gradient elevator algorithm, like other Boosting algorithms, integrates a better performing model by combining several models (usually a fixed depth decision tree) that perform generally together. AdaBoost locates the deficiency of the model by raising the weight of the misclassified data points, Gradient Boosting identifies problems by negative gradients, and the model is improved by calculating the negative gradients, i.e. by repeatedly selecting a function pointing in the direction of the negative gradients, the algorithm can be regarded as optimizing the objective function in the function space. The GBDT needs to traverse the entire training data multiple times at each iteration. If the whole training data is loaded into the memory, the size of the training data is limited; repeated reading and writing of training data can consume significant time if not loaded into memory. Especially, in the case of industrial massive data, the ordinary GBDT algorithm cannot meet the requirements. The main reason for the proposal of LightGBM is to solve the problems encountered by GBDT in massive data, making GBDT better and faster to use in industrial practice. LightGBM has the advantages of faster training efficiency, low memory usage, better accuracy and capability of processing large-scale data.
The basic idea of the K-Nearest Neighbor (KNN) algorithm is that if most of the K most similar (i.e., Nearest neighbors in feature space) instances in feature space belong to a certain class, then the instance also belongs to this class. When classification prediction is performed on KNN, a majority voting method is generally selected, namely K samples in a training set, which are closest to the predicted sample characteristics, are predicted to be the classes with the largest number of classes in the K samples. The basic algorithm flow is to calculate the distance from the unknown point to all the known classification points, sort according to the distance, select k points which are closest to the unknown point, and count the classification which has the highest frequency of occurrence as the unknown point. The KNN algorithm has the advantages of simplicity, effectiveness and easy understanding, but has large and time-consuming memory consumption.
As shown in FIG. 2, the SVM is a discriminative classifier defined by a classification hyperplane. That is, given a set of labeled training samples, the algorithm will output an optimal hyperplane to classify the new samples (test samples). It is a hyperplane that is found to be the largest distance from the training sample that is closest to him. I.e. optimal segmentation hyperplane maximization training sample boundary. The above is for a linearly separable data set. For a nonlinear data set, a kernel function conversion space is needed to have nonlinear data processing capability.
AdaBoost is a typical Boosting algorithm, belonging to a member of the Boosting family. The Boosting algorithm is a process of promoting a "weak learning algorithm" to a "strong learning algorithm". In general, it is relatively easy to find a weak learning algorithm, and then a series of weak classifiers is obtained by repeated learning, and a strong classifier is obtained by combining the weak classifiers. The Boosting algorithm involves two parts, an additive model and a forward stepwise algorithm. The additive model means that the strong classifier is formed by linearly adding a series of weak classifiers.
AdaBoost is the Boosting algorithm where the loss function is exponential loss. AdaBoost changes the weight of the training data, namely the probability distribution of the samples, and the idea is to put the attention point on the samples which are classified wrongly, reduce the weight of the samples which are classified correctly in the previous round, and improve the weight of the samples which are classified wrongly.
Then, learning is performed according to some basic machine learning algorithms, such as logistic regression.
The core idea of the classification decision tree is to find an optimal feature in a data set, then find an optimal candidate value from the selected values of the feature, divide the data set into two sub-data sets according to the optimal candidate value, and then recurse the above operations until the specified conditions are satisfied. Decision trees typically have three steps: feature selection, decision tree generation and decision tree pruning. Classification with decision trees: starting from a root node, testing a certain characteristic of the example, distributing the example to child nodes according to a test result, wherein each child node corresponds to a value of the characteristic, recursively testing and distributing the example until a leaf node is reached, and finally distributing the example to the class of the leaf node. FIG. 3 is a schematic diagram of a decision tree in which dots represent internal nodes and boxes represent leaf nodes.
FIG. 4 is a flow chart of the method for constructing the feature library in the face classification process of the constitutions in traditional Chinese medicine.
As shown in fig. 4, acquiring a face sample image for training, and constructing a feature library, further includes:
s402, obtaining a face sample image, carrying out feature extraction and stereo matching, and carrying out stereo matching on the face sample
Carrying out binocular vision three-dimensional reconstruction on the image to obtain a human face sample model;
s404, receiving a face type class label which is labeled on the face sample image and is related to the traditional Chinese medicine constitution;
s406, measuring n key points of the face from the face sample model and
selecting m face features with discriminative power by using a feature selection mode, and establishing a corresponding relation between the m face features and a face type label related to the traditional Chinese medicine constitution to obtain a training sample;
s408, collecting a plurality of training samples and constructing the feature library.
According to the embodiment of the invention, the obtaining of the face image to be classified further comprises:
acquiring a face image to be classified, performing feature extraction and stereo matching, and performing binocular vision three-dimensional reconstruction on the face image to obtain a face model;
and acquiring the face features of the face model to be classified.
According to the embodiment of the present invention, the method for classifying face images to be classified by using the facial form classification model related to the traditional Chinese medicine constitution to obtain and display corresponding classification result information further includes:
and classifying the face images to be classified by adopting the face classification model related to the traditional Chinese medicine constitution according to the face characteristics to obtain and display corresponding classification result information.
The face features are one or more of forehead width, cheekbone width, chin width, forehead length, atrium length, lower court length, eye width, nose length, nose width, nose height, human middle length, lip width, upper lip height, lower lip height, and chin. But is not limited thereto.
FIG. 5 is a block diagram of a face classification system according to the present invention.
As shown in fig. 5, the second aspect of the present invention further provides a face classification system 5 for constitutions in traditional Chinese medicine, wherein the face classification system 5 for constitutions in traditional Chinese medicine comprises: a memory 51 and a processor 52, wherein the memory 51 comprises a face classification method program related to the physique of traditional Chinese medicine, and the face classification method program related to the physique of traditional Chinese medicine realizes the following steps when being executed by the processor 52:
acquiring a face sample image for training, and constructing a feature library;
training by adopting a machine learning algorithm according to the face sample images in the feature library to obtain a face type classification model about the traditional Chinese medicine constitution;
acquiring a face image to be classified;
and classifying the face images to be classified by adopting the face classification model about the traditional Chinese medicine constitution to obtain and display corresponding classification result information.
It should be noted that the system of the present invention can be operated in a terminal device such as a PC, a mobile phone, a PAD, etc.
It should be noted that the Processor may be a Central Processing Unit (CPU), other general-purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, a discrete hardware component, and so on. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
It should be noted that the system may further include a display, and the classification result information is fed back to the user through the display. The display may also be referred to as a display screen or display unit. In some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an Organic Light-emitting diode (OLED) touch panel, or the like. The display is used for displaying information processed in the system and for displaying a visual work interface.
It should be noted that machine learning (machine learning) is a multi-domain cross discipline, and relates to multiple disciplines such as probability theory, statistics, algorithm complexity, and the like. It is specialized to study how computers simulate or implement human learning behavior, and it is able to discover and mine the potential value contained in the data. Machine learning has become a branch of artificial intelligence, and potential rules of data are discovered and mined through a self-learning algorithm, so that unknown data are predicted. Machine learning has been widely used in the fields of computer science research, natural language processing, machine vision, speech, games, and the like. The methods of machine learning are mainly divided into three types: supervised learning, unsupervised learning, and reinforcement learning.
The training data in supervised learning is labeled with classes. Supervised learning builds a model by using training data with class targets, and unknown data can be predicted by the trained model. For example, the machine learning algorithm used for handwritten number recognition belongs to supervised learning, and before training a model, a number table represented by the picture needs to be defined so that a computer can extract better feature similarity marks from data. Supervised learning can be divided into classification and regression.
Reinforcement learning is the process of building a system to improve the performance of the system during interaction with the environment.
The current state information of the environment may include a feedback signal by which the current system may be evaluated to improve the system. Through interaction with the environment, the system can obtain a series of behaviors through reinforcement learning, and forward feedback is maximized through the design of an incentive system. Reinforcement learning is often used in the field of games, such as go games, where the system determines the next step according to the current game status on the board, and the win or loss at the end of the game is used as the motivation signal.
Unsupervised learning deals with the problem that there is no class label or the general trend of data is not clear, and unsupervised learning is achieved by unsupervised learning
Learning can look for these unknown classes and output scalars without feedback signals
Underlying regularities in the data. Unsupervised learning can be divided into clustering and dimensionality reduction.
It should be noted that the machine learning algorithm used in the present invention may be one or more of a random forest algorithm, a gradient elevator algorithm, a K-nearest neighbor algorithm, a support vector machine algorithm, an AdaBoost algorithm, and a classification decision tree algorithm. But is not limited thereto.
Random forest is a supervised learning method, which combines several decision trees to obtain more accurate and stable prediction. Inputting: training a data set, and the number T of sample subsets;
and (3) outputting: and (5) a final strong classifier. Randomly extracting m sample points from an original sample set to obtain a training set; a CART decision tree is trained by using a training set, wherein in the training process, a segmentation rule for each node is to randomly select k features from all the features and then select an optimal segmentation point from the k features to divide left and right subtrees. If the leaf node is a classification algorithm, the predicted final class is the class with the largest vote number in the leaf node where the sample point is located; in the case of a regression algorithm, the final class is the mean of the leaf nodes to which the sample point belongs.
The gradient hoist algorithm (lightGBM) is a gradient Boosting framework that uses decision trees based on learning algorithms. It contains two key points: light is a lightweight class, and GBM is a gradient hoist. The gradient elevator algorithm, like other Boosting algorithms, integrates a better performing model by combining several models (usually a fixed depth decision tree) that perform generally together. AdaBoost locates the deficiency of the model by raising the weight of the misclassified data points, Gradient Boosting identifies problems by negative gradients, and the model is improved by calculating the negative gradients, i.e. by repeatedly selecting a function pointing in the direction of the negative gradients, the algorithm can be regarded as optimizing the objective function in the function space. The GBDT needs to traverse the entire training data multiple times at each iteration. If the whole training data is loaded into the memory, the size of the training data is limited; repeated reading and writing of training data can consume significant time if not loaded into memory. Especially, in the case of industrial massive data, the ordinary GBDT algorithm cannot meet the requirements. The main reason for the proposal of LightGBM is to solve the problems encountered by GBDT in massive data, making GBDT better and faster to use in industrial practice. LightGBM has the advantages of faster training efficiency, low memory usage, better accuracy and capability of processing large-scale data. The basic idea of the K-Nearest Neighbor (KNN) algorithm is that if most of the K most similar (i.e., Nearest neighbors in feature space) instances in feature space belong to a certain class, then the instance also belongs to this class. When classification prediction is performed on KNN, a majority voting method is generally selected, namely K samples in a training set, which are closest to the predicted sample characteristics, are predicted to be the classes with the largest number of classes in the K samples. The basic algorithm flow is to calculate the distance from the unknown point to all the known classification points, sort according to the distance, select k points which are closest to the unknown point, and count the classification which has the highest frequency of occurrence as the unknown point. The KNN algorithm has the advantages of simplicity, effectiveness and easy understanding, but has large and time-consuming memory consumption.
An SVM is a discriminative classifier defined by a classification hyperplane. That is, given a set of labeled training samples, the algorithm will output an optimal hyperplane to classify the new samples (test samples).
It is a hyperplane that is found to be the largest distance from the training sample that is closest to him. I.e. the optimal cut hyperplane maximizes the training sample boundary. The above is for a linearly separable data set. For a nonlinear data set, a kernel function conversion space is needed to have nonlinear data processing capability.
AdaBoost is a typical Boosting algorithm, belonging to a member of the Boosting family. The Boosting algorithm is a process of promoting a "weak learning algorithm" to a "strong learning algorithm". In general, it is relatively easy to find a weak learning algorithm, and then a series of weak classifiers is obtained by repeated learning, and a strong classifier is obtained by combining the weak classifiers. The Boosting algorithm involves two parts, an additive model and a forward stepwise algorithm. The additive model means that the strong classifier is formed by linearly adding a series of weak classifiers.
AdaBoost is the Boosting algorithm where the loss function is exponential loss. AdaBoost changes the weight of the training data, namely the probability distribution of the samples, and the idea is to put the attention point on the samples which are classified wrongly, reduce the weight of the samples which are classified correctly in the previous round, and improve the weight of the samples which are classified wrongly.
Then, learning is performed according to some basic machine learning algorithms, such as logistic regression.
The core idea of the classification decision tree is to find an optimal feature in a data set, then find an optimal candidate value from the selected values of the feature, divide the data set into two sub-data sets according to the optimal candidate value, and then recurse the above operations until the specified conditions are satisfied. Decision trees typically have three steps: feature selection, decision tree generation and decision tree pruning. Classification with decision trees: starting from a root node, testing a certain characteristic of the example, distributing the example to child nodes according to a test result, wherein each child node corresponds to a value of the characteristic, recursively testing and distributing the example until a leaf node is reached, and finally distributing the example to the class of the leaf node.
According to the embodiment of the invention, the face sample image for training is obtained, the feature library is constructed, and the method also comprises the steps of
The method comprises the following steps:
acquiring a face sample image, performing feature extraction and stereo matching, and performing binocular vision three-dimensional reconstruction on the face sample image to obtain a face sample model;
receiving a face type class label which is marked on the face sample image and is related to the traditional Chinese medicine constitution;
measuring n key points of the face under the face sample model and
selecting m face features with discriminative power by using a feature selection mode, and establishing a corresponding relation between the m face features and a face type label related to the traditional Chinese medicine constitution to obtain a training sample;
a plurality of training samples are collected and the feature library is constructed.
According to the embodiment of the invention, the obtaining of the face image to be classified further comprises:
acquiring a face image to be classified, performing feature extraction and stereo matching, and performing binocular vision three-dimensional reconstruction on the face image to obtain a face model;
and acquiring the face features of the face model to be classified.
According to the embodiment of the present invention, the method for classifying face images to be classified by using the facial form classification model related to the traditional Chinese medicine constitution to obtain and display corresponding classification result information further includes:
and classifying the face images to be classified by adopting the face classification model related to the traditional Chinese medicine constitution according to the face characteristics to obtain and display corresponding classification result information.
The face features are one or more of forehead width, cheekbone width, chin width, forehead length, atrium length, lower court length, eye width, nose length, nose width, nose height, human middle length, lip width, upper lip height, lower lip height, and chin. But is not limited thereto.
The third aspect of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium
The medium includes a program of a face classification method for constitutions of traditional Chinese medicine, which when executed by the processor, implements the steps of the face classification method for constitutions of traditional Chinese medicine as described above.
In order to better explain the technical solution of the present invention, the following detailed description will be made by an embodiment.
As shown in fig. 6, the facial form classification method for constitutions in traditional Chinese medicine of this embodiment includes two main stages: an off-line training phase and an on-line testing phase.
An off-line testing stage:
acquiring a face image, calibrating a camera, extracting features, performing stereo matching, performing binocular vision three-dimensional reconstruction on the face, and labeling the face type of the corresponding traditional Chinese medicine constitution of the image by using experienced traditional Chinese medicine;
step two, measuring n key points of the human face from the three-dimensional model, andthe distance is selected by using a feature selection method to select m features with the most discriminating power, and the m features are selected as classification features of a discriminator, for example: selecting 17 characteristics of forehead width, cheekbone width, chin width, forehead length, atrium length, lower court length, eye width, nose length, nose width, nose height, human middle length, lip width, upper lip height, lower lip height, lip height and chin as the human face characteristics, but not limited to the characteristics;
collecting a large number of training samples of the facial features and facial types of the traditional Chinese medicine constitutions, and constructing a feature library;
training on a training library by using a supervised learning algorithm to obtain a face classifier (a face classification model for the traditional Chinese medicine constitution);
it should be noted that the supervised learning algorithm may be any one of a k-nearest neighbor algorithm, a support vector machine algorithm, an AdaBoost algorithm, and a random forest algorithm.
And (3) an online testing stage:
acquiring a face image, calibrating a camera, extracting features, performing stereo matching, and performing binocular vision three-dimensional reconstruction on the face;
acquiring feature information of five sense organs under the three-dimensional face;
and step three, the human face features are sent to a trained face classifier related to the traditional Chinese medicine constitution to obtain final classification result information.
The facial features mainly include: center point of the eye (2), position of nose tip, position of mouth corner (2); after the five key point positions of the face are obtained, adjusting the connecting line of the two eyes to a horizontal position, and zooming to a fixed length; the method comprises the steps of extracting the outer contour features of a human face by using an image segmentation algorithm, then uniformly sampling n points on a contour line, calculating the distances from the n points to the nose tip, and using the distances as the features of the human face. Five key points of the face are shown as red crosses in fig. 7.
As shown in fig. 8, discriminative feature data such as forehead width, cheekbone width, chin width, forehead length, atrium length, chin length, eye width, nose length, nose width, nose height, midlength, lip width, upper lip height, lower lip height, and chin are obtained on the three-dimensional model.
The invention obtains the characteristics of the facial form of the traditional Chinese medicine constitution through the sensor and automatically learns the facial form classifier of the traditional Chinese medicine constitution by utilizing a machine learning method. When the face type classifying device is used by a user, the corresponding face type classifying result about the traditional Chinese medicine constitution can be obtained only by sending the face type information about the traditional Chinese medicine constitution obtained by the sensor into the classifier.
The method comprises the steps of obtaining a face sample image for training and constructing a feature library; training by adopting a machine learning algorithm according to the face sample images in the feature library to obtain a face type classification model about the traditional Chinese medicine constitution; acquiring a face image to be classified; and classifying the face images to be classified by adopting the face classification model about the traditional Chinese medicine constitution to obtain and display corresponding classification result information. The invention can automatically output the classification result of the facial form of the traditional Chinese medicine constitution according to the facial image data with discrimination. Compared with the traditional artificial (traditional Chinese medicine expert) classification mode, the facial type classification method for the traditional Chinese medicine constitutions has high automation degree, and effectively saves labor cost; meanwhile, the method is not influenced by artificial subjective factors, and the classification accuracy is high.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described device embodiments are merely illustrative, for example, the division of the unit is only a logical functional division, and there may be other division ways in actual implementation, such as: multiple units or components may be combined, or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the coupling, direct coupling or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between the devices or units may be electrical, mechanical or other forms.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units; can be located in one place or distributed on a plurality of network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, all the functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately regarded as one unit, or two or more units may be integrated into one unit; the integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
Those of ordinary skill in the art will understand that: all or part of the steps for realizing the method embodiments can be completed by hardware related to program instructions, the program can be stored in a computer readable storage medium, and the program executes the steps comprising the method embodiments when executed; and the aforementioned storage medium includes: a mobile storage device, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
Alternatively, the integrated unit of the present invention may be stored in a computer-readable storage medium if it is implemented in the form of a software functional module and sold or used as a separate product. Based on such understanding, the technical solutions of the embodiments of the present invention may be essentially implemented or a part contributing to the prior art may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the methods described in the embodiments of the present invention. And the aforementioned storage medium includes: a removable storage device, a ROM, a RAM, a magnetic or optical disk, or various other media that can store program code.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.
Claims (10)
1. A facial form classification method for constitutions in traditional Chinese medicine is characterized by comprising the following steps:
acquiring a face sample image for training, and constructing a feature library;
training by adopting a machine learning algorithm according to the face sample images in the feature library to obtain a face type classification model about the traditional Chinese medicine constitution;
acquiring a face image to be classified;
and classifying the face images to be classified by adopting the face classification model about the traditional Chinese medicine constitution to obtain and display corresponding classification result information.
2. The face classification method according to the constitutions of traditional Chinese medicine, as claimed in claim 1, wherein the face sample images for training are obtained and a feature library is constructed, further comprising:
acquiring a face sample image, performing feature extraction and stereo matching, and performing binocular vision three-dimensional reconstruction on the face sample image to obtain a face sample model;
receiving a face type class label which is marked on the face sample image and is related to the traditional Chinese medicine constitution;
measuring n key points of the face under the face sample model and
selecting m face features with discriminative power by using feature selection mode, and establishing corresponding relation between the m face features and face type labels related to traditional Chinese medicine constitution to obtain a distanceTraining a sample;
a plurality of training samples are collected and the feature library is constructed.
3. The method of claim 1, wherein the facial form is obtained
The classified face image further comprises:
acquiring a face image to be classified, performing feature extraction and stereo matching, and performing binocular vision three-dimensional reconstruction on the face image to obtain a face model;
and acquiring the face features of the face model to be classified.
4. The method of claim 3, wherein the facial classification model for constitutions is used to classify facial images to be classified, and the classification result information is displayed, further comprising:
and classifying the face images to be classified by adopting the face classification model related to the traditional Chinese medicine constitution according to the face characteristics to obtain and display corresponding classification result information.
5. The method for classifying facial shapes according to constitutions of any one of claims 2 to 4, wherein said facial features are one or more of frontal width, cheekbone width, chin width, frontal length, atrium length, lower court length, eye width, nose length, nose width, nose height, human middle length, lip width, upper lip height, lower lip height, and chin.
6. The facial form classification method according to the constitutions of traditional Chinese medicine according to any one of claims 1 to 5, wherein said machine learning algorithm is one or more of random forest algorithm, gradient elevator algorithm, K-nearest neighbor algorithm, support vector machine algorithm, AdaBoost algorithm, and classification decision tree algorithm.
7. A facial form classification system for constitutions in traditional Chinese medicine, comprising:
a memory and a processor, wherein the memory includes a face type classification method program related to the physique of traditional Chinese medicine, and the face type classification method program related to the physique of traditional Chinese medicine realizes the following steps when being executed by the processor:
acquiring a face sample image for training, and constructing a feature library;
training by adopting a machine learning algorithm according to the face sample images in the feature library to obtain a face type classification model about the traditional Chinese medicine constitution;
acquiring a face image to be classified;
and classifying the face images to be classified by adopting the face classification model about the traditional Chinese medicine constitution to obtain and display corresponding classification result information.
8. The system of claim 7, wherein the face classification system for traditional Chinese medicine constitution comprises a face sample image for training and a feature library, and further comprises:
acquiring a face sample image, performing feature extraction and stereo matching, and performing binocular vision three-dimensional reconstruction on the face sample image to obtain a face sample model;
receiving a face type class label which is marked on the face sample image and is related to the traditional Chinese medicine constitution;
measuring n key points of the face under the face sample model and
selecting m face features with discriminative power by using a feature selection mode, and establishing a corresponding relation between the m face features and a face type label related to the traditional Chinese medicine constitution to obtain a training sample;
a plurality of training samples are collected and the feature library is constructed.
9. The system of claim 7, wherein the face image to be classified is obtained, and further comprising:
acquiring a face image to be classified, performing feature extraction and stereo matching, and performing binocular vision three-dimensional reconstruction on the face image to obtain a face model;
and acquiring the face features of the face model to be classified.
10. A computer-readable storage medium, comprising a program for a face classification method for constitutions in traditional chinese medicine, wherein the program for a face classification method for constitutions in traditional chinese medicine is executed by a processor, and wherein the steps of a face classification method for constitutions in traditional chinese medicine according to any one of claims 1 to 6 are performed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910729339.8A CN110633634A (en) | 2019-08-08 | 2019-08-08 | Face type classification method, system and computer readable storage medium for traditional Chinese medicine constitution |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910729339.8A CN110633634A (en) | 2019-08-08 | 2019-08-08 | Face type classification method, system and computer readable storage medium for traditional Chinese medicine constitution |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110633634A true CN110633634A (en) | 2019-12-31 |
Family
ID=68969302
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910729339.8A Pending CN110633634A (en) | 2019-08-08 | 2019-08-08 | Face type classification method, system and computer readable storage medium for traditional Chinese medicine constitution |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110633634A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111340129A (en) * | 2020-03-09 | 2020-06-26 | 南京览笛信息科技有限公司 | Efficient classification method for multi-angle estimation of 3D structure depth network images |
| CN112084956A (en) * | 2020-09-11 | 2020-12-15 | 上海交通大学烟台信息技术研究院 | Special face crowd screening system based on small sample learning prototype network |
| CN112434970A (en) * | 2020-12-12 | 2021-03-02 | 广东电力信息科技有限公司 | Qualification data verification method and device based on intelligent data acquisition |
| CN112597942A (en) * | 2020-12-29 | 2021-04-02 | 深圳市华尊科技股份有限公司 | Face clustering method, electronic equipment and related products |
| CN112786201A (en) * | 2021-01-24 | 2021-05-11 | 武汉东湖大数据交易中心股份有限公司 | Hand form cognition-based health prediction model construction method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101398886A (en) * | 2008-03-17 | 2009-04-01 | 杭州大清智能技术开发有限公司 | Rapid three-dimensional face identification method based on bi-eye passiveness stereo vision |
| CN107169399A (en) * | 2016-08-25 | 2017-09-15 | 北京中医药大学 | A kind of face biological characteristic acquisition device and method |
| CN109657655A (en) * | 2019-01-30 | 2019-04-19 | 吴长汶 | A kind of five-element's face classification method and storage equipment |
| CN113591763A (en) * | 2021-08-09 | 2021-11-02 | 平安科技（深圳）有限公司 | Method and device for classifying and identifying face shape, storage medium and computer equipment |
-
2019
- 2019-08-08 CN CN201910729339.8A patent/CN110633634A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101398886A (en) * | 2008-03-17 | 2009-04-01 | 杭州大清智能技术开发有限公司 | Rapid three-dimensional face identification method based on bi-eye passiveness stereo vision |
| CN107169399A (en) * | 2016-08-25 | 2017-09-15 | 北京中医药大学 | A kind of face biological characteristic acquisition device and method |
| CN109657655A (en) * | 2019-01-30 | 2019-04-19 | 吴长汶 | A kind of five-element's face classification method and storage equipment |
| CN113591763A (en) * | 2021-08-09 | 2021-11-02 | 平安科技（深圳）有限公司 | Method and device for classifying and identifying face shape, storage medium and computer equipment |
Non-Patent Citations (2)
| Title |
|---|
| 周佳立等: "基于双目被动立体视觉的三维人脸重构与识别", 《自动化学报》 * |
| 崔笛: ""中医阴阳五行人体质辨识方案方法研究"", 《中国博士学位论文全文数据库 医药卫生科技辑》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111340129A (en) * | 2020-03-09 | 2020-06-26 | 南京览笛信息科技有限公司 | Efficient classification method for multi-angle estimation of 3D structure depth network images |
| CN112084956A (en) * | 2020-09-11 | 2020-12-15 | 上海交通大学烟台信息技术研究院 | Special face crowd screening system based on small sample learning prototype network |
| CN112434970A (en) * | 2020-12-12 | 2021-03-02 | 广东电力信息科技有限公司 | Qualification data verification method and device based on intelligent data acquisition |
| CN112597942A (en) * | 2020-12-29 | 2021-04-02 | 深圳市华尊科技股份有限公司 | Face clustering method, electronic equipment and related products |
| CN112786201A (en) * | 2021-01-24 | 2021-05-11 | 武汉东湖大数据交易中心股份有限公司 | Hand form cognition-based health prediction model construction method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Wang et al. | Towards complete and accurate iris segmentation using deep multi-task attention network for non-cooperative iris recognition | |
| Yao et al. | Revisiting co-saliency detection: A novel approach based on two-stage multi-view spectral rotation co-clustering | |
| CN110633634A (en) | Face type classification method, system and computer readable storage medium for traditional Chinese medicine constitution | |
| Durand et al. | Weldon: Weakly supervised learning of deep convolutional neural networks | |
| Sharma et al. | Classification in pattern recognition: A review | |
| Galleguillos et al. | Context based object categorization: A critical survey | |
| Batra et al. | Learning class-specific affinities for image labelling | |
| Elguebaly et al. | Simultaneous high-dimensional clustering and feature selection using asymmetric Gaussian mixture models | |
| Marini et al. | Bird species classification based on color features | |
| CN110880007A (en) | Automatic selection method and system for machine learning algorithm | |
| Li et al. | Dating ancient paintings of Mogao Grottoes using deeply learnt visual codes | |
| Wang et al. | S 3 D: Scalable pedestrian detection via score scale surface discrimination | |
| Li et al. | Texture analysis of remote sensing imagery with clustering and Bayesian inference | |
| Almuashi et al. | Siamese convolutional neural network and fusion of the best overlapping blocks for kinship verification | |
| Singh et al. | Apple Disease Classification Built on Deep Learning | |
| CN112561935B (en) | Intelligent classification method, device and equipment for brain images | |
| Subasi | Applications of artificial intelligence in medical imaging | |
| Goundar | Improved deep learning model based on integrated convolutional neural networks and transfer learning for shoeprint image classification | |
| Blandon et al. | An enhanced and interpretable feature representation approach to support shape classification from binary images | |
| Pan et al. | A two-stage shape retrieval (TSR) method with global and local features | |
| Veloso et al. | Learning to annotate clothes in everyday photos: Multi-modal, multi-label, multi-instance approach | |
| Sun et al. | A dynamic programming approach for accurate content-based retrieval of ordinary and nano-scale medical images | |
| Phan et al. | Hierarchical sparse autoencoder using linear regression-based features in clustering for handwritten digit recognition | |
| Paul et al. | A simple interpretable transformer for fine-grained image classification and analysis | |
| Chakraborty et al. | A framework for multi-lingual scene text detection using k-means++ and memetic algorithms |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191231 |