CN110210448B - Intelligent face skin aging degree identification and evaluation method - Google Patents
Intelligent face skin aging degree identification and evaluation method Download PDFInfo
- Publication number
- CN110210448B CN110210448B CN201910508742.8A CN201910508742A CN110210448B CN 110210448 B CN110210448 B CN 110210448B CN 201910508742 A CN201910508742 A CN 201910508742A CN 110210448 B CN110210448 B CN 110210448B
- Authority
- CN
- China
- Prior art keywords
- wrinkle
- area
- image
- human face
- wrinkles
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/28—Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
Abstract
The invention discloses an intelligent face skin aging identification and evaluation method, which comprises the following steps: and (5) image acquisition and preprocessing. Acquiring a human face image for evaluating the aging degree of human face skin, converting the image from an RGB (red, green and blue) image into a gray image, and performing Gaussian convolution and Hessian matrix algorithm calculation on the gray image to obtain a binary image; and (5) detecting and screening wrinkles. The area with the median of 1 in the binary image is a suspected wrinkle area, whether the suspected wrinkle area is a real wrinkle is judged according to the minimum bounding rectangle and the inclination angles of the long side, the short side and the long side, and a new wrinkle binary image is obtained through processing; and extracting wrinkle features. The area with the median value of 1 in the wrinkle binary image is a real wrinkle area, and the wrinkle characteristics of the human face, including the number of wrinkles, the maximum connected length and the maximum width of the wrinkles, the color depth degree of the wrinkles and the minimum circumscribed rectangular area of the wrinkles, are extracted; and evaluating the aging degree of the human face skin. And obtaining the human face skin aging degree and the human face skin visual aging degree through the human face wrinkle characteristic weighting comprehensive calculation. The invention realizes comprehensive judgment of the human face skin aging degree by rapidly and accurately detecting the human face wrinkle index, and provides a quantitative index.
Description
Technical Field
The invention relates to the field of image processing, in particular to an image processing method for identifying and evaluating the aging degree of human face skin.
Background
With the continuous development of the beauty industry, people pay more attention to the skin quality, and how to detect and quantify the skin indexes of the human face aging degree becomes a key technology. At present, the domestic evaluation of the aging degree of the human face skin is mostly based on the subjective evaluation of clinicians rather than the objective evaluation of computer vision, mainly focuses on evaluating the severity degree of wrinkles, and cannot well and accurately position the position of the facial wrinkles and provide quantitative indexes. In view of the above-mentioned problems, it is necessary to design a method for rapidly and accurately detecting wrinkles on human face skin and analyzing the wrinkles to provide a quantitative index of aging degree.
In the evaluation of the human face on the aging degree, wrinkles account for the main proportion, and the existing patents do not relate to the identification and evaluation method on the human face aging degree, such as CN109299633A proposes a wrinkle detection method based on gray classification, but does not describe a specific wrinkle and aging degree index evaluation method; for example, CN108334836A proposes a skin wrinkle evaluation method and system, but the described method is based on cloud and the adopted method is photometric stereo method, which has high requirement on environment and poor applicability; for example, CN109086688A proposes a method, apparatus, computer device and storage medium for detecting facial wrinkles, which do not extract detailed features such as the number of wrinkles, the maximum connection length of wrinkles, the maximum width, and the color depth.
1) And a wrinkle detection method, system, device and medium, patent No. CN 109299633A. The invention discloses a wrinkle detection method, system, device and medium. Wherein the wrinkle detection method comprises: performing a directional gray-scale filtering process on the acquired face image to highlight a gray scale of pixels extending in a corresponding direction; carrying out gray-level-based classification processing on the face image after the filtering processing to obtain a candidate wrinkle image; the obtained candidate wrinkle image is evaluated to obtain a target wrinkle image. This method mentions a method of extracting a wrinkle image, but it does not describe a specific evaluation method.
2) And "a skin wrinkle evaluation method and system", patent No. CN 108334836A. The invention discloses a method and a system for evaluating skin wrinkles, belonging to the technical field of skin detection: the method comprises the steps that under the irradiation of different light sources, face images related to the same face are shot by an image acquisition device respectively, and a skin detection mirror sends the face images to a cloud server; the cloud server obtains a unit normal vector of each pixel point through photometric stereo processing; the cloud server obtains surface depth information of each pixel point according to unit normal vector processing, and then forms a face three-dimensional image of a face; the cloud server judges the face three-dimensional image by adopting a skin wrinkle evaluation model formed by pre-training so as to obtain and output a corresponding skin wrinkle evaluation result. The method provides a wrinkle detection method based on cloud deep learning, but the adopted method is a photometric stereo method, and is high in environmental requirement and poor in applicability.
3) And "facial wrinkle detection method, apparatus, computer device, and storage medium", patent No. CN 109086688A. The invention relates to a method and a device for detecting facial wrinkles, a computer device and a storage medium. The method comprises the steps of acquiring a face image, marking face characteristic points of the face image, dividing a wrinkle region of the face image according to the face characteristic points of the face image, determining the wrinkle type of the wrinkle region according to the wrinkle type of the wrinkle region, extracting a region image in the wrinkle region, acquiring wrinkle characteristic components of the region image, and performing wrinkle detection according to the wrinkle characteristic components of the region image. According to the method and the device, even if fine wrinkles exist in a certain area of the face or irregular facial features exist in the certain area of the face, accurate detection can be performed, and the accuracy of wrinkle detection is improved.
Disclosure of Invention
In order to solve the above technical problems, the present invention aims to provide an intelligent method for identifying and evaluating the aging degree of human face skin, so as to meet the requirements of rapidly and accurately detecting the aging degree of human face skin and providing quantitative indexes, and improve the measurement efficiency and the applicability of the measurement method.
The purpose of the invention is realized by the following technical scheme:
an intelligent face skin aging degree identification and evaluation method comprises the following steps: collecting and preprocessing images; detecting and screening wrinkles; extracting wrinkle characteristics and evaluating the aging degree of the human face skin; the method specifically comprises the following steps:
a, acquiring a face image for evaluating the aging degree of the face skin, converting the image from an RGB (red, green and blue) image into a gray image, and performing Gaussian convolution and Hessian matrix algorithm calculation on the gray image to obtain a binary image.
And B, determining whether the suspected wrinkle area is a real wrinkle area according to the minimum bounding rectangle of the suspected wrinkle area and the inclination angles of the long side, the short side and the long side of the suspected wrinkle area, and processing to obtain a new wrinkle binary image, wherein the area with the median value of 1 in the binary image is the suspected wrinkle area.
And C, taking the area with the median value of 1 in the wrinkle binary image as a real wrinkle area, and extracting the wrinkle characteristics of the human face, including the number of wrinkles, the maximum connected length and the maximum width of the wrinkles, the color depth degree of the wrinkles and the minimum circumscribed rectangular area.
And D, obtaining the human face skin aging degree and the human face skin visual aging degree by the human face wrinkle feature weighting comprehensive calculation.
One or more embodiments of the present invention may have the following advantages over the prior art: the method is used for identifying and evaluating the human face skin aging degree, has good applicability, improves the measurement precision and the detection efficiency on the premise of ensuring the timeliness, and provides an index for quantifying the human face aging degree.
Drawings
FIG. 1 is a flow chart of a method for identifying and evaluating the degree of skin aging of an intelligent human face;
FIG. 2 is a face skin region extraction gray scale image;
FIG. 3 is a face skin gray scale gradient image;
FIG. 4 is a diagram of a face wrinkle binary image region judgment and recognition effect;
FIG. 5 is a schematic diagram of the results of face wrinkle screening;
fig. 6 is a program framework of the method for identifying and evaluating the skin aging degree of the intelligent human face.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail with reference to the following embodiments and accompanying drawings.
As shown in fig. 1, a workflow of a method for identifying and evaluating the skin aging degree of an intelligent face includes the following steps:
As shown in fig. 6, the program framework of the intelligent human face skin aging degree identification and evaluation method includes the following steps:
collecting a face image I for evaluating the aging degree of the face skin face Converting the image from RGB image to gray image I grey As shown in FIG. 2, the gray gradient map is calculated as shown in FIG. 3, in the gray image I grey Performing Gaussian convolution and Hessian matrix algorithm to calculate to obtain binary image I det As shown in fig. 4, the specific process is as follows:
wherein; h a 、H b And H c Is an image I grey Second derivative output of H a 、H b Is an image I grey And two Gaussian kernels g 1 (σ)、g 2 Convolution of (σ), H c Is H a σ is the scale of the detection kernel generated by the second derivative of the gaussian kernel.
And finding the maximum filter response in the Hessian matrix H (x, y, sigma) of different scales and carrying out binarization processing.
σ min And σ max Is the minimum and maximum scale expected to find the relevant structure from H (x, y, σ), I det The term (x, y) denotes a binary image obtained by binarizing L (x, y).
As shown in the program frame of the intelligent human face skin aging degree identification and evaluation method of fig. 6, the step 20 specifically includes:
FIG. 4 wrinkle region identification binary image I det The area with the median value of 1 is a suspected wrinkle area R det n Manually setting the minimum wrinkle length limit l min Maximum short to long edge ratio r max In a binary image I det The total number of the extracted suspected wrinkle areas is N, and R is respectively det 1 ,R det 2 …R det n …R det N Let the nth suspected wrinkle region R det n Minimum circumscribed rectangle r det n 。
To obtain r det n Long side l of det n Short side b det n And angle of inclination of long side theta det n Only if the length is not less than the wrinkle length limit l min The ratio does not exceed the limit of the short-to-long edge ratio r max Region of suspected wrinkles R det n The pixel value of the area is kept as 1, otherwise, the pixel value of the area is converted into 0. As shown in fig. 4, 401 is a real wrinkle region, 402 is a region with a ratio exceeding the short-to-long edge ratio limit, and 403 is a wrinkle length non-conforming region;
therefore, all the N suspected wrinkle areas R are determined det n Then, a new wrinkle binary image I can be obtained W As shown in fig. 5, 501 for retaining the real wrinkle area, 502 and 503 non-conforming areas are filtered and removed. At this time, the wrinkle binary image I W The area with the upper value of 1 is large and recognized as a wrinkle.
Intelligent face skin aging degree identification and evaluation method as shown in FIG. 6As shown in the program framework, the step 30 specifically includes: wrinkle binary image I W The area with the median value of 1 is the real wrinkle area R W m In a binary image I W The number of the extracted wrinkle areas is M, and R is respectively W 1 ,R W 2 …R W m …R W M If the face is not wrinkled, the face is wrinkled for M times; let the mth real wrinkle region R W m Maximum connected length L of wrinkles W m Maximum width B W m Degree of depth of color g W m (ii) a Minimum circumscribed rectangle is r W m Long side is l W m Short side is b W m Then the real wrinkle area s W m ＝l W m ×b W m ；
Constructing 1 x (4M +1) human face wrinkle feature F:
F＝{M,(L W 1 ,B W 1 ,g W 1 ,s W 1 ),(L W 2 ,B W 2 ,g W 2 ,s W 2 )…(L W M ,B W M ,g W M ,s W M )}
m in the human face wrinkle characteristic F represents the number of all wrinkles on the human face, and the maximum connected length L of the wrinkles W m Maximum width B W m Degree of color depth g W m Is a real wrinkle parameter, characterizing the real degree of wrinkles; minimum circumscribed rectangular area s W n The visually affected area of the wrinkle is characterized.
As shown in the program frame of the intelligent human face skin aging degree identification and evaluation method of fig. 6, the step 40 specifically includes: setting a wrinkle influence weight rho according to the wrinkle characteristic F of the face W Wrinkle depth color influence weight rho g Calculating the aging degree S of the human face skin W ：
Setting wrinkle influence weight rho W And the visual influence weight rho vis Calculating the visual aging degree S of the human skin through the human face wrinkle characteristic F vis ：
Although the embodiments of the present invention have been described above, the above descriptions are only for the convenience of understanding the present invention, and are not intended to limit the present invention. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (4)
1. An intelligent face skin aging identification and evaluation method is characterized by comprising the following steps: collecting and preprocessing an image; detecting and screening wrinkles; extracting wrinkle characteristics and evaluating the aging degree of the human face skin; the method specifically comprises the following steps:
a, collecting a face image I for evaluating the aging degree of the face skin face Converting the image from RGB image to gray image I grey In a gray scale image I grey Performing Gaussian convolution and Hessian matrix algorithm to calculate to obtain binary image I det ；
B binary image I det The area with the median value of 1 is a suspected wrinkle area R detn According to the minimum bounding rectangle r of the suspected wrinkle area detn Based on the long side l of the suspected wrinkle area detn Short side b detn And angle of inclination of long side theta detn Determining the suspected wrinkle area R detn Whether the image is a real wrinkle area or not, processing to obtain a new wrinkle binary image I W ；
C wrinkle binary image I W The area with the median value of 1 is the real wrinkle area R Wm Extracting the human face wrinkle characteristics F including the number M of wrinkles and the maximum connection length L of wrinkles Wm Maximum width B Wm Degree of depth g of wrinkle color Wm Minimum bounding rectangle area of wrinkle s Wm ；
D is obtained by weighting and comprehensively calculating the face wrinkle characteristics FDegree of aging of human face skin S W Human face skin visual aging degree S vis ；
In the step B, the suspected wrinkle area R is judged detn Whether it is a real wrinkle region R Wm The method comprises the following steps:
setting a wrinkle minimum length limit l min Maximum short to long edge ratio r max In a binary image I det The total number of the suspected wrinkle areas extracted above is N, and the number of the suspected wrinkle areas is R det1 ,R det2 …R detn …R detN Let the nth suspected wrinkle area R detn Minimum circumscribed rectangle r detn ；
To obtain r detn Long side l of detn Short side b detn And angle of inclination of long side theta detn Only if the length is not less than the wrinkle length limit l min The ratio does not exceed the limit of the short-to-long edge ratio r max Region of suspected wrinkles R detn If the wrinkles exist, keeping the pixel value of the area as 1, otherwise, converting the pixel value of the area into 0;
therefore, all the N suspected wrinkle areas R are determined detn Then, a new wrinkle binary image I can be obtained W At this time, the wrinkle binary image I W The area with the value of 1 is the real wrinkle.
2. The intelligent human face skin aging identification and evaluation method as claimed in claim 1, wherein the step A is implemented by displaying a gray image I grey Performing Gaussian convolution and Hessian matrix algorithm to calculate to obtain binary image I det The specific process comprises the following steps:
wherein: h a 、H b And H c Is an image I grey Second derivative output of H a 、H b Is an image I grey And two Gaussian nuclei g 1 (σ)、g 2 Convolution of (σ), H c Is H a The sigma is the scale of a detection kernel generated by the second derivative of the Gaussian kernel;
finding the maximum filter response in Hessian matrixes H (x, y, sigma) with different scales and carrying out binarization processing;
wherein σ min And σ max Is the minimum and maximum scale expected to find the relevant structure from H (x, y, σ), I det The term (x, y) denotes a binary image obtained by binarizing L (x, y).
3. The intelligent human face skin aging identification and evaluation method as claimed in claim 1, wherein the wrinkle binary image I in the step C W The area with the median value of 1 is the real wrinkle area R Wm Extracting a human face wrinkle feature F;
in a binary image I W The number of the extracted wrinkle areas is M, and R is respectively W1 ,R W2 …R Wm …R WM If the face is not wrinkled, the face is wrinkled for M times;
let m' th real wrinkle region R Wm Maximum connected length L of wrinkles Wm Maximum width B Wm Degree of depth of color g Wm ；
The minimum circumscribed rectangle is r Wm Long side is l Wm Short side is b Wm Then the area of the real wrinkle region s Wm ＝l Wm ×b Wm ；
Constructing 1 x (4M +1) human face wrinkle feature F:
F＝{M,(L W1 ,B W1 ,g W1 ,s W1 ),(L W2 ,B W2 ,g W2 ,s W2 )…(L WM ,B WM ,g WM ,s WM )} (4)
m in the human face wrinkle characteristic F represents the number of all wrinkles on the human face, and the maximum connected length L of the wrinkles Wm Maximum width B Wm Degree of color depth g Wm The real wrinkle parameter represents the real degree of wrinkles; minimum circumscribed rectangular area s Wm The visually affected area of the wrinkle is characterized.
4. The intelligent human face skin aging identification and evaluation method as claimed in claim 1, wherein in the step D:
calculating the aging degree S of the human face skin through the human face wrinkle characteristics F W The calculation formula of (c) is:
where ρ is W Influencing the weight, rho, for wrinkles g The wrinkle depth color influence weight value;
calculating the visual aging degree S of the human skin through the human face wrinkle characteristic F vis The calculation formula of (c) is:
where ρ is vis Is the visual impact weight.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910508742.8A CN110210448B (en) | 2019-06-13 | 2019-06-13 | Intelligent face skin aging degree identification and evaluation method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910508742.8A CN110210448B (en) | 2019-06-13 | 2019-06-13 | Intelligent face skin aging degree identification and evaluation method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110210448A CN110210448A (en) | 2019-09-06 |
| CN110210448B true CN110210448B (en) | 2022-09-13 |
Family
ID=67792354
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910508742.8A Active CN110210448B (en) | 2019-06-13 | 2019-06-13 | Intelligent face skin aging degree identification and evaluation method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110210448B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112603259B (en) * | 2019-09-18 | 2022-04-19 | 荣耀终端有限公司 | Skin roughness detection method and electronic equipment |
| CN110929681B (en) * | 2019-12-05 | 2023-04-18 | 南京所由所以信息科技有限公司 | Wrinkle detection method |
| CN111009006A (en) * | 2019-12-10 | 2020-04-14 | 广州久邦世纪科技有限公司 | Image processing method based on human face characteristic points |
| CN113128376A (en) * | 2021-04-02 | 2021-07-16 | 西安融智芙科技有限责任公司 | Wrinkle recognition method based on image processing, wrinkle recognition device and terminal equipment |
| CN113160224B (en) * | 2021-05-18 | 2021-11-26 | 中国医学科学院皮肤病医院(中国医学科学院皮肤病研究所) | Artificial intelligence-based skin aging degree identification method, system and device |
| CN116091487B (en) * | 2023-03-07 | 2023-06-23 | 深圳市宝安区石岩人民医院 | Skin wrinkle analysis comparison method and system based on image recognition |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20050039765A (en) * | 2005-02-21 | 2005-04-29 | 주식회사 아이티펌 | Method for extracting face outline and face feature points |
| CN109427053A (en) * | 2017-08-31 | 2019-03-05 | 丽宝大数据股份有限公司 | Skin aging state evaluating method and electronic device |
| CN109730637A (en) * | 2018-12-29 | 2019-05-10 | 中国科学院半导体研究所 | A kind of face face-image quantified system analysis and method |
-
2019
- 2019-06-13 CN CN201910508742.8A patent/CN110210448B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20050039765A (en) * | 2005-02-21 | 2005-04-29 | 주식회사 아이티펌 | Method for extracting face outline and face feature points |
| CN109427053A (en) * | 2017-08-31 | 2019-03-05 | 丽宝大数据股份有限公司 | Skin aging state evaluating method and electronic device |
| CN109730637A (en) * | 2018-12-29 | 2019-05-10 | 中国科学院半导体研究所 | A kind of face face-image quantified system analysis and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110210448A (en) | 2019-09-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110210448B (en) | Intelligent face skin aging degree identification and evaluation method | |
| CN110097034B (en) | Intelligent face health degree identification and evaluation method | |
| CN110348319B (en) | Face anti-counterfeiting method based on face depth information and edge image fusion | |
| WO2021000524A1 (en) | Hole protection cap detection method and apparatus, computer device and storage medium | |
| CN107169953B (en) | Bridge concrete surface crack detection method based on HOG characteristics | |
| CN102426649B (en) | Simple steel seal digital automatic identification method with high accuracy rate | |
| CN113592845A (en) | Defect detection method and device for battery coating and storage medium | |
| CN105427275B (en) | Crop field environment wheat head method of counting and device | |
| CN103034838A (en) | Special vehicle instrument type identification and calibration method based on image characteristics | |
| CN104880389A (en) | Mixed crystal degree automatic measurement and fine classification method for steel crystal grains, and system thereof | |
| CN113240626A (en) | Neural network-based method for detecting and classifying concave-convex flaws of glass cover plate | |
| CN111639629A (en) | Pig weight measuring method and device based on image processing and storage medium | |
| CN111507426A (en) | No-reference image quality grading evaluation method and device based on visual fusion characteristics | |
| CN112862744B (en) | Intelligent detection method for internal defects of capacitor based on ultrasonic image | |
| CN111161237A (en) | Fruit and vegetable surface quality detection method, storage medium and sorting device thereof | |
| CN104574312A (en) | Method and device of calculating center of circle for target image | |
| Patki et al. | Cotton leaf disease detection & classification using multi SVM | |
| CN110766657B (en) | Laser interference image quality evaluation method | |
| CN106600615A (en) | Image edge detection algorithm evaluation system and method | |
| CN110059573A (en) | Wild ginseng based on image recognition is classified calibration method | |
| KR101782366B1 (en) | Vision inspection method based on learning data using normalization of sample scale | |
| CN115511814A (en) | Image quality evaluation method based on region-of-interest multi-texture feature fusion | |
| CN112884002B (en) | Intelligent electric meter version classification method | |
| CN114881984A (en) | Detection method and device for rice processing precision, electronic equipment and medium | |
| CN111523605B (en) | Image identification method and device, electronic equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |