CN110837782A - Method for identifying fracture information according to material stretching process monitoring video - Google Patents
Method for identifying fracture information according to material stretching process monitoring video Download PDFInfo
- Publication number
- CN110837782A CN110837782A CN201910982235.8A CN201910982235A CN110837782A CN 110837782 A CN110837782 A CN 110837782A CN 201910982235 A CN201910982235 A CN 201910982235A CN 110837782 A CN110837782 A CN 110837782A
- Authority
- CN
- China
- Prior art keywords
- fracture
- image
- region
- matched
- axis
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
Abstract
A method for identifying fracture information according to a material stretching process monitoring video belongs to the technical field of material fracture analysis and image identification. And then, performing mixed Gaussian model modeling on the small region extracted from the fracture region, and analyzing the internal stress condition and the overall change trend. Compared with the traditional digital image correlation method, the method has the advantages that the technical effect is achieved, under the condition that the accuracy is guaranteed, the more detailed internal change condition of the material can be obtained through the analysis of the parameters of the established model, and the method has good development prospect.
Description
Technical Field
The invention belongs to the technical field of material fracture analysis and image recognition, and particularly relates to a method for recognizing fracture information according to a monitoring video of a material stretching process.
Background
Digital Image Correlation (DIC) technology, also known as Digital speckle Correlation, was originally proposed by Yamaguchi and Peters in the 80 th 20 th century, and two Digital images before and after deformation of a test piece were subjected to Correlation calculation to obtain deformation information of a region of interest. The basic principle of the method is that the interesting region in the image before deformation is subjected to grid division, each sub-region is taken as rigid motion, then correlation calculation is carried out on each sub-region according to a predefined correlation function through a certain search method, a region with the maximum mutual pivot correlation coefficient with the sub-region, namely the position of the sub-region after deformation is searched in the image after deformation, the displacement of the sub-region is further obtained, all the sub-regions are calculated, and the deformation information of the whole field can be obtained. Nowadays, digital image correlation techniques have been widely applied to the fields of aviation, aerospace, materials, biology and the like due to a series of advantages of simple equipment, high measurement precision, low vibration isolation requirement and the like, and become the most popular morphology, deformation and strain measurement method in the field of optical measurement experimental mechanics.
At present, digital image correlation technology is mainly used for fracture identification of various materials, but the method still has some defects in the basic principle aspect of 1) shape function error cannot be effectively estimated; 2) for complex deformation, the traditional algorithm cannot give consideration to both speed and precision; 3) the deep mechanism of interpolation deviation is unknown, and an experimental measurement means is lacked; 4) lack of complete speckle quality evaluation criteria; 5) lack of theoretical models for speckle optimization; 6) the traditional two-dimensional DIC compensation method is very sensitive to image noise, the defects cause that the digital image correlation technology cannot adapt to various conditions, the obtained conclusion cannot always ensure the precision, and the method is greatly influenced by external factors, the method cannot obtain the stress condition and other internal information inside the material while the material appearance is deformed, and only can process the macroscopic image of the material, and the limitations make a new method urgently needed to solve the problems at present.
Disclosure of Invention
The invention aims to solve the technical problem of providing a method for modeling based on SIFT feature points and a Gaussian mixture model, monitoring videos according to a material stretching process and identifying fracture positions and internal stress conditions in the material stretching process. According to the invention, under the condition that the material is not required to be sprayed with speckles, the change of the monitoring video image information in the stretching process of the material is directly analyzed through SIFT feature points and Gaussian mixture model parameters, so that the time point and the position of the fracture of the material in the stretching process and the stress information causing the fracture are identified.
1. A method for identifying fracture information according to a monitoring video of a material stretching process comprises the following steps:
1.1, extracting feature points of each frame of image in a monitoring video by using an SIFT algorithm;
1.2 matching the characteristic points to determine the time node of fracture, comprising the following steps:
1.2.1, matching feature points extracted from every two adjacent frames of images in the monitored video;
1.2.2, counting the number of the feature points matched between each frame of image and the adjacent image, and if the ratio of the number of the matched feature points to the total number of the feature points of the frame of image is less than 30%, judging that the frame of image is a time node of fracture occurrence;
1.3 for the first frame image in the time node video with fracture, determining the specific area with fracture by using the number of matched characteristic points, including the following steps:
1.3.1 counting the number of matched characteristic points in each region with the width of x pixel points from a first column in the horizontal direction of the image;
1.3.2, translating to the right by one row, counting the number of the matched characteristic points in each region with the width of x pixel points again, and finding out the region which is matched except the region where the image edge is located and has the minimum number of the characteristic points after translating for x-1 times;
1.3.3 counting the number of matched characteristic points in each region with the width of y pixel points from the first line in the vertical direction of the image;
1.3.4, translating downwards for one line, counting the number of the matched characteristic points in each region with the width of y pixel points again, and finding out the region which is matched except the region where the image edge is located and has the minimum number of the characteristic points after translating for y-1 times;
1.3.5 judging the intersection part of the two minimum areas determined in the steps 1.3.2 and 1.3.4 as an area where the fracture occurs;
1.4 for the fracture occurrence region, identifying fracture information by modeling of a Gaussian mixture model, comprising the following steps:
1.4.1 extracting n image blocks with the size of k multiplied by k and containing matched feature points in the area where the fracture occurs;
1.4.2 carrying out mixed Gaussian model modeling on the extracted image blocks;
1.4.3 identifying fracture information by Gaussian mixture model parameters, comprising the following steps:
1.4.3.1, setting the covariance matrix of any model in the Gaussian mixture model as follows:
1.4.3.2 the model corresponding to the covariance matrix is represented by an ellipse, and the equation of the ellipse is calculated according to the value of the covariance matrix as follows:
Ax2+Dxy+By2＝1
the major axis inclination angle theta of the 1.4.3.3 ellipse is calculated by the formula:
the inclination angle θ of the major axis of the ellipse means: an angle formed by the long axis or the short axis of the ellipse and the X axis in the horizontal direction is less than 45 degrees, the angle of one axis of the long axis or the short axis is determined, and the angle of the other axis is theta +90 degrees;
1.4.3.4 the major axis length a and the minor axis length b of the ellipse are calculated as:
the coordinate of the central point of the 1.4.3.3 ellipse is the stress center of the model under the action of tensile resultant force, the length of the major axis is consistent with the tensile force of the material, and the direction of the force is consistent with theta; the length of the short axis is consistent with the magnitude of the pressed force, and the direction of the force is consistent with theta + 90.
According to the invention, SIFT feature points and a Gaussian mixture model are modeled and applied to a microscopic video passing through the material stretching process, the fracture time and area of the material and the stress information of the material are identified, and a new thought is provided for the microscopic video analysis of the material stretching. Compared with the traditional Digital Image Correlation (DIC), the method can provide more internal information of the material during fracture on the premise of ensuring the accuracy, can be applied to the subject fields of mechanics and other aspects, and has good development prospect.
Drawings
FIG. 1 is a flow chart of a method for identifying fracture information from a surveillance video of a material drawing process
FIG. 2 shows the matching of two image feature points before fracture
FIG. 3 shows the fracture zone cut out in step 1.3.5
FIG. 4 is a schematic drawing of a small block taken for modeling
Wherein: the size of the small block is 32 pixel points multiplied by 32 pixel points
FIG. 5 is an ellipse drawn from the model parameters of each model of the first patch
Wherein: (a) respectively representing ellipses drawn according to model parameters of 5 models established by the first small block of the previous frame
FIG. 6 is a schematic diagram showing the results of the tensile forces experienced by each model of the first patch during the experiment
Detailed Description
The core content of the invention is as follows: the method comprises the steps of monitoring a video according to the material stretching process, identifying the fracture position and the internal stress condition of the material in the stretching process, directly extracting and matching image SIFT feature points in the change process and modeling the Gaussian mixture model under the condition that speckles do not need to be sprayed on the material, obtaining the change condition of the interior of the material when the material is fractured through parameter analysis of the final model, and providing more effective information.
1. A method for identifying fracture information according to a monitoring video of a material stretching process is characterized by comprising the following steps:
1.1, extracting feature points of each frame of image in a monitoring video by using an SIFT algorithm;
1.2 matching the characteristic points to determine the time node of fracture, comprising the following steps:
1.2.1, matching feature points extracted from every two adjacent frames of images in the monitored video;
1.2.2, counting the number of the feature points matched between each frame of image and the adjacent image, and if the ratio of the number of the matched feature points to the total number of the feature points of the frame of image is less than 30%, judging that the frame of image is a time node of fracture occurrence;
1.3 for the first frame image in the time node video with fracture, determining the specific area with fracture by using the number of matched characteristic points, including the following steps:
1.3.1 counting the number of matched characteristic points in each region with the width of x pixel points from a first column in the horizontal direction of the image;
1.3.2, translating to the right by one row, counting the number of the matched characteristic points in each region with the width of x pixel points again, and finding out the region which is matched except the region where the image edge is located and has the minimum number of the characteristic points after translating for x-1 times;
1.3.3 counting the number of matched characteristic points in each region with the width of y pixel points from the first line in the vertical direction of the image;
1.3.4, translating downwards for one line, counting the number of the matched characteristic points in each region with the width of y pixel points again, and finding out the region which is matched except the region where the image edge is located and has the minimum number of the characteristic points after translating for y-1 times;
1.3.5 judging the intersection part of the two minimum areas determined in the steps 1.3.2 and 1.3.4 as an area where the fracture occurs;
1.4 for the fracture occurrence region, identifying fracture information by modeling of a Gaussian mixture model, comprising the following steps:
1.4.1 extracting n image blocks with the size of k multiplied by k and containing matched feature points in the area where the fracture occurs;
1.4.2 carrying out mixed Gaussian model modeling on the extracted image blocks;
1.4.3 identifying fracture information by Gaussian mixture model parameters, comprising the following steps:
1.4.3.1, setting the covariance matrix of any model in the Gaussian mixture model as follows:
1.4.3.2 the model corresponding to the covariance matrix is represented by an ellipse, and the equation of the ellipse is calculated according to the value of the covariance matrix as follows:
Ax2+Dxy+By2＝1
the major axis inclination angle theta of the 1.4.3.3 ellipse is calculated by the formula:
the inclination angle θ of the major axis of the ellipse means: an angle formed by the long axis or the short axis of the ellipse and the X axis in the horizontal direction is less than 45 degrees, the angle of one axis of the long axis or the short axis is determined, and the angle of the other axis is theta +90 degrees;
1.4.3.4 the major axis length a and the minor axis length b of the ellipse are calculated as:
the coordinate of the central point of the 1.4.3.3 ellipse is the stress center of the model under the action of tensile resultant force, the length of the major axis is consistent with the tensile force of the material, and the direction of the force is consistent with theta; the length of the short axis is consistent with the magnitude of the pressed force, and the direction of the force is consistent with theta + 90.
1. Working conditions
The experimental platform adopts Intel (R) core (TM) i5-8500 CPU @3.10GHz 3.10GHz, the memory is 4GB, a PC running Windows 7 is adopted, and the programming language is MATLAB language.
2. Analysis of experimental content and results
The time node at which the fracture starts to occur can be determined according to steps 1.1 to 1.2.2, and the specific result is shown in fig. 2. The area of the entire picture where the fracture occurred was then determined according to step 1.3, the specific results are shown in fig. 3. This achieves the first objective of finding the time and area where the fracture occurred.
Next, according to the distribution of the matching feature points in the fracture region of the fracture frame picture, a small block of 5 32 pixels × 32 pixels intercepted in the fracture region is determined for modeling analysis, as shown in fig. 4. Parametric analysis was performed using the first of the 5 patches taken as an example, and the covariance matrices of the 5 models are shown in table 1:
TABLE 1
In table 1, X in each matrix represents coordinates of each model in the X-axis direction, Y represents coordinates in the Y-axis direction, and Z represents gray scale values. The 5 matrices are the autocorrelations of the 5 models of the first patch shown in fig. 5, which are real symmetric matrices, and the parameters of the matrices represent the cross correlation coefficients between each two of the 3 elements, which reflects the correlation between each 2 elements. The average of the coordinates and gray scale for each model is shown in table 2:
TABLE 2
| 1 | 2 | 3 | 4 | 5 | |
| X | 6.3287 | 21.4286 | 6.0548 | 23.4383 | 19.4099 |
| Y | 11.9661 | 25.4717 | 27.7177 | 9.4309 | 18.3279 |
| Z | 50.5216 | 53.1159 | 43.7124 | 69.4701 | 110.5273 |
In table 2, X and Y represent the average of 2-dimensional coordinates of all points included in each model, respectively, and Z represents the average of gray values of all points included in each model. And then, according to the self mathematical relationship of the ellipse, calculating the ellipse formed by each model generating deformation on the basis of the unit circle. The parameters of the ellipses represented by the 5 models are shown in table 3:
TABLE 3
| 1 | 2 | 3 | 4 | 5 | |
| Long shaft | 13.3825 | 11.3508 | 6.6351 | 11.6380 | 17.9725 |
| Short shaft | 7.0235 | 7.3429 | 5.1660 | 10.5746 | 14.0405 |
| Inclination angle of long axis | -0.0607 | -9.1717 | -9.0269 | 33.0082 | -14.3673 |
The ellipse which can describe the shape of each model as shown in fig. 5 is obtained according to the above steps, and parameters such as the major axis, the minor axis, and the major axis inclination angle θ of each ellipse are calculated by using a mathematical method, as shown in table 3. By analyzing these parameters, the stress condition and the deformation tendency of the material in the time interval represented by the two images can be obtained, and finally the labeled graph shown in fig. 6 can be obtained. Fig. 6 is a graph of the result of labeling the tensile force applied to the model, the white point is the center of each model, the black line segment with arrows is the variation trend of the model, and the length and direction of the line segment represent the magnitude and direction of the applied force.
Claims (1)
1. A method for identifying fracture information according to a monitoring video of a material stretching process is characterized by comprising the following steps:
1.1, extracting feature points of each frame of image in a monitoring video by using an SIFT algorithm;
1.2 matching the characteristic points to determine the time node of fracture, comprising the following steps:
1.2.1, matching feature points extracted from every two adjacent frames of images in the monitored video;
1.2.2, counting the number of the feature points matched between each frame of image and the adjacent image, and if the ratio of the number of the matched feature points to the total number of the feature points of the frame of image is less than 30%, judging that the frame of image is a time node of fracture occurrence;
1.3 for the first frame image in the time node video with fracture, determining the specific area with fracture by using the number of matched characteristic points, including the following steps:
1.3.1 counting the number of matched characteristic points in each region with the width of x pixel points from a first column in the horizontal direction of the image;
1.3.2, translating to the right by one row, counting the number of the matched characteristic points in each region with the width of x pixel points again, and finding out the region which is matched except the region where the image edge is located and has the minimum number of the characteristic points after translating for x-1 times;
1.3.3 counting the number of matched characteristic points in each region with the width of y pixel points from the first line in the vertical direction of the image;
1.3.4, translating downwards for one line, counting the number of the matched characteristic points in each region with the width of y pixel points again, and finding out the region which is matched except the region where the image edge is located and has the minimum number of the characteristic points after translating for y-1 times;
1.3.5 judging the intersection part of the two minimum areas determined in the steps 1.3.2 and 1.3.4 as an area where the fracture occurs;
1.4 for the fracture occurrence region, identifying fracture information by modeling of a Gaussian mixture model, comprising the following steps:
1.4.1 extracting n image blocks with the size of k multiplied by k and containing matched feature points in the area where the fracture occurs;
1.4.2 carrying out mixed Gaussian model modeling on the extracted image blocks;
1.4.3 identifying fracture information by Gaussian mixture model parameters, comprising the following steps:
1.4.3.1, setting the covariance matrix of any model in the Gaussian mixture model as follows:
1.4.3.2 the model corresponding to the covariance matrix is represented by an ellipse, and the equation of the ellipse is calculated according to the value of the covariance matrix as follows:
Ax2+Dxy+By2＝1；
the major axis inclination angle theta of the 1.4.3.3 ellipse is calculated by the formula:
the inclination angle θ of the major axis of the ellipse means: an angle formed by the long axis or the short axis of the ellipse and the X axis in the horizontal direction is less than 45 degrees, the angle of one axis of the long axis or the short axis is determined, and the angle of the other axis is theta +90 degrees;
1.4.3.4 the major axis length a and the minor axis length b of the ellipse are calculated as:
the coordinate of the central point of the 1.4.3.3 ellipse is the stress center of the model under the action of tensile resultant force, the length of the major axis is consistent with the tensile force of the material, and the direction of the force is consistent with theta; the length of the short axis is consistent with the magnitude of the pressed force, and the direction of the force is consistent with theta + 90.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910982235.8A CN110837782B (en) | 2019-10-16 | 2019-10-16 | Method for identifying fracture information according to material stretching process monitoring video |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910982235.8A CN110837782B (en) | 2019-10-16 | 2019-10-16 | Method for identifying fracture information according to material stretching process monitoring video |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110837782A true CN110837782A (en) | 2020-02-25 |
| CN110837782B CN110837782B (en) | 2022-06-03 |
Family
ID=69575325
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910982235.8A Active CN110837782B (en) | 2019-10-16 | 2019-10-16 | Method for identifying fracture information according to material stretching process monitoring video |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110837782B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111442986A (en) * | 2020-03-27 | 2020-07-24 | 河南中烟工业有限责任公司 | Method for detecting tensile property of cut tobacco |
| CN113378820A (en) * | 2021-07-02 | 2021-09-10 | 深圳市东亿健康服务有限公司 | Method and system for identifying digital pathological section target area |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105654461A (en) * | 2014-12-02 | 2016-06-08 | 中国科学院沈阳自动化研究所 | Machine vision detection method for fracture of multi-bundle conductor spacer |
| CN106530274A (en) * | 2016-10-11 | 2017-03-22 | 昆明理工大学 | Steel bridge crack positioning method |
| CN106529591A (en) * | 2016-11-07 | 2017-03-22 | 湖南源信光电科技有限公司 | Improved MSER image matching algorithm |
| CN107292917A (en) * | 2017-06-15 | 2017-10-24 | 武汉理工大学 | A kind of Dragging recognition methods based on Computer Vision |
| US20170343481A1 (en) * | 2016-05-27 | 2017-11-30 | Purdue Research Foundation | Methods and systems for crack detection |
| CN107610094A (en) * | 2017-08-02 | 2018-01-19 | 长安大学 | Based on the three-dimensional crack 3 D detection method characterized of ellipsoid |
| CN107610092A (en) * | 2017-08-01 | 2018-01-19 | 长安大学 | Pavement crack dynamic testing method based on video flowing |
| CN108009535A (en) * | 2017-11-21 | 2018-05-08 | 武汉中元华电科技股份有限公司 | A kind of simple pointer meter reading method based on machine vision |
-
2019
- 2019-10-16 CN CN201910982235.8A patent/CN110837782B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105654461A (en) * | 2014-12-02 | 2016-06-08 | 中国科学院沈阳自动化研究所 | Machine vision detection method for fracture of multi-bundle conductor spacer |
| US20170343481A1 (en) * | 2016-05-27 | 2017-11-30 | Purdue Research Foundation | Methods and systems for crack detection |
| CN106530274A (en) * | 2016-10-11 | 2017-03-22 | 昆明理工大学 | Steel bridge crack positioning method |
| CN106529591A (en) * | 2016-11-07 | 2017-03-22 | 湖南源信光电科技有限公司 | Improved MSER image matching algorithm |
| CN107292917A (en) * | 2017-06-15 | 2017-10-24 | 武汉理工大学 | A kind of Dragging recognition methods based on Computer Vision |
| CN107610092A (en) * | 2017-08-01 | 2018-01-19 | 长安大学 | Pavement crack dynamic testing method based on video flowing |
| CN107610094A (en) * | 2017-08-02 | 2018-01-19 | 长安大学 | Based on the three-dimensional crack 3 D detection method characterized of ellipsoid |
| CN108009535A (en) * | 2017-11-21 | 2018-05-08 | 武汉中元华电科技股份有限公司 | A kind of simple pointer meter reading method based on machine vision |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111442986A (en) * | 2020-03-27 | 2020-07-24 | 河南中烟工业有限责任公司 | Method for detecting tensile property of cut tobacco |
| CN113378820A (en) * | 2021-07-02 | 2021-09-10 | 深圳市东亿健康服务有限公司 | Method and system for identifying digital pathological section target area |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110837782B (en) | 2022-06-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Guo et al. | Graph attention tracking | |
| Lee et al. | Regularization strategy for point cloud via rigidly mixed sample | |
| CN106204638B (en) | It is a kind of based on dimension self-adaption and the method for tracking target of taking photo by plane for blocking processing | |
| US20190318158A1 (en) | Multi-pose face feature point detection method based on cascade regression | |
| CN106934800B (en) | Metal plate strip surface defect detection method and device based on YOLO9000 network | |
| CN105117720B (en) | Target scale adaptive tracking method based on space-time model | |
| CN108171133B (en) | Dynamic gesture recognition method based on characteristic covariance matrix | |
| CN104700412B (en) | A kind of calculation method of visual saliency map | |
| CN111507976B (en) | Defect detection method and system based on multi-angle imaging | |
| CN110837782B (en) | Method for identifying fracture information according to material stretching process monitoring video | |
| CN112949408B (en) | Real-time identification method and system for target fish passing through fish channel | |
| CN110008828B (en) | Pairwise constraint component analysis measurement optimization method based on difference regularization | |
| CN108446583A (en) | Human bodys' response method based on Attitude estimation | |
| CN111368682B (en) | Method and system for detecting and identifying station caption based on master RCNN | |
| CN111738344A (en) | Rapid target detection method based on multi-scale fusion | |
| CN104794714A (en) | Image segmentation quality evaluating method based on ROC Graph | |
| CN104318559A (en) | Quick feature point detecting method for video image matching | |
| CN103903280A (en) | Subblock weight Mean-Shift tracking method with improved level set target extraction | |
| CN114199892B (en) | Plate measuring method and system based on machine vision | |
| CN109584206B (en) | Method for synthesizing training sample of neural network in part surface flaw detection | |
| Liang et al. | Automatic defect detection of texture surface with an efficient texture removal network | |
| CN104036528A (en) | Real-time distribution field target tracking method based on global search | |
| Blackledge et al. | A surface inspection machine vision system that includes fractal texture analysis | |
| CN116958052A (en) | Printed circuit board defect detection method based on YOLO and attention mechanism | |
| CN112633327B (en) | Staged metal surface defect detection method, system, medium, equipment and application |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |